Title: README.txt Description: README for the virtual machine images distribution for replication of ManyBugs-based experiments. Author: Claire Le Goues Contact: claire.legoues@gmail.com Original: March 19, 2012 Last Revised: January 25, 2016 ******************************************************************************** * Overview ******************************************************************************** This directory contains VM images and pointers to AMIs for use in running repair experiments on the ManyBugs scenarios. We conducted these experiments on the Amazon EC2 cloud, using a custom Amazon Machine Image (AMI). At a high level, an AMI is a bundle of a disk image, a ramdisk, and a kernel, configured to work with Xen. We additionally converted the 32-bit Fedora AMI into an image suitable for running in the VirtualBox virtualization software, which has the benefits of being free and relatively easy to use/well-documented. If you would like to try the 64-bit Fedora 20 AMI with the 32-bit Fedora 13 chroot jail, you must do so on EC2. This README also tells you how to get the VirtualBox machine image and use it to run the experiments we did on AWS. Running such experiments locally is slower and significantly less scalable than doing so on AWS, but it is free. You are welcome to use any other VM environment, but it's on you to do the conversion. We include the raw disk image, kernel, and ramdisk, which can theoretically be manipulated into basically any kvm- or xen-based cloud framework, though this can be irritatingly difficult and we do not provide support for it. This directory therefore contains: genprog_icse2012_virtualbox/ --> genprog_icse2012_virtualbox.vmdk --> genprog_icse2012_virtualbox.ovf The .ovf file describes the machine, .mf is the manifest, the .vmdk is the hard drive. genprog_icse2012_aws/ --> genprog_icse2012_fedora13.raw --> vmlinuz-2.6.34.9-69.fc13.i686.PAE --> initramfs-2.6.34.9-69.fc13.i686.PAE.img The bulk of these directions refer to the two images (.raw and .vmdk) in the subdirectories here. These are still accurate and apply as well to virtual machines launched on Amazon's EC2. As of January 25, 2016, we have made available public amis that can be accessed from ec2 in the us-east (N. Virginia) region: AMI, AMI-Name, Description ami-525eb23a, GenProg May 2014, GenProg May 2014 32-bit Fedora-13 Benchmarks and Experiments ami-e746658d, autobugfixbenchmarks-ManyBugs-64bit-ami, GenProg May 2014 64-bit Fedora-20 w/Fedora 13 chroot Benchmarks and Experiments (if you search for some combination of the AMI number, name, or description on the "Community AMI" pane in launch instances, you should find them.) The first AMI is an almost-copy of the 32-bit fedora 32 image, of which you can find alternative copies in this directory. It is somewhat cleaned up as compared to the VirtualBox image (matching the two is on the todo list). The genprog code checked out on it is version 1688, the snapshot released on October 2015. It contains both CIL 1.3.7 and CIL 1.5.1; the latter is necessary to compile more recent version of GenProg. The second AMI is an image running 64-bit Fedora 20 with a 32-bit Fedora chroot jail. When you login, sh mount.sh; sh chroot.sh, and you will be dropped into the jail. The intention behind this AMI is to allow you to compile your own repair technique using modern libraries and additional memory requirements and still use the ManyBugs benchmarks in the 32-bit Fedora 13 environment in which they were developed. This is particularly important if you'd like to compare to old versions of GenProg built in the 32-bit environment, or if you'd like to run on the fbc scenarios. You just need to link against the Fedora 13 32-bit libraries when building outside the jail and copy it over. Other than that, once you're in some environment that looks like 32-bit fedora, the directions below apply. ******************************************************************************** * Why a VM? ******************************************************************************** Bugs are surprisingly tricky to replicate, despite their ubiquity. In the interest of reproducibility of the tests, compilation processes, and bugs, we constructed a VM with a disk image based on the (lab-) machine on which we initially found the bugs. It happened to be running fedora 13. Additionally, GenProg makes random changes to the input program, Sandboxing is thus a legitimate concern to guard against runaway behavior. A VM is nice because it doesn't matter if you accidentally rm -rf /. Not that we have reason to believe this will ever happen, but we did accidentally make a couple of spambots while trying to repair the email module in Python (Amazon really doesn't like this, by the way), so never say never. ******************************************************************************** * Notes on experimental replication ******************************************************************************** You must untar the bug scenarios into /root/genprog-many-bugs/. For example, given scenario fbc-bug-5458-5459.tar.gz, your directory structure should look like /root/genprog-many-bugs/fbc-bug-5458-5459/. Please ignore anything in genprog-many-bugs other than what you put there (some of the images have leftover scripts that are not useful to you; one of these days, I'll remove them from all of the images). To run a repair scenario, cd into the untarred directory and run repair with appropriate configuration options. All runs of genprog print out all configured options and their parameters in the log files. You may also want to look at default-configuration or oracle-configuration in the scenarios to see other options. Note that all experiments since ICSE 2012 have used additional arguments beyond what is in default-configuration, but since default-configuration contains at least what is necessary to specify the location of the source code, test cases, and build files, we tend to pass default-configuration to repair and then additional configuration options at the command line. Any options passed after a config file take precedence over the config file. For example, you might run /path/to/repair default-configuration --appp 0.7 --repp 1.0 --swapp 0.0 to change the probability of the append, repair, and swap operators from their default values. The source code (/root/mountpoint-genprog/genprog-code) and binary (/root/mountpoint-genprog/genprog-code/repair/repair) on these images are note the most recent version of the code; you must update it yourself. On the VirtualBox images, they are the code used for ICSE 2012; see notes above for the AMI information. In general, we provide them for demonatration and encourage you to put your own versions on there instead. ******************************************************************************** * VirtualBox ******************************************************************************** Download and install VirtualBox for your machine: https://www.virtualbox.org/wiki/Downloads I exported this machine using version 3.2.8. VirtualBox is pretty good about maintaining backwards compatibility, but I can't make promises In general, there are several ways to do any of the things I tell you about in this README; these are all "one way", there are be others. Consult the high-quality VirtualBox documentation for more. === High-level Notes === * The virtual machine username is root, password is "password". * In VirtualBox, the "host" machine is the machine running VirtualBox and the "guest" machine is the virtual machine you're running in VirtualBox. Guest machines have names in a given instance of VirtualBox, like "genprog_icse2012_fedora13". When I say or anything similar in the following instructions, I'm referring to the name of the guest machine according to VirtualBox. By default, for the machine associated with this README, this name will be genprog_icse2012_fedora13. ******************************************************************************** * Import the Virtual Machine ******************************************************************************** With the GUI: go to File -> import, choose the .ovf file corresponding to the machine you want to import, hit next, hit finish, wait. From the command line: # import the virtual machine image # if successful output ends in "Successfully imported the appliance" VBoxManage import /path/to/genprog_icse2012_fedora13.ovf ******************************************************************************** * Run the Virtual Machine ******************************************************************************** === GUI === Select the machine in the list on the left, hit start. A window should pop up. === Command line === Run in "headless" mode: > VBoxHeadless --startvm "" --vrdp=off & No window will pop up; you will have to interact with your machine over ssh (see below). (Digression: vrdp lets you connect with some program from some other (non-host) machine and see the GUI; I don't use it so I turn it off in the interest of speed.) To poweroff a machine in headless mode: > VBoxManage controlvm poweroff ******************************************************************************** * Settings you may want to change ******************************************************************************** A guest machine must not be running or paused (must be powered off) when you change these settings. === Name === * GUI: Select the memory whose name you want to change, select Settings (the big gear up top), in the first dialogue ("General"), change the name in the name text box. * Command line: > VBoxManage modifyvm "" --name "new vm name" === System memory === VirtualBox will not allow you to allocate more than 75% of the host machine's memory to the guest machine, since the memory in use by the guest is unavailable to the host while you're running the guest. Our EC2 machines had 1.7 GB of memory, however. The machine image here is set to 768 MB, which is rather more reasonable and has worked for me in the past. You're welcome to set it as high as you can. * GUI: Select the machine whose Base Memory you want to change, select Settings (the big gear up top), select System in the left-hand list, set "Base Memory" to whatever you want it to be (within reason for your Host machine), hit OK. * Command Line: > VBoxManage modifyvm "" --memory "memory size in MB" ******************************************************************************** * Networking/Interacting with your VM over SSH ******************************************************************************** I have set the VirtualBox image to use bridged networking. If you want to ssh in, however, you can't use port 22 (the ssh default), because port 22 on your machine will probably be talking to your host machine's ssh server, not the one on the vm. The standard solution to this is to set the VirtualBox networking to forward requests to some other port (2222 is pretty canonical) on the host machine to port 22 on the guest machine. To ssh in, direct your ssh request to port 2222 on the host machine to get to the guest. From the host: > ssh -p 2222 root@localhost From some other machine: > ssh -p 2222 root@host.machine.address.edu The same trick applies to scp, except the port argument is specified with a capital P: > scp -P 2222 root@localhost To change this setting to pick a different port: > VBoxManage modifyvm "VM name" --natpf1 "guestssh,tcp,,PORT_TO_FORWARD_TO,,PORT_TO_FORWARD_FROM" To illustrate, I set up the machine image with: > VBoxManage modifyvm "VM name" --natpf1 "guestssh,tcp,,2222,,22"