Click here to see the new installation guide for OMF 5.2

OMF installation guide for version 5.1 (previously known as 1.3)¶

We are currently restructuring our software packages. Please report any issues on the issue tracker. Thanks!¶

The following steps are necessary to install and configure the OMF Aggregate Manager on an Ubuntu or Debian box. We will gradually automate some of these steps and remove them from this guide in the future to simplify the installation.

Alongside this quick installation guide, we strongly recommend that you read the (slightly outdated) deployment manual (PDF) which can be downloaded on the Installation page.

System Requirements¶

• For the aggregate manager: A x86-compatible PC with at least two network interfaces. The control network interface should be Ethernet, while the experiment network interface can be Ethernet, Wireless LAN or any other technology.
• A number of testbed nodes. Currently we only supply precompiled binaries for x86. Each node should have one control network interface (Ethernet) and one or more experiment network interfaces.
• An Ethernet switch for the control network. Connect the control interfaces of the Aggregate Manager and of each node to this switch.
• If your experiment network is using Ethernet, you need another switch (or separate VLAN) to link the experiment network interfaces of the AM and the the nodes.

Suggested Testbed Layout¶

Configuration of the Aggregate Manager¶

• Install Ubuntu 8.04 (Hardy Heron) or Debian 5.0 (Lenny) on the AM PC
• There is no need to install any operating system on the nodes, since we will upload an image onto their drives later
• Open a terminal
• sudo su on Ubuntu, su on Debian

Network Setup¶

• ensure you have a working Internet connection on your eth0 interface (ifconfig eth0)
• apt-get remove network-manager (Ubuntu only)
• open the file /etc/network/interfaces and configure eth0 for DHCP (or static IP address, depending on your network):

auto eth0                                                                                      
iface eth0 inet dhcp

• ifup eth0
• again, ensure you have a working Internet connection on your eth0 interface (ifconfig eth0)
• apt-get install ssh atftpd syslinux dhcp3-server ntp
• ignore the error message "invoke-rc.d: initscript dhcp3-server, action "start" failed."
• configure eth1 to 10.0.1.200. To do that, open /etc/network/interfaces and replace any existing "eth1" section with:

auto eth1                                                                                      
iface eth1 inet static
address 10.0.1.200
netmask 255.255.255.0

• of course you can use a different IP address or a different subnet for your control network. This guide will assume 10.0.1.200 for the AM IP address and 10.0.1.0/24 as the subnet.
• then run ifdown eth1; ifup eth1 and check the interface configuration (ifconfig eth1)

Configure the DHCP server¶

• edit /etc/default/dhcp3-server and change the last line to:

    INTERFACES="eth1" 

• edit /etc/dhcp3/dhcpd.conf and comment out the following lines:

    # option domain-name ...
    # option domain-name-servers ...

• Change the default-lease-time in that file to 6000
• At the end of that file, append a subnet declaration:

subnet 10.0.1.0 netmask 255.255.255.0 {
  filename "/pxelinux.0";
  next-server 10.0.1.200;
  option ntp-servers 10.0.1.200;
  use-host-decl-names on;

  host node001 {
    hardware ethernet 00:03:2D:0D:30:C1;
    fixed-address 10.0.1.1;
  }
  host node002 {
    hardware ethernet 00:03:2D:0D:30:D5;
    fixed-address 10.0.1.2;
  }
  host node003 {
    hardware ethernet 00:03:2D:0D:30:CD;
    fixed-address 10.0.1.3;
  }
  host node004 {
    hardware ethernet 00:03:2D:0D:30:CF;
    fixed-address 10.0.1.4;
  }
}

• and replace the MAC addresses in this part with the ones from your nodes (control interfaces).
• /etc/init.d/dhcp3-server restart

Set up PXE booting¶

• mkdir -p /tftpboot/pxelinux.cfg; ln -s /usr/lib/syslinux/pxelinux.0 /tftpboot/
• edit /etc/default/atftpd and change USE_INETD=true to USE_INETD=false
• in the same file, add --daemon --bind-address 10.0.1.200 to the OPTIONS line
• also check that the directory given at the end of the OPTIONS line is /tftpboot
• /etc/init.d/atftpd restart
• drop the pxe kernel and root filesystem into the /tftpboot directory. Build them yourself from the git master or download a precompiled release here
• create the file /tftpboot/pxelinux.cfg/pxeconfig with the following contents:

SERIAL 0 38400 0
DEFAULT linux
LABEL linux
KERNEL linux-orbit-pxe-2.6.29.2
APPEND console=ttyS0,38400n8 console=tty0 vga=normal root=/dev/rd/0 rw ramdisk_size=32768 initrd=initramfs-orbit-pxe-2.0.4.gz control=eth0
PROMPT 0

• be careful not to wrap any of those lines in the editor
• make sure the initrd and kernel file names match the ones you've placed in the /tftpboot directory
• match the control=eth0 parameter to your setup. Use the Linux device name of your PXE interface (control interface) here. If you only have one Ethernet card, use eth0. If you have more than one card and you don't know the Linux interface name of your control interface, try eth0 first. If you get a message saying "ifconfig: eth0: error fetching interface information: Device not found" when you later on boot from PXE, try eth1 instead (and then eth2, eth3 and so on if you have 3 or 4 or more Ethernet cards).
• create the file /tftpboot/pxelinux.cfg/default and add:

DEFAULT harddrive
LABEL harddrive
localboot 0

• change directory to /tftpboot/pxelinux.cfg/ and for each node, run ln -s pxeconfig 01-00-03-2D-0D-30-C1 (replace with control network interface MAC address of the node and insert a "01-" in front of it)

At this point you should be able to network-boot your testbed nodes. Enable PXE on the control network interface on each of the nodes via their BIOS and move it to the top of the boot device order. Set your primary harddrive to the second place in the boot order. Now the nodes should receive an IP address via BOOTP and look for a PXE configuration file on the TFTP server. The first filename it looks for is its own MAC address. You have created that symlink in the steps above. If it cannot find such a file on the TFTP server, it will eventually fall back to "default". The "default" file simply instructs the system to boot from the harddrive and not pull any kernel or filesystem image from the TFTP server.

A "tail -f /var/log/syslog" can help debugging BOOTP and TFTP. Use telnet to log on to the network-booted nodes. After you've confirmed that PXE booting works, remove the symlinks created in the previous step. Upon rebooting, the nodes should now get instructions via PXE to boot from their local hard drives.

PXE booting works? Fantastic! Continue by following the next steps:¶

Install the FTP server (for image uploads)¶

• apt-get install vsftpd
• mkdir -m 777 /home/ftp/upload
• chown ftp. -R /home/ftp
• chmod 555 /home/ftp
• Then uncomment these lines in /etc/vsftpd.conf:

write_enable=YES
anon_upload_enable=YES

• and add the line

anon_umask=000

• /etc/init.d/vsftpd restart

Inventory installation¶

• apt-get install mysql-server libdb4.4 phpmyadmin (replace libdb4.4 with libdb4.6 on Debian)
• when prompted, enter a mysql root password of your choice or just press enter for no password
• when prompted, configure phpmyadmin to use apache2
• run mysql -u root -p and insert the following command sequence:

use mysql;
create user 'orbit'@'localhost';
grant all on *.* to 'orbit'@'localhost';
set password for 'orbit'@'localhost'=password('orbit');
quit;

• you may obviously use a different user and password, but don't forget to add these to the gridservice configuration files later
• run mysql -u orbit -porbit and insert the following command sequence (after authenticating with password "orbit"):

create database inventory;
quit;

• to populate the inventory database, you may import this sql file. It contains all the necessary tables and is filled with sample entries from on of our sandboxes. You may edit it to your needs before or after importing it.
• to import our sample inventory, run mysql -u orbit -porbit inventory < inventory-sample.sql
• you should now be able to open your browser at http://localhost/phpmyadmin/ and log on as orbit/orbit. Have a look at the inventory database tables to get a feel for what goes in there. Check InstallationInventory_ for a detailed description of each table. Modify the inventory contents to match your setup accordingly.
• your inventory database must hold information about the testbed name (e.g. the 'testbeds' table must have an entry corresponding to this name)
• For every node in your testbed, you need to add a database record in each of the tables devices, locations, motherboards and nodes
• The inventory requires some manual work, and describing each step would take this quick installation guide too far. If you are reasonably convinced that you've filled the tables correctly, please proceed with the guide. You can optimize the inventory at a later stage.*

Configuring the gridservices¶

• add this line to your /etc/apt/sources.list:

deb http://pkg.mytestbed.net/ubuntu hardy/ 

• apt-get update
• apt-get install libsqlite3-ruby1.8 libldap-ruby1.8 liblog4r-ruby1.8 libmysql-ruby1.8 libruby1.8 ruby1.8 libmarkaby-ruby libcoderay-ruby1.8 rubygems gridservices2 nodehandler4 oml-server
• update-rc.d gridservices2 defaults
• in the directory /etc/gridservices2/available, copy each file that is named file.yaml.nicta to file.yaml, unless you are at Winlab, where you copy each file.yaml.winlab to file.yaml
• enable each gridservice by creating the following symlinks:

mkdir /etc/gridservices2/enabled
cd /etc/gridservices2/enabled
ln -s ../available/cmcStub.yaml
ln -s ../available/frisbee.yaml
ln -s ../available/pxe.yaml
ln -s ../available/oml2Server.yaml
ln -s ../available/inventory.yaml
ln -s ../available/result.yaml

• Modify the configuration file called /etc/gridservices2/available/inventory.yaml to reflect your mySQL database configuration
• Modify the configuration file called /etc/gridservices2/available/frisbee.yaml to set up the directory containing the testbed filesystem images, and the multicast interface (set this to your AM's control network IP address)

...
imageDir: /home/ftp/upload
...
multicastIF: 10.0.1.200
...

• Modify the configuration file called /etc/gridservices2/available/oml2Server.yaml and change the testbed name ("npc" in this example) to your testbed name. Also make sure the AM IP address is correct here. Also set the path of the oml2-server binary to /usr/bin/oml2-server.

...
   npc:
      # Address for collection server to bind to.
      serverAddress: '10.0.1.200'
      serverBin: '/usr/bin/oml2-server'
...

• /etc/init.d/gridservices2 start
• whenever you edit a .yaml file, be sure to run /etc/init.d/gridservices2 restart to apply the changes
• test the CMC gridservice with wget -qO- http://localhost:5022/cmc/allStatus?domain=your_testbed_name

Configuring the Nodehandler¶

• If you are not at Winlab and you do not have hardware with our custom CMC card installed (to trigger hardware reboots over the network), you must use the cmcStub gridservice instead of the cmc gridservice. Enter the directory /opt/nodehandler4-4.4.0/app/ and create two softlinks: ln -s omf_load.nicta omf_load and ln -s omf_save.nicta omf_save. These .nicta files contain a hack to soft-reboot the nodes remotely during image load and save operation. If you are at Winlab, please create the soft links to the .winlab files.
• In /etc/nodehandler4/nodehandler.yaml, enter your testbed name ("npc" in this example):

...
default:
      name: 'npc'

      inventory:
        url: 'http://localhost:5022/inventory'
...

Loading and saving disk images¶

• warning: all partitions and all data on the node's disk will be overwritten when you load an image!
• check the testbeds table in the inventory and make sure you have the correct Linux device name of the node's hard drive in the column "frisbee_default_disk" of your testbed
• download our default baseline image from here and place it in /home/ftp/upload
• load an image onto a node by running omf load [x,y] where x and y correspond to the coordinates configured in the locations table in the inventory (e.g. omf load [1,1])
• after loading the image, the node should boot from its local hard drive. SSH into the node as user "root" with password "voyage".
• save an image from a node (will be uploaded to the /home/ftp/upload directory) with the command omf save [x,y]

Head over to Basic_Tutorials_for_Developing_Experiments and follow the tutorial to run your first "Hello World" experiment!

The following instructions are not (yet) necessary for the current testbed:¶

• apt-get install sun-java5-jre
• download the openfire debian package from http://www.igniterealtime.org/downloads/index.jsp and install it using dpkg -i