How to install and use the FTB software
Step 1. Download FTB-0.6 from the CIFTS website
Alternatively, you can download it from the SVN TAGS directory at: https://svn.mcs.anl.gov/repos/cifts/ftb/tags/
The code in the SVN trunk (https://svn.mcs.anl.gov/repos/cifts/ftb/trunk) refers to the current development code. While we attempt to keep the trunk code stable (and working) at most times - please be aware that the under-development code might cause a few hiccups.
To download from the SVN trunk, you can run the following command:
svn co https://svn.mcs.anl.gov/repos/cifts/ftb/trunk ftb
If you are not familiar with using SVN, please refer to (http://svn.collab.net/svn-doxygen/) for further guidance.
Preparing your build environment (Linux, IBM BG series, Cray XT)
The FTB software supports linux (Ubuntu) clusters, IBM BG/L, IBM BG/P and the Cray XT series. To get the FTB software working, you may need to install additional libraries or configure certain minor aspects of your cluster - especially for the peta-scale machines. Please read the following to ensure that these dependencies are taken care of before using FTB.
Configuration and Make
Configuring FTB is simple. It consists of the following steps
Step 0: For BG/L and BG/P machines:: Setup the BG environment file 'bg_setup_env'. For ex: For BG/P, set 'ZOID_HOME' to '$HOME/zoid' and 'ZOID_LIBC_PATH' to '$HOME/zoid/cnl/client'. Use the sample files provided under the 'docs' directory for reference.
Step 1: Run autogen.sh to create the configure script
Step 2: Run the configure scripts
#user-xyz# ./configure Other options to ./configure in addition to above: (a) --with-bstrap-server=<server_ipaddress> The --with-bstrap-server=<server_ipaddress> is used to specify the ip address of the bootstrap server. Please note that the bootstrap server is required by FTB during run-time. If it is not passed as a parameter to configure, then it can be specified as an environment variable or can be specified in a config file. Please refer to Section XXXXX in this file for more information on this topic. (b) --with-bstrap-port=<port_number> Default value: 14455 This is the port used by the FTB agent and perhaps the FTB clients to communicate to the bootstrap server. This value can be overridden at run-time by specifying the environment variable FTB_BSTRAP_PORT. Please refer to Section XXXXX in this file for more information on this topic. (c) --with-agent-por[[IBM Blue Gene/L series]]t=<port_number> t=<port_number> Default value: 10809 This is the port used by the FTB clients to connect to an FTB agent. This value can be overriden at run-time by specifying the environment variable FTB_AGENT_PORT. Please refer to Section XXXXX in this file for more information on this topic. (d) --enable-debug=<Log Files Directory> Default: no debug This option enables debug messages on the client as well as the FTB side. The log files will be written in the specified directory. (e) --with-platform=<linux or bgl or bgp> Default: linux This option is used by the user to specify the platform for this FTB build. Only BGL, BGP and linux are supported at this point. (d) In addition, the user can use other standard options like --with-prefix etc. ./configure --help will provide more options
Step 4: If you have specified the --prefix option during configure time.
Step 5: For BG/L machines: : the following steps ensure that the libraries are loaded correctly and the ftb agent starts up smoothly on the I/O nodes
(a) Copy the ftb_zoid_client_impl.so and ftb_zoid_client_preload.so from ftb/lib/ to /bgl/argonne-utils/profiles/user-xyz/BGL_SITEDIST/dist/lib (b) Copy the ftb/sbin/ftb_agent daemon to /bgl/argonne-utils/profiles/user-xyz/BGL_SITEDIST/dist/sbin (c) Copy startup scripts as per below steps (0) Set the log file path to point to correct directory in init.d scripts for ftb agent (1) copy ftb/src/client_lid/platforms/bgl/extras/ftb_agent to bgl/argonne-utils/profiles/user-xyz/BGL_SITEDIST/dist/etc/rc.d/init.d (2) cd bgl/argonne-utils/profiles/user-xyz/BGL_SITEDIST/dist/etc/rc.d/rc3.d (3) ln -s ../init.d/ftb_agent S36ftb_agent (4) ln -s ../init.d/ftb_agent K64ftb_agent (d) Make changes in bgl/argonne-utils/profiles/user-xyz/BGL_SITEDIST/dist/etc/sysconfig/zoid by replacing/adding the following to ZOID_MODULES: "ZOID_MODULES="$BGL_SITEDISTDIR/lib/unix_impl.so:$BGL_SITEDISTDIR/lib/unix_preload.so:$BGL_SITEDISTDIR/lib/ ftb_zoid_client_impl.so:$BGL_SITEDISTDIR/lib/ftb_zoid_client_preload.so""
Step 5: For BG/P machines: : the following steps ensure that the libraries are loaded correctly, ZeptoOs and ZOID is correctly built and the ftb agent starts up smoothly on the I/O nodes
(a) Copy the FTB Agent # cp ~/FTB/sbin/ftb_agent ~/ZeptoOS/ramdisk/BGP/ION/ramdisk-add/bin/ (b) Copy the FTB Zoid Client Libraries # cp ~/FTB/lib/ftb_zoid_client* ~/ZeptoOS/ramdisk/BGP/ION/ramdisk-add/lib/ (c) Copy the FTB Agent startup script # cp ~/FTB/src/client_lib/platforms/bgp/init_scripts/ftb_agent ~/ZeptoOS/ramdisk/BGP/ION/ramdisk-add/etc/init.d/S66FTB_Agent OR # ln -s ~/FTB/src/client_lib/platforms/bgp/init_scripts/ftb_agent ~/ZeptoOS/ramdisk/BGP/ION/ramdisk-add/etc/init.d/S66FTB_Agent (d) Edit Zoid startup to include FTB Libraries # vi ~/ZeptoOS/ramdisk/BGP/ION/ramdisk-add/etc/init.d/rc3.d/S51zoid Append 'ftb_zoid_client_impl.so:ftb_zoid_client_preload.so' to the list of libraries. Common error message seen due to not following Step 4 is: Server stub backend not loaded! in the BGP logs (e) Build the I/O Node and Compute Node Kernels cd ZeptoOS-directory make bgp-cn-linux make bgp-ion-ramdisk-cnl make bgp-ion-linux (f) Preparing the Kernel Profile directory The '/bgsys/argonne-utils/profiles/<username>' directory should contain the I/O Node and Compute Node Kernels that was built in the previous step. Since you will most probably require administrative privileges to modify this directory, it is advisable that you request your system admin to perform the following one-time operation. # cd /bgsys/argonne-utils/profiles/<user-name> # ln -s CNK ~username/ZeptoOS/BGP-CN-zImage-with-initrd.elf # ln -s INK ~username/ZeptoOS/BGP-ION-zImage.elf # ln -s ramdisk ~username/ZeptoOS/BGP-ION-ramdisk-for-CNL.elf # ln -s CNS ../factory-default/CNS # ln -s uloader ../factory-default/uloader
To launch FTB, you need to start your bootstrap database server on the Boostrap server, specified during configuration time. The ftb_database_server maintains the FTB agent topology on your machine. It can be found in your sbin directory
For BG/L and BG/P machines, the ftb_database_server can be started on the login node (as specified during configuration time).
Once the database server has started, you need to start your FTB agents on your nodes.
For BG/L and BG/P machines, the agent will automatically get started on your IO nodes during the I/O image installation time.
#user-xyz# ./sbin/ftb_agent &
In order to see activity from the ftb_database_server and the ftb_agent, it will be useful to use the --enable-debug option during configure time.
Configuring FTB-enabled software
The FTB software comes with independent FTB-enabled software in order to assist users to test their FTB installation, as well as provide a starting point for users to start writing their own FTB-enabled software.
This software is a part of the source code and can be found in the 'components' directory.
Configuring the source code is easy.
#user-xyz/ftb#cd components #user-xyz/ftb/components#./autogen.sh
Step 2: Configuring the components
#user-xyz/ftb/components# ./configure --with-ftb=<path_to_ftb_binaries> --with-bstrap-server=IP address of the boot-strap server The default directory to --with-ftb is /usr/local/ftb. Other options to above are: (a) --with-platform=<linux,bgl> Default: linux This option is used to specify the platform for which the components are to be built for (b) In addition, the user can use other standard option like --with-prefix etc. ./configure --help will provide more options
#user-xyz/ftb/components# make install
Running FTB-enabled software
Step 1: Ensure that LD_LIBRARY_PATH environment variable points to the lib directory
#user-xyz# export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/user-xyz/ftb/lib
Step 2: Ensure that the ftb_agent is launched somewhere on the cluster
Step 3: To run a pre-provided FTB-enabled component, simply launch it. The binaries for the software can be found in the components/examples directory.
Sample FTB-enabled software
The current FTB software encompasses the following independent FTB-enabled software. These software components can be found in the components/examples directory of the FTB source code. Note that not all of these softwares might run on the BGL or BGP machines.
(1) ftb_simple_publisher and ftb_simple_subscriber
Demonstrates: Connecting to FTB and publish event, Connecting to FTB and catching specific events using polling : This is the only working example on BGL for now. Description : Start the ftb_simple_subscriber and ftb_simple_publisher on two different machines. ftb_simple_subscriber subscribes and polls events published by ftb_simple_publisher
Demonstrates: Connecting to FTB, Declare publishable events using a schema file, Subscribe (using polling) and Publish an event Description : This component publishes the watch_dog_event and subscribes to catch the same event. This can be used to confirm the working of FTB
Demonstrates: Subscribe to all events and record them, using polling mechanism Description : This component calls and records *every* event sent to the FTB
Demonstrates: Subscribe to all events and record them, using callback mechanism Description : This component calls and records *every* event sent to the FTB
Demonstrates: Multi-threading: Publish an event from the callback function, which is called when an subscribed event arrives Description : This example is composed of the server and the client components. The client publishes an event, the server recives the event and responds with its own events. The client receives the server events and throws its own event. Events, thus, pingpong between the client and the server
Demonstrates: Multiple FTB-enabled components in a single software stack Description : Refer to the code documentation for an understanding of how this example works
Demonstrates: Multi-threading: Calling unsubscribe from the callback function, triggered when a specific event, matching subscription string arrives. Description : Refer to the code documentation for an understanding of how this example works
Demonstrates: Multithreading and Event association using event responses Description : Refer to the code documentation for an understanding of how this example works
Demonstrates: An example of an FTB-enabled InfiniBand network library Description : This example throws an event when the InfiniBand port becomes active. The example needs the OFED stack to be present
Demonstrates: An example of how to FTb-enabled MPI application Description : This example calculates the amount of time needed to publish events. The example needs the MPI software to be present