How to install and use the FTB software
Contents |
Downloading FTB
Step 1. Download FTB-0.6 from the CIFTS website
Alternatively, you can download it from the SVN TAGS directory at: https://svn.mcs.anl.gov/repos/cifts/ftb/tags/
The code in the SVN trunk (https://svn.mcs.anl.gov/repos/cifts/ftb/trunk) refers to the current development code. While we attempt to keep the trunk code stable (and working) at most times - please be aware that the under-development code might cause a few hiccups.
To download from the SVN trunk, you can run the following command:
svn co https://svn.mcs.anl.gov/repos/cifts/ftb/trunk ftb
If you are not familiar with using SVN, please refer to (http://svn.collab.net/svn-doxygen/) for further guidance.
Preparing your build environment (Linux, IBM BG series, Cray XT)
The FTB software supports linux (Ubuntu) clusters, IBM BG/L, IBM BG/P and the Cray XT series. To get the FTB software working, you may need to install additional libraries or configure certain minor aspects of your cluster - especially for the peta-scale machines. Please read the following to ensure that these dependencies are taken care of before using FTB.
Configuration and Make
Configuring FTB is simple. It consists of the following steps
Step 0: For BG/L and BG/P machines:: Setup the BG environment file 'bg_setup_env'.
For ex: For BG/P, set 'ZOID_HOME' to '$HOME/zoid' and 'ZOID_LIBC_PATH' to '$HOME/zoid/cnl/client'. Use the sample files provided under the 'docs' directory for reference.
Step 1: Run autogen.sh to create the configure script
#user-xyz# ./autogen.sh
Step 2: Run the configure scripts
#user-xyz# ./configure
Other options to ./configure in addition to above:
(a) --with-bstrap-server=<server_ipaddress>
The --with-bstrap-server=<server_ipaddress> is used to specify
the ip address of the bootstrap server. Please note that the
bootstrap server is required by FTB during run-time. If it is
not passed as a parameter to configure, then it can be specified
as an environment variable or can be specified in a config
file. Please refer to Section XXXXX in this file for more
information on this topic.
(b) --with-bstrap-port=<port_number>
Default value: 14455
This is the port used by the FTB agent and perhaps the FTB
clients to communicate to the bootstrap server. This value
can be overridden at run-time by specifying the environment
variable FTB_BSTRAP_PORT. Please refer to Section XXXXX in this
file for more information on this topic.
(c) --with-agent-por[[IBM Blue Gene/L series]]t=<port_number>
t=<port_number>
Default value: 10809
This is the port used by the FTB clients to connect to an FTB
agent. This value can be overriden at run-time by specifying
the environment variable FTB_AGENT_PORT. Please refer to Section
XXXXX in this file for more information on this topic.
(d) --enable-debug=<Log Files Directory>
Default: no debug
This option enables debug messages on the client as well as
the FTB side. The log files will be written in the specified
directory.
(e) --with-platform=<linux or bgl or bgp>
Default: linux
This option is used by the user to specify the platform for this
FTB build. Only BGL, BGP and linux are supported at this point.
(d) In addition, the user can use other standard options
like --with-prefix etc. ./configure --help will provide
more options
Step 3:
#user-xyz#make
Step 4: If you have specified the --prefix option during configure time.
#user-xyz#make install
Step 5: For BG/L machines: : the following steps ensure that the libraries are loaded correctly and the ftb agent starts up smoothly on the I/O nodes
(a) Copy the ftb_zoid_client_impl.so and ftb_zoid_client_preload.so from ftb/lib/ to /bgl/argonne-utils/profiles/user-xyz/BGL_SITEDIST/dist/lib
(b) Copy the ftb/sbin/ftb_agent daemon to /bgl/argonne-utils/profiles/user-xyz/BGL_SITEDIST/dist/sbin
(c) Copy startup scripts as per below steps
(0) Set the log file path to point to correct directory in init.d scripts for ftb agent
(1) copy ftb/src/client_lid/platforms/bgl/extras/ftb_agent to
bgl/argonne-utils/profiles/user-xyz/BGL_SITEDIST/dist/etc/rc.d/init.d
(2) cd bgl/argonne-utils/profiles/user-xyz/BGL_SITEDIST/dist/etc/rc.d/rc3.d
(3) ln -s ../init.d/ftb_agent S36ftb_agent
(4) ln -s ../init.d/ftb_agent K64ftb_agent
(d) Make changes in bgl/argonne-utils/profiles/user-xyz/BGL_SITEDIST/dist/etc/sysconfig/zoid by replacing/adding
the following to ZOID_MODULES:
"ZOID_MODULES="$BGL_SITEDISTDIR/lib/unix_impl.so:$BGL_SITEDISTDIR/lib/unix_preload.so:$BGL_SITEDISTDIR/lib/
ftb_zoid_client_impl.so:$BGL_SITEDISTDIR/lib/ftb_zoid_client_preload.so""
Step 5: For BG/P machines: : the following steps ensure that the libraries are loaded correctly, ZeptoOs and ZOID is correctly built and the ftb agent starts up smoothly on the I/O nodes
(a) Copy the FTB Agent
# cp ~/FTB/sbin/ftb_agent ~/ZeptoOS/ramdisk/BGP/ION/ramdisk-add/bin/
(b) Copy the FTB Zoid Client Libraries
# cp ~/FTB/lib/ftb_zoid_client* ~/ZeptoOS/ramdisk/BGP/ION/ramdisk-add/lib/
(c) Copy the FTB Agent startup script
# cp ~/FTB/src/client_lib/platforms/bgp/init_scripts/ftb_agent ~/ZeptoOS/ramdisk/BGP/ION/ramdisk-add/etc/init.d/S66FTB_Agent
OR
# ln -s ~/FTB/src/client_lib/platforms/bgp/init_scripts/ftb_agent ~/ZeptoOS/ramdisk/BGP/ION/ramdisk-add/etc/init.d/S66FTB_Agent
(d) Edit Zoid startup to include FTB Libraries
# vi ~/ZeptoOS/ramdisk/BGP/ION/ramdisk-add/etc/init.d/rc3.d/S51zoid
Append 'ftb_zoid_client_impl.so:ftb_zoid_client_preload.so' to the list of libraries.
Common error message seen due to not following Step 4 is: Server stub backend not loaded! in the BGP logs
(e) Build the I/O Node and Compute Node Kernels
cd ZeptoOS-directory
make bgp-cn-linux
make bgp-ion-ramdisk-cnl
make bgp-ion-linux
(f) Preparing the Kernel Profile directory
The '/bgsys/argonne-utils/profiles/<username>' directory should contain the I/O Node and Compute Node Kernels that was built in the
previous step. Since you will most probably require administrative privileges to modify this directory, it is advisable that you request your system admin to perform the following one-time operation.
# cd /bgsys/argonne-utils/profiles/<user-name>
# ln -s CNK ~username/ZeptoOS/BGP-CN-zImage-with-initrd.elf
# ln -s INK ~username/ZeptoOS/BGP-ION-zImage.elf
# ln -s ramdisk ~username/ZeptoOS/BGP-ION-ramdisk-for-CNL.elf
# ln -s CNS ../factory-default/CNS
# ln -s uloader ../factory-default/uloader
Launching FTB
To launch FTB, you need to start your bootstrap database server on the Boostrap server, specified during configuration time. The ftb_database_server maintains the FTB agent topology on your machine. It can be found in your sbin directory
For BG/L and BG/P machines, the ftb_database_server can be started on the login node (as specified during configuration time).
#user-xyz#./sbin/ftb_database_server &
Once the database server has started, you need to start your FTB agents on your nodes.
For BG/L and BG/P machines, the agent will automatically get started on your IO nodes during the I/O image installation time.
#user-xyz# ./sbin/ftb_agent &
In order to see activity from the ftb_database_server and the ftb_agent, it will be useful to use the --enable-debug option during configure time.
Configuring FTB-enabled software
The FTB software comes with independent FTB-enabled software in order to assist users to test their FTB installation, as well as provide a starting point for users to start writing their own FTB-enabled software.
This software is a part of the source code and can be found in the 'components' directory.
Configuring the source code is easy.
Step 1:
#user-xyz/ftb#cd components #user-xyz/ftb/components#./autogen.sh
Step 2: Configuring the components
#user-xyz/ftb/components# ./configure --with-ftb=<path_to_ftb_binaries> --with-bstrap-server=IP address of the boot-strap server
The default directory to --with-ftb is /usr/local/ftb.
Other options to above are:
(a) --with-platform=<linux,bgl>
Default: linux
This option is used to specify the platform for which the
components are to be built for
(b) In addition, the user can use other standard option
like --with-prefix etc. ./configure --help will provide
more options
Step 3:
#user-xyz/ftb/components#make
Step 4:
#user-xyz/ftb/components# make install
Running FTB-enabled software
Step 1: Ensure that LD_LIBRARY_PATH environment variable points to the lib directory
#user-xyz# export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/user-xyz/ftb/lib
Step 2: Ensure that the ftb_agent is launched somewhere on the cluster
Step 3: To run a pre-provided FTB-enabled component, simply launch it. The binaries for the software can be found in the components/examples directory.
Sample FTB-enabled software
The current FTB software encompasses the following independent FTB-enabled software. These software components can be found in the components/examples directory of the FTB source code. Note that not all of these softwares might run on the BGL or BGP machines.
(1) ftb_simple_publisher and ftb_simple_subscriber
Demonstrates: Connecting to FTB and publish event, Connecting to FTB and catching specific events using polling
: This is the only working example on BGL for now.
Description : Start the ftb_simple_subscriber and ftb_simple_publisher on two different machines.
ftb_simple_subscriber subscribes and polls events published by ftb_simple_publisher
(2) ftb_watchdog
Demonstrates: Connecting to FTB, Declare publishable events using a schema
file, Subscribe (using polling) and Publish an event
Description : This component publishes the watch_dog_event and subscribes to catch the same event.
This can be used to confirm the working of FTB
(3) ftb_polling_logger
Demonstrates: Subscribe to all events and record them, using polling mechanism Description : This component calls and records *every* event sent to the FTB
(4) ftb_notify_logger
Demonstrates: Subscribe to all events and record them, using callback mechanism Description : This component calls and records *every* event sent to the FTB
(5) ftb_pingpong
Demonstrates: Multi-threading: Publish an event from the callback function,
which is called when an subscribed event arrives
Description : This example is composed of the server and the client components.
The client publishes an event, the server recives the event and responds
with its own events. The client receives the server events and throws its
own event. Events, thus, pingpong between the client and the server
(6) ftb_multiplecomp
Demonstrates: Multiple FTB-enabled components in a single software stack Description : Refer to the code documentation for an understanding of how this example works
(7) ftb_example_callback_unsubscribe
Demonstrates: Multi-threading: Calling unsubscribe from the callback
function, triggered when a specific event, matching subscription
string arrives.
Description : Refer to the code documentation for an understanding of how this example works
(8) ftb_eventhandle_example
Demonstrates: Multithreading and Event association using event responses Description : Refer to the code documentation for an understanding of how this example works
(9) ftb_ib_port_status_publisher
Demonstrates: An example of an FTB-enabled InfiniBand network library
Description : This example throws an event when the InfiniBand port becomes active. The example
needs the OFED stack to be present
(10) ftb_throw_delay_mpi
Demonstrates: An example of how to FTb-enabled MPI application
Description : This example calculates the amount of time needed to publish events. The example
needs the MPI software to be present
