Difference between revisions of "ZOID"

From ZeptoOS
Jump to navigationJump to search
Line 36: Line 36:
  
 
===Pset broadcast===
 
===Pset broadcast===
 +
A <tt>/bin.rd/f2cn</tt> command is available on I/O nodes for a very efficient (hardware-assisted) broadcasting of files to all the compute nodes handled by the given I/O node.
 +
 +
The command takes two arguments:
 +
* absolute pathname to the input file on the I/O node,
 +
* absolute pathname to the output file on the compute nodes.
 +
 +
The input file does not need to be physically on the I/O node; it can be on a network filesystem mounted on the node.  The file will be created in the ramdisk of each compute node.
 +
 +
The throughput is in practice limited by how fast the input file can be read; we have seen results in excess of 300&nbsp;MB/s for files residing in the I/O node ramdisk.
 +
 +
'''Note:''' all the compute nodes in the pset must be up and running.  Do not use this command on ''incomplete'' partitions (e.g., a one-process job on a 64-node partition); you will likely hang the ZOID daemon if you try.
  
 
===Performance counters===
 
===Performance counters===

Revision as of 17:05, 22 April 2009

Other Packages and Utilities | Top | KTAU


Introduction

ZOID is an I/O forwarding component of the ZeptoOS project. Any communication between the compute nodes and I/O nodes (job management, file I/O, sockets) is facilitated by ZOID.

ZOID infrastructure consists of:

  • Multithreaded zoid daemon on the I/O nodes which performs I/O forwarding for the compute nodes and which also communicates with the service node to perform job management,
  • control daemon on the compute nodes which is responsible for job management tasks such as launching application processes, for the forwarding of stdin/out/err data, and for forwarding of IP packets,
  • zoid-fuse daemon on the compute nodes which performs file I/O forwarding for POSIX-compliant applications.

User interface

User script

Right before a job starts running, and right after the last process of a job has terminated, ZOID daemon attempts to invoke a user script on I/O nodes. By default, the daemon invokes $HOME/zoid-user-script.sh (this pathname can be changed by an administrator). A single parameter is passed to the script: 1 at the job startup, and 0 at the termination.

Information about the job will be passed to the script in the following environment variables:

ZOID_JOB_EXEC
name of the job executable,
ZOID_JOB_ARGS
job arguments, separated by a :
ZOID_JOB_ENV
job environment variables, separated by a :
ZOID_JOB_ID
BG/P control system job id (Note: this is generally different from the Cobalt job ID; see FAQ for the latter),
ZOID_JOB_GLOBAL_SIZE
number of processes in the job (size of MPI_COMM_WORLD),
ZOID_JOB_LOCAL_SIZE
number of job processes handled by this I/O node,
ZOID_JOB_MODE
0 for SMP, 1 for VN, and 2 for DUAL,
SHELL, PATH, USER, and HOME
will also be set...

Pset broadcast

A /bin.rd/f2cn command is available on I/O nodes for a very efficient (hardware-assisted) broadcasting of files to all the compute nodes handled by the given I/O node.

The command takes two arguments:

  • absolute pathname to the input file on the I/O node,
  • absolute pathname to the output file on the compute nodes.

The input file does not need to be physically on the I/O node; it can be on a network filesystem mounted on the node. The file will be created in the ramdisk of each compute node.

The throughput is in practice limited by how fast the input file can be read; we have seen results in excess of 300 MB/s for files residing in the I/O node ramdisk.

Note: all the compute nodes in the pset must be up and running. Do not use this command on incomplete partitions (e.g., a one-process job on a 64-node partition); you will likely hang the ZOID daemon if you try.

Performance counters

Administrator interface

Command line options

-u

Programmer interface

Building a new plugin

Replacement libc?


Other Packages and Utilities | Top | KTAU