BBCP

For larger files, the multistreaming transfer utility BBCP is recommended. The BBCP utility is capable of breaking up your transfer into multiple simultaneously transferring streams, thereby transferring data much faster than single-streaming utilities such as SCP and SFTP.

Installation

Before you can use the BBCP utility, it must be installed on both the local and remote systems. It is currently available on each NCCS system and should be a part of each user’s default environment. Several precompiled binaries as well as the source can be downloaded from the Stanford Linear Accelerator Center (SLAC) BBCP download page.

Here at the NCCS, we have recently added the -r (recursive) option to the version of BBCP deployed on our machines to support tranfer of entire directory trees. Until SLAC incorporates this update into its source code and binaries, be aware that binaries downloaded from SLAC will not support recursive directory-tree transfers.

 

Installation from Source Tips

  • Download the BBCP from the download page.

  • Decompress the downloaded file into a directory on the local system.

> gunzip bbcp.tar.Z
> tar -xvf bbcp.tar

  • From within the decompressed BBCP directory, run make. This should build the BBCP executable into the created bin directory. The build has been tested on Linux-based systems and should build with few or no modifications. If you system’s uname command does not return Linux, you may need to modify the Makefile.

> cd bbcp
> uname
Linux
> gmake

Common variable modifications

  • In MakeSname, the test command is hard coded to /usr/bin/test. If this is not the location of test on your system, you can change the following line to the correct path (which test should return the path to test on your system):

  if /usr/bin/test -${1} $2; then

  • If the uname command is not in /bin on your system, change the uname variable in the MakeSname file. You will also need to change the following line in the file Makefile:

 @cd src;$(MAKE) make`/bin/uname` OSVER=`../MakeSname`

  • If the libz.a library is not located at /usr/local/lib/libz.a on your system, change the libzMakefile file.

  • The file Makefile contains compiler and compiler flag options for the BBCP build. You can change the compilers and flags by modifying variables in this file. For example, to change the compilers used on a Linux system, modify the variables LNXCC and LNXcc.

Usage

To transfer the local file /local/path/largefile.tar to the remote system remotesystem as /remote/path/largefile.tar, use the following:

bbcp -P 2 -V -w 8m -s 16 /local/path/largefile.tar remotesystem:/remote/path/largefile.tar

where
-P 2 produces progress messages every 2 seconds.
-V produces verbose output, including detailed transfer-speed statistics.
-w 8m sets the size of the disk input/output (I/O) buffers.
-s 16 sets the number of parallel network streams to 16.

BBCP assumes the remote system’s noninteractive environment contains the path to the BBCP utility. This can be determined with the following command:

ssh remotesystem which bbcp

If this is not the case, the -T BBCP option can be used to specify how to start BBCP on the remote system. For example, you could use the following:

bbcp -P 2 -V -w 8m -s 16 -T 'ssh -x -a -oFallBackToRsh=no %I -l %U %H /remote/path/to/bbcp' /local/path/largefile.tar remotesystem:/remote/path/largefile.tar

Often, during large transfers the connection between the transferring systems is lost. The -a options gives BBCP the ability to pick up where it left off. For example, you could use the following:

bbcp -k -a /remotesystem/homedir/.bbcp/ -P 2 -V -w 8m -s 16 /local/path/largefile.tar remotesystem:/remote/path/largefile.tar

To transfer an entire directory tree, use the following:

bbcp -r -P 2 -V -w 8m -s 16 /local/path/* remotesystem:/remote/path

When transferring files to the Cray XT (Jaguar) at the NCCS, it is necessary to specify a particular Jaguar node as the destination host because the host name jaguar.ccs.ornl.gov actually points to a server load-balancing device that returns node addresses in a round-robin fashion. For example, you could use the following:

bbcp -r -P 2 -V -w 8m -s 16 /local/path/* jaguar3.ccs.ornl.gov:/remote/path

Documentation

More information on BBCP can be found by typing “bbcp -h” on the NCCS systems as well as on the BBCP site.