Back to BDM Home

Requirements and Rational

Space verification at target

• BDM client at target should find size from source node, and check space availability at target node
• Note: this applies to use case 1; could also be a requirement for use case 2.

Recovery and restart

• Recovery for transfer failure is a requirement for BDM client
• Restart mechanism should be provided by BDM client

Size of transfer request

• BDM client should support any size, even if it takes it days to complete.
• Initial goal: 50 TBs, 5,000 files (up to 10 GBs each)

On-demand transfer request status

• Should be supported asynchronously
• Mechanism: use request status to be used at anytime
• Only command line support needed
• Only summary needed (number of files and total size transferred/remaining, failures)
• Web access to summary status to be considered at a later stage

Support for checksum

• Should be available by default
• Requires checksums to be pre-computed and available at source nodes
• Not decided where checksums will be stored (metadata?)
• Option to disable checksum

Statistics collection

• Statistics should be collected at target site by BDM client
• At first - stored locally
• Later - may be provided to statistics server (metrix group issue)
• To be used for request estimation

Request estimation

• Prior to transfer - use recent statistics
• In-progress request estimation - use statistics of transfers so far to predict "time to completion"
• This feature is desirable - should be developed last

Take advantage of ESnet provisioning

• Highly desirable, start experimenting, add later

All bulk data movement (BDM) will be done in "pull mode"

• Allows "client BDM" to work behind a firewall
• Client DBM is lightweight and can include a transfer client
• Leaves control to target site

For writing data only disk-to-disk transfers will be supported

• Applies to use case 1
• Writing to data archiving is left as a local task

Some sites may support reading from archives

• Applies to use case 2
• Such sites may have to setup special security arrangements
• Site URLs may be used in such cases as the physical locations