Bulk Data Movement

Bulk Data Mover (BDM) is a scalable data transfer management tool for GridFTP transfer protocol. The goal is to manage as much as 1+ PB with millions of files transfers reliably.

Bulk Data Movement is an ESG (http://www.earthsystemgrid.org) project.

Links

• License agreement
• Copyright notice


• Download BDM
• BDM Command line Options
• Frequently Asked Questions


• Transfer performance plots on sample transfers


• Sample Commands to test ESG Federation
• Sample Commands in general
• Sample Commands to download IPCC datasets


• Current Status
• Current Issues and Discussion Items


• Use Cases: 1. Move data between nodes, 2. Move data from a node to a client (user) site
• Requirements and Rational
• Dependency Requirements
• Implementation Details and Time-lines
• Interaction Specification


• Refer to NCAR page for more project description:
• https://wiki.ucar.edu/display/esgcet/Bulk+Data+Movement

Other References

• DataMover-Lite (DML)
• SRM-Lite
• ESG Data Replication Use Case description: https://wiki.ucar.edu/display/esgcet/Data+Replication+Use+Case
• Earth System Grid (http://www.earthsystemgrid.org)
• Climate 100

Support

• Email: sdmsupportlbl.gov