SDM
People
Publications
Projects
August 3, 2010
This document summarizes test cases for StorNet prototype testbed setup between University of Michigan and Brookhaven National Laboratory.
We will perform 15 minutes worth of transfers initially. Provided that there will be 10 Gbps dedicated circuit on the testbed, 1.125TB can be moved during 15 minutes at the maximum.
We plan to have two kinds of data files: one with all large files such as 2GB in sizes and one with smaller files, as it is easier to watch sustained bandwidth usage with larger files. We assume to have all data files already on the source storage under the management of the source BeStMan.
The current network maximum bandwidth on the testbed is 10 Gbps, and we plan to reserve up to the allowed bandwidth through the negotiation with ESnet and Internet2. It would be most likely somewhere around 3 Gbps. We will start with 1 Gbps and increase gradually.
We plan to have three test request cases for each kind of data: a simple request, a larger request, and multiple requests. The following test cases are targeted for the StorNet testbed between University of Michigan and Brookhaven National Laboratory.
A simple request would contain one large file that would be copied from the source to the target. Figure 1 shows the overview of the communication flow between components on the testbed for the test case.
Client submits the simple request to the target BeStMan for srmCopy.
Target BeStMan checks the local disks for the capacity and availability.
Target BeStMan communicates and negotiates with source BeStMan for the capacity and availability.
Source BeStMan checks the local disks for the capacity and availability.
Target BeStMan communicates with target TeraPaths to check the bandwidth.
Target TeraPaths communicates and negotiates with source TeraPaths for capacity and availability.
Target TeraPaths communicates with OSCARS for the bandwidth capacity and availability.,
Target BeStMan communicates with target TeraPaths to reserve the bandwidth.
Target TeraPaths reserves network.
Target BeStMan initiates and completes GridFTP file transfer to the target disk from the source disk.
A large request test case consists of many files in a request that would be copied from the source to the target.
A multiple requests test case consists of mixture of small and large requests, so that both storage and network bandwidth reservations can overlap the requests with different capacity, as shown in Figure 2. When the second network bandwidth reservation goes to the TeraPaths, BeStMan may receive “network busy” status that updating the same network bandwidth is not possible at this moment. This may result in StorNet interface updates for the status.
We plan to have a web-based control center to visualize the data transfer plots, display various critical parameters and status along the network paths to show the state of the data transfer and performance of various components in the data transfer.
This web interface can be used in a demo at the exhibition floor of the Supercomputing 2010 on a large flat panel, allowing demo to start and stop the data transfers. A few exemplary plots are shown in Figure 3.