BeStMan 2 Frequently Asked Questions

Support email : srmlbl.gov

Introduction
What happens when BeStMan2 server configuration parameter is not recognized?
Does BeStMan2 work with srmcp from FNAL dCache?
Does BeStMan2 work with lcg-cp or lcg-ls from lcg-utils?
Does BeStMan2 work with FTS?
Where can I find BeStMan2?
GUMS connection from BeStMan2 is not working.
Can BeStMan2 server handle large numbers in file size or space reservation?
I have a customized mass storage system. Can I use BeStMan2 on top of our MSS?
How much memory does srm-copy use?
How can I provide multiple sources to srm-ls?
How do I do for our SRM server to be monitored for its functionality and inter-operation?
I have a firewall. What needs to be done extra?
How do I configure BeStMan2 scalable?
How to provide external commands for space token usage?
I have a customized gridftp server with DSI backend. Can I have BeStMan2 use my gridftp server?
I have a few gridftp servers to work with. Can I have BeStMan2 use the all of those gridftp servers?
I have other transfer protocols to work with. Can I have BeStMan2 use the other transfer protocols such as root?

Introduction

BeStMan2 is a full implementation of SRM v2.2, developed by Lawrence Berkeley National Laboratory, for disk based storage systems and mass storage systems such as HPSS. End users may have their own personal BeStMan2 that manages and provides an SRM interface to their local disks or storage systems. It works on top of existing disk-based unix file system, and has been reported so far to work on file systems such as NFS, PVFS, AFS, GFS, GPFS, PNFS, HFS+, LustrE, Ibrix, and XrootdFS. It also works with any file transfer service, such as gsiftp, http, https and ftp, as supported file transfer protocols. It requires minimal administrative efforts on the deployment and maintenance. BeStMan2 also has a gateway mode by configuration that would provide a full SRM interface with small footprints on any existing file system without queuing or space management.

SRM v2.2 specification can be found here. BeStMan2 downloads and instructions can be found here.

What happens when BeStMan2 server configuration parameter is not recognized?

Those unrecognized entries in the bestman2.rc configuration file would be ignored when running BeStMan2 server. So, having conf parameters for full management mode would not affect the server running on gateway mode.

Does BeStMan2 work with srmcp from FNAL dCache?

Yes. Refer to sections 7.5-7.9 on BeStMan2 guide for sample commands.

Does BeStMan2 work with lcg-cp or lcg-ls from lcg-utils?

Yes. Refer to sections 7.10-7.11 on BeStMan2 guide for sample commands.

Does BeStMan2 work with FTS?

Yes. It works with FTS with or without space token. glite-url-copy also works.

Where can I find BeStMan2?

http://sdm.lbl.gov/bestman contains the latest gzipped tar pkg for direct downloading and installation. VDT contains the BeStMan2 package, and pacman installation includes gridftp server. Sample pacman command is following: pacman -get http://vdt.cs.wisc.edu/vdt_1101_cache:Bestman. Pacman BeStMan2 installation does not include EDG-mkgridmap, and you can install it separately: pacman -get http://vdt.cs.wisc.edu/vdt_1101_cache:EDG-Make-Gridmap. OSG BeStMan2 VDT installation guide can be found here.

GUMS connection from BeStMan2 is not working.

When GUMS connection is successful, srm-ping to the BeStMan2 server will display the properly mapped local id. When GUMS connection is not working, localIDMapped will be null instead. e.g.

Key=clientDN
Value=/DC=org/DC=doegrids/OU=People/CN=Arie Shoshani 12345
Key=localIDMapped
Value=osgatlas004

Check proper GUMSserviceURL in conf/bestman.rc.

e.g. GUMSserviceURL=https://gums.lbl.gov:8443/gums/services/GUMSAuthorizationServicePort

GUMSCurrHostDN should be the DN that GUMS server allows to decide which mapping group you are in. Many times, it's the host DN where BeStMan server runs: e.g. openssl x509 -in /etc/grid-security/hostcert.pem -noout -subject returns subject=/DC=org/DC=doegrids/OU=Services/CN=srm.lbl.gov.
An example of host mapping on a typical GUMS server is: /?.lbl.gov
The local mapping can be checked on the web to your GUMS server by clicking on the "host to group mappings" link or directly at https://yourgumsserver.domain.tld:8443/gums/hostToGroupMappings.jsp
Sometimes, local system where BeStMan2 server runs needs the existence of the following path: /etc/grid-security/vomsdir/vdt_empty.pem. Try the following two commands:

% mkdir /etc/grid-security/vomsdir
% touch /etc/grid-security/vomsdir/vdt_empty.pem

BeStMan2 GUMS connection does not depend on /etc/grid-security/gsi-authz.conf and /etc/grid-security/prima-authz.conf files.

Can BeStMan2 server handle large numbers in file size or space reservation?

As long as there is enough space on the storage, BeStMan2 can handle as much as java allows. Currently it is 18,446,744,073,709,551,615.

I have a customized mass storage system. Can I use BeStMan2 on top of our MSS?

If your customized mass storage system has POSIX compliant file system, BeStMan2 as is would work on top of your MSS. If you have special clients for your MSS access, you can extend BeStMan MSS plugin library to support your customized MSS as custodial quality storage. For more info on how to extend the plugin library, send an email to srmlbl.gov.

How much memory does srm-copy use?

Because LBNL BeStMan SRM-Client tools are based on java, there is a java vm overhead. However, when one file is requested, it is somewhat optimized so that less memory (about 30MB) is used. It is still more than C-based SRM copy client such as lcg-cp, but considering modern computer hardware systems, this memory usage would be okay. Our work on the memory usage optimization will be done continuously.

How can I provide multiple sources to srm-ls?

LBNL BeStMan SRM-Client tools support multiple "-s' options so that users can provide multiple source urls.

How do I do for our SRM server to be monitored for its functionality and inter-operation?

Register your SRM endpoint here and follow the instructions. There would be daily testing and results would be posted on the web.http://datagrid.lbl.gov/v22 or http://datagrid.lbl.gov/osg

I have a firewall. What needs to be done extra?

If you have firewall, the gridftp port range should be properly set in VDT installation. In order to do so, you need to modify vdt-local-setup.sh.

edit $VDT_LOCATION/vdt/etc/vdt-local-setup.sh
GLOBUS_TCP_SOURCE_RANGE= low_port,high_port
GLOBUS_TCP_PORT_RANGE= low_port,high_port
export GLOBUS_TCP_SOURCE_RANGE
export GLOBUS_TCP_PORT_RANGE

Where low_port,high_port - controls all outbound globus connections for gridftp (e.g GLOBUS_TCP_PORT_RANGE=40000,49150). This low_port,high_port must correspond to --globus-tcp-port-range of VDT configuration of BeStMan2 (or --with-globus-tcp-port-range and --with-globus-tcp-source-range from the BeStMan2 manual configuration). Also, make sure that two ports for BeStMan2 (--http-port and --https-port from the VDT configuration) are open.

How do I configure BeStMan2 scalable?

Refer to Notes for Scalability.

How to provide external commands for space token usage?

If staticTokenList is used for space tokens in Gateway mode, [usedBytesCommand:External_command] and [totalBytesCommand:External_command] can be used to provide the size information from the external commands. For example:

staticTokenList=ATLASHOTDISK[desc:ATLASHOTDISK][retention:REPLICA][latency:ONLINE][path:/data2/atlasdata/atlashotdisk][usedBytesCommand:/bin/echo

Note that /bin/echo with ` and ` around the external command provides a way to java based bestman process to capture the displayed information.
Note that the path information given in the option per space token is appended to the external command. If the external command does not require the path information, do not set the [path:...] option.

I have a customized gridftp server with DSI backend. Can I have BeStMan2 use my gridftp server?

Yes, for full management mode and for gateway mode. During the configuration, provide your customized gridftp server url --with-transfer-servers. E.g. --with-transfer-servers=gsiftp://mygsiftp.domain.tld:2812 You can confirm or modify by checking conf/bestman.rc for entry: supportedProtocolList=gsiftp://mygsiftp.domain.tld:2812

I have a few gridftp servers to work with. Can I have BeStMan2 use the all of those gridftp servers?

Yes, for full management mode and for gateway mode. During the configuration, provide your gridftp server urls --with-transfer-servers with semi-colon separated entries. Those gsiftp servers would be used in round-robin bases for TURLs. E.g.

--with-transfer-servers="gsiftp://gsiftp1.domain.tld;gsiftp://gsiftp2.domain.tld;gsiftp://gsiftp3.domain.tld"; You can confirm or modify by checking conf/bestman.rc for entry:

supportedProtocolList=gsiftp://gsiftp1.domain.tld;gsiftp://gsiftp2.domain.tld;gsiftp://gsiftp3.domain.tld

I have other transfer protocols to work with. Can I have BeStMan2 use the other transfer protocols such as root?

Yes, add root://-style URLs to supportedProtocolList, as well as matching entry in acceptProtocols. E.g.

supportedProtocolList=gsiftp://eosppsftp.cern.ch;root://eospps.cern.ch

acceptProtocols=root[/eos /eos]