Berkeley Storage Manager (BeStMan),
SRM-Client Tools,
and SRM-Tester

 

Frequently Asked Questions

 

July 31, 2008

 

Alex Sim, Junmin Gu, Vijaya Natarajan, Arie Shoshani
Lawrence Berkeley National Laboratory

http://datagrid.lbl.gov/bestman
[email protected]


Table of contents

 

Copyright Notice. 3

1������ Introduction. 4

2������ Frequently Asked Questions. 4

2.1�������� What happens when BeStMan server configuration parameter is not recognized?. 4

2.2�������� Does BeStMan work with srmcp from FNAL dCache?. 4

2.3�������� Does BeStMan work with lcg-cp or lcg-ls from lcg-utils?. 4

2.4�������� Does BeStMan work with FTS?. 4

2.5�������� Where can I find BeStMan?. 4

2.6�������� GUMS connection from BeStMan is not working. 5

2.7�������� I have a customized gridftp server with DSI backend. Can I have BeStMan use my gridftp server?. 5

2.8�������� I have a few gridftp servers to work with. Can I have BeStMan use the all of those gridftp servers?. 6

2.9�������� Can BeStMan server handle large numbers in file size or space reservation?. 6

2.10����� I have a customized mass storage system. Can I use BeStMan on top of our MSS?. 6

2.11����� How much memory does srm-copy use?. 6

2.12����� How can I provide multiple sources to srm-ls?. 6

2.13����� How do I do for our SRM server to be monitored for its functionality and inter-operation?. 6

 


 

 

Copyright Notice

 

BeStMan Copyright (c) 2007,2008, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Dept. of Energy).� All rights reserved.

 

If you have questions about your rights to use or distribute this software, please contact Berkeley Lab's Technology Transfer Department at [email protected] .

 

NOTICE:�

This software was developed under partial funding from the U.S. Department of Energy.� As such, the U.S. Government has been granted for itself and others acting on its behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software to reproduce, prepare derivative works, and perform publicly and display publicly.� Beginning five (5) years after the date permission to assert copyright is obtained from the U.S. Department of Energy, and subject to any subsequent five (5) year renewals, the U.S. Government is granted for itself and others acting on its behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software to reproduce, prepare derivative works, distribute copies to the public, perform publicly and display publicly, and to permit others to do so.

 

For the end user license agreement file for BeStMan for non-commercial research use, go to http://datagrid.lbl.gov/bestman/license-nc.html.

For the end user license agreement file for BeStMan for commercial research use, go to http://datagrid.lbl.gov/bestman/license-c.html.


1         Introduction

BeStMan is a full implementation of SRM v2.2, developed by Lawrence Berkeley National Laboratory, for disk based storage systems and mass storage systems such as HPSS. End users may have their own personal BeStMan that manages and provides an SRM interface to their local disks or storage systems. It works on top of existing disk-based unix file system, and has been reported so far to work on file systems such as NFS, PVFS, AFS, GFS, GPFS, PNFS, HFS+ and Lustre. It also works with any file transfer service, such as gsiftp, http, https and ftp, as supported file transfer protocols. It requires the minimal administrative efforts on the deployment and maintenance. BeStMan also has a gateway mode by configuration that would provide a lightweight SRM interface on any existing file system without queuing or space management.

SRM v2.2 specification can be found on http://sdm.lbl.gov/srm-wg/doc/SRM.v2.2.html.
BeStMan downloads and instructions can be found on http://datagrid.lbl.gov/bestman.

2         Frequently Asked Questions

2.1       What happens when BeStMan server configuration parameter is not recognized?

Those unrecognized entries in the bestman.rc configuration file would be ignored when running BeStMan server. So, having conf parameters for full management mode would not affect the server running on gateway mode.

2.2       Does BeStMan work with srmcp from FNAL dCache?

Yes. Refer to sections 7.5-7.9 on BeStMan guide for sample commands.

2.3       Does BeStMan work with lcg-cp or lcg-ls from lcg-utils?

Yes. Refer to sections 7.10-7.11 on BeStMan guide for sample commands.

2.4       Does BeStMan work with FTS?

Yes. It works with FTS with or without space token. glite-url-copy also works.

2.5       Where can I find BeStMan?

http://datagrid.lbl.gov/bestman contains the latest gzipped tar pkg for direct downloading and installation.� VDT contains the BeStMan package, and pacman installation includes gridftp server. Sample pacman command is following: pacman -get http://vdt.cs.wisc.edu/vdt_1101_cache:Bestman
Pacman BeStMan installation does not include EDG-mkgridmap, and you can install it separately: pacman -get http://vdt.cs.wisc.edu/vdt_1101_cache:EDG-Make-Gridmap
The OSG pacman cache does not contain BeStMan package currently, and you need to get it from VDT directly.

2.6       GUMS connection from BeStMan is not working.

         When GUMS connection is successful, srm-ping to the BeStMan server will display the properly mapped local id. When GUMS connection is not working, localIDMapped will be null instead.
e.g.� Key=clientDN
�� �����Value=/DC=org/DC=doegrids/OU=People/CN=Arie Shoshani 12345
������� Key=localIDMapped
������� Value=osgatlas004

         Check proper GUMSserviceURL in conf/bestman.rc.
e.g. GUMSserviceURL=https://gums.lbl.gov:8443/gums/services/GUMSAuthorizationServicePort

         Check proper GUMSCurrHostDN in conf/bestman.rc.
e.g. GUMSCurrHostDN=/DC=org/DC=doegrids/OU=Services/CN=srm.lbl.lbl.gov

         GUMSCurrHostDN should be the DN that GUMS server allows to decide which mapping group you are in. Many times, it�s the host DN where BeStMan server runs: e.g. openssl x509 -in /etc/grid-security/hostcert.pem -noout -subject� returns subject=/DC=org/DC=doegrids/OU=Services/CN=srm.lbl.gov.

         An example of host mapping on a typical GUMS server is: �*/?*.lbl.gov

         The local mapping can be checked on the web to your GUMS server by clicking on the �host to group mappings� link or directly at https://yourgumsserver.domain.tld:8443/gums/hostToGroupMappings.jsp

         Sometimes, local system where BeStMan server runs needs the existence of the following path: /etc/grid-security/vomsdir/vdt_empty.pem. Try the following two commands:
% mkdir /etc/grid-security/vomsdir
% touch /etc/grid-security/vomsdir/vdt_empty.pem

         BeStMan GUMS connection does not depend on /etc/grid-security/gsi-authz.conf and /etc/grid-security/prima-authz.conf files.

2.7       I have a customized gridftp server with DSI backend. Can I have BeStMan use my gridftp server?

Yes, for full management mode and for gateway mode. During the configuration, provide your customized gridftp server url --with-transfer-servers.
E.g. �with-transfer-servers=gsiftp://mygsiftp.domain.tld:2812
You can confirm or modify by checking conf/bestman.rc for entry:
supportedProtocolList=gsiftp:// mygsiftp.domain.tld:2812

2.8       I have a few gridftp servers to work with. Can I have BeStMan use the all of those gridftp servers?

Yes, for full management mode and for gateway mode. During the configuration, provide your gridftp server urls --with-transfer-servers with semi-colon separated entries. Those gsiftp servers would be used in round-robin bases for TURLs.
E.g. �with-transfer-servers=�gsiftp://gsiftp1.domain.tld;gsiftp://gsiftp2.domain.tld;gsiftp://gsiftp3.domain.tld�
You can confirm or modify by checking conf/bestman.rc for entry:
supportedProtocolList=gsiftp://gsiftp1.domain.tld;gsiftp://gsiftp2.domain.tld;gsiftp://gsiftp3.domain.tld

2.9       Can BeStMan server handle large numbers in file size or space reservation?

As long as there is enough space on the storage, BeStMan can handle as much as java allows. Currently it is 18,446,744,073,709,551,615.

2.10  I have a customized mass storage system. Can I use BeStMan on top of our MSS?

If your customized mass storage system has POSIX file system, BeStMan as is would work on top of your MSS. If you have special clients for your MSS access, you can extend BeStMan MSS plugin library to support your customized MSS as custodial quality storage. For more info on how to extend the plugin library, send an email to [email protected].

2.11  How much memory does srm-copy use?

Because LBNL SRM-Client tools are based on java, there is a java vm overhead. However, when one file is requested, it is somewhat optimized so that less memory (about 30MB) is used. It is still more than C-based SRM copy client such as lcg-cp, but considering modern computer hardware systems, this memory usage would be okay. Our work on the memory usage optimization will be done continuously.

2.12  How can I provide multiple sources to srm-ls?

LBNL SRM-Client tools support multiple �-s� options so that users can provide multiple source urls.

2.13  How do I do for our SRM server to be monitored for its functionality and inter-operation?

Register your SRM endpoint here and follow the instruction. There would be daily testing and results would be posted on the web: http://datagrid.lbl.gov/v22 or http://datagrid.lbl.gov/osg