Berkeley Storage Manager

BeStMan Copyright (c) 2007,2009, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Dept. of Energy).� All rights reserved.

If you have questions about your rights to use or distribute this software, please contact Berkeley Lab's Technology Transfer Department at [email protected] .

NOTICE:�

This software was developed under partial funding from the U.S. Department of Energy.� As such, the U.S. Government has been granted for itself and others acting on its behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software to reproduce, prepare derivative works, and perform publicly and display publicly.� Beginning five (5) years after the date permission to assert copyright is obtained from the U.S. Department of Energy, and subject to any subsequent five (5) year renewals, the U.S. Government is granted for itself and others acting on its behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software to reproduce, prepare derivative works, distribute copies to the public, perform publicly and display publicly, and to permit others to do so.

For the end user license agreement file for BeStMan for non-commercial research use, go to http://datagrid.lbl.gov/bestman/license-nc.html.

For the end user license agreement file for BeStMan for commercial research use, go to http://datagrid.lbl.gov/bestman/license-c.html.

1 Introduction

BeStMan is a full implementation of SRM v2.2, developed by Lawrence Berkeley National Laboratory, for disk based storage systems and mass storage systems such as HPSS. End users may have their own personal BeStMan that manages and provides an SRM interface to their local disks or storage systems. It works on top of existing disk-based unix file system, and has been reported so far to work on file systems such as NFS, PVFS, AFS, GFS, GPFS, PNFS, HFS+, NGFS, Hadoop, Ibrix and Lustre. It also works with any file transfer service, such as gsiftp, http, https and ftp, as supported file transfer protocols. It requires the minimal administrative efforts on the deployment and maintenance. BeStMan also has a gateway mode by configuration that would provide a lightweight SRM interface on any existing file system without queuing or space management and provide great performance.

SRM v2.2 specification can be found on http://sdm.lbl.gov/srm-wg/doc/SRM.v2.2.html.
BeStMan downloads and instructions can be found on http://datagrid.lbl.gov/bestman.

1.1 Why use BeStMan?

� BeStMan is a simple, lightweight implementation, and minimal administrative effort on the deployment and maintenance is needed.

� BeStMan and LBNL SRM clients are full SRM v2.2 compliant implementations, and interoperable with all other known SRM V2.2 implementations (CASTOR, dCache, DPM, StoRM). The compatibility and interoperability are continuously checked.

� BeStMan works with an existing file system and storage to provide an SRM v2.2 interface. Nothing needs to be changed on the storage or file system to support SRM interface with BeStMan.

� BeStMan works with any existing file transfer servers, and there is no need to install additional file transfer servers as long as storage already supports at least one. If there is no existing file transfer server, VDT installation of BeStMan will include GridFTP server installation.

� BeStMan manages its assigned cache for its files and spaces, according to the customizable policy.

� BeStMan supports access to the user managed storage space through SRM interface with a few configurable parameters.�

� BeStMan supports a gateway mode for SRM interface on any existing file system through configuration without internal queuing or space management.� For example, BeStMan in gateway mode works with Xrootd.

� BeStMan works under GSI security with either grid-mapfile or GUMS server.

� BeStMan supports secure directory management where files are accessible only by the owner, according to the customizable policy.

� BeStMan supports customized plug-in capability for special underlying storage, such as locally developed mass storage system.

1.2 BeStMan version

� This guide is for BeStMan version 2.2.1.2 or later.

2 Requirements

� SUN Java 1.6.0_01 or higher versions such as 1.6.0_07.

� Valid grid host certificate or service certificate

3 Installation

� Manual installation from a web downloadable file
After downloading from the web (http://datagrid.lbl.gov/bestman) and expanding from the gzipped tar file, it creates bestman directory.

� VDT packman installation
VDT (http://vdt.cs.wisc.edu) provides a package for installing BeStMan, using pacman.
e.g. from the VDT 1.10.1,
pacman -get http://vdt.cs.wisc.edu/vdt_1101_cache:Bestman

3.1 Installation Preparation

� Java 1.6.0_x or higher Java home path
e.g. /sandbox/jdk1.6.0_07

� BeStMan installation directory
This directory may be created during the installation process,
e.g. /opt/bestman

� Grid Service Certificate
The existing host certificate can be used, or a new service certificate can be obtained. These service certificates must be accessible by the BeStMan process owner. Note: WLCG experiments require host certificates.
e.g. /DC=org/DC=doegrids/OU=Services/CN=myhost.lbl.gov
�� in /etc/grid-security/hostcert.pem and /etc/grid-security/hostkey.pem
e.g. /DC=org/DC=doegrids/OU=Services/CN=srm/myhost.lbl.gov
�� in /opt/srm/demo/srmcert.pem and /opt/srm/demo/srmkey.pem

� Grid CA certificate directory
The existing CA certificate directory must be used, and /etc/grid-security/certificates is used by default.
e.g. /etc/grid-security/certificates

� GridFTP server hostname and port number if different from 2811
This is needed for putting files into BeStMan managed storage system.
e.g. srm.lbl.gov

� GLOBUS_TCP_PORT_RANGE
These ports have to be opened when there is a firewall on the system.
e.g. GLOBUS_TCP_PORT_RANGE=48001,48999

� Two open port numbers to be assigned to BeStMan
These ports have to be opened when there is a firewall on the system.
e.g. http=8080, https=8443

� Local disk path and size information to be managed by BeStMan
e.g.� /data/bestman/cache : 20000MB

� log file path information for BeStMan (recommended using different disk partition from the cache)
e.g. /data2/bestman/log

� Decision to run BeStMan in gateway mode without any managements or in full management mode
e.g. gateway mode

3.2 Installing and Running BeStMan for the impatients

3.2.1 Full management mode

� For those who do not have time to follow the whole page and whose host machine meets the following assumptions, the easy three steps will have BeStMan server ready on the host machine.

o Assumptions

� root is being used to configure and run the BeStMan server

� host certificate exists in /etc/grid-security/hostcert.pem

� the machine does not have a firewall

� all default configurable values are used

1. Decide your SRM managed cache path and its size and run configure in bestman/setup directory:
e.g.� % ./configure \
--with-replica-storage-path=/data/bestman/cache \
--with-replica-storage-size=20000

2. Start the server in bestman/sbin directory:
e.g.� % SXXbestman start

3. Check the server:
e.g. % bestman/bin/srm-ping srm://hostname:8443/srm/v2/server

� When checking the server prints out BeStMan server version information, it is up and running and ready for your work.

3.2.2 Getaway mode

� For those who do not have time to follow the whole page and whose host machine meets the following assumptions, the easy three steps will have BeStMan server ready on the host machine.

o Assumptions

� root is being used to configure and run the BeStMan server in gateway mode

� host certificate exists in /etc/grid-security/hostcert.pem

� the machine does not have a firewall

� all default configurable values are used

1. Run configure in bestman/setup directory:
e.g.� % ./configure� --enable-gateway-mode

2. Start the server in bestman/sbin directory:
e.g.� % SXXbestman start

3. Check the server:
e.g. % bestman/bin/srm-ping srm://hostname:8443/srm/v2/server

� When checking the server prints out BeStMan server version information, it is up and running and ready for your work.

4 Configuration and notes

a. Change directory to the installed bestman/setup, and run configure with options.

b. Minimum options for full management mode are --with-replica-storage-path and --with-replica-storage-size when other options use default values. For gateway mode, --enable-gateway-mode is minimal. configure --help to see options with default values.

c. Upon successful running of configure, bestman/conf/bestman.rc would be created.

d. If automatic generation of configuration file does not fit your needs, you can go to section 4.3 for manual creation or modification of the configuration file.

e. srmcache keyword is on by default since v2.2.0.6 for full management mode, so that user controlled disk space can be accessible through SRM interface.
If it is not desired (no access to the user controlled disk path through SRM interface), you can disable by providing -disable-srmcache-keyword or edit conf/bestman.rc for srmcacheKeywordOn=false.

f. When space reservation needs to be configured other than the default for full management mode, consider adding/updating the following parameters in conf/bestman.rc after configure:

# for 5 GB quota allocation per user
ReplicaQualityStorageUserQuotaMB=5000
# 1GB quota allocation per request
DefaultMBPerToken=1000
# for 40% of the storage that can be reserved by space reservation
PublicSpaceProportion=60

g. When support for access to user managed space is needed,� --enable-sudofsmng option will set the proper configuration parameters (accessFileSysViaSudo=true). In this case, if BeStMan server runs under other than root account, /etc/sudoers file needs to be modified (visudo) as following. Suppose bestman process runs under daemon account:

Cmnd_Alias�� SRM_CMD = /bin/rm, /bin/mkdir, /bin/rmdir, /bin/mv, /bin/cp, /bin/ls
Runas_Alias�� SRM_USR = ALL, !root
daemon�� ALL=(SRM_USR) NOPASSWD: SRM_CMD

Some OS systems such as Fedora Core and RHEL5 may require additional entry in the /etc/sudoers for tty access (Defaults !requiretty). Some Redhat-like and Ubuntu/Debian distribution do not require this entry.

h. When multiple transfer servers are supported, they can be defined with --with-transfer-servers for configure command or configuration parameter supportedProtocolList in conf/bestman.rc as following:

supportedProtocolList=gsiftp://host1.domain.tld;gsiftp://host2.domain.tld

Multiple transfer servers are separated by semi-colon.

i. In gateway mode, static (pre-allocated) space tokens can be defined with --with-tokens-list for configure command or configuration parameter staticTokenList in conf/bestman.rc as following:

staticTokenList=token_name[desc:token_desc][token_size_in_GB]

�desc� is the keyword that cannot be changed, and multiple tokens are separated by semi-colon.
e.g.� staticTokenList=mytoken[desc:my_tokendesc][12];mytoken2[desc:my_tokendesc2][34]

This staticTokenList does not work in full management mode, and will be ignored if defined.

j. When some local paths need to be blocked for user access, they can be defined with --with-blocked-paths� for configure command or configuration parameter localPathListToBlock in conf/bestman.rc as following:

localPathListToBlock=/data/path1;/data/path2;/data/path3

Multiple paths are separated by semi-colon. Defined paths and their recursive sub-paths will be blocked for client access.

k. When only some local paths need to be allowed for user access, they can be defined with --with-allowed-paths for configure command or configuration parameter localPathListAllowed in conf/bestman.rc as following:

localPathListAllowed=/data/path1;/data/path2;/data/path3

Multiple paths are separated by semi-colon. Defined paths and their recursive sub-paths will be accessible by clients, and no other paths are accessible by clients.

4.1 Sample Configuration options

These below examples with the same configuration may not work for your environment.� For each option, choose the value that fits your environment.
Typical case where default values are used,
% configure \
--with-replica-storage-path=/data/bestman/cache \
--with-replica-storage-size=20000
Typical case for gateway mode where default values are used,
% configure \
--enable-gateway-mode
Typical� configure example,
% configure \
--with-replica-storage-path=/data/bestman/cache \
--with-replica-storage-size=20000 \
--with-globus-tcp-port-range=48001,48999 \
--with-cacert-path=/etc/grid-security/certificates \
--with-certfile-path=/opt/srm/demo/srmcert.pem \
--with-keyfile-path=/opt/srm/demo/srmkey.pem \
--with-eventlog-path=/data2/destman/log \
--with-cachelog-path=/data2/bestman/log
Another example with specified open ports for BeStMan server,
% configure \
--with-replica-storage-path=/data/bestman/cache \
--with-replica-storage-size=20000 \
--with-http-port=8080 \
--with-https-port=8443 \
--with-globus-tcp-port-range=48001,48999 \
--with-cacert-path=/etc/grid-security/certificates \
--with-certfile-path=/opt/srm/demo/srmcert.pem \
--with-keyfile-path=/opt/srm/demo/srmkey.pem \
--with-eventlog-path=/data2/destman/log \
--with-cachelog-path=/data2/bestman/log
Another example with an external globus location,
% configure \
--with-replica-storage-path=/data/bestman/cache \
--with-replica-storage-size=20000 \
--with-globus-tcp-port-range=48001,48999 \
--with-http-port=48080 \
--with-https-port=48443 \
--with-eventlog-path=/data2/bestman/log \
--with-cachelog-path=/data2/bestman/log \
--with-certfile-path=/opt/srm/demo/srmcert.pem \
--with-keyfile-path=/opt/srm/demo/srmkey.pem \
--with-globus-location=/software/globus-4.2.1
An example with GUMS connection,
% configure \
--with-replica-storage-path=/data/bestman/cache \
--with-replica-storage-size=20000 \
--with-globus-tcp-port-range=48001,48999 \
--with-http-port=48080 \
--with-https-port=48443 \
--with-eventlog-path=/data2/bestman/log \
--with-cachelog-path=/data2/bestman/log \
--with-certfile-path=/opt/srm/demo/srmcert.pem \
--with-keyfile-path=/opt/srm/demo/srmkey.pem \
--with-globus-location=/software/globus-4.0.3 \
--enable-gums \
--with-gums-url="https://gums-server.lbl.gov:8443/gums/services/GUMSAuthorizationServicePort"
An example with GUMS connection for gateway mode,
% configure \
--enable-gateway-mode \
--with-globus-tcp-port-range=48001,48999 \
--with-http-port=48080 \
--with-https-port=48443 \
--with-eventlog-path=/data2/bestman/log \
--with-certfile-path=/etc/grid-security/hostcert.pem \
--with-keyfile-path=/etc/grid-security/hostkey.pem \
--with-globus-location=/software/globus-4.2.1 \
--enable-gums \
--with-gums-url="https://gums-server.lbl.gov:8443/gums/services/GUMSAuthorizationServicePort"
An example with pre-allocated space tokens and sudo access to the underlying file system for gateway mode,
% configure \
--enable-gateway-mode \
--with-http-port=48080 \
--with-https-port=48443 \
--enable-sudofsmng \
--with-tokens-list="data[desc:mydata][10];data2[desc:mydata2][12]"
An example with access blocked paths and sudo access to the underlying file system for gateway mode,
% configure \
--enable-gateway-mode \
--with-http-port=48080 \
--with-https-port=48443 \
--enable-sudofsmng \
--with-blocked-paths=�/projects/blocked;/projects2/blocked2� \
--with-tokens-list="data[desc:mydata][10];data2[desc:mydata2][12]"
An example with allowed paths and sudo access to the underlying file system for gateway mode,
% configure \
--enable-gateway-mode \
--with-http-port=48080 \
--with-https-port=48443 \
--enable-sudofsmng \
--with-aloowed-paths=�/projects/data;/projects2/data2� \
--with-tokens-list="data[desc:mydata][10];data2[desc:mydata2][12]"
An example with extended static token information, allowed paths and sudo access to the underlying file system for gateway mode,
% configure \
--enable-gateway-mode \
--with-globus-tcp-port-range=48001,48999 \
--with-http-port=48080 \
--with-https-port=48443 \
--with-eventlog-path=/data2/bestman/log \
--enable-sudofsmng \
--with-blocked-paths=�/projects/blocked;/projects2/blocked2� \
--with-tokens-list="DT1[desc:DT1][owner:exprt][retention:REPLICA][latency:ONLINE][path:/data][12]"

4.2 Configure options

4.2.1 Required options for full management mode

--with-replica-storage-path=<PATH>	Replica Quality Storage directory path
--with-replica-storage-size=<INT>	Replica Quality Storage size in MB

4.2.2 Required options for gateway mode

--enable-gateway-mode

Enable BeStMan in gateway mode. Gateway mode provides lightweight SRM interface to any existing file system with faster request handling performance. There will be no management for space or queuing.

4.2.3 Other options for both full management mode and gateway mode

--with-srm-home=<PATH>	Installation path for BeStMan. If not given, it will be guessed based on the current working directory.
--enable-serveronly	Installation for BeStMan server only (default=no). By default, all server, client and tester are installed.
--enable-clientonly	Installation for SRM clients only (default=no). By default, all server, client and tester are installed.
--enable-testeronly	Installation for SRM tester only (default=no). By default, all server, client and tester are installed.
--enable-verbose	Print output to the standard output during the configuration
--enable-backup	Enable backup before running a new configuration if there is a previous configuration (default=no)
--enable-gums	Enable GUMS interface (default=no)
--enable-sudofsmng	Enable SUDO access for local MKDIR, RMDIR, RM, MV and CP to the user managed spaces (default=no)
--enable-gsiftpfsmng	Enable GridFTP access for local MKDIR, RMDIR, RM, MV, CP and LS to the user managed spaces (default=no)
--enable-sudols	Enable SUDO access for local LS to the user managed spaces (default=no)
--with-java-home=<PATH>	Specify the JAVA_HOME directory
--enable-java-version-check	Enable java version check (default=yes). It checks if java version is 1.6.0_01 or higher.
--with-globus-tcp-port-range=<VALUES>	Specify the GLOBUS_TCP_PORT_RANGE when firewall is enabled. E.g. 6201,6299
--with-globus-tcp-source-range=<VALUES>	Specify the GLOBUS_TCP_SOURCE_RANGE when necessary
--with-gums-url=<URL>	Specify GUMS server service URL with service handle
--with-gums-dn=<DN>	Specify the GUMS client service DN that GUMS server would recognize (default=SRM service DN)
--with-srm-owner=<LOGIN>	Specify the BeStMan SRM server process owner (default=root)
--with-max-java-heap=<INT>	Specify the max java heap size in MB (default=512)
--with-min-java-heap=<INT>	Specify the min java heap size in MB (default=16)
--with-http-port=<PORT>	Specify the http port (default=8080)
--with-https-port=<PORT>	Specify the https port (default=8443)
--enable-eventlog	Enable event logging (default=yes). When disabled, there is no logging performed.
--with-eventlog-path=<PATH>	Specify the EventLogFile� directory path (default=/var/log)
--with-cachelog-path=<PATH>	Specify the CacheLogFile directory path (default=/var/log)
--with-output-storage-path=<PATH>	Specify the OutputQualityStorage directory path
--with-output-storage-size=<INT>	Specify the OutputQualityStorage Size in MB
--with-custodial-storage-path=<PATH>	Specify the CustodialQualityStorage directory path
--with-custodial-storage-size=<INT>	Specify the CustodialQualityStorage Size in MB
--with-max-users=<INT>	Specify the maximum number of active users (default=100)
--with-max-filerequests=<INT>	Specify the maximum number of active file requests (default=1000000)
--with-concurrency=<INT>	Specify the number of concurrent requests (default=40)
--with-concurrent-filetransfer=<INT>	Specify the number of concurrent file transfers (default=10)
--with-gridftp-parallel-streams=<INT>	Specify the number of gridftp parallel streams (default=2)
--with-gridftp-buffersize=<INT>	Specify the gridftp buffer size in bytes (default=1048576)
--with-default-filesize=<INT>	Specify the default file size in MB (default=500)
--with-volatile-file-lifetime=<INT>	Specify the default lifetime of volatile files in seconds (default=1800)
--with-space-file-lifetime=<INT>	Specify the default lifetime of files in public space in seconds (default=1800)
--with-inactive-transfer-timeout=<INT>	Specify the default time out value for inactive user file transfer in seconds (default=300)
--with-public-space-size=<INT>	Specify the default size for SRM owned volatile space in MB
--with-public-space-proportion=<INT>	Specify default size for SRM owned volatile space in percentage (default=80)
--with-default-space-size=<INT>	Specify the default size for space reservation in MB (default=1000)
--with-cacert-path=<PATH>	Specify the Grid CA Certificate directory path (default=/etc/grid-security/certificates)
--with-certfile-path=<PATH>	Specify the Grid Certificate file path (default=/etc/grid-security/hostcert.pem)
--with-keyfile-path=<PATH>	Specify the Grid Certificate Key file path (default=/etc/grid-security/hostkey.pem)
--with-proxyfile-path=<PATH>	Specify the Grid proxy file path
--with-gums-certfile-path=<PATH>	Specify the GUMS client Grid Certificate file path (default=same as �with-certfile-path)
--with-gums-keyfile-path=<PATH>	Specify the GUMS client Grid Certificate Key file path (default=same as �with-keyfile-path)
--with-gums-proxyfile-path=<PATH>	Specify the GUMS client Grid proxy file path
--with-gridmap-path=<PATH>	Specify the grid-mapfile path (default=/etc/grid-security/grid-mapfile)
--with-max-mss-connection=<INT>	Specify the maximum MSS file transfers when supported (default=5)
--with-mss-timeout=<INT>	Specify the MSS connection timeout in seconds when supported (default=600)
--with-plugin-path=<PATH>	Specify the plug-in library directory path when supported
--with-globus-location=<PATH>	Specify the GLOBUS_LOCATION path
--with-transfer-servers=<STRING>	Specify supported file transfer servers
--with-blocked-paths=<PATH>	Specify Non-accessible paths (in addition to /;/etc;/var). Multiple entries are separated by semi-colon.
--with-allowed-paths=<PATH>	Specify accessible paths only (separated by semi-colon)

4.2.4 Other options for gateway mode only

--enable-checkfile-fs	Enable use of file system to check file size (default=yes)
--enable-checkfile-gsiftp	Enable use of GridFTP to check file size (default=no). This option may not work with LCG-utils because of delegation issues.
--enable-pathfortoken	Enable PathForToken mode (default=yes)
--with-tokens-list=<STRING>	Specify pre-allocated static space tokens list with their sizes when supported. Format: token_name[KEY:VALUE][size_in_GB] �� KEY = desc, owner, retention, latency, �� path, usedBytesCommand �� retention avail values = REPLICA, OUTPUT, CUSTODIAL �� latency avail values = ONLINE, NEARLINE �� usedBytesCommand = e.g. some custom script or �� du �s �b�. Its output must have the �� available bytes as the first value

4.3 Configuration file

Upon successful configuration, bestman/conf/bestman.rc would be created.�� Each entry has the following meaning.

ReplicaQualityStorageMB	� Replica Quality Storage Size and Path � For more than one path, use ";" to seperate them on one line. � Size information is specified in "[###]" before the path where �###� is the value of the size in MB. � Replica quality has the highest probability of loss such as disks, but is appropriate for data that can be replaced because other copies can be accessed from somewhere.
	e.g. ReplicaQualityStorageMB=[5100]path=/bestman/cache ReplicaQualityStorageMB=[300]path=/bestman/cache;[200]path=/bestman2/cache
OutputQualityStorageMB	� Output Quality Storage Size and Path � For more than one path, use ";" to seperate them on one line. � Size information is specified in "[###]" before the path where �###� is the value of the size in MB. � Output quality is an intermediate level and refers to the data which can be replaced by lengthy or effort-full processes. � [We currently do not support OutputQualityStorage]
	e.g. OutputQualityStorageMB=[2000]path=/bestman/cached
CustodialQualityStorageMB	� Custodial Quality Storage (e.g. disk spaces for permanent files) � For more than one path, use ";" to seperate them on one line. � Size information is specified in "[###]" before the path where �###� is the value of the size in MB. � Custodial quality provides low probability of loss such as tapes.
	e.g. CustodialQualityStorageMB=[1000]path=/bestman/pcache CustodialQualityStorageMB=[200]path=/bestman/cache/p;[200]path=/bestman2/cache
	� When Custodial Quality Storage supports mass storage system such as HPSS, Zero (0) size indicates indefinite.
	e.g. For user specified MSS path access CustodialQualityStorageMB=[0]path=&type=gov.lbl.srm.transfer.mss.hpss.SRM_MSS_HPSS&host=garchive.nersc.gov&conf=hpss.datagrid.rc
	e.g. For bestman owned MSS path access: Only when specific path on MSS is used as custodial storage CustodialQualityStorageMB=[0]path=/nersc/bestman/&type=gov.lbl.srm.transfer.mss.hpss.SRM_MSS_HPSS&host=garchive.nersc.gov&conf=hpss.datagrid.rc
	e.g. For other customized MSS plugins CustodialQualityStorageMB=[0]path=/lstore/bestman&type=plugin.lstore.SRM_MSS_LSTORE&jarFile=lstore.jar&host=lstore.domain.edu&conf=lstore.rc

4.3.2 Related to the server control

These entries have the default values when configured.

supportedProtocolList	� List of the supported file transfer protocol list. � Use �;� to separate multiple entries
	e.g. supportedProtocolList= gsiftp://machA.domain/;gsiftp://machB.domain:2812/;ftp://machC.domain/;http://machD.domain:9123/
protocolSelectionPolicy	� Custom policy for transfer protocol selection. � Default is round robin.
	e .g. protocolSelectionPolicy=class=edu.unl.rcf.BestmanGridftpSelector.BestmanGridftp&jarFile=UNLGangliaBestman.jar&name=gsiftp
MaxNumberOfUsers	� Number of active users limit. Beyond the limit, the request will get SRM_FAILURE with explanations
	e .g. MaxNumberOfUsers=100
MaxNumberOfFileRequests	� Number of active and queued file requests limit. Beyond the limit, the request will get SRM_FAILURE with explanations
	e .g. MaxNumberOfFileRequests =1000000
Concurrency	� Number of file requests that BeStMan server processes at a time. Beyond the limit, file requests will wait on the queue for any of the completed requests
	e.g. Concurrency=20
MaxConcurrentFileTransfer	� Maximum concurrent file transfers
	e.g. MaxConcurrentFileTransfer=10
GridFTPNumStreams	� Number of parallel streams per gridftp file transfer
	e.g. GridFTPNumStreams=2
GridFTPBufferSizeBytes	� Buffer size of the gridftp file transfer in bytes
	e.g. GridFTPBufferSizeBytes=2097152
GridFTPBufferSizeMB	� Buffer size of the gridftp file transfer in MB. � GridFTPBufferSizeMB takes priority from GridFTPBufferSizeBytes when both are defined
	e.g.� GridFTPBufferSizeMB=2
GridFTPDcauOn	� Enable DCAU for GridFTP (Default: true)
	e.g. GridFTPDcauOn=true
DefaultVolatileFileLifeTimeInSeconds	� Default lifetime of volatile files in seconds
	e.g. DefaultVolatileFileLifeTimeInSeconds=1800
DefaultFileSizeMB	� Default file size in MB (default=1/10 of cache size)
	e.g. DefaultFileSizeMB =1000
PublicTokenMaxFileLifetimeInSeconds	� Max file lifetime that can be granted in the unreserved "public" storage space
	e.g. PublicTokenMaxFileLifetimeInSeconds=600
PublicTokenMaxMBPerUser	� Max file size in the unreserved "public" storage space per user
	e.g. PublicTokenMaxMBPerUser=300
PublicTokenMaxNumFilesPerUser	� Max number of files in the unreserved "public" storage space per user
	e.g. PublicTokenMaxNumFilesPerUser =100
InactiveTxfTimeOutInSeconds	� Default time out value for inactive user file transfer in case user puts a file into the BeStMan cache, in seconds
	e.g. InactiveTxfTimeOutInSeconds=900
PublicSpaceInMB	� Size of the BeStMan owned public volatile storage space in MB � When both PublicSpaceProportion and PublicSpaceInMB are defined, PublicSpaceInMB takes priority and is effective.
	e.g. PublicSpaceInMB=1000
PublicSpaceProportion	� Size of the BeStMan owned public volatile storage space in percentage � When both PublicSpaceProportion and PublicSpaceInMB are defined, PublicSpaceInMB takes priority and is effective.
	e.g. PublicSpaceProportion=80
DefaultMBPerToken	� Default space reservation size when user requests without specific size info, in MB
	e.g. DefaultMBPerToken=1000
WorldPermission	� File readable permission in BeStMan cache. � By default, all files are accessible (read-only) by others. � It can be disabled by setting this to �None�, and no one other than the owners can read their files. � Other options are R, W, or None.
	e.g. WorldPermission=None
silent	� When set to true, minimum output will be displayed on the console (default = false)
	e.g. silent=true
retryGsiftp	� Retry options for BeStMan initiated file transfers. � Value is specified as (seconds/maxRetry) � Default is 120 seconds apart between each retry, and maximum 2 retries of failed gsiftps. � If maxRetry value ismissing, the default is assumed to be 2.
	e.g. retryGsiftp=120/2 e.g. retryGsiftp=200
guc_path	� Sets the path to the globus-url-copy to be used in gsiftp file transfers, rather than gsiftp client lib calls when defined.
	e.g. guc_path=/sandbox/globus/bin/globus-url-copy
uploadQueueParameter	� Sets a balance between read and write of the file transfers. � When not set, all files transfers use one queue. � FORMAT: N[:M] where N is number of threads for the queue and M is number of file transfers allowed for the queue
	e.g. uploadQueueParameter=40:10
srmcacheKeywordOn	� If set to true, then �srmcache� is a required prefix to refer to the srm cache files. � For example,� to refer to a file �myfile� owned by �uid� in srm,� it needs to be look like srm://host:port/srm/v2/server?SFN=/srmcache/uid/myfile. � While without using /srmcache, the surl srm://host:port/srm/v2/server?SFN=/tmp/myfile will refer to the file /tmp/myfile on the local disk that runs the srm server. � Server default is false
	e.g. srmcacheKeywordOn=true
markupPingMsg	� When set to true, some of the extra information from srmPing do not get returned to the client. � Default=false
	e.g. markupPingMsg=true
localPathListToBlock	� Blocked list of the local directory path for user access � Default=/;/etc/;/var � Any definition will include the default block list � Multiple entries are separated by semi-colon
	e.g. localPathListToBlock=/home/secret;/data/secret2
localPathListAllowed	� Allowed list of the local directory path for user access � Any definition will include the default block list � If a path is listed both on blocked and allowed list, blocked takes priority. � Multiple entries are separated by semi-colon
	e.g. localPathListAllowed=/home/data;/data/public
disableSpaceMgt	� Disable space management � Default=false � All space related functions are not supported when it is set to true. E.g. srmReserveSpace � Gateway mode must have it true
	e.g. disableSpaceMgt=true
disableDirectoryMgt	� Disable directory management � Default=false � All directory related functions is not supported when it is set to true. E.g. srmMkdir, srmRmdir, srmLs
	e.g. disableDirectoryMgt=true
disableLocalAuthorization	� Disable SRM Permission functions � Default=true � All permission related functions is not supported when it is set to true. E.g. srmSetPermission
	e.g. disableLocalAuthorization=false
disableSrmCopy	� Disable SRM remote copy function � Default=false � srmCopy function is not supported when it is set to true.
	e.g. disableSrmCopy=true
GUMSserviceURL	� GUMS server service endpoint
	e.g. GUMSserviceURL= https://gumsserver.lbl.gov:8443/gums/services/GUMSAuthorizationServicePort
GUMSCurrHostDN	� GUMS client service dn � This DN is provided to GUMS server so that it can decide which mapping group the calling client (bestman) is in. � When not provided, server extracts DN information from the service certificate. � This DN takes priority than �with-gums-certfile-path and �with-gums-keyfile-path. � Default=/DC=org/�/CN hostname
	e.g. GUMSCurrHostDN=/DC=org/DC=doegrids/OU=Services/CN=gums-client.lbl.gov
ReplicaQualityStorageUserQuotaMB	� User quota for reserving replica quality storage, in MB � Default=no limit
	e.g. ReplicaQualityStorageUserQuotaMB=1000
accessFileSysViaGsiftp	� Allows BeStMan access file system through gsiftp on behalf of the user, upon user request � Default=false � When both accessFileSysViaGsiftp and accessFileSysViaSudo are defined, accessFileSysViaGsiftp takes priority. � This may not work with LCG-utils because of the delegation issues.
	e.g. accessFileSysViaGsiftp=true
accessFileSysViaSudo	� Allows BeStMan access file system through sudo on behalf of the user, upon user request � Default=false � This option is recommended when BeStMan is used to provide SRM interface to user defined storage space. � /etc/sudoers must be modified for BeStMan running under other than root � e.g. Recommended modification on the /etc/sudoers when BeStMan runs under daemon Cmnd_Alias SRM_CMD = /bin/rm, /bin/mkdir, /bin/rmdir, /bin/mv, /bin/ls, /bin/cp Runas_Alias SRM_USR = ALL, !root daemon�� ALL=(SRM_USR) NOPASSWD: SRM_CMD � Some OS systems such as Fedora Core and RHEL5 may require additional entry in the /etc/sudoers for tty access. Some Redhat-like and Ubuntu/Debian distribution do not require this entry. Defaults !requiretty � When both accessFileSysViaGsiftp and accessFileSysViaSudo are defined, accessFileSysViaGsiftp takes priority.
	e.g. accessFileSysViaSudo=true
noSudoOnLs	� When false, Allows BeStMan access file system for ls through sudo on behalf of the user, upon user request � Default=true � To have this option effective, accessFileSysViaSudo must be true. � This option is recommended when BeStMan is used to provide SRM interface to user defined storage space. � /etc/sudoers must be modified for BeStMan running under other than root � e.g. Recommended modification on the /etc/sudoers when BeStMan runs under daemon Cmnd_Alias SRM_CMD = /bin/rm, /bin/mkdir, /bin/rmdir, /bin/mv, /bin/ls, /bin/cp Runas_Alias SRM_USR = ALL, !root daemon�� ALL=(SRM_USR) NOPASSWD: SRM_CMD � Some OS systems such as Fedora Core and RHEL5 may require additional entry in the /etc/sudoers for tty access. Some Redhat-like and Ubuntu/Debian distribution do not require this entry. Defaults !requiretty
	e.g. noSudoOnLs=true
userSpaceKeywords	� Allows pre-defined space tokens. BeStMan does not manage these spaces, but provides access to users through SRM interface. � Format is (spacetoken1=/dirpath1)(spacetoken2=/dirpath2) � When these re-defined space tokens are used in a request, the SFN should not include the full path.
	e.g. refer to 7.10 userSpaceKeywords=(SPT1=/data/dirpath1)(SPT2=/data2/dirpath2)(SPT3=/data3/dirpath3)

These entries have the default values when configured.

securePort	� Secure ports for SRM service endpoint. � These ports must be open for firewall. � The service end point will be: srm://hostname.domain:securePort/srm/v2/FactoryID
	e.g. securePort=8443
publicPort	� Ports for SRM service endpoint. � These ports must be open for firewall.
	e.g. publicPort=8080
protocol	� Protocol for service endpoint. � This is fixed for httpg in SRM v2.2.
	e.g. protocol=httpg
EventLogLocation	� Path for service event log. � This can be either specific file path or directory path. � Default=/var/log/event.bestman.log
	e.g. EventLogLocation=/tmp/bestman/event.bestman.log e.g. EventLogLocation=/tmp/bestman
noEventLog	� When enabled, no event log is written. � Default=false
	e.g. noEventLog=true
CacheLogLocation	� Path for cache event log. � Default=/var/log/cache.bestman.log. � This can be either specific file path or directory path. � When useBerkeleyDB is true (by default), DB files are written in /var/log by default. If CacheLogLocation is defined when useBerkeleyDB is defined as true, CacheLogLocation must be a directory path.
	e.g. CacheLogLocation=/tmp/bestman/cache.bestman.log e.g. CacheLogLocation=/tmp/bestman
noCacheLog	� When enabled, no cache log is written. � Default=false � For gateway mode, this option must be true.
	e.g. noCacheLog=true
CertFileName	� Grid service certifiticate file path � When the entry is missing, server will try to use the user grid proxy, and make a prompt for the proxy password every time. � Those cert/key files must be readable by the BeStMan process owner.
	e.g. CertFileName=/etc/grid-security/hostcert.pem
KeyFileName	� Grid service certifiticate Key file path
	e.g. KeyFileName=/etc/grid-security/hostkey.pem
ProxyFileName	� Grid proxy file path � If provided, proxy will take priority than cert/key files. � When user proxy is used, only the particular user may access the BeStMan server.
	e.g. ProxyFileName=/tmp/proxyFile
GridMapFileName	� Provide GridMapFileName if it is not in the default location, /etc/grid-security/grid-mapfile
	e.g. GridMapFileName=/etc/grid-security/grid-mapfile
useBerkeleyDB	� use of Berkeley DB as an internal management component. � Default=true. � When it�s false, text file based CacheLogLocation will be used.
	e.g. useBerkeleyDB=true
FactoryID	� FactoryID is for web service end point. � Recommended to be �server�. � The service end point will be srm://hostname.domain:secure_port/srm/v2/FactoryID � When this has different name than �server�, server-config.wsdd file needs to be updated too.
	e.g. FactoryID=server

4.3.4 Related to the gateway mode only

These entries would only be effective, when gateway mode is enabled.

pathForToken	� Gateway mode supports a path defined as a space token. � When this option is defined as true, BeStMan server checks the available space in this path for the expected file size. � When this option is defined as true, the TURL becomes a combination of space token and SFN. E.g. when space token is /data/scratch1 and SFN is /mydir/myfile, the TURL becomes, when /data/scracth1 has enough space for the file, gsiftp://hostname//data/scrach1/mydir/myfile.
	e.g. pathForToken=true
staticTokenList	� Specifies pre-allocated space tokens with their size info � Format is token_name[KEY:VALUE][token_size_inGB] KEY = desc, owner, retention, latency, path, usedBytesCommand a keyword cannot be changed. retention available values = REPLICA, OUTPUT, CUSTODIAL latency available values = ONLINE, NEARLINE usedBytesCommand = e.g. some custom script or �du �s �b�. �� Its output must have the available bytes as the first value � Multiple tokens are separated by semi-colon
	e.g. staticTokenList=mytoken[desc:my_tokendesc][12];mytoken2[desc:my_tokendesc2][34] e.g. staticTokenList=DATA1[desc:DATA1][owner:projects][retention:REPLICA][latency:ONLINE][path:/projects/data/][usedBytesCommand:/usr/bin/du -s -b][12]
checkSizeWithFS	� Enables file size browsing through file system � Default=true � When both checkSizeWithFS and checkSizeWithGsiftp are defined to be true, checkSizeWithFS takes priority. � When both checkSizeWithFS and checkSizeWithGsiftp are defined to be false, file size browsing such as srmLs would fail.
	e.g. checkSizeWithFS=true
checkSizeWithGsiftp	� Enables file size browsing through gridftp server � Default=false � When both checkSizeWithFS and checkSizeWithGsiftp are defined to be true, checkSizeWithFS takes priority. � When both checkSizeWithFS and checkSizeWithGsiftp are defined to be false, file size browsing such as srmLs would fail. � This may not work with LCG-utils because of the delegation issues.
	e.g. checkSizeWithGsiftp=false

4.3.5 Related to the MSS connection

When backend MSS is supported, these entries would affect the its connection to MSS.

MaxMSSConnections	� maximum MSS transfers � Non MSS File Transfers = MaxConcurrentFileTransfer - MaxMSSConnections
	e.g. MaxMSSConnections=5
mssTimeOutSeconds	� MSS connection timeout in seconds
	e.g. mssTimeOutSeconds=3600
pluginLib	� If plugin is provided, then pluginLib needs to be defined for the directory to look for the plugin libraries. � This is usually for the customized BeStMan for localized MSS. � Libraries are expected to be jar files for dynamic loading.
	e.g. pluginLib=/opt/bestman/plugin/lib

5 Runnign BeStMan server

5.1 Running BeStMan server manually

From the bestman installation directory, run sbin/bestman.server.

SXXbestman.personal may be used for testing purpose only that it saves all screen outputs to /tmp/bestman-.log.� SXXbestman.personal has four options: start, stop, check and checkjava.

� �SXXbestman.personal start� will run the bestman server and output goes into /tmp/bestman-.log as well as on the screen.

� �SXXbestman.personal stop� will stop the running server.

� �SXXbestman.personal check� will check the process to see if it is running.

� �SXXbestman.personal checkjava� will check java version that the server runs with.

5.2 Running BeStMan server from VDT installation

From the bestman installation directory (bestman/sbin/SXXbestman), VDT-control will run BeStMan server process under daemon by default.

5.3 Running BeStMan server under root or installing as a startup process

As root, make a link or a copy of bestman/sbin/SXXbestman in /etc/rc3.d as S97bestman.

SXXbestman has three options: start, stop and check.

� �SXXbestman start� will run the bestman server in the background.

� �SXXbestman stop� will stop the running server.

� �SXXbestman check� will check the process to see if it is running.

SXXbestman has the BeStMan server process owner login embedded. By default, �root� will be used.
If --with-srm-owner were defined during the configuration, the specific login will be used.
If a modification is needed later, edit bestman/sbin/SXXbestman, and modify for SRM_OWNER.

6 Sample configuration file

6.1 bestman.rc from VDT installed BeStMan in full mode

#####################################################################

ReplicaQualityStorageMB=[200000]path=/project/projectdirs/osg/srm/cache;

#####################################################################

MaxNumberOfUsers=100

MaxNumberOfFileRequests=1000000

Concurrency=40

MaxConcurrentFileTransfer=5

GridFTPNumStreams=2

GridFTPBufferSizeBytes=1048576

DefaultFileSizeMB=1000

DefaultVolatileFileLifeTimeInSeconds=1800

PublicTokenMaxFileLifetimeInSeconds=1800

InactiveTxfTimeOutInSeconds=300

PublicSpaceProportion=80

DefaultMBPerToken=1000

publicPort=62080

securePort=62443

EventLogLocation=/export/data/bestman/log

CacheLogLocation=/export/data/bestman/log

useBerkeleyDB=true

CertFileName=/export/data/bestman/.globus/bestman-cert.pem

KeyFileName=/export/data/bestman/.globus/bestman-key.pem

GridMapFileName=/etc/grid-security/grid-mapfile

FactoryID=server

srmcacheKeywordOn=true

uploadQueueParameter=40:5

6.2 bestman.rc for BeStMan in full mode with GUMS interface

###########################################################

ReplicaQualityStorageMB=[200000]path=/data/bestman/cache;

###########################################################

MaxNumberOfUsers=100

MaxNumberOfFileRequests=1000000

Concurrency=40

MaxConcurrentFileTransfer=10

GridFTPNumStreams=2

GridFTPBufferSizeBytes=1048576

DefaultFileSizeMB=500

DefaultVolatileFileLifeTimeInSeconds=1800

PublicTokenMaxFileLifetimeInSeconds=1800

InactiveTxfTimeOutInSeconds=300

PublicSpaceProportion=80

DefaultMBPerToken=1000

publicPort=6286

securePort=6288

EventLogLocation=/opt/bestman/log

CacheLogLocation=/opt/bestman/log

useBerkeleyDB=true

srmcacheKeywordOn=true

CertFileName=/home/gridtest/srmcert.pem

KeyFileName=/home/gridtest/srmkey.pem

GridMapFileName=/opt/bestman/conf/grid-mapfile.empty

GUMSserviceURL=https://gums-server.lbl.gov:8443/gums/services/GUMSAuthorizationServicePort

GUMSCurrHostDN=/DC=org/DC=doegrids/OU=Services/CN=http/bestman.lbl.gov

FactoryID=server

uploadQueueParameter=40:10

6.3 bestman.rc for BeStMan in full mode from manual configure

###########################################################

ReplicaQualityStorageMB=[200000]path=/data/bestman/cache;

###########################################################

MaxNumberOfUsers=100

MaxNumberOfFileRequests=1000000

Concurrency=40

MaxConcurrentFileTransfer=10

GridFTPNumStreams=2

GridFTPBufferSizeBytes=1048576

DefaultFileSizeMB=500

DefaultVolatileFileLifeTimeInSeconds=1800

PublicTokenMaxFileLifetimeInSeconds=1800

InactiveTxfTimeOutInSeconds=300

PublicSpaceProportion=80

DefaultMBPerToken=1000

publicPort=6286

securePort=6288

EventLogLocation=/opt/bestman/log

CacheLogLocation=/opt/bestman/log

useBerkeleyDB=true

srmcacheKeywordOn=true

CertFileName=/home/gridtest/srmcert.pem

KeyFileName=/home/gridtest/srmkey.pem

GridMapFileName=/etc/grid-security/grid-mapfile

FactoryID=server

uploadQueueParameter=40:10

6.4 bestman.rc for BeStMan in full mode from manual configure with more options

####################################################################################

ReplicaQualityStorageMB=[200000]path=/data/bestman/cache;[500000]path=/data4/bestman/cache

CustodialQualityStorageMB=[2000]path=/data2/bestman/pcache;[4000]path=/data3/bestman/pcache

supportedProtocolList=gsiftp://thost1.lbl.gov;gsiftp://thost2.lbl.gov;gsiftp://thost3.lbl.gov

userSpaceKeywords=(ATLASUSERDISK1=/project/atlas/user1)(ATLASUSERDISK2=/project/atlas/user2)

###################################################################################

MaxNumberOfUsers=100

MaxNumberOfFileRequests=1000000

Concurrency=40

MaxConcurrentFileTransfer=10

GridFTPNumStreams=2

GridFTPBufferSizeBytes=1048576

DefaultFileSizeMB=500

DefaultVolatileFileLifeTimeInSeconds=1800

PublicTokenMaxFileLifetimeInSeconds=1800

InactiveTxfTimeOutInSeconds=300

PublicSpaceProportion=80

DefaultMBPerToken=1000

publicPort=6286

securePort=6288

EventLogLocation=/opt/bestman/log

CacheLogLocation=/data/bestman/log

useBerkeleyDB=true

srmcacheKeywordOn=true

CertFileName=/home/gridtest/srmcert.pem

KeyFileName=/home/gridtest/srmkey.pem

GridMapFileName=/etc/grid-security/grid-mapfile

FactoryID=server

uploadQueueParameter=40:10

6.5 bestman.rc for BeStMan in gateway mode

###########################################################

MaxNumberOfUsers=100

MaxNumberOfFileRequests=1000000

Concurrency=40

MaxConcurrentFileTransfer=10

publicPort=6286

securePort=6288

supportedProtocolList=gsiftp://gserver1.lbl.gov;gsiftp://gserver2.lbl.gov;gsiftp://gserver3.lbl.gov

CertFileName=/etc/grid-security/hostcert.pem

KeyFileName=/etc/grid-security/hostkey.pem

GridMapFileName=/opt/bestman/conf/grid-mapfile.empty

GUMSserviceURL=https://gums-server.lbl.gov:8443/gums/services/GUMSAuthorizationServicePort

FactoryID=server

accessFileSysViaSudo=true

accessFileSysViaGsiftp=true

noEventLog=false

EventLogLocation=/opt/bestman/log

noCacheLog=true

disableSpaceMgt=true

pathForToken=true

checkSizeWithFS=true

checkSizeWithGsiftp=false

staticTokenList=data[desc:mydata][10];data2[desc:mydata2][12]

6.6 bestman.rc for BeStMan in gateway mode with GUMS interface

###########################################################

MaxNumberOfUsers=100

MaxNumberOfFileRequests=1000000

Concurrency=40

MaxConcurrentFileTransfer=10

publicPort=6286

securePort=6288

supportedProtocolList=gsiftp://gserver1.lbl.gov;gsiftp://gserver2.lbl.gov;gsiftp://gserver3.lbl.gov

CertFileName=/etc/grid-security/hostcert.pem

KeyFileName=/etc/grid-security/hostkey.pem

GridMapFileName=/etc/grid-security/grid-mapfile

FactoryID=server

accessFileSysViaSudo=true

accessFileSysViaGsiftp=true

noEventLog=false

EventLogLocation=/data/asim/bestman/log

noCacheLog=true

disableSpaceMgt=true

pathForToken=true

checkSizeWithFS=true

checkSizeWithGsiftp=false

staticTokenList=data[desc:mydata][10];data2[desc:mydata2][12]

7 Sample SRM client commands related to BeStMan configuration

SRM client commands can be found in bestman/bin directory. They are full implementations of SRM v2.2 as generic SRM v2.2 clients, developed by Lawrence Berkeley National Laboratory. They have been tested for all current SRM v2.2 implementations such as BeStMan, CASTOR, dCache, DPM, SRM/iRODS-SRB and StoRM. They are continuously being tested for compatibility and interoperability.

Assume the BeStMan instance is on srm://bestman.lbl.gov:8443/srm/v2/server.
Assume the local file path is used as /tmp/my.test.file.
Assume the user grid proxy exists and is valid, and the user is grid-mapped on the server side as arie.

7.1 SRM-COPY

a. When BeStMan is configured with default values, and a file is requested to put into the BeStMan managed storage cache:

srm-copy \
�� file:////tmp/my.test.file \
�� srm://bestman.lbl.gov:8443/srm/v2/server\?SFN=/srmcache/~/my.test.file

Note that �~� is used for user�s top directory under BeStMan managed storage cache.
�~� is translated into the locally mapped account from grid mapping. When users do not know their grid mappings, they can use �~�.
e.g. in grid-mapfile, �/DC=org/DC=doegrids/OU=People/CN=Arie Shoshani 245501" arie,
�~� is translated into �arie� which makes the full SFN as /srmcache/arie/my.test.file.
Both SFNs srm://bestman.lbl.gov:8443/srm/v2/server\?SFN=/srmcache/~/my.test.file� and srm://bestman.lbl.gov:8443/srm/v2/server\?SFN=/srmcache/arie/my.test.file� would work.

b. When BeStMan is configured with default values, and a file is requested to get from the BeStMan managed storage cache:

srm-copy \
�� srm://bestman.lbl.gov:8443/srm/v2/server\?SFN=/srmcache/arie/my.test.file \
�� file:////tmp/my.test.file2

c. When BeStMan is configured with default values, and a file is requested to put into the user managed storage paths (e.g. /myproject/mydir):

srm-copy \
�� file:////tmp/my.test.file \
�� srm://bestman.lbl.gov:8443/srm/v2/server\?SFN=/myproject/mydir/my.test.file

Note that BeStman does not manage these files, but provides an access to the user controlled storage space with SRM interfaces.

7.2 SRM-LS

a. When BeStMan is configured with default values, and a file is requested from the BeStMan� managed storage cache:

srm-ls \
�� srm://bestman.lbl.gov:8443/srm/v2/server\?SFN=/srmcache/arie/my.test.file

b. When BeStMan is configured with default values, and a file is requested from the user managed storage paths (e.g. /myproject/mydir):

srm-ls \
�� srm://bestman.lbl.gov:8443/srm/v2/server\?SFN=/myproject/mydir/my.test.file

7.3 SRM-RM

a. When BeStMan is configured with default values, and a file is requested to be removed from the BeStMan� managed storage cache:

srm-rm \
�� srm://bestman.lbl.gov:8443/srm/v2/server\?SFN=/srmcache/arie/my.test.file

b. When BeStMan is configured with default values, and a file is requested to be removed from the user managed storage paths (e.g. /myproject/mydir):

srm-rm \
�� srm://bestman.lbl.gov:8443/srm/v2/server\?SFN=/myproject/mydir/my.test.file

Note that BeStman does not manage these files, but provides an access to the user controlled storage space with SRM interfaces.
For permission issues, accessFileSysViaSudo or accessFileSysViaGsiftp needs to be defined.
When accessFileSysViaGsiftp is defined, only LBNL or FNAL implementation of srm-rm or srmrm can be used because of the delegation issue. LCG-utils do not delegate credentials and BeStMan cannot access gsiftp server to remove the requested files on behalf of the user.
When accessFileSysViaSudo is defined, any clients implementing srmRm can be used, and �/etc/sudoers needs proper entry for BeStMan server. Refer to 4.3.2 for recommended entry.

7.4 SRM-MKDIR and SRM-RMDIR

a. When BeStMan is configured with default values, and a directory is requested to be created or removed from the BeStMan managed storage cache:

srm-mkdir (or srm-rmdir) \
�� srm://bestman.lbl.gov:8443/srm/v2/server\?SFN=/srmcache/arie/my.test.dir

b. When BeStMan is configured with default values, and a directory is requested to be created or removed from the user managed storage paths (e.g. /myproject/mydir):

srm-mkdir (or srm-rmdir)� \
�� srm://bestman.lbl.gov:8443/srm/v2/server\?SFN=/myproject/mydir/my.test.dir

Note that BeStman does not manage these files, but provides an access to the user controlled storage space with SRM interfaces.
For permission issues, accessFileSysViaSudo or accessFileSysViaGsiftp needs to be defined.
When accessFileSysViaGsiftp is defined, only LBNL or FNAL client implementation can be used because of the delegation issue. LCG-utils do not delegate credentials and BeStMan cannot access gsiftp server to create or remove the requested files on behalf of the user.
When accessFileSysViaSudo is defined, any clients implementing srmRm can be used, and /etc/sudoers needs proper entry for BeStMan server. Refer to 4.3.2 for recommended entry.

7.5 SRMPING from FNAL client

a. When BeStMan is configured with default values, and FNAL SRM client srmping needs some extra flags:

srmping �-2 �-webservice_path=srm/v2/server �-debug srm://bestman.lbl.gov:8443
Or srmping �-2 -debug srm://bestman.lbl.gov:8443/srm/v2/server

Note that some of FNAL SRM clients have a default GSS type to support in a request call. By default it is fixed to �host�. Many of SRM deployment use other service type certificates such as �http� or �srm�. In such cases, FNAL SRM clients need �-gss_expected_name=srm� or �-gss_expected_name=http� as an option.

7.6 SRMLS from FNAL client

a. When BeStMan is configured with default values, and FNAL SRM client srmls needs some extra flags:

srmls �-webservice_path=srm/v2/server \
�� srm://bestman.lbl.gov:8443/srm/v2/server\?SFN=/srmcache/arie/my.test.file

Note that all FNAL SRM clients have default SRM service handle as srm/managerv2, and it needs to be specified in the option with �-webservice_path�.

7.7 SRMCP from FNAL client

a. When BeStMan is configured with default values, and FNAL SRM client srmcp needs some extra flags:

srmcp -2 �-webservice_path=srm/v2/server \
�� srm://bestman.lbl.gov:8443 /srm/v2/server\?SFN=/srmcache/arie/my.test.file \
�� file:////tmp/my.test.file

Or srmcp -2 ��srm://bestman.lbl.gov:8443 /srm/v2/server\?SFN=/srmcache/arie/my.test.file \
�� file:////tmp/my.test.file

Note that all FNAL SRM clients have default SRM service handle as srm/managerv2, and it needs to be specified in the option with �-webservice_path� for older versions. Also, some of FNAL SRM clients support both SRM v1.1 and SRM v2.2 with default pointing to v1.1, and it needs to be specified in the option with �-2�.
Extra command line options such as �-access_latency=ONLINE -retention_policy=CUSTODIAL� can be provided when supported in BeStMan configuration.

7.8 SRMRM from FNAL client

a. When BeStMan is configured with default values, and FNAL SRM client srmrm needs some extra flags:

srmrm -2 �-webservice_path=srm/v2/server \
�� srm://bestman.lbl.gov:8443/srm/v2/server\?SFN=/srmcache/arie/my.test.file

Note that all FNAL SRM clients have default SRM service handle as srm/managerv2, and it needs to be specified in the option with �-webservice_path�. Also, some of FNAL SRM clients support both SRM v1.1 and SRM v2.2 with default pointing to v1.1, and it needs to be specified in the option with �-2�.

7.9 SRMMKDIR and SRMRMDIR from FNAL client

a. When BeStMan is configured with default values, and FNAL SRM client srmmkdir needs some extra flags:

srmmkdir� (or srmrmdir) �-webservice_path=srm/v2/server \
�� srm://bestman.lbl.gov:8443/srm/v2/server\?SFN=/srmcache/arie/my.test.dir

Note that all FNAL SRM clients have default SRM service handle as srm/managerv2, and it needs to be specified in the option with �-webservice_path�.

7.10 LCG-CP from GLITE LCG-UTILS client

a. lcg-cp only works with SRM servers running with host certficate. If BeStMan server runs with http or srm service certificate, lcg-cp would not work and give an error.

b. When BeStMan is configured with default values, and a file is requested to put into the user managed storage paths (e.g. /myproject/atlas/mydir). GLITE SRM client lcg-cp needs some extra flags:

lcg-cp�� -v�� -b�� -U srmv2�� --vo atlas �\
�� file:////tmp/my.test.file \
�� srm://bestman.lbl.gov:8443/srm/v2/server\?SFN=/myproject/atlas/mydir/my.test.file

Note that the user file is put into the non-BeStMan-managed storage space.

Note that lcg-cp only works with �host� type of GSS certificates.

c. When BeStMan is configured with userSpaceKeywords, and a file is requested to put into the user managed storage paths (e.g. /myproject/atlas/mydir). GLITE SRM client lcg-cp needs some extra flags:

In bestman.rc, userSpaceKeywords=(ATLASUSERSPACE1=/myproject/atlas)

lcg-cp�� -v�� -b�� -U srmv2�� --vo atlas �-S ATLASUSERSPACE1� \
�� file:////tmp/my.test.file \
�� srm://bestman.lbl.gov:8443/srm/v2/server\?SFN=/mydir/my.test.file

Note that the user file is put into the non-BeStMan-managed storage space, and the space has a space token named �ATLASUSERSPACE1�.� The pre-defined space token is provided with �-S� option.� Upon successful request, the user file resides in /myproject/atlas/mydir/my.test.file.

With srm-copy, the following command has the same effect, and the user file will be found in /project/atlas/mydir/my.test.file upon successful request:

srm-copy\
�� file:////tmp/my.test.file \
�� srm://bestman.lbl.gov:8443/srm/v2/server\?SFN=/mydir/my.test.file \
�� -spacetoken ATLASUSERSPACE1

7.11 LCG-LS from� GLITE LCG-UTILS client

a. lcg-ls only works with SRM servers running with host certficate. If BeStMan server runs with http or srm service certificate, lcg-ls would not work and give an error.

b. Examples:

lcg-ls� -b�� -D srmv2�� -l �\
�� srm://bestman.lbl.gov:8443/srm/v2/server\?SFN=/myproject/atlas/mydir/my.test.file

Note that lcg-ls only works with �host� type of GSS certificates.

8 Customizations and Plug-ins

BeStMan has ways for sites to customize the behavior and to plug in specialized mass storage systems.

8.1 Customized Transfer Protocol Selection Policy

a. User can define transfer selection plugin for BeStMan.

b. Configuration entries �protocolSelectionPolicy� and �pluginLib�
Each plugin entry must define the class name, jar file name, and the procotol name the policy applies to. Key values are "class", "jarFile" and "name", and they are separated by "&". Multiple plugin entries for different protocols can be defined, and must be seperated by ";".� When two or more plugin entries are defined for the same protocol, the last entry will be used.

For example, when two entries are defined for gsiftp and http respectively; protocolSelectionPolicy=class=plugin.NotRoundRobin1&jarFile=p1.jar&name=gsiftp;class=plugin.NotRoundRobin2&jarFile=p2.jar&name=http

If key "name=" is missing, and there is only one plugin entry, then the policy would be used on all protocols.

Another configuration entry "pluginLib" must be defined to provide the directory where the user defined jar files are located.� e.g. pluginLib=/opt/bestman/lib/plugin

c. Interface implementation
The interface that need to implement is:
package gov.lbl.srm.policy;
public interface ISRMSelectionPolicy {
�� public Object getNext();
�� public void setItems(Object[] objs);
}

BeStMan server uses getNext() to retrieve the next available "host:port" or "host" list. For example, "gsiftp1.mydomain.edu:2811" or "gsiftp1.mydomain.edu" when default port is used.
BeStMan server calls the setItems() to supply the list of available selections, if defined by the supportedProtocols entry in the conf file. supportedProtocols is optional, and when it is not defined, setItems() is called with the default value which is the host name where bestman server process is running on. Although setItems() sets the list of transfer server choices, getNext() may have more than what is provided in supportedProtocols, specially when a new transfer server is added to the list.

Sample implementation is following:
public class NotRoundRobin implements gov.lbl.srm.policy.ISRMSelectionPolicy {
�� int _count = -1;
�� Object[] _itemArray = null;
�� public Object getNext() {
�� Object result = null;
�� if (_itemArray != null) {
�� result = _itemArray[0];
�� }
�� return result;
�� }
�� public void setItems(Object[] col) {
�� _itemArray = col;
�� _count = 0;
�� }
}

The values of objects are like "host:port" e.g. "bestman.lbl.gov:2811".

8.2 Customized MSS Support

a. User can extend MSS support for BeStman

b. Follow the instruction below to write your custom plug-in. Pack it as a jar file.

c. Assuming that custom mss plug-in is stored in "/my/foo/dir/foo.jar" and your name space is "plugin.my.mss", modify conf/bestman.rc entries as following:

pluginLin=/my/foo/dir
CustodialQualityStorageMB=[0]path=/any/path&type=plugin.my.mss&jarFile=foo.jar&host=msshost.domain.tld&conf=mymss.rc

The keywords for CustodialQualityStorageMB is:
[0]:� indicates that no need for BeStMan to keep track of the usage of the plug-in device.
path: is the home path for each usage. This can be left as an empty string if not needed.
type: is the name space of customized mss plug-in
jarFile: is name of the mss plug-in jar file
host: is the host name that the device is on, e.g. archive.nersc.gov
conf: is the configuration file that the plug-in library needs, example: hpss.init.rc

d. Instructions for extending the MSS plug-in class with BeStMan

� Look at this java file to extend and implement: SRM_MSSIntf.java

� If queue support is needed, extend the super class SRM_MSS too. If queue support is not needed, then there is no need to extend SRM_MSS class.

� For a plain sample files, look into this file SRM_MSS_TEST.java. This is a plain implementation without extending SRM_MSS.

� For extending super class sample, look into this file SRM_MSS_TESTE.java. This is an implementation with extending SRM_MSS classs.

� To test the plug-in, see this example Test_MSS_TEST.java

9 Frequently Asked Questions

Click here for the frequently asked questions.

10 Troubleshooting

Check the frequently asked questions for more info.

1 GUMS connection from BeStMan is not working

a. Check 2.6 from the frequently asked questions.

11 Reference

1. SRM v2.2 specification, http://sdm.lbl.gov/srm-wg/doc/SRM.v2.2.html

2. BeStMan Download, http://datagrid.lbl.gov/bestman

3. BeStMan Guide, http://datagrid.lbl.gov/bestman/docs/bestman-guide.html

4. BeStMan Release Document in OSG, https://twiki.grid.iu.edu/bin/view/ReleaseDocumentation/Bestman

5. BeStMan Gateway with Xrootd at SLAC, http://wt2.slac.stanford.edu/xrootdfs/bestman-gateway.html

6. BeStMan Gateway for US ATLAS, https://www.usatlas.bnl.gov/twiki/bin/view/Admins/BestMan

7. BeStMan Status Document, https://twiki.grid.iu.edu/bin/view/Storage/BeStMan

8. BeStMan Gateway Installation for OSG-Tier2 and OSG-Tier3, https://twiki.grid.iu.edu/bin/view/Documentation/BestmanGateway

9. BeStMan Gateway + Xrootd Installation for OSG-Tier2 and OSG-Tier3, https://twiki.grid.iu.edu/bin/view/Documentation/BestmanGateway-Xrootd

10. How To Guide for Admins of UMD HEP T3 Cluster, http://hep-t3.physics.umd.edu/HowToForAdmins.html#osgInstall