SDM
People Publications Projects

Configuration and Notes

          # for 5 GB quota allocation per user
          ReplicaQualityStorageUserQuotaMB=5000
          # 1GB quota allocation per request
           DefaultMBPerToken=1000
          # for 40% of the storage that can be reserved by space reservation
          PublicSpaceProportion=60
        Cmnd_Alias     SRM_CMD = /bin/rm, /bin/mkdir, /bin/rmdir, /bin/mv, /bin/cp, /bin/ls
        Runas_Alias    SRM_USR = ALL, root
        daemon           ALL=(SRM_USR) NOPASSWD: SRM_CMD
         supportedProtocolList=gsiftp://host1.domain.tld;gsiftp://host2.domain.tld
         staticTokenList=token_name[desc:token_desc][token_size_in_GB]
          localPathListToBlock=/data/path1;/data/path2;/data/path3
          localPathListAllowed=/data/path1;/data/path2;/data/path3

Sample Configuration options

These below examples with the same configuration may not work for your environment. For each option, choose the value that fits your environment.

% configure \
--enable-gateway-mode 
% configure \
--enable-full-mode \
--with-replica-storage-path=/data/bestman/cache \
--with-replica-storage-size=20000 
% configure \
--with-globus-tcp-port-range=62001,62999 \
--with-cacert-path=/etc/grid-security/certificates \
--with-certfile-path=/opt/srm/demo/srmcert.pem \
--with-keyfile-path=/opt/srm/demo/srmkey.pem \
--with-eventlog-path=/data2/destman/log \
--with-cachelog-path=/data2/bestman/log
% configure \
--with-https-port=8443 \
--with-globus-tcp-port-range=48001,48999 \
--with-cacert-path=/etc/grid-security/certificates \
--with-certfile-path=/opt/srm/demo/srmcert.pem \
--with-keyfile-path=/opt/srm/demo/srmkey.pem \
--with-eventlog-path=/data2/destman/log \
--with-cachelog-path=/data2/bestman/log
% configure \
--with-globus-tcp-port-range=48001,48999 \
--with-https-port=48443 \
--with-eventlog-path=/data2/bestman/log \
--with-cachelog-path=/data2/bestman/log \
--with-certfile-path=/opt/srm/demo/srmcert.pem \
--with-keyfile-path=/opt/srm/demo/srmkey.pem \
--with-globus-location=/software/globus-4.2.1
% configure \
--with-globus-tcp-port-range=48001,48999 \
--with-https-port=48443 \
--with-eventlog-path=/data2/bestman/log \
--with-cachelog-path=/data2/bestman/log \
--with-certfile-path=/opt/srm/demo/srmcert.pem \
--with-keyfile-path=/opt/srm/demo/srmkey.pem \
--with-globus-location=/software/globus-4.2.1 \
--enable-gums \
--with-gums-url="https://gums-server.lbl.gov:8443/gums/services/GUMSAuthorizationServicePort"
% configure \
--enable-gateway-mode \
--with-globus-tcp-port-range=48001,48999 \
--with-https-port=48443 \
--with-eventlog-path=/data2/bestman/log \
--with-certfile-path=/etc/grid-security/hostcert.pem \
--with-keyfile-path=/etc/grid-security/hostkey.pem \
--with-globus-location=/software/globus-4.2.1 \
--enable-gums \
--with-gums-url="https://gums-server.lbl.gov:8443/gums/services/GUMSXACMLAuthorizationServicePort"
% configure \
--with-https-port=48443 \
--enable-sudofsmng \
--with-tokens-list="data[desc:mydata][10];data2[desc:mydata2][12]"
% configure \
--with-https-port=48443 \
--enable-sudofsmng \
--with-blocked-paths=î/projects/blocked;/projects2/blocked2î \
--with-tokens-list="data[desc:mydata][10];data2[desc:mydata2][12]"
% configure \
--with-https-port=48443 \
--enable-sudofsmng \
--with-aloowed-paths=î/projects/data;/projects2/data2î \
--with-tokens-list="data[desc:mydata][10];data2[desc:mydata2][12]"
% configure \
--enable-gateway-mode \
--with-globus-tcp-port-range=48001,48999 \
--with-https-port=48443 \
--with-eventlog-path=/data2/bestman/log \
--enable-sudofsmng \
--with-blocked-paths=î/projects/blocked;/projects2/blocked2î \
--with-tokens-list="DT1[desc:DT1][owner:exprt][retention:REPLICA][latency:ONLINE][path:/data][12]"
% configure \
--enable-gateway-mode \
--with-globus-tcp-port-range=48001,48999 \
--with-https-port=48443 \
--with-eventlog-path=/data2/bestman/log \
--with-eventlog-size=500000000 \
--with-eventlog-num=10 \
--enable-sudofsmng \
--with-blocked-paths=î/projects/blocked;/projects2/blocked2î \
--with-tokens-list="DT1[desc:DT1][owner:exprt][retention:REPLICA][latency:ONLINE][path:/data][12]"
% configure \
--enable-gateway-mode \
--with-connector-queue-size=512 \
--with-connection-acceptor-thread-size=8 \
--with-max-container-threads=2048 \
--with-max-java-heap=4096 \
--with-globus-tcp-port-range=48001,48999 \
--with-https-port=48443 \
--with-eventlog-path=/data2/bestman/log \
--with-eventlog-size=500000000 \
--with-eventlog-num=10 \
--enable-sudofsmng \
--enable-sudols \
--with-allowed-paths=î/projects/data/allowed;/projects2/mnt/data/allowed2î \
--with-tokens-list="DT1[desc:DT1][owner:exprt][retention:REPLICA][latency:ONLINE][path:/data][12]" \
--enable-gums \
--with-gums-url="https://gums-server.lbl.gov:8443/gums/services/GUMSXACMLAuthorizationServicePort"

Configure options

Required options for full management mode

--enable-full-mode Enable BeStMan in Full mode (default=no) --with-replica-storage-path=<PATH> Replica Quality Storage directory path --with-replica-storage-size=<INT> Replica Quality Storage size in MB

Other options for both full management mode and gateway mode

--with-srm-home=<PATH> Installation path for BeStMan2. If not given, it will be guessed based on the current working directory.
--enable-serveronly Installation for BeStMan server only (default=no). By default, all server, client and tester are installed.
--enable-clientonly Installation for SRM clients only (default=no). By default, all server, client and tester are installed.
--enable-testeronly Installation for SRM tester only (default=no). By default, all server, client and tester are installed.
--enable-verbose Print output to the standard output during the configuration
--enable-backup Enable backup before running a new configuration if there is a previous configuration (default=no)
--enable-checksum-listing Enable checksum returns in file browsing (default=no)
--enable-debug-jetty Enable debugging Jetty requests (default=no).
--enable-eventlog Enable event logging (default=yes). When disabled, there is no logging performed.
--enable-gsiftpfsmng Enable GridFTP access for local MKDIR, RMDIR, RM, MV, CP and LS to the user managed spaces (default=no)
--enable-gums Enable GUMS interface (default=no)
--enable-java-version-check Enable java version check (default=yes). It checks if java version is 1.6.0_01 or higher.
--enable-sudofsmng Enable SUDO access for local MKDIR, RMDIR, RM, MV and CP to the user managed spaces (default=no)
--enable-sudols Enable SUDO access for local LS to the user managed spaces (default=no)
--enable-voms-validation Enable VOMS validation (default=no)
--with-allowed-paths=<PATH> Specify accessible paths only (separated by semi-colon)
--with-backup-tag=<STRING> Specify the tag for backups during configure
--with-blocked-paths=<PATH> Specify Non-accessible paths (in addition to /;/etc;/var). Multiple entries are separated by semi-colon.
--with-cacert-path=<PATH> Specify the Grid CA Certificate directory path (default=/etc/grid-security/certificates)
--with-cached-id-lifetime=<INT> Specify the lifetime of cached id mapping in seconds (default=1800)
--with-certfile-path=<PATH> Specify the Grid Certificate file path (default=/etc/grid-security/hostcert.pem)
--with-checksum-callout=<PATH> Specify path for checksum call-out command
--with-checksum-type=<STRING> Specify the checksum type (default=adler32) from adler32, md5, crc32
--with-concurrent-fs=<INT> Specify the number of concurrent file system involved operations processing
--with-connector-queue-size=<INT> Specify the size of the Jetty http connector queue size
--with-connection-acceptor-thread-size=<INT> Specify the number of acceptor threads available for the Jetty server's channel connector
--with-eventlog-level=<STRING> Specify the event log level (default=INFO) from INFO and DEBUG
--with-eventlog-num=<INT> Specify the maximum total number of event log files
--with-eventlog-path=<PATH> Specify the event log file directory path (default=/var/log)
--with-eventlog-size=<INT> Specify the maximum size of event log files in bytes
--with-extra-libs=<PATH> Specify the extra libraries definitions
--with-globus-tcp-port-range=<VALUES> Specify the GLOBUS_TCP_PORT_RANGE when firewall is enabled. E.g. 62001,62999
--with-globus-tcp-source-range=<VALUES> Specify the GLOBUS_TCP_SOURCE_RANGE when necessary
--with-gridmap-path=<PATH> Specify the grid-mapfile path (default=/etc/grid-security/grid-mapfile)
--with-gums-certfile-path=<PATH> Specify the GUMS client Grid Certificate file path (default=same as –with-certfile-path)
--with-gums-dn=<DN> Specify the GUMS client service DN that GUMS server would recognize (default=SRM service DN)
--with-gums-keyfile-path=<PATH> Specify the GUMS client Grid Certificate Key file path (default=same as –with-keyfile-path)
--with-gums-proxyfile-path=<PATH> Specify the GUMS client Grid proxy file path
--with-gums-url=<URL> Specify GUMS server service URL with service handle
--with-https-port=<PORT> Specify the https port (default=8443)
--with-java-home=<PATH> Specify the JAVA_HOME directory
--with-keyfile-path=<PATH> Specify the Grid Certificate Key file path (default=/etc/grid-security/hostkey.pem)
--with-max-container-threads=<INT> Specify the max thread pool size for the web service container (default=256)
--with-max-java-heap=<INT> Specify the max java heap size in MB (default=1024)
--with-min-container-threads=<INT> Specify the min thread pool size for the web service container (default=10)
--with-min-java-heap=<INT> Specify the min java heap size in MB (default=32)
--with-plugin-path=<PATH> Specify the plug-in library directory path when supported
--with-protocol-selection-policy=<STRING> Specify the definition of transfer protocol selection policy
--with-proxyfile-path=<PATH> Specify the Grid proxy file path
--with-srm-owner=<LOGIN> Specify the BeStMan SRM server process owner (default=root)
--with-tokens-list= Specify pre-allocated static space tokens list with their sizes when supported.
Format: token_name[KEY:VALUE][size_in_GB]
KEY = desc, owner, retention, latency, path, usedBytesCommand
retention avail values = REPLICA, OUTPUT, CUSTODIAL
latency avail values = ONLINE, NEARLINE
usedBytesCommand = e.g. some custom script or "du -s -b".
Its output must have the available bytes as the first value.
--with-transfer-servers=<STRING> Specify supported file transfer servers
--with-user-space-key=<STRING> Specify user space keys format: (key1=/path1)(key2=/path2)
--with-vomsdir-path=<PATH> Specify the VOMS directory path

Other options for gateway mode only

--enable-checkfile-fs Enable use of file system to check file size (default=yes)
--enable-checkfile-gsiftp Enable use of GridFTP to check file size (default=no). This option may not work with LCG-utils because of delegation issues.
--enable-gateway-mode Enable BeStMan in gateway mode (default=yes). Gateway mode provides an SRM interface to any existing file system with faster request handling performance. There will be no management for space or queuing.
--enable-pathfortoken Enable PathForToken mode (default=yes)

Other options for full mode only

--with-cachelog-path=<PATH> Specify the CacheLogFile directory path (default=/var/log)
--with-concurrency=<INT> Specify the number of concurrent requests (default=40)
--with-concurrent-filetransfer=<INT> Specify the number of concurrent file transfers (default=10)
--with-custodial-storage-path=<PATH> Specify the CustodialQualityStorage directory path
--with-custodial-storage-size=<INT> Specify the CustodialQualityStorage Size in MB
--with-default-filesize=<INT> Specify the default file size in MB (default=500)
--with-default-space-size=<INT> Specify the default size for space reservation in MB (default=1000)
--with-globus-location=<PATH> Specify the GLOBUS_LOCATION path
--with-gridftp-buffersize=<INT> Specify the gridftp buffer size in bytes (default=1048576)
--with-gridftp-parallel-streams=<INT> Specify the number of gridftp parallel streams (default=2)
--with-inactive-transfer-timeout=<INT> Specify the default time out value for inactive user file transfer in seconds (default=300)
--with-max-filerequests=<INT> Specify the maximum number of active file requests (default=1000000)
--with-max-mss-connection=<INT> Specify the maximum MSS file transfers when supported (default=5)
--with-max-users=<INT> Specify the maximum number of active users (default=100)
--with-mss-timeout=<INT> Specify the MSS connection timeout in seconds when supported (default=600)
--with-output-storage-path=<PATH> Specify the OutputQualityStorage directory path
--with-output-storage-size=<INT> Specify the OutputQualityStorage Size in MB
--with-public-space-proportion=<INT> Specify default size for SRM owned volatile space in percentage (default=80)
--with-public-space-size=<INT> Specify the default size for SRM owned volatile space in MB
--with-space-file-lifetime=<INT> Specify the default lifetime of files in public space in seconds (default=1800)
--with-volatile-file-lifetime=<INT> Specify the default lifetime of volatile files in seconds (default=1800)

Configuration file

Upon successful configuration, bestman2/conf/bestman2.rc would be created. Each entry has the following meaning, and it's for both gateway mode and full mode unless noted otherwise.

Related to the server connection and logging

These entries have the default values when configured.

CacheLogLocation
  • Path for cache event log for full mode.
  • Default=/var/log/cache.bestman.log.
  • This can be either specific file path or directory path.
  • When useBerkeleyDB is true (by default), DB files are written in /var/log by default. If CacheLogLocation is defined when useBerkeleyDB is defined as true, CacheLogLocation must be a directory path.
^ e.g. CacheLogLocation=/tmp/bestman/cache.bestman.log
e.g. CacheLogLocation=/tmp/bestman
CertFileName
  • Grid service certifiticate file path
  • When the entry is missing, server will try to use the user grid proxy, and make a prompt for the proxy password every time.
  • Those cert/key files must be readable by the BeStMan process owner.
^ e.g. CertFileName=/etc/grid-security/hostcert.pem
EventLogLocation
  • Path for service event log.
  • This can be either specific file path or directory path.
  • Default=/var/log/event.bestman.log
^ e.g. EventLogLocation=/tmp/bestman/event.bestman.log
e.g. EventLogLocation=/tmp/bestman
FactoryID
  • FactoryID is for web service end point.
  • Recommended to be “server”.
  • The service end point will be srm://hostname.domain:secure_port/srm/v2/FactoryID
  • When this has different name than “server”, server-config.wsdd file needs to be updated too.
^ e.g. FactoryID=server
GridMapFileName
  • Provide GridMapFileName if it is not in the default location, /etc/grid-security/grid-mapfile
^ e.g. GridMapFileName=/etc/grid-security/grid-mapfile
KeyFileName
  • Grid service certifiticate Key file path
^ e.g. KeyFileName=/etc/grid-security/hostkey.pem
noCacheLog
  • When enabled, no cache log is written.
  • Default=false
  • For gateway mode, this option must be true.
^ e.g. noCacheLog=true
noEventLog
  • When enabled, no event log is written.
  • Default=false
^ e.g. noEventLog=true
protocol
  • Protocol for service endpoint.
  • This is fixed for httpg in SRM v2.2.
^ e.g. protocol=httpg
ProxyFileName
  • Grid proxy file path
  • If provided, proxy will take priority than cert/key files.
  • When user proxy is used, only the particular user may access the BeStMan server.
^ e.g. ProxyFileName=/tmp/proxyFile
securePort
  • Secure ports for SRM service endpoint.
  • These ports must be open for firewall
  • The service end point will be: srm://hostname.domain:securePort/srm/v2/FactoryID
^ e.g. securePort=8443
useBerkeleyDB
  • use of Berkeley DB for full mode as an internal management component.
  • Default=true.
  • When it’s false, text file based CacheLogLocation will be used.
^ e.g. useBerkeleyDB=true

Related to the server control

These entries have the default values when configured.

accessFileSysViaGsiftp
  • Allows BeStMan access file system through gsiftp on behalf of the user, upon user request
  • Default=false
  • When both accessFileSysViaGsiftp and accessFileSysViaSudo are defined, accessFileSysViaGsiftp takes priority.
  • This may not work with LCG-utils because of the delegation issues.
^ e.g. accessFileSysViaGsiftp=true
accessFileSysViaSudo
  • Allows BeStMan access file system through sudo on behalf of the user, upon user request
  • Default=false
  • This option is recommended when BeStMan is used to provide SRM interface to user defined storage space.
  • /etc/sudoers must be modified for BeStMan running under other than root
  • e.g. Recommended modification on the /etc/sudoers when BeStMan runs under daemon
    Cmnd_Alias SRM_CMD = /bin/rm, /bin/mkdir, /bin/rmdir, /bin/mv, /bin/ls, /bin/cp
    Runas_Alias SRM_USR = ALL, root
    daemon ALL=(SRM_USR) NOPASSWD: SRM_CMD
  • Note: Some OS systems such as Fedora Core and RHEL5 may require additional entry in the /etc/sudoers for tty access. Some Redhat-like and Ubuntu/Debian distribution do not require this entry.
    Defaults requiretty
  • When both accessFileSysViaGsiftp and accessFileSysViaSudo are defined, accessFileSysViaGsiftp takes priority.
^ e.g. accessFileSysViaSudo=true
Concurrency
  • Number of file requests that BeStMan server processes at a time. Beyond the limit, file requests will wait on the queue for any of the completed requests
^ e.g. Concurrency=20
DefaultFileSizeMB
  • Default file size in MB (default=1/10 of cache size)
^ e.g. DefaultFileSizeMB =1000
DefaultMBPerToken
  • Default space reservation size when user requests without specific size info, in MB
^ e.g. DefaultMBPerToken=1000
DefaultVolatileFileLifeTimeInSeconds
  • Default lifetime of volatile files in seconds
^ e.g. DefaultVolatileFileLifeTimeInSeconds=1800
disableDirectoryMgt
  • Disable directory management
  • Default=false
  • All directory related functions is not supported when it is set to true. E.g. srmMkdir, srmRmdir, srmLs
^ e.g. disableDirectoryMgt=true
disableLocalAuthorization
  • Disable SRM Permission functions
  • Default=true
  • All permission related functions is not supported when it is set to true. E.g. srmSetPermission
^ e.g. disableLocalAuthorization=false
disableSpaceMgt
  • Disable space management
  • Default=false
  • All space related functions are not supported when it is set to true. E.g. srmReserveSpace
  • Gateway mode must have it true
^ e.g. disableSpaceMgt=true
disableSrmCopy
  • Disable SRM remote copy function
  • Default=false
  • srmCopy function is not supported when it is set to true.
^ e.g. disableSrmCopy=true
GridFTPBufferSizeBytes
  • Buffer size of the gridftp file transfer in bytes
^ e.g. GridFTPBufferSizeBytes=2097152
GridFTPBufferSizeMB
  • Buffer size of the gridftp file transfer in MB.
  • GridFTPBufferSizeMB takes priority from GridFTPBufferSizeBytes when both are defined
^ e.g. GridFTPBufferSizeMB=2
GridFTPDcauOn
  • Enable DCAU for GridFTP (Default: true)
^ e.g. GridFTPDcauOn=true
GridFTPNumStreams
  • Number of parallel streams per gridftp file transfer
^ e.g. GridFTPNumStreams=2
guc_path
  • Sets the path to the globus-url-copy to be used in gsiftp file transfers, rather than gsiftp client lib calls when defined.
^ e.g. guc_path=/sandbox/globus/bin/globus-url-copy
GUMSCurrHostDN
  • GUMS client service dn
  • This DN is provided to GUMS server so that it can decide which mapping group the calling client (bestman) is in.
  • When not provided, server extracts DN information from the service certificate.
  • This DN takes priority than –with-gums-certfile-path and –with-gums-keyfile-path.
  • Default=/DC=org/…/CN hostname
^ e.g. GUMSCurrHostDN=/DC=org/DC=doegrids/OU=Services/CN=gums-client.lbl.gov
GUMSserviceURL
  • GUMS server service endpoint
^ e.g. GUMSserviceURL= https://gumsserver.lbl.gov:8443/gums/services/GUMSAuthorizationServicePort
GUMSserviceURL= https://gumsserver.lbl.gov:8443/gums/services/GUMSXACMLAuthorizationServicePort
InactiveTxfTimeOutInSeconds
  • Default time out value for inactive user file transfer in case user puts a file into the BeStMan cache, in seconds
^ e.g. InactiveTxfTimeOutInSeconds=900
localPathListAllowed
  • Allowed list of the local directory path for user access
  • Any definition will include the default block list
  • If a path is listed both on blocked and allowed list, blocked takes priority.
  • Multiple entries are separated by semi-colon
^ e.g. localPathListAllowed=/home/data;/data/public
localPathListToBlock
  • Blocked list of the local directory path for user access
  • Default=/;/etc/;/var
  • Any definition will include the default block list
  • Multiple entries are separated by semi-colon
^ e.g. localPathListToBlock=/home/secret;/data/secret2
markupPingMsg
  • When set to true, some of the extra information from srmPing do not get returned to the client.
  • Default=false
^ e.g. markupPingMsg=true
MaxConcurrentFileTransfer
  • Maximum concurrent file transfers
^ e.g. MaxConcurrentFileTransfer=10
MaxNumberOfFileRequests
  • Number of active and queued file requests limit for full mode. Beyond the limit, the request will get SRM_FAILURE with explanations
^ e .g. MaxNumberOfFileRequests =1000000
MaxNumberOfUsers
  • Number of active users limit for full mode. Beyond the limit, the request will get SRM_FAILURE with explanations
^ e .g. MaxNumberOfUsers=100
noSudoOnLs
  • When false, Allows BeStMan access file system for ls through sudo on behalf of the user, upon user request
  • Default=true
  • To have this option effective, accessFileSysViaSudo must be true.
  • This option is recommended when BeStMan is used to provide SRM interface to user defined storage space
  • /etc/sudoers must be modified for BeStMan running under other than root
  • Recommended modification on the /etc/sudoers when BeStMan runs under daemon
    Cmnd_Alias SRM_CMD = /bin/rm, /bin/mkdir, /bin/rmdir, /bin/mv, /bin/ls, /bin/cp
    Runas_Alias SRM_USR = ALL, root
    daemon ALL=(SRM_USR) NOPASSWD: SRM_CMD
  • Some OS systems such as Fedora Core and RHEL5 may require additional entry in the /etc/sudoers for tty access. Some Redhat-like and Ubuntu/Debian distribution do not require this entry.
    Defaults requiretty
^ e.g. noSudoOnLs=true
protocolSelectionPolicy
  • Custom policy for transfer protocol selection.
  • Default is round robin.
^ e .g. protocolSelectionPolicy=class=edu.unl.rcf.BestmanGridftpSelector.BestmanGridftp&jarFile=UNLGangliaBestman.jar&name=gsiftp
PublicSpaceInMB
  • Size of the BeStMan owned public volatile storage space in MB
  • When both PublicSpaceProportion and PublicSpaceInMB are defined, PublicSpaceInMB takes priority and is effective.
^ e.g. PublicSpaceInMB=1000
PublicSpaceProportion
  • Size of the BeStMan owned public volatile storage space in percentage
  • When both PublicSpaceProportion and PublicSpaceInMB are defined, PublicSpaceInMB takes priority and is effective.
^ e.g. PublicSpaceProportion=80
PublicTokenMaxFileLifetimeInSeconds
  • Max file lifetime that can be granted in the unreserved "public" storage space
^ e.g. PublicTokenMaxFileLifetimeInSeconds=600
PublicTokenMaxMBPerUser
  • Max file size in the unreserved "public" storage space per user
^ e.g. PublicTokenMaxMBPerUser=300
PublicTokenMaxNumFilesPerUser
  • Max number of files in the unreserved "public" storage space per user
^ e.g. PublicTokenMaxNumFilesPerUser =100
ReplicaQualityStorageUserQuotaMB
  • User quota for reserving replica quality storage, in MB
  • Default=no limit
^ e.g. ReplicaQualityStorageUserQuotaMB=1000
retryGsiftp
  • Retry options for BeStMan initiated file transfers.
  • Value is specified as (seconds/maxRetry)
  • Default is 120 seconds apart between each retry, and maximum 2 retries of failed gsiftps.
  • If maxRetry value ismissing, the default is assumed to be 2.
^ e.g. retryGsiftp=120/2
e.g. retryGsiftp=200
silent
  • When set to true, minimum output will be displayed on the console (default = false)
^ e.g. silent=true
srmcacheKeywordOn
  • If set to true, then “srmcache” is a required prefix to refer to the srm cache files.
  • For example, to refer to a file “myfile” owned by “uid” in srm, it needs to be look like srm://host:port/srm/v2/server?SFN=/srmcache/uid/myfile.
  • While without using /srmcache, the surl srm://host:port/srm/v2/server?SFN=/tmp/myfile will refer to the file /tmp/myfile on the local disk that runs the srm server.
  • Server default is false
^ e.g. srmcacheKeywordOn=true
supportedProtocolList
  • List of the supported file transfer protocol list.
  • Use “;” to separate multiple entries
^ e.g. supportedProtocolList= gsiftp://machA.domain/;gsiftp://machB.domain:2812/;ftp://machC.domain/;http://machD.domain:9123/
uploadQueueParameter
  • Sets a balance between read and write of the file transfers.
  • When not set, all files transfers use one queue.
  • FORMAT: N[:M] where N is number of threads for the queue and M is number of file transfers allowed for the queue
^ e.g. uploadQueueParameter=40:10
userSpaceKeywords
  • Allows pre-defined space tokens. BeStMan does not manage these spaces, but provides access to users through SRM interface.
  • Format is (spacetoken1=/dirpath1)(spacetoken2=/dirpath2)
  • When these re-defined space tokens are used in a request, the SFN should not include the full path.
^ e.g. refer to 7.10 userSpaceKeywords=(SPT1=/data/dirpath1)(SPT2=/data2/dirpath2)(SPT3=/data3/dirpath3)
WorldPermission
  • File readable permission in BeStMan cache.
  • By default, all files are accessible (read-only) by others.
  • It can be disabled by setting this to “None”, and no one other than the owners can read their files.
  • Other options are R, W, or None.
^ e.g. WorldPermission=None

Related to the gateway mode only

These entries would only be effective, when gateway mode is enabled.

pathForToken
  • Gateway mode supports a path defined as a space token.
  • When this option is defined as true, BeStMan server checks the available space in this path for the expected file size.
  • When this option is defined as true, the TURL becomes a combination of space token and SFN. E.g. when space token is /data/scratch1 and SFN is /mydir/myfile, the TURL becomes, when /data/scracth1 has enough space for the file, gsiftp://hostname//data/scrach1/mydir/myfile.
^ e.g. pathForToken=true
staticTokenList
  • Specifies pre-allocated space tokens with their size info
  • Format is token_name[KEY:VALUE][token_size_inGB]
    KEY = desc, owner, retention, latency, path, usedBytesCommand
    a keyword cannot be changed.
    retention available values = REPLICA, OUTPUT, CUSTODIAL
    latency available values = ONLINE, NEARLINE
    usedBytesCommand = e.g. some custom script or "du -s -b".
    Its output must have the available bytes as the first value
  • Multiple tokens are separated by semi-colon
^ e.g. staticTokenList=mytoken[desc:my_tokendesc][12];mytoken2[desc:my_tokendesc2][34]
e.g. staticTokenList=DATA1[desc:DATA1][owner:projects][retention:REPLICA][latency:ONLINE][path:/projects/data/][usedBytesCommand:/usr/bin/du -s -b][12]
checkSizeWithFS
  • Enables file size browsing through file system
  • Default=true
  • When both checkSizeWithFS and checkSizeWithGsiftp are defined to be true, checkSizeWithFS takes priority.
  • When both checkSizeWithFS and checkSizeWithGsiftp are defined to be false, file size browsing such as srmLs would fail.
^ e.g. checkSizeWithFS=true
checkSizeWithGsiftp
  • Enables file size browsing through gridftp server
  • Default=false
  • When both checkSizeWithFS and checkSizeWithGsiftp are defined to be true, checkSizeWithFS takes priority.
  • When both checkSizeWithFS and checkSizeWithGsiftp are defined to be false, file size browsing such as srmLs would fail.
  • This may not work with LCG-utils because of the delegation issues.
^ e.g. checkSizeWithGsiftp=false

Related to the Quality of the Storage

ReplicaQualityStorageMB
  • Replica Quality Storage Size and Path
  • For more than one path, use ";" to seperate them on one line.
  • Size information is specified in "[###]" before the path where "###" is the value of the size in MB.
  • Replica quality has the highest probability of loss such as disks, but is appropriate for data that can be replaced because other copies can be accessed from somewhere.
^ e.g. ReplicaQualityStorageMB=[5100]path=/bestman/cache
ReplicaQualityStorageMB=[300]path=/bestman/cache;[200]path=/bestman2/cache
OutputQualityStorageMB
  • Output Quality Storage Size and Path
  • For more than one path, use ";" to seperate them on one line.
  • Size information is specified in "[###]" before the path where "###" is the value of the size in MB.
  • Output quality is an intermediate level and refers to the data which can be replaced by lengthy or effort-full processes.
  • Note: We currently do not support OutputQualityStorage
^ e.g. OutputQualityStorageMB=[2000]path=/bestman/cached
CustodialQualityStorageMB
  • Custodial Quality Storage (e.g. disk spaces for permanent files)
  • For more than one path, use ";" to seperate them on one line.
  • Size information is specified in "[###]" before the path where "###" is the value of the size in MB.
  • Custodial quality provides low probability of loss such as tapes.
^ e.g. CustodialQualityStorageMB=[1000]path=/bestman/pcache
CustodialQualityStorageMB=[200]path=/bestman/cache/p;[200]path=/bestman2/cache
^
  • When Custodial Quality Storage supports mass storage system such as HPSS, Zero (0) size indicates indefinite.
e.g. For user specified MSS path access
CustodialQualityStorageMB=[0]path=&type=gov.lbl.srm.transfer.mss.hpss.SRM_MSS_HPSS&host=garchive.nersc.gov&conf=hpss.datagrid.rc
^ e.g. For bestman owned MSS path access: Only when specific path on MSS is used as custodial storage
CustodialQualityStorageMB=[0]path=/nersc/bestman/&type=gov.lbl.srm.transfer.mss.hpss.SRM_MSS_HPSS&host=garchive.nersc.gov&conf=hpss.datagrid.rc
^ e.g. For other customized MSS plugins
CustodialQualityStorageMB=[0]path=/lstore/bestman&type=plugin.lstore.SRM_MSS_LSTORE&jarFile=lstore.jar&host=lstore.domain.edu&conf=lstore.rc

Related to the MSS connection

When backend MSS is supported, these entries would affect the its connection to MSS.

MaxMSSConnections
  • maximum MSS transfers
  • Non MSS File Transfers = MaxConcurrentFileTransfer - MaxMSSConnections
^ e.g. MaxMSSConnections=5
mssTimeOutSeconds
  • MSS connection timeout in seconds
^ e.g. mssTimeOutSeconds=3600
pluginLib
  • If plugin is provided, then pluginLib needs to be defined for the directory to look for the plugin libraries.
  • This is usually for the customized BeStMan for localized MSS.
  • Libraries are expected to be jar files for dynamic loading.
^ e.g. pluginLib=/opt/bestman/plugin/lib