The Storage Resource Manager
Functional Interface Specification
Version 3.0�
Draft PR.1
�
8 September 2005
|
Collaboration Web: |
http://sdm.lbl.gov/srm-wg |
|
Document Location: |
http://sdm.lbl.gov/srm-wg/doc/SRM.v3.0.0509.pdf |
|
|
|
|
|
|
|
Editors: |
|
|
Arie Shoshani |
Lawrence Berkeley National Laboratory |
|
Alex Sim |
Lawrence Berkeley National Laboratory |
|
|
|
|
|
|
|
Contributors: |
|
|
Peter Kunszt |
EGEE Project (Enabling Grids for E-sciencE), CERN |
|
Don Petravick |
Fermi National Accelerator Laboratory (FNAL) |
|
Junmin Gu |
Lawrence Berkeley National Laboratory (LBNL) |
|
Jean-Philippe Baud |
LHC Computing Project (LCG), CERN |
|
Jens Jensen |
Rutherford Appleton Laboratory (RAL), England |
|
Bryan Hess |
Thomas Jefferson National Accelerator Facility (TJNAF) |
Copyright Notice
� Copyright Lawrence Berkeley National Laboratory (LBNL), Fermi National Accelerator Laboratory(FNAL), Jefferson National Accelerator Facility (JLAB), Rutherford Appleton Laboratory (RAL) and European Organization for Nuclear Research (CERN) 200, 2001, 2002, 2003, 2004, 2005. All Rights Reserved.
Permission to copy and display this �The Storage Resource Manager Functional Interface Specification� (�this paper"), in any medium without fee or royalty is hereby granted, provided that you include the following on ALL copies of this paper, or portions thereof that
you make:
1. A link or URL to this paper at this location.
2. This Copyright Notice as shown in this paper.
THIS WHITEPAPER IS PROVIDED "AS IS," AND Lawrence Berkeley National Laboratory, Fermi National Accelerator Laboratory, Jefferson National Accelerator Facility, Rutherford Appleton Laboratory and European Organization for Nuclear Research (COLLECTIVELY, THE �COMPANIES�) MAKE NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, WARRANTIES OF �MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, NON-INFRINGEMENT OR TITLE; THAT THE CONTENTS OF THIS PAPER ARE SUITABLE FOR ANY PURPOSE; NOR THAT THE
IMPLEMENTATION OF SUCH CONTENTS WILL NOT INFRINGE ANY THIRD PARTY PATENTS, COPYRIGHTS, TRADEMARKS OR OTHER RIGHTS.
THE COMPANIES WILL NOT BE LIABLE FOR ANY DIRECT, INDIRECT, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF OR RELATING TO ANY USE OR DISTRIBUTION OF THIS WHITEPAPER.
The names and trademarks of the Companies may NOT be used in any manner, including advertising or publicity pertaining to this paper or its contents, without specific, written prior permission. Title to copyright in this paper will at all times remain with the Companies.
No other rights are granted by implication, estoppel or otherwise.
PORTIONS OF THIS MATERIAL WERE PREPARED AS AN ACCOUNT OF WORK FUNDED BY U.S. Department of Energy AT UNIVERSITY OF CALIFORNIA'S LAWRENCE BERKELEY NATIONAL LABORATORY. NEITHER THE AUTHORS, NOR THE UNITED STATES GOVERNMENT OR ANY AGENCY THEREOF, NOR THE UNIVERSITY OF CALIFORNIA, NOR ANY OF THEIR EMPLOYEES OR OFFICERS, NOR ANY OTHER COPYRIGHT HOLDERS OR CONTRIBUTORS, MAKES ANY WARRANTY, EXPRESS OR IMPLIED, OR ASSUMES ANY LEGAL LIABILITY OR RESPONSIBILITY FOR THE ACCURACY, COMPLETENESS, OR USEFULNESS OF ANY INFORMATION, APPARATUS, PRODUCT, OR PROCESS DISCLOSED, OR REPRESENTS THAT ITS USE WOULD NOT INFRINGE PRIVATELY OWNED RIGHTS. REFERENCE HEREIN TO ANY SPECIFIC COMMERCIAL PRODUCT, PROCESS, OR SERVICE BY TRADE NAME, TRADEMARK, MANUFACTURER, OR OTHERWISE, DOES NOT NECESSARILY CONSTITUTE OR IMPLY ITS ENDORSEMENT, RECOMMENDATION, THE UNITED STATES GOVERNMENT OR ANY AGENCY THEREOF OR ANY OTHER COPYRIGHT HOLDERS OR CONTRIBUTORS. THE VIEW AND OPINIONS OF AUTHORS EXPRESSED HEREIN DO NOT NECESSARILY STATE OR REFLECT THOSE OF THE UNITED STATES GOVERNMENT OR ANY AGENCY THEREOF, OR THE ENTITY BY WHICH AN AUTHOR MAY BE EMPLOYED.
This manuscript has been supported �by the Office of Energy Research, Office of Computational and Technology Research, Division of Mathematical, Information, and Computational Sciences, of the U.S. Department of Energy under Contract No. DE-AC03-76SF00098.
The U.S. Government retains for itself, and others acting on its behalf, a paid-up, nonexclusive, irrevocable worldwide license in said article to reproduce, prepare derivative works, distribute copies to the public, and perform publicly and display publicly, by or on behalf of the Government.
Table of Contents
I.1. Extending parameters according to features
I.3. Releasing and removing files
I.4. Reserving and releasing spaces
I.7. The history of SRM versions
2.4. srmChangeFileStorageTypeStatus
3. Advanced feature set 1 : Remote Access Functions
3.2. srmStatusOfRemoteCopyRequest
4. Advanced feature set 2 : Space Management Functions
5. Advanced feature set 3 : Directory Management Functions
6. Advanced feature set 4 : Authorization Functions
7. Advanced feature set 5 : Request Administration Functions
8.1. Appendix A : StatusCode� Specification
This document contains the functional interface specification of SRM 3.0.� It is designed to support the functionality of previous SRM versions (specifically, v1.1 and v2.1.1) but is organized to support the functionality by �core features�, and �advanced features� functions.
Storage Resource Managers (SRMs) are middleware components whose function is to provide dynamic space allocation and file management of shared storage components on the Grid.� Introductory information about SRM concepts and the design of their functionality can be found in http://sdm.lbl.gov/srm-wg/papers/SRM.book.chapter.pdf.
�In this introduction we describe the organization of SRM v3.0 specification.� We start with description of how to represent �core features� and advanced features� in the specification.� Specifically, we address the issue of having functions that may be involved in multiple features.� This is followed file a section on space and file types (referred to as �volatile�, durable� and �permanent�), and the discovery of which features and types are supported by a certain SRM implementation.� Next, are sections that describe the semantics of releasing and removing files, as well as reserving and releasing spaces.� Finally, we describe the behavior of directory management when multiple spaces are managed for the client.
Following the introductory section, we included a section that describes the evolution of SRM versions and the relationship between functional specifications and operational specifications.� The detailed specification of SRM v3.0 follows.
I.1. Extending parameters according to
features
The functional design of SRM v3.0 calls for having �core� functions that all SRM implementations must support, and �advanced Features� functions that are optional.� Thus, it is inevitable that some of the core functions will have additional functionality if some advanced feature is supported by the SRM.� For example, a �srmPrepareToGet� as a core function does not have to specify a space_token, but if the �space reservation� feature is supported, the client may want to specify the space that the files will go into, and thus the parameter for the space_token must exist.
The advanced features supported in this version include: space management, directory management, authorization management, remote accessfunctions, and request administrative functions.
If an SRM supports an advanced feature, then all the functions in that feature must be supported.
The relationship between core functions and advanced feature functions is shown in the figure below.� The core functions in the center can be affected by certain features.� For example, the figure shows that Feature 6 effects function 1 of core, and Feature 1 effects functions 3 and 7 of core.� Note that two or more features can affect the same function, as is the case with function 7 of core.� In addition, it is also possible, although not very common, that a feature can affect the function of another feature.� This is shown with the broken line from Feature 2 that effects function 2 of Feature 3.
We represent this in this specification document is by using �behavior� sub-sections.� For the core functions there are two subsections: �core behavior�, and �behavior with advanced feature�. Under �behavior with advanced features� we have subsections for each of the features that affect particular functions.
I.2.a. File Types
The concepts of permanent and temporary file types are familiar concepts, but when dealing with shared storage on a Grid, temporary files require additional functionality.� Temporary files on shared Grid spaces cannot be removed arbitrarily.� Some minimal amount of time must be guaranteed by the SRM for the client to rely on.� This is the reason for a lifetime of a file.� This feature of a lifetime for a file is associated with each client accessing a file.� That is, a lifetime is associated with a file for a client when the file is made available to the client.� If another client requests the same file later, the file gets a fresh lifetime associated with that client (regardless of whether the SRM chooses to replicate the file to a separate space or not).� We refer to a file that is temporary in nature, but has a lifetime guarantee as a volatile file.� The concept of volatile files is very useful for dynamic space usage, automatic garbage collection, and sharing of files on a temporary basis.� In contrast, Permanent files have no lifetime associated with them, and can only be removed by the owner of the file
For grid applications, one needs another type of a file that has properties of both permanent and volatile files.� A durable file has the behavior of a volatile file in that it has a lifetime associated with it, but also the behavior of a permanent file in that when the lifetime expires the file is not automatically eligible for removal.� Instead, the SRM may advise the client or an administrator that the lifetime expired or take some other action.� For example, the SRM may move the file to a permanent space, release the durable space, and notify the client.� A durable file type is necessary in order to provide temporary space for files (such as files generated by large simulations), which need to be eventually archived.� However, the archiving process is usually too slow, thus slowing down and wasting the computational resources.� By storing durable files into available shared disk caches, the simulation can continue efficiently, yet it is guaranteed that the files will not be removed prematurely, and the files can be moved to the archive as a secondary task.� Similar to volatile files, durable files can be released by the client as soon as the files have been moved to a permanent location, or as soon as the client has no need for the files.
Permanent, durable, and volatile files are file types that provide the flexibility needed to manage files on the Grid for various use cases.� Permanent files are files that need to be stored for the long term, such as the results of a long running simulation.� Durable files are useful for dumping files as soon as possible to disk caches in order not to slow down computation resources.� Volatile files are useful in analysis use cases, where files have to be replicated temporarily but a lifetime duration is guaranteed.� Another advantage of volatile files is that they can be shared at the discretion of the SRM.
I.2.b. Space types
SRMs manage shared spaces.� If the �space reservation� feature is supported, then clients can negotiate and acquire space that the SRM assigns to them.� Otherwise, the SRM assigns a default space size that depends on its policy.� Similar to file types, the spaces assigned by the SRM have types.� Thus, Volatile and Durable space types have a lifetime associated with them.� Permanent space has unlimited lifetime.
If the �space reservation� feature is supported, a client can acquire multiple spaces of the same type.� For this reason, spaces are assigned a space_token.� Generally, when a request is made to bring a file into a space managed by the SRM, a space token is expected.�� If it is not provided, and the client has only one space of that type, the file is put into that space.
Files of a certain type can only be assigned to a space of the same type.� Furthermore, the lifetime of a file cannot exceed the lifetime of the space it is assigned to.
I.2.c. The support of file types by core and advanced features
An SRM can support any combination of the three space (and file) types: Volatile, Durable, and Permanent.� Given that an SRM supports a particular combination of space types, it may choose to support any subset of that combination for the core functions and� a different subset for any of its advanced features functions.� However, if certain space types are supported for some feature, then all of the functions of that feature must support that space type.
For example, an SRM that supports the space types of Volatile and Permanent, and the advance feature �space reservation�, may choose to support Volatile and Permanent for the core functions, and only Volatile for �space reservation�.� This flexibility is necessary in order to allow various SRMs to be developed for different types of storage systems.
The types of files supported for each feature must be discoverable when invoking the feature discovery function.� For the above example, the discovery function will return: {core [volatile, permanent], space_reservation [volatile]}.
I.3. Releasing and removing files
srmPrepareToGet and� srmPrepareToPut put or get local files only.� In order to get files from remote sites or put files into remote sites, the srmRemoteCopy function has to be used.� From the client�s point of view files brought in by the SRM as a result of srmPrepareToGet, srmPrepareToPut, and srmRemoteCopy requests are always brought into one or more local spaces that are assigned to the client.� The spaces can be assigned by default, as in the �core� feature, or acquired explicitely if the �space reservation� feature is supported.� Consequently, releasing and removing files refers to local spaces only.
We use the term �releasing a file� to mean that the file is marked as eligible for removal.� The �release a file� action takes place when a client no longer needs the file.� The �release� action takes place either by a direct request to release the file using the release command, or implicitly when abort or cleanup commands are issued.�� Released files are removed by the SRM only when space is needed or when the removal of the files is explicitly requested by the client.� We explain next the behavior of release, cleanup and abort functions.
srmReleaseFiles (Request_Token, SURLs) can only be used for files previously pinned as a result of srmPrepareToGet, srmPrepareToPut, or srmRemoteCopy.� The files designated by SURLs are marked as released, and are removed only when space is needed.� For example, if additional files need to be brought into the space the SRM will remove one or more of the released files to make space for the additional files.� To remove files, the srmRm function has to be used.
srmCleanupFilesFromSpace (Space_Token) can be used to release files regardless of the request that brought them in.� It has a remove-parameter (flag) that can be set to remove the files from the space, rather than only releasing them.
Aborting files is possible at the request-level � for aborting the entire request, and at the file-level � for aborting a specified subset of the files in the request.� Abort functions release files that are in spaces, and remove files from the queue of files that were not brought into the space yet.�� Both file-level and request-level aborts have a remove-parameter (flag) that can be set if the client wishes the files to be removed, not only released.�
Aborting an srmPrepareToGet and srmPrepareToPut has only a local effect.� However, aborting an srmRemoteCopy has to be propagated to the remote site.� In the case of aborting an srmPrepareToGet and srmPrepareToPut the files released are files that were brought into the local space (assigned to the client) as a result of that function.� If the remove-flag is set, the files that were brought into local space are removed.
Aborting an srmRemoteCopy request has two cases to consider.� When the srmRemoteCopy function is from the remote SRM to local SRM (also referred to as a �copy-pull�), and vice versa, if the srmRemoteCopy command is from local SRM to a remote SRM (also referred to as a �copy-push�).� For copy-pull, the local SRM issues an srmPrepareToGet request to the remote SRM, and for copy-push an srmPrepareToPut request is issued.�� When the srmRemoteCopy is aborted, it is propagated to the remote site by aborting the srmPrepareToGet or the srmPrepareToPut request as well.� Furthermore, if the abort function has the remove-flag set, then the propagated abort should have this flag set, too.� In the case of copy-push, the srmPrepareToPut gets aborted with the remove-flag set, which has the effect of removing the copied files from the remote SRM.� In the case of a copy-pull, the srmPrepareToGet to the remote site is aborted, but the remove-flag effects only the local site.
I.4. Reserving and releasing spaces
I.4.a. Reserving spaces
Given that an SRM support the Space Management feature for certain space types, it is possible to make space reservation requests for spaces of these types.� The space reservation request can specify a minimum �guaranteed� space, and a �total� space (which is larger than the �guaranteed�).� The difference between total and guaranteed is referred to as �best effort�.� The SRM can honor the request, or return lower values than requested.�� At this point the SRM assumes that the offer is accepted.� The client can refuse the space offer, by simply releasing that space (see below).
Different space reservation requests are needed for each space requested.� It is possible to make reservations to multiple spaces of each type, as long as the total does not exceed the SRM allocation.� The SRM allocation per client may be an internal feature, or may be dictated by the virtual organization (VO) that owns the space.� Currently, there are no mechanisms (interface) in place to communicate the VO�s allocation per client.
Space reservations to volatile and durable spaces are made for a lifetime.� The lifetime is subject to the SRM granting it.� Here again, the SRM may return a shorter lifetime for a space.� If the client wishes to refuse the modified lifetime, the space should be released, otherwise it is considered as granted.
There is no way of consolidating spaces.� However, one can request to increase or shrink an existing space as well as change the lifetime of a space with the srmUpdateSpace function.
As mentioned above all files assigned to a space must have the same file type as the space, and the lifetime of a file put in a space must be shorter than the lifetime of the space it is put into.
For core functionality, space reservation is performed by default.� That is, space is allocated to the client by the SRM according to its internal policies.� The policy for default space allocation and space types can be found in the discovery function.� The default space amount can be expressed as �guaranteed� and �total�.� As, above, the difference between total and guaranteed is referred to as �best effort�.�� The actual amount of space that was assigned can be found dynamically with the srmGetSpaceMetadata function.� The same default behavior applied in the case that the Space Management feature is supported, but no space request was made.
I.4.b. Releasing spaces
Releasing a space requires a space token and of course only the owner of the space can release it.�� Releasing a space, that has no files in it, is straightforward � it has the action that the space is no longer available.� It is not possible to request re-use of a space that was released.� Instead, a new request must be made.
If the release space has files in it, the space that is not occupied by files is released immediately.� However, the action regarding the files that are in the space depends on the type of space.�� We describe the action for each next.
� For volatile space, the files that were released prior to the space release request, will be removed from the space.� Files that were not released, will be left in the space till their lifetime expires or till they are released, and the space they occupy is released.� The space not occupied by files will be released.� Therefore, it is good to �cleanup� the space before releasing it.
� For Durable file, the files that were released prior to the space release request, will be removed from the space.� Files that were not released, will be left in the space till their lifetime expires, and then the files archived (and client is notified), and only then the space they occupy is released.�
� For permanent files, the files must be removed or released before the space they occupy is released.� Otherwise, the files stay in the space and the space they occupy is not released.
The directory management feature is intended to provide the client the capability to organize files managed by the SRM in directories and to assign names to the files that reside in the directories.� The SRM supports the usual functions: srmLs, srmMkdir, srmRmdir, srmRm, srmMv, and srmCp.� srmLs and srmRm are considered core functions.� No �link� functions are supported.
In the case that the SRM supports a single space for for each client, the behavior of the above functions is straightforward.� However, what is the behavior when there are multiple space types and/or multiple spaces per type?� There are two options: to support a directory for each space, or a single directory over all the spaces.� SRMs support the second option.� Thus, an SRM can have files of different types in a single directory, while these files may belong to different spaces.� This is illustrated in the diagram below in part (1) where (D), (V), and (P) represent file types.� The spaces that files belong to are not shown in the diagam, but are known to the SRM.� Note that this choice of a single directory over all the spaces, also gives the client the flexibility of how to organize the directories. For example, the diagram in part (2) below is a choice of having each sub-directory D2, D3, and D4, contain different file types.
The advantage of a single directory for all space and file types, is that it is possible to move files between spaces and even change their type without changing the directory structure.�� Thus, this permits the SRM to manage all the spaces virtually, regardless of where they are physically placed.� The issue of whether files are physically copied when they are moved between spaces or when their type changes, is an implementation choice.� The same behavior of files assigned to spaces according to space type is visible to the clients regardless of the implementation choice.
I.6. �Acknowledgements
While the initial ideas of SRMs were first introduced by people from the Lawrence Berkeley National Laboratory (LBNL), many of the ideas and concepts described in this paper were developed over time and suggested by various people involved in collaboration meetings and discussions.� In particular, we would like to acknowledge the contributions of people from Fermi National Accelerator Laboratory (FNAL), Thomas Jefferson National Accelerator Facility (Jlab), European Organization for Nuclear Research (CERN, including European Data Grid, LCG and EGEE), and Rutherford Appleton Laboratory (RAL).� Finally, the people who participated in meetings and discussions over the last few years also have contributed to the development and refinement of the concepts described here.� We apologize for any names that we might have overlooked in the list of contributors.
I.7. The history of SRM versions
This document contains the functional interface specification of SRM 3.0. It is designed to support the functionality of SRM 2.1.1 and SRM 1.1 (see http://sdm.lbl.gov/srm-wg/documents.html), but but is organized to support the functionality by �core features�, and �advanced features� functions.
This version is designed to be backward-compatible from the later versions to previous versions . Backward-compatibility is defined as �the ability of a previous version client to access a successive version server�.�
I.7.a. The relationship between functional and operational specifications
The functional specification document is to have an independent functional interface specification that operational specifications may be derived from this document. Figure 1 shows the relationship between specifications.
Figure 1 Relationship betweeen specifications
SRM�s �local� storage and �local� files mean that the storage spaces and files are under the SRM�s administrative control.
The file storage type must match the space that it resides.
Core function may support Space and File Storage Type of volatile and permanent. Advanced Space Function must support durable space and file storage type.
When an advanced feature function is not supported and an option for the advanced feature is provided to the server, a fault must be returned from the server
SiteURL(SURL) or StorageFileName(StFN) must be anyURI in the operational specification.
� The definition of the type �anyURI� used below is compliant with the XML standard. See http://www.w3.org/TR/xmlschema-2/#anyURI.�� It is defined as: "The lexical space of anyURI is finite-length character sequences which, when the algorithm defined in Section 5.4 of [XML Linking Language] is applied to them, result in strings which are legal URIs according to [RFC 2396], as amended by [RFC 2732]".
� The definition SURL is used for both site URL and the �Storage File Name� (stFN). This was done in order to simplify the notation.� Recall that stFN is the file path/name of the intended storage location when a file is put (or copied) into an SRM controlled space.� Thus, a stFN can be thought of a special case of an SURL, where the protocol is �srm� and the machine:port is local to the SRM.� For example, when the request srmRemoteCopy� is made, the source file is specified by a site URL, and the target location can be optionally specified as a stFN.� By considering the stFN a special case of an SURL, an srmRemoteCopy takes SURLs as both the source and target parameters.
Transfer URL (TURL) must be anyURI in the operational specification.
Request tokens and space tokens are strings, assigned by SRM, and are unique and immutable (non-reusable).� For example, if the date:time is part of the request reference it will be immutable.
User ID is in any form of user authentication information in the operational specification.
Authorization ID is in any form of user authorization information in the operational specification.
�
Authorization ID : from the SASL
RFC 2222
During the authentication protocol exchange, the mechanism performs
authentication, transmits an authorization identity (frequently known as a user
id) from the client to server�. The transmitted authorization identity may be
different than the identity in the client�s authentication credentials. This
permits agents such as proxy servers to authenticate using their own
credentials, yet request the access privileges of the identity for which they
are proxying. With any mechanism, transmitting an authorization identity of the
empty string directs the server to derive an authorization identity from the
client�s authentication credentials.
Storage system info is string.
o Storage system info may contain but is not limited to the following: storage device, storage login ID, storage login authorization.
o Storage system info may be a string that contains the login and password required by the storage system.� For example, it might have a form of login:passwd@hostname, where �:� is a reserved separator between login and passwd. If hostname is not provided, it is defaulted to what�s in the accompanying site URL or the host of SRM.
o Storage system info is added in the parameters of functions where storage access is needed. This is to simplify the case so that all files sent to the request share the same storage system info. If storage system info is different for each file, SRM needs a different request for those files.
Regarding file sharing by the SRM, it is an implementation choice.� An SRM can choose to share files by providing multiple clients access to the same physical file, or by copying a file into another� space.� Either way, if an SRM chooses to share a file (that is, to avoid reading a file over again from the source site) the SRM should check with the source site whether the client has a read/write permission. �Only if permission is granted, the file can be shared.
The keywords "MUST", "MUST
NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED",
"MAY", and "OPTIONAL" in this document are to be
interpreted as described in [RFC 2119].
This specification uses a notational convention, refered to as �Pseudo-schemas�. A Pseudo-schema uses a BNF-style convention to describe attributes and elements:
�
element� � a named variable that has a type: string, integer,
boolean, or enumerated.
������������������� e.g. �string SURL�
� item� � a single element, a tuple, a list, or a set
� {item, item, �}�� � tuple of items
� item()��� � list of items (ordered collection)
� item[] ���� set of items (unordered collection, array)�
� _���� � underline as required (as opposed to optional)�
� ::=� � definition operator
� {item | item | �} ����� choice operator
�
{item} � a way of enclosing a complex item for the list or set
notation.�
���������������� e.g. {MY_TYPE()}[]� is used to denote a set of MY_TYPE lists
� String � set of characters
� Enum � enumerated strings
� Int � number (to be defined in the operational specification, e.g. long or unsigned long)
� Boolean � true (1) or false (0)
Underlined attributes are REQUIRED. If a set is not required but some of attirubutes are underlined, then those underlined attributes are required when the set is provided: For example, �ReturnStatusForSURL { String� SURL, EnumStatusCode statusCode, string explanation } represents that ReturnStatusForSURL is not required, but when it is provided, SURL and statusCode are required to be provided.
notes:
� The following definitions are not data type definitions nor bound to any programming languages. When defined in operational specifications, the exact operational data type name shall be different.
� Here only repeating definitions are listed. Those definitions that appeared only once are listed under the associated function.
Namespace SRM
EnumFileStorageType���������� ::= VOLATILE |
�PERMANENT |
�DURABLE�������
EnumFileType����������������������� ::= FILE |
�DIRECTORY | ��
EnumSpaceType������������������� ::= VOLATILE |��
�DURABLE |���
�PERMANENT��
EnumOverwriteMode����������� ::= NEVER |
�ALWAYS |
�WHENFILESAREDIFFERENT
EnumPermissionMode��������� ::= NONE | X | W | WX | R | RX | RW | RWX
EnumPermissionType����������� ::= ADD | REMOVE | CHANGE
EnumRequestType��������������� ::= PREPARETOGET |
�PREPARETOPUT |
�REMOTECOPY ��������������������������
GMTtime������������������ ����������� ::= date and time in GMT with no local time extention
StatusCode� ::= �SRM_SUCCESS |
SRM_FAILURE |
������������� ���� SRM_AUTHENTICATION_FAILURE |
������������� ���� SRM_INVALID_REQUEST |
������������� ���� SRM_INVALID_SURL |
������������� ���� SRM_FILE_LIFETIME_EXPIRED |
������������� ���� SRM_EXCEED_ALLOCATION |
������������� ���� SRM_NO_USER_SPACE |
������������� ���� SRM_NO_FREE_SPACE |
������������� ���� SRM_DUPLICATION_ERROR |
������������� ���� SRM_TOO_MANY_RESULTS |
������������� ���� SRM_INTERNAL_ERROR |
������������� ���� SRM_FATAL_INTERNAL_ERROR |
������������� ���� SRM_NOT_SUPPORTED |
������������� ���� SRM_REQUEST_QUEUED |
������������� ���� SRM_REQUEST_INPROGRESS |
������������� ���� SRM_REQUEST_FINISHED |
������������� ���� SRM_REQUEST_SUSPENDED |
������������� ���� SRM_ABORTED |
������������� ���� SRM_RELEASED |
������������� ���� SRM_FILE_PINNED |
������������� ���� SRM_FILE_IN_SPACE |
������������� ���� SRM_FILE_IN_CACHE |
������������� ���� SRM_FILE_IN_USE |
SRM_CUSTOM_STATUS |
����������������� ����� SRM_AUTHORIZATION_FAILURE |�
SRM_SPACE_LIFETIME_EXPIRED |�
������������� ���� SRM_SPACE_AVAILABLE |������������
������������� ���� SRM_LOWER_SPACE_GRANTED |��
������������� ���� SRM_NON_EMPTY_DIRECTORY�����
o The SRM status codes are explained in Appendix A.
summary:
srmChangeFileStorageTypeStatus
details:
srmAbortRequest terminates the previously submitted request.
|
In |
Out |
|
String userID String authorizationID String storageSystemInfo String requestToken Boolean remove |
ReturnStatusForRequest { ������� EnumStatusCode statusCode, ������� string ������������������ ������������explanation } |
2.1.1. Core Behavior
a) Terminate all files in the request regardless of the file state. Remove files from the queue, and� release cached files if applicable. Expired files are released, too.
b) �remove� flag is false by default. When remove flag is true, all cached files for the request associated with the request token must be removed.
c) Abort must be allowed to all requests with requestToken, including requests such as srmLs or srmRemoteCopy.�
d) If any file transfers are active, SRM should attempt to stop the transfer. If not possible, abort takes place after the pending transfers are completed.
Behavior
With Advanced Features
2.1.2. �Remote Access Feature
a) srmAbortRequest with the remove flag on must be propagated for any calls to remote SRMs.
srmAbortRequestedFiles is to abort selected files from the request.
|
In |
Out |
|
String userID String authorizationID String storageSystemInfo String requestToken String SURL[] Boolean remove |
ReturnStatusForRequest { ������� EnumStatusCode statusCode, ������� string ������������������ ������������explanation } ReturnStatusForSURL { ������� String����������������������������� SURL, ������� EnumStatusCode statusCode, ������� string ������������������ ������������explanation } [] |
2.2.1. Behavior
a) Abort all previously requested files in the request regardless of the file status. Files must be removed from the queue, and cached files must be released, when applicable.
b) If SURLs are empty, a fault must be returned.
c) �remove� flag is false by default. If �remove� is true, then cached files must be removed.
Behavior
With Advanced Features
2.2.2. �Remote AccessFeature
a) srmAbortRequestedFiles with the �remove� flag on must be propagated for any calls to remote SRMs.
srmChangeFileStorageType changes file storage types.
|
Int offset Int count EnumFileStorageType� desiredFileStorageType String spaceToken |
������� EnumStatusCode statusCode, ������� string ������������������ ������������explanation ������� String������������������� SURL, ������� Int������������������������ newLifetime, ������� EnumStatusCode� statusCode, |
�����������
2.3.1. Core Behavior
a) SURL must be a reference to a local file only.
b) The file storage type indicates the type of space it goes into.
Behavior
With Advanced Features
2.3.2. �Directory Management Feature
a) SURL must be applied to both directories and files.
b) If SURL refers to a directory, then the effect is recursive for all the files under the directory.
2.3.3. �Space Management Feature
a) Space allocation and de-allocation may be involved.
b) Space management allows support for durable space.
c) The file storage type must match the space type associated with the spaceToken if spaceToken is provided.
2.4. srmChangeFileStorageTypeStatus
srmChangeFileStorageTypeStatus is for getting the response to the srmChangeFileStorageType.
|
In |
Out |
|
String userID String authorizationID String requestToken Int offset Int count |
ReturnStatusForRequest { ������� EnumStatusCode statusCode, ������� string ������������������ ������������explanation } ReturnStatusForSURLwLifetime{ ������� String������������������� SURL, ������� Int������������������������ newLifetime, ������� EnumStatusCode� statusCode, ������� string ������������������ ��explanation } [] |
�����������
2.4.1. Core Behavior
a) �Offset and count to may be used in case the list is too long to return.
srmExtendFileLifetime is for requesting extention of lifetime for volatile and durable files.
|
������� EnumStatusCode�� statusCode, ������� string ������������������ ���explanation ������� String������������������� SURL, ������� Int����� �������������������newLifetime, ������� EnumStatusCode� statusCode, |
2.5.1. Core Behavior
a) newLifetime is relative to the calling time.
b) newLifetime is the lifetime duration requested.
c) SRM may refuse the request, and failure error code is returned.
d) The number of lifetime extensions may be limited by SRM according to its policies.
e) If remaining lifetime is longer than the requested one, then the requested one will be assigned.
f) If newLifetime is not specified, the SRM may use its default to assign the newLifetime.
g) The life time of expired files and released files may be extended.
srmGetFeatures is a discovery function for clients to find out which features an SRM supports. This function returns the space type for each feature that SRM supports.
|
|
������� EnumStatusCode statusCode, ������� string ������������������ ������������explanation ReturnFeatureSet { ������� EnumSpaceType spaceType, ������� EnumSRMFeatures supportedFeature[] }[] |
EnumSRMFeatures�� :=�������� SRM_CORE |
SRM_REMOTE_ACCESS |
SRM_SPACE_MANAGEMENT |
SRM_DIRECTORY_MANAGEMENT |
SRM_AUTHORIZATION |
SRM_ADMINISTRATION |
SRM_ACCOUNTING
srmGetRequestSummary is to retrieve a summary of the submitted request.
|
������� EnumStatusCode��� statusCode, ������� string ������������������ ����explanation EnumRequestType����� requestType Int� numFilesFailed |
2.7.1. Core Behavior
a) numFilesFailed refers to failure to get the file such as file not found or file transfer failed.
b) numFilesInProgress refers to files that are in transfer, or in cache but not released yet.
srmGetRequestTokens retrieves request tokens for the client�s requests, given client provided request description. This is to accommodate lost request tokens. This can also be used for getting all request tokens.
|
������� EnumStatusCode statusCode, |
2.8.1. Core Behavior
a) If userRequestDescription is not provided, all requests that belong to the client are returned.
b) If the client assigned the same request description to multiple requests, multiple request tokens each with the time the request was made are returned.
�����
srmGetSRMStorageInfo retrieves SRM storage information, such as storage capacity, client quota, default lifetime, etc.
|
������� EnumStatusCode statusCode, |
|
|
������ EnumStorageAttributes storageAttr, ������ String value, ������ String valueType |
EnumStorageAttributes�������� :=�������� SRM_STORAGE_TYPE |
SRM_STORAGE_CAPACITY |
SRM_STORAGE_ACCESS_TYPE |
SRM_USER_STORAGE_MAX |
SRM_USER_STORAGE_MIN |
SRM_USER_STORAGE_DEFAULT_LIFETIME |
SRM_DEFAULT_FILE_LIFETIME |
SRM_DEFAULT_FILE_STORAGE_TYPE |
SRM_REQUEST_INFO_AVAILABILITY |
SRM_DEFAULT_RETURN_COUNT
2.9.1. Core Behavior
a) valueType in SupportedAttributes is designated for the returned value. For example, int, long, string, boolean, etc.
b) SRM_REQUEST_INFO_AVAILABILITY describes how long SRM will keep the status of the request after completion or termination.
c) SRM_DEFAULT_RETURN_COUNT shows the default number of entries returned starting from the first available entry in case the result is too large.
srmGetTransferProtocols returns the supported file transfer protocols of the SRM.
|
������� EnumStatusCode statusCode, |
|
srmLs returns a list of files in the space.
|
In |
Out |
|
String userID |
String requestToken |
|
String authorizationID String storageSystemInfo { String� SURL[]� | String spaceToken | EnumFileStorageType fileStorageType } Boolean fullDetailedList Boolean allLevelRecursive Int numOfLevels Int offset Int count |
ReturnStatusForRequest { ������� EnumStatusCode statusCode, ������� string ������������������ ������������explanation } SURLMetaData { String SURL, EnumStatusCode statusCode, String explanation, Int size,� GMTtime lastModificationTime, EnumFileType �fileType, String ownerID, EnumPermissionMode ownerPermission, String permittedGroupID, EnumPermittedMode groupPermission, EnumPermissionMode otherPermission, SURLMetaData[] subpath } [] Boolean partialList |
2.11.1. Behavior
a) Either SURLs, spaceToken or fileStorageType is needed.
b) SURL refers to files only.
c) If output parameter partialList is true, it indicates that only part of the result was returned. In this case, a requestToken is returned.
d) If the entire result is returned, then output parameter requestToken is optional.
e) fullDetailedList is false by default. In this case, SURL, size and fileType are returned in the status.
f) If fullDetailedList is true, additional items may be returned by the SRM.
g) When listing for a particular type specified by �fileStorageType�, all the files of that type must be returned.
Behavior
with other advanced features
2.11.2. Directory Management Feature
a) SURLs may refer to either directory or file.
b) Files are returned in width first order.
c) List of directories come before list of files in the return order.
d) If SURL refers to a directory, the returned value size must be 0.
e) If allLevelRecursive is "true", it dominates, i.e. ignore numOfLevels.�
f) If allLevelRecursive is "false" or missing, then numOfLevels must be honored.� If numOfLevels is "0" (zero) or missing, a single level is assumed.�
g) Directory names are returned even if they are empty.
2.11.3. Space Management Feature
a) If spaceToken is provided, all files that belong to the spaceToken are returned.
b) If space token is not provided and fileStorageType is provided, then all the files in each space of that type must be returned.
srmLsStatus is an asynchronous call for srmLs that is large.
|
In |
Out |
|
String userID String authorizationID String requestToken Int offset Int count |
ReturnStatusForRequest { ������� EnumStatusCode statusCode, ������� string ������������������ ������������explanation } SURLMetaData { String SURL, EnumStatusCode statusCode, String explanation, Int size,� GMTtime lastModificationTime, EnumFileType� fileType, String ownerID, EnumPermissionMode ownerPermission, String permittedGroupID, EnumPermittedMode groupPermission, EnumPermissionMode otherPermission, MetaDataDetails[] subpath } [] |
srmPrepareToGet gets files from the local space of SRM.
|
EnumOverwriteMode overwriteOption, GetFileRequest { ������ String������� fromSURL, ������ String������� toSURL, ������� int desiredLifetime, ������� EnumFileStorageType toFileStorageType, ������� String ������fromStorageSystemInfo, ������� String������ toStorageSystemInfo, ������� Int����������� knownSizeOfFile, ������� String������ spaceToken,���� ������� Boolean��� isSourceADirectory, ��� ����Boolean��� allLevelRecursive, ������� int numOfLevels Int desiredLifetime, EnumFileStorageType fromFileStorageType, EnumFileStorageType toFileStorageType, String toStorageSystemInfo, String�� spaceToken� |
������� EnumStatusCode��� statusCode, |
|
|
2.13.1. Core Behavior
a) This is an asynchronous (non-blocking) call. To get status and results, separate calls may be made with the request token.
b) fromSURL and toSURL must be local to SRM.
c) The userRequestDescription is a client designated name for the request.� It can be used in the getRequestToken function to get back the request tokens for requests made by the client.
d) SRM assigns the requestToken for the request.
e) storageSystemInfo, fileStorageType or desiredLifetime may be provided at the request level and the file level. In that case, SRM uses the one provided at the file level.
f) After the file is brought into the cache, it is pinned, and the lifetime clock starts at that time.
g) When the streaming option is true, and only part of the files fit into the space, the SRM wait until file are released and continue to bring files in. When the streaming option is false, and only part of the files fit into the space, SRM must return failure for the request.
h) If some file transfers fail, and all the other files transferred successfully, then �totalRetryTime� is the amount of time that SRM should try to transfer failed files.
i) In case that the retries fail, the return values in status call include an explanation of why the retries failed.
j) srmReleaseFile() is expected for files that are no longer needed. Otherwise, file lifetime expires. If all the files� lifetime in the space expires, then the request is terminated.
Behavior
with Advanced Features
2.13.2. Space Management Feature
a) spaceToken must be a valid input parameter.
b) If spaceToken is provided at the request level and also at the file level, SRM uses the one provided at the file level.
c) The default value of� �lifetime� for volatile or durable files must be equal to or less than the lifetime left in the space of the corresponding file type. The default value of fileType is �volatile�.
2.13.3. Directory Management Feature
a) numOfLevels must be a valid input parameter, and the default value is 1.
b) The default value for allLevelResursive is false.
srmPrepareToPut is a request to the SRM to prepare local space for files, so that the client may push files to the allocated space. It allocates a local space, and return TransferURL (TURL) for each file to the client.
|
EnumOverwriteMode overwriteOption ������ String������� toSURL, ������� int desiredLifetime, ������� EnumFileStorageType�� toFileStorageType, ������� String �����toStorageSystemInfo, ���� ���Int���������� knownSizeOfFile ������� String������ spaceToken Int desiredLifetime, EnumFileStorageType�� toFileStorageType String�� spaceToken� |
������� EnumStatusCode� statusCode, |
�����������
2.14.1. Core Behavior
a) SURLs must be limited to the local files.
b) Only push mode from the client to SRM is supported.
c) When the input parameter �toSURL� is not specified, the SRM names it automatically. The name is returned as part of the transferURL (TURL) in the srmStatusOfPutRequest call.
d) srmPutFileDone is expected from the client after each file is �put� into the allocated space.� The srmPutRequestDone tells the SRM that the srmPrepareToPut is completed.
e) When the �streaming� flag is true, and the allocated space is full, the SRM waits till files in the space are released, and then continues. When the �streaming� flag is false, and if there is not enough space to accommodate the entire request, the SRM returns a failure code.
f) The lifetime of the file starts as soon as the SRM receives srmPutFileDone or srmPutRequestDone.� If srmPutFileDone or srmPutRequestDone is not provided, then the files in the space are subject to removal when the space lifetime expires.
g) storageSystemInfo, fileStorageType or desiredLifetime may be provided at the request level and the file level. In that case, SRM uses the one provided at the file level.
Behavior
with Advanced Features
2.14.2. Space Management Feature
a) If spaceToken is provided at the request level and the file level, the SRM uses the one provided at the file level.
b) The default value of� �lifetime� for volatile or durable files must be equal to or less than the lifetime left in the space of the corresponding file type. The default value of fileType is �volatile�.
srmPutFileDone is used to notify the SRM that the client completed a file transfer to the TransferURL in the allocated space. This call should normally follow srmPrepareToPut.
|
|
������� EnumStatusCode statusCode, ������� string ������������������ ������������explanation ������� String����������������������������� SURL, ������� EnumStatusCode statusCode, |
2.15.1. Core Behavior
a) If SURL is empty, fault is returned.
srmPutRequestDone is used to notify the SRM that the client completed all file transfers in the request. All file transfers associated with the requestToken are considered to be completed. If srmPutFileDone is was issued for all files, this call is not needed.�
|
In |
Out |
|
String userID String authorizationID String storageSystemInfo String requestToken |
ReturnStatusForRequest { ������� EnumStatusCode statusCode, ������� string ������������������ ������������explanation } ReturnStatusForSURL { ������� String����������������������������� SURL, ������� EnumStatusCode statusCode, ������� string ������������������ ������������explanation } [] |
srmReleaseFiles releases the files in a space. �The file is not removed, but the space occupied by the released file is eligible for removal if space is needed.
|
Boolean releaseAllCurrentlyPinnedFiles |
������� EnumStatusCode statusCode, ������� string ������������������ ������������explanation ������ �String����������������������������� SURL, ������� EnumStatusCode statusCode, |
2.17.1. Core Behavior
a) The default value of releaseAllCurrentlyPinnedFiles is false.
b) If releaseAllCurrentlyPinnedFiles is true, none of SURL must be provided, or all of SURLs must be provided.
c) If none of SURLs are provided and releaseAllCurrentlyPinnedFiles is true, then all currently pinned files in the client�s request are released.
d) Once a file is released, its lifetime cannot be extended with the srmExtendLifetime function. If the released file is needed again, it should be requested again.�
Behavior
with Advanced Features
2.17.2. Directory Management Feature
The SURL may be a directory. In this case, all files under the directory that belong to the request (i.e. same request token) are released.
srmRm removes local files regardless of the requests that got the file into the space.
|
In |
Out |
|
String userID String authorizationID String storageSystemInfo String� SURL[] Boolean recursive Int offset Int count |
String requestToken ReturnStatusForRequest { ������� EnumStatusCode statusCode, ������� string ������������������ ������������explanation } ReturnStatusForSURL { ������� String����������������������������� SURL, ������� EnumStatusCode statusCode, ������ �string ������������������ ������������explanation } [] |
2.18.1. Behavior
a) Consistent with unix.
b) If file removal can be handled quickly, the requestToken may not be returned.
�
Behavior
with other advanced features
2.18.2. Directory Management Feature
a) To remove empty directories, srmRmdir() must to be used.
b) SURLs can be either directories or files. However, if a directory is provided, all files under the directory will be removed, but not the directory.
c) The default value for the recursive option is false.
d) If the recursive option is true, all the files in the directory and subdirectories will be removed, but not the directories themselves.� Recursive removal of empty directories is supported with� srmRmdir().
2.18.3. Authorization Feature
a) Authorization checks are performed on SURLs before files can be removed.
srmRmStatus is a status call for srmRm in case that the srmRm was not performed synchronously.
|
In |
Out |
|
String userID String authorizationID String� requestToken Int offset Int count |
ReturnStatusForRequest { ������� EnumStatusCode statusCode, ������� string ������������������ ������������explanation } ReturnStatusForSURL { ������� String����������������������������� SURL, ������� EnumStatusCode statusCode, ������� string ������������������ ������������explanation } [] |
srmStatusOfGetRequest is the status call for srmPrepareToGet.
|
String authorizationID String requestToken ����� String toSURL } [] Int offset Int count |
������� EnumStatusCode statusCode, ������� string ������������������ ������������explanation String ������ fromSURL, String������ localReferenceSURL, String������� localTransferURL, EnumStatusCode�� statusCode, String������ explanation, int������������ fileSize, int������������ estimatedWaitTimeOnQueue, int������������ estimatedProcessingTime, int������������ remainingLifetime, EnumFileStorageType toFileStorageType, String������ spaceToken� |
2.20.1. Core Behavior
a) If RequestedSURL is not provided, the status for all the files associated with the request token is returned.
b) The output parameter localReferenceSURL is the same as the input parameter �toSURL� if the client provided it with srmPrepareToGet().� Otherwise, it is the local reference SURL that SRM assigned.
c) Output parameter fromSURL is the same as the fromSURL that client provided with srmPrepareToGet().
Behavior
with Advanced Features
2.20.2. Space Management Feature
a) spaceToken may be returned for each file in the request, since a single request can have files in multiple spaces.
srmStatusPutRequest is the status call for srmPrepareToPut.
|
Int offset Int count |
������� EnumStatusCode statusCode, ������� string ������������������ ������������explanation String ������ localReferenceSURL,�� String������ localTransferURL, int������������ fileSize, EnumStatusCode�� statusCode, String������ explanation, int������������ estimatedWaitTimeOnQueue, int������������ estimatedProcessingTime, int������������ remainingLifetime, EnumFileStorageType toFileStorageType, String�� ����spaceToken� |
2.21.1. Core Behavior
a) If input parameter �toSURL� is empty, it returns status for all files associated with the request token.
b) The output parameter �localReferenceSURL� must be the input parameter �toSURL� of srmPrepareToPut if the client provided it.� Otherwise, it is SURL that SRM assigned for the reference.
c) When the space to put a file is allocated by SRM, localReferenceSURL, localTransferURL, toFileStorageType and fileSize must be returned.
d) Returned file size is initially the default allocated space size, unless a specific size was requested for the file. After the srmPutFileDone() or srmPutRequestDone() is issued, the file size is set to the actual file size that occupies the location.
e) While the srmRequestToPut is processed, the output parameter �remainingLifetime� is null until the srmPutFileDone (or srmPutRequestDone) is issued by the client.� After an srmPutFileDone is issued the remainingLifetime indicates the remaining lifetime of the file.
Behavior
with Advanced Features
2.21.2. Space Management Feature
a) spacetToken must be a valid output parameter.
3. Advanced feature set 1 : Remote Access Functions
summary:
details:
srmRemoteCopy replicates files from one site to another.
|
EnumOverwriteMode overwriteOption ������� String������ fromSURL, ������� String������ toSURL, ������� Int���������� knownSize, ������� int fileLifetime, ������� EnumFileStorageType�� toFileStorageType, ������� String fromStorageSystemInfo, ������� String toStorageSystemInfo, ������� String������ spaceToken,���� ������� Boolean��� isSourceADirectory, ������� Boolean��� allLevelRecursive, ������� int numOfLevels String spaceToken EnumFileStorageType toFileStorageType |
������� EnumStatusCode���� statusCode, |
3.1.1. Behavior
a) Third party copy are not supported, from a remote location to another remote location. Either source or target must be local to the SRM where the request is submitted.
b) srmRemoteCopy can be done in a pull ot a push mode.� Pull mode is to copy from remote location to local SRM. Push mode is to copy from local SRM to remote location.� The mode is determined by the source and target SURLs.
c) Always release files from source after copy is done.
d) When removeSourceFiles is true, SRM removes the source files on behalf of the client after the copy is done.
e) The client may release the file local to the SRM after the copy is completed in push mode. If the client releases a file being copied to another location before the transfer is completed, then the release fails.
f) When the streaming option is true, and the target space is full, the copy operation is suspended till more space is made available. When the streaming option is false, and if there is not enough space to accommodate the entire request, the SRM returns failure.
g) There is no protocol negotiation with the client.
h) �totalRetryTime� represents the length of time that the SRM will try to copy file whose transfer previously failed. This action takes place after all the other file transfers for the request completed.
i) In case that retries fail, the return should include an explanation of why the retries failed.
j) srmRemoteCopy performs a copy from or to remote sites only.� Thus, when both fromSURL and toSURL are local, an error is returned.� A copy of a local file to another can be done by the srmCp function if the �directory feature� is supported.�
Behavior
with other advanced features
3.1.2. Space Management Feature
a) The default value of� �lifetime� for volatile or durable files must be equal to or less than the lifetime left in the space of the corresponding file type. The default value of fileType is �volatile�.
3.1.3. Directory Management Feature
a) Empty directories must be copied as well.
3.2. srmStatusOfRemoteCopyRequest
srmStatusOfRemoteCopyRequest is the status call for srmRemoteCopy.
|
|
������� EnumStatusCode statusCode, |
|
RequestedSURLs { ������ String fromSURL, ������ String toSURL } [] Int offset Int count |
RequestStatusForCopy { String fromSURL, String toSURL, ������ EnumFileStorageType� toFileStorageType, Int fileSize, EnumFileType fileType, ������ Int fileLifetime, EnumStatusCode statusCode, String explanation, Int estimatedWaitTimeOnQueue, Int estimatedProcessingTime, ������ String spaceToken } [] |
3.2.1. Behavior
a) If RequestedSURLs are not provided, the status for all the files associated with the request token is returned.
b) If input parameter �toSURL� is provided, the output parameter �toSURL� is the same as the input parameter �toSURL�. If input parameter �toSURL� is not provided, the SRM assigned one.
4. Advanced feature set 2 : Space Management Functions
summary:
���������
details:
srmCleanupFilesFromSpace releases all files from the space associated with the space token. The spaces of released files can be used when space is needed.� The space is not released.
|
In |
Out |
|
String userID String authorizationID String storageSystemInfo String� spaceToken Boolean remove |
ReturnStatusForRequest { ���� ���EnumStatusCode statusCode, ������� string ������������������ ������������explanation } ReturnStatusForSURL { ������� String����������������������������� SURL, ������� EnumStatusCode statusCode, ������� string ������������������ ������������explanation } [] |
4.1.1. Behavior
a) The default value of the remove parameter is false.� If remove flag is true, then all files in the spaceToken or SURLs are removed.
b) If any of the files in the space associated with the space token cannot be removed, then ReturnStatusForSURL must be returned for the status and the explanation.
srmCompactSpace is to reclaim the space for all released files in spaces.
|
ReturnStatusForRequest { ������� EnumStatusCode statusCode, |
4.2.1. Behavior
a) During the compacting process, files not released must not be removed (even if lifetime expired).
b) The default value for doDynamicCompactFromNowOn is false, which implies that compactSpace will take place only once at the time it is issued.
c) If the doDynamicCompactFromNowOn is true, then the space of released files will be automatically compacted.� The value of doDynamicCompactFromNowOn can be set to false again with another srmCompactSpace call.
d) When space is compacted, the files in that space do not have to be removed by the SRM, only the corresponding space released.� For example, the SRM may choose to move files from durable to volatile space.� The client will only perceive that the compacted space is now available to them.
e) In order to force a physical removal of a file, the client must use srmRm.
srmGetSpaceMetaData is used to retrieve meta information of a space.
|
������� EnumStatusCode statusCode, |
|
|
EnumStatusCode statusCode, String explanation, Boolean expired, |
4.3.1. Behavior
a) SpaceMetaData may refer to a single space of each type (i.e. volatile, durable, or permanent).
b) The returned size does not include the extra space needed to hold the directory structures.
Behavior
with other advanced features
4.3.2. Space Management Feature
a) If multiple spaces per space type exist, the returned metadata is for each space using the space token.
srmGetSpaceToken returns space tokens for currently allocated spaces.
|
������� EnumStatusCode statusCode, |
4.4.1. Behavior
a) If userSpaceTokenDescription is null, the SRM returns all space tokens that the client owns.
b) If the client assigned the same name to multiple space reservations, the client will get back multiple space tokens.
srmReleaseSpace releases an occupied space.
|
ReturnStatusForRequest { ������� EnumStatusCode statusCode, |
4.5.1. Behavior
a) The parameter forceFileRelease is false by default.� This means that the space will not be released if it has files that are still pinned in the space.� To release the space regardless of the files it contains and their status, forceFileRelease must be specified as true.
b) All files must be released in the specified space before the space can be released, unless the forceFileRelease is set.
c) A request to release a reserved space that has on-going file transfers will be postponed until after the transfers complete (if the transfers cannot be interrupted).� The files will then be released, and the space released.
d) When space is releasable and forceFileRelease is true, all the files in the space are released, even in durable or permanent space.
e) srmReleaseSpace may not complete right away because of the lifetime of files in the space.� When space is released, the files in that space are treated according to their types: If file storage types are permanent, keep them until further operation such as srmRm is issued by the client. If file storage types are durable, perform necessary actions at the end of their lifetime. If file storage types are volatile, release those files at the end of their lifetime.
srmReserveSpace facilitates negotiation of space reservation.
|
Int ExpectedFileSize[] String userSpaceTokenDescription |
������� EnumStatusCode statusCode, ������� string ������������������ ������������explanation EnumSpaceType typeOfReservedSpace |
4.6.1. Behavior
a) lifetimeOfSpaceToReserve is not needed when requesting permanent space. It is� ignored for permanent space.
b) An SRM may provide default size and lifetime if not supplied.
c) storageSystemInfo is optional and used for the case that the storage system requires an additional security check.
d) If sizeOfTotalSpaceDesired is not specified, the SRM must return its default quota.
e) The difference between sizeOfTotalReservedSpace and sizeOfGuaranteedReservedSpac is on best effort basis.
srmUpdateSpace is to resize the space and/or extend the lifetime of a space.
|
Int newSizeOfTotalSpaceDesired Int newSizeOfGuaranteedSpaceDesired Int newLifetime |
������� EnumStatusCode statusCode, ������� string ������������������ ������������explanation String spaceToken |
4.7.1. Behavior
a) Function call must include size and/or lifetime.
b) If neither size nor lifetime is supplied in the input, then return will be an error.
c) newSize is the new actual size of the space, and it must be positive.
d) newLifetime is the new lifetime requested regardless of the previous lifetime, and it must be positive.� It may even be shorter than the remaining lifetime at the time of the call.
5. Advanced feature set 3 : Directory Management Functions
summary:
details:
5.1. �srmCp
srmCp is to copy SRM�s local file to another space in the same SRM.
|
In |
Out |
|
String userID |
String requestToken |
|
String authorizationID String fromStorageSystemInfo String toStorageSystemInfo String userRequestDescription Int����� totalRetryTime EnumOverwriteMode overwriteOption CopyFileRequest { ������� String������ fromSURL, ������� String������ toSURL, ������� Int���������� knownSize ������� int fileLifetime, ������� EnumFileStorageType toFileStorageType, ������� String fromStorageSystemInfo, ������� String toStorageSystemInfo, ������� String������ spaceToken,���� ������� Boolean��� isSourceADirectory, ������� Boolean��� allLevelRecursive, ������� int numOfLevels } String spaceToken EnumFileStorageType�� toFileStorageType |
ReturnStatusForRequest { ������� EnumStatusCode���� statusCode, ��� ����string ������������������ �����explanation }
|
5.1.1. Behavior
a) Output parameter requestToken is optional, when the copying files are small enough to be handled at once.
b) srmCp does not support remote copy, but only local copies; from a local location to another local location.
c) Same amount of space as the source files is required to copy to the target.
d) There is no protocol negotiation with the client for the request.
e) �totalRetryTime� means that if all the file transfers for the request are complete, then try previously failed transfers for a time period of �totalRetryTime�.
f) In case that the retries fail, the return should include an explanation of why the retries failed.
g) Empty directories are copied as well.
h) The default value of� �lifetime� for volatile or durable files must be equal to or less than the lifetime left in the space of the corresponding file type. The default value of fileType is �volatile�.
Behavior
with other advanced features
5.1.2. Space Management Feature
a) spaceToken must be a valid input parameter.
b) spaceToken or toFileStorageType or both must be provided. When spaceToken is not provided and toFileStorageType is provided, space allocation is needed in the toFileStorageType.
srmCpStatus is a status call for srmCp.
|
In |
Out |
|
String userID String authorizationID String requestToken Int offset Int count
|
ReturnStatusForRequest { ������� EnumStatusCode statusCode, ������� string ������������������ ������������explanation } RequestStatusForCopy { String fromSURL, String toSURL, ������ EnumFileStorageType� toFileStorageType, Int fileSize, EnumFileType fileType, ������ Int fileLifetime, EnumStatusCode statusCode, String explanation, Int estimatedWaitTimeOnQueue, Int estimatedProcessingTime, ������ String spaceToken } |
Behavior
with other advanced features
5.2.1. Space Management Feature
a) spaceToken must be valid.
srmMkdir create a directory in a local space.
|
String userID |
String requestToken |
|
������� EnumStatusCode statusCode, |
5.3.1. Behavior
a) The output parameter requestToken is optional, when making a directory is fast enough to be returned at once (synchroneously).
b) Consistent with unix
c) Recursive creation of directories is not supported.
srmMkdirStatus is the status call for srmMkdir.
|
In |
Out |
|
String userID String authorizationID String� requestToken |
ReturnStatusForRequest { ������� EnumStatusCode statusCode, ������� string ������������������ ������������explanation } |
srmMv is to move a file from one local directory to another.
|
ReturnStatusForRequest { ������� EnumStatusCode statusCode, |
�
5.5.1. Behavior
a) The output parameter requestToken is optional, when moving the file is fast enough to be returned at once.
b) SURL may be applied to both directory and file.
Behavior
with other advanced features
5.5.2. Authorization Feature
a) Authorization checks need to be performed on both fromSURL and toSURL.
srmMvStatus is the status call for srmMv.
|
In |
Out |
|
String userID String authorizationID String requestToken |
ReturnStatusForRequest { ������� EnumStatusCode statusCode, ������� string ������������������ ������������explanation } |
�
srmRmdir removes a local empty directory.
|
Boolean recursiveRemoval |
������� EnumStatusCode statusCode, |
5.7.1. Behavior
a) SURL must be an empty directory only.
b) The default value of recursiveRemoval is false.
c) If the value of recursiveRemoval is true, the entire directory with its subdirectories will be removed, provided that they are all empty.� The srmRm function can be used recursively to remove all files in a directory recursively.
6. Advanced feature set 4 : Authorization Functions
summary:
details:
���������������������
srmCheckPermission is used to check the client permissions on the SURLs.
|
In |
Out |
|
String userID String authorizationID String storageSystemInfo String� SURL[] Boolean localCheckOnly |
ReturnStatusForRequest { ������� EnumStatusCode statusCode, ������� string ������������������ ������������explanation } CheckedPermission { String SURL, EnumStatusCode statusCode, String explanation, EnumPermissionType userPermission } [] |
6.1.1. Behavior
a) The default value of localCheckOnly is true, and SRM only checks files in its local space. Otherwise, if a file is not in its local space, then SRM goes to the SURL location to check the client permission.
b) If localCheckOnly is false, SRM may choose to always check the SURL for client permission of each file. It may be okay if SRM choose to check its local cache first.
srmReassigneToUser is used to reassign SURLs to anther client.
|
ReturnStatusForRequest { ������� EnumStatusCode statusCode, |
6.2.1. Behavior
a) lifetimeOfAssignment is the lifetime of the file that the current file owner is willing to provide, in terms of accountings, etc. The reassignment must occur within the lifetime of the assignment.
b) This function implies actual lifetime of file and space involved, and is extended up to the lifetimeOfAssignment.
c) After the lifetimeOfAssignment time period expires, or when newAssignedUserID or when owner issues srmAbortRequest with the requestToken, the files involved are released, may be removed, and space is compacted automatically, which ever is first.
d) The client must be the current owner of the files.
e) lifetimeOfAssignment is relative to the calling time.
f) If there are any previously released files involved before the request, then the files cannot not be involved in reassignment, even if they are still in the space.
Behavior
with other advanced features
6.2.2. Space Management Feature
a) If a srmCompactSpace is requested before the srmReassigneToUser request is complete, then the request has priority over srmCompactSpace.� Compact must be done automatically as soon as files are assigned to the new client. Whether to compact dynamically or not is an implementation choice.
6.2.3. Directory Management Feature
a) If SURL is a directory, then all the files under the SURL must be included recursively.
srmReassignToUserStatus is the status call to srmReassignToUser.
|
In |
Out |
|
String userID String authorizationID String requestToken
|
ReturnStatusForRequest { ��� ����EnumStatusCode statusCode, ������� String ����������������� ������������explanation } RequestStatusForSURL { ������� String SURL, ������� EnumStatusCode statusCode, ������� String� explanation }[] String newAssignedUserID Int lifetimeOfAssignment |
srmSetPermission is to set permission on local SURLs. This is similar to unix style permissions.
|
������� EnumStatusCode statusCode, |
|
|
EnumPermissionMode ������groupPermission EnumPermissionMode������ otherPermission Boolean recursive |
RequestStatusForSURL { ������� String SURL, ������� EnumStatusCode statusCode, ������� String explanation }[] |
6.4.1. Behavior
a) EnumPermissionMode is similar to Unix permission modes.
b) User permissions are not supported in this version for dynamic user-level permission assignment similar to Access Control Lists (ACLs).
c) Permissions must be assigned to a single owner and a single group, similar to unix permission.
d) SRMs do not provide any group operations (setup, modify, remove, etc.).
e) Groups must be assumed to be set up separately, before srmSetPermission is used. The owner must be a member of the group.
f) If EnumPermissionType is ADD or CHANGE, and EnumPermissionMode is null, then it must be assumed that EnumPermissionMode is READ only.
g) If EnumPermissionType is REMOVE, then the EnumPermissionMode is ignored.
Behavior
with other advanced features
6.4.2. Directory Management Feature
a) SURL must be either directory or file.
7. Advanced feature set 5 : Request Administration Functions
summary:
details:
srmResumeRequest is to resume previously suspended requests.
|
ReturnStatusForRequest { ������� EnumStatusCode statusCode, |
7.1.1. Behavior
a) Resume suspended files that belong to the request associated with the requestToken.
srmSuspendedRequest is to suspend a request.
|
ReturnStatusForRequest { ������� EnumStatusCode statusCode, |
7.2.1. Behavior
a) Suspend all files in the request until srmResumeRequest is issued. Local policy may be enforced for the duration of suspended time period. If the suspended time period expires and there has been no action from the client on the request, the SRM may choose to abort the request.
b) In order to avoid space charges on pinned files, the client must release those pinned files before or after suspending the request.
c) Lifetime of files not released in will continue until its expiration.
d) If lifetime of files expires for the suspended request, then those files will be put back on the queue for the client.
e) File release requests are performed even if the request is suspended. Releasing a request can be done after suspending the request because some files may be brought into the cache between release and suspend.
f) Upon suspending the request, new files must not be brought into the SRM space for the request .
g) srmAbortRequest may be performed to end the suspended request.
8.1. Appendix A : StatusCode� Specification
Note:
� Status codes represent errors, warnings and status.
Status code����� Explanation
SRM_SUCCESS:���
� SRM request was successful
�����������
Errors:
SRM_FAILURE :�������
� Requested operation failed for unspecified reason, and additional info is in the explanation string.
SRM_AUTHENTICATION_FAILURE:
� Requester has an invalid authentication information.
SRM_AUTHORIZATION_FAILURE:
� Requester has no permissions for the operation (although the user could have a valid authentication information).
SRM_INVALID_REQUEST:
� The request is invalid, and additional information may be provided in the explanation string.�� For example,
o The request token is invalid
o The requested life time of a file is longer than the lifetime of the space.
SRM_INVALID_SURL:
� The requested file/directory path or SURL is invalid.
SRM_FILE_LIFETIME_EXPIRED:
� The life time on the pinned file has expired
SRM_SPACE_LIFETIME_EXPIRED:
� The life time on the reserved space has expired
SRM_EXCEED_ALLOCATION:
� Requester exceeded allocation (number of requests, files or spaces), and the request cannot be placed.
SRM_NO_USER_SPACE:
� The requester does not have enough space to put the file into that space.
SRM_NO_FREE_SPACE:
� SRM has not more space.
SRM_DUPLICATION_ERROR :
� Requester tried to create a new file or directory that already exists.
SRM_NON_EMPTY_DIRECTORY:
� Requester tried to remove a non-empty directory without the recursive option set.
SRM_TOO_MANY_RESULTS:
� The request produced too many results;� for example, as a result of srmLs. The term �too many� is determined by each SRM , and the detailed information, such as the supported max number of results can be returned in the explanation string.
SRM_INTERNAL_ERROR:
� SRM has an internal error temporarily.� Client may try again.
SRM_FATAL_INTERNAL_ERROR:
� SRM has a severe internal error that cannot be recovered for an extended period of time.
SRM_NOT_SUPPORTED:
� SRM implementation does not support this functionality that client requested.
Status:
SRM_REQUEST_QUEUED
SRM_REQUEST_INPROGRESS
SRM_REQUEST_FINISHED
SRM_REQUEST_SUSPENDEND
SRM _ABORTED
SRM _RELEASED
SRM_FILE_PINNED
� The requested file is pinned
SRM_FILE_IN_CACHE
� The file is in cache, but not pinned
SRM_FILE_IN_SPACE
� The file is in space. Will be used with srmPutFileDone() or srmPutRequestDone().
SRM_SPACE_AVAILABLE
� The requested space is reserved and ready to be used
SRM_LOWER_SPACE_GRANTED
� The requested space is not ready, but lower sized space is granted.
SRM_CUSTOM_STATUS:
� SRM has a site specific status information. The details are described in the explanation string.
References and Related Papers
[1] Storage Resource Management working group: http://sdm.lbl.gov/srm-wg
[2] GGF Grid Storage Management working group: http://sdm.lbl.gov/gsm
[3] StorageManager.org: http://www.storagemanager.org
[4] Storage Resource Managers: Middleware Components for Grid Storage, Arie Shoshani, Alex Sim, Junmin Gu, Nineteenth IEEE Symposium on Mass Storage Systems, 2002 (MSS '02)
�
[5] Disk Cache Replacement Algorithm for Storage Resource Managers in Data Grids, Ekow J. Otoo, Frank Olken and Arie Shoshani, Supercomputing Conference, 2002.
[6] The Anatomy of the Grid: Enabling Scalable Virtual Organizations. I. Foster, C. Kesselman, S. Tuecke. International J. Supercomputer� Applications, 15(3), 2001.
[7] Best-effort versus reservations: A simple comparative analysis, Lee Breslau and Scott Shenker, ACM Computer Communication Review, 28(4):3--16, September 1998.
[8] Concurrency Control and Recovery In Database Systems, Philip A. Bernstein, Vassos Hadzilacos, and Nathan Goodman, Addison-Wesley Longman, 1987.
[9] Transaction Processing: Techniques and Concepts, J. Gray and A. Reuter, Morgan Kaufmann, San Mateo, CA, 1994.
[10] DataMover: Robust Terabyte-Scale Multi-file Replication over Wide-Area Networks, Alex Sim, Junmin Gu, Arie Shoshani, Vijaya Natarajan, Proceedings of the 16th International Conference on Scientific and Statistical Database Management (SSDBM 2004), Greece
[11] Storage Resource Managers: Essential Components for the Grid, Arie Shoshani, Alexander Sim, and Junmin Gu, in Grid Resource Management: State of the Art and Future Trends, Edited by Jarek Nabrzyski, Jennifer M. Schopf, Jan weglarz, Kluwer Academic Publishers, 2003