Issues List in
SRM v2.2 Proposed Changes
Compiled by Alex Sim at LBNL
Reference to Proposed_SRM_V2.2
May 18, 2006
Abstract
This is a list for issues on the proposed SRM v2.2 specification. These issues may be resolved by the time the SRM v2.2 specification has to be released, or maybe postponed until the SRM v3.0 specification is discussed.
Resolved issues will be consolidated into the SRM specification to be a part of SRM v2.2 specification.
This document is a set of issues that have been identified on the SRM v2.2 specification. These issues were found during review of the specifications, email discussions, interoperability testing, and collaboration discussions about the specification.
The process described below is used by several groups in OASIS and found to be useful to keep track of issues and recommendations.
We propose the following process for handling issues.
1) Any collaboration member can propose a new issue. This is done by sending an email to the mailing list where the subject is "ISSUE: <title of issue>". The title of the issue should be a short description of what the issue is about. The body of the message should contain a description of the issue in detail and if possible a proposed resolution. Updates to existing issues can be done in the same way, and make sure the body of the message indicates that it is an update and include the issue number. The person sending the email will be listed as the issue contact, unless noted otherwise in the message.
2) The issues coordinator will collect all new issues as proposed issues. This list will be reviewed at the next collaboration meeting, during the conference calls, or through emails.
3) Issues on the list will have one of the following status:
4) An issue is added to the list with a status of Open. �If it does not become an issue because of any reasons, it will be removed from the list. An issue moves from Open to Resolved after discussion and agreement of a resolution of the collaboration. The changes will be reflected on the revision of the specification.
Summary:� Retention Policy may not be enough for some client communities.� Additional information about retention policy may be needed to describe the community needs.
Details:� If detailed Retention Policy needs to be described in addition to the coarse granularity Retention Policy (i.e. custodial, output, replica), such �retention policy details� will be represented as a community-defined string set.
Proposed Resolution: to define a string of RetentionPolicyDetails. It is defined by the client communities.
Comments:� Jean-Philippe suggested postponing this for v3.
Status: Open 5/12/2006 by Arie
����������� Deferred for v3, 5/15/2006
Final Recommendation: postpone for v3, as clients do not need this at this time.
Summary:� Should we have retention policy information for space reservation only?
Details: Retention policy information is needed to reserve a space. In srmPrepareToPut, if a space reservation was not performed in advance, the space token cannot be provided by the client. So, retention policy should be provided as input parameter. Should we have retention policy as an input parameter in srmPrepareToPut, as well as space token as an input parameter?
Proposed Resolution:� to have retention policy in space reservation only, and not in srmPrepareToPut.
Status: Open 5/15/2006 by Alex
����������� Resolved 5/16/2006
Final Recommendation: �to have both space token and file retention policy as parameters of the target specified with data movement functions, such as srmPrepareToPut, srmPrepareToGet, srmCopy, and srmBringOnline.� If both file retention policy and space token are provided, the file retention policy must match the space retention policy.
Summary:� During the space reservation, a client needs to reserve a particular size of consecutive storage on SRM.
Details: The available space may be fragmented over multiple devices/servers, and the storage system may not have the required file size to write into. An array of expected file sizes needs to be provided by the client so that SRM can allocate the consecutive storage on a storage media for the client. Some systems that reserve spaces based on a total size that the client request will not make use of the array of expected file sizes. �However, if the underlying storage system is capable of �locking� consecutive space at the time of space reservation, this expected file size information can contribute to making a better usage of the storage space by avoiding fragmentation.
Proposed Resolution: to have an array of expected file size as an input parameter to srmReserveSpace.
Comments: Shaun agrees that some may find it useful.
Status: Open 5/12/2006 by Maarten, Olof
����������� Resolved for v2.2 5/15/2006, and may come back for v3.0
Final Recommendation: keep it for now
Summary:� Do we need a function to change the retention policy of all files in a storage space without specifying the SURLs?
Details: A new function (srmChangeRetentionPolicy) can be used to change the retention policy of files to another retention policy without specifying source urls.� All files that are associated with the retention policy in a specified space will have a new retention policy.� Asynchronous operation may be necessary for some SRMs, and in such case, a request token is returned for later status inquiry (srmGetStatusOfChangeRetentionPolicy).
Proposed Resolution: to have a new function srmChangeRetentionPolicy and srmGetStatusOfChangeRetentionPolicy with retention policy without array of SURLs.
Comments: Shaun agrees this is needed, but a storage tokens for the original space as well as the new space (that supports the requested retention policy) should be provided.
Status: Open 5/12/2006 by Alex
����������� Resolved 5/16/2006
Final Recommendation: to have new functions srmChangeRetentionPolicy and srmGetStatusOfChangeRetentionPolicy with optional array of SURLs. �The optional array of SURLs can be used to perform this operation selectively, rather than all the files in the space.� Space tokens for the original space and the new space need to be provided.
Summary:� Do we have a need to extend lifetime for all files in a space, without specifying all files?
Details: All files that are associated with the space token will have a new extended lifetime through this function.� If we do have a need for this function, do we need, as input parameters, space token and retention policy?� If both retention policy AND space token are provided, either the types have to get matched, or one of them has to be ignored.
Proposed Resolution: to have a new function srmExtendFileLifeTimeInSpace with space token and retention policy as input parameters. If both retention policy and space token are provided, the retention policy must match the type of space that is associated with the space token.
Status: Open 5/12/2006 by Alex
����������� Resolved 5/15/2006
Final Recommendation: to have a new function srmExtendFileLifeTimeInSpace with space token only.
Summary:� Do we need a space token in srmRemoveFiles?
Details: Files are brought online to specific spaces that are associated with space tokens, for different purposes or for different clients, through srmPrepareToGet and srmBringOnline. The online files could be physical copies or state representations of SUSLs, depending on the implementations.
There is a need to remove a "copy" or a "state" of the SURL in a specific space that is associated with a space token. �This function will be used to remove previously requested files (online "copies" or "states") specified by SURLs, through srmPrepareToGet and srmBringOnline. If space token is provided, it will be used to remove previously requested files (online "copies" or "states") specified by SURLs in a specific space that is associated with the space token. This function will not remove the SURLs, but only the "copies" or "states" of the SURLs.� srmRm should be used to remove SURLs and all their copies or �states�.
Proposed Resolution: to have a space token as an input parameter.
Opinions: Jean-Phililppe, Maarten and Arie agree to specify a space token in srmRemoveFiles.
Status: Open 5/12/2006 by Arie
����������� Resolved 5/16/2006
Final Recommendation: to have a space token as an input parameter, as proposed.
Summary:� Do we need access pattern and connection type information provided by the client?
Details: The access pattern has two values: TransferMode and ProcessingMode. The connection type has two values: WAN and LAN. These two types are used to select the appropriate storage sub-system returned through the Transfer URLs so that SRM may optimize the storage usage. However, these enumerated values may not be sufficient to represent the client access or connection types. During the lifetime of files, the access pattern and connection type from the client(s) may change as well. Different client communities may have different representation of access patterns and connection types.
Proposed Resolution: to have an open-ended string type of access/connection information, similar to the way the desired transfer protocol is specified.
Comments: Jean-Philippe suggested postponing this issue until v3 because it is not urgent to have this feature although this additional information could be useful for some SRMs in the future. Nick Brook, Tony Cass and Federico Carminati expressed the opinion that this is not of interest as an experiment. Stefano Belforte stated that CMS does not have the concept of WAN vs. LAN data. Shaun agrees to these types, provided we can agree on the attributes. �Timur mentioned that this will be very useful to SRM/dCache.
Status: Open 5/12/2006 by Alex
����������� Resolved for v2.2 5/15/2006, and may come back for v3.0
Recommendation: to have access pattern and connection type information provided by the client in srmPrepareToGet, srmPrepareToPut, and srmBringOnline.� The access pattern values are: TransferMode and ProcessingMode. The connection type values are: WAN and LAN.
Summary:� Do we need access pattern information in srmPrepareToPut?
Details: access pattern has two values: TransferMode and Processing Mode. These two values are used to compose Transfer URLs so that SRM may optimize the storage usage. srmPrepareToPut prepares TURL for clients to write data into the TURL, and there is no use case of "random seek" like protocol access as processing mode indicates. Do we still need to have an option for processing mode in srmPrepareToPut?�
Proposed Resolution: not to have an access pattern as an input parameter.
Comments: Shaun mentioned that slow writing may benefit the other transfers. Alex mentioned that slowness may be handled by the writing buffer, but network slowness may have be the problem. In such case, those two enumerated values may not properly represent the case. Maarten mentioned that slow network and small buffered host need to be allocated if network slowness is a problem.
Status: Open 5/12/2006 by Alex
����������� Resolved 5/15/2006
Final Recommendation: to have an access pattern as an input parameter.
Cross-Reference: [ISSUE 7]
Summary:� Do we need an assigned retention policy and current retention policy in TMetaDataPathDetails to show the current location of the file?
Details: TMetaDataPathDetails describes properties of a file. We agreed to have an assigned retention policy to show where the file is assigned to when it was written into the underlying storage system. However, there could be a need to show �current retention policy� because the file is on transit to the final destination. There is interest in finding out if the file reached the bottom of the hierarchy (MSS, a nearline storage), or is still at the top of the hierarchy (disk cache, online).� We could have a file location, which describes the location of the files; online, nearline or both. Is file location sufficient to indicate such needs, or do we need to have a "current" retention policy? [ Retention policy has three values; replica, output, custodial. ] If we need a "current" retention policy to indicate such needs, who allocates/reserves/owns the "current" retention policy which is different from the assigned retention policy, and what it means to the client?
Proposed Resolution: to have file location only to represent this information.
Status: Open 5/12/2006 by Alex
����������� Resolved 5/15/2006
Final Recommendation: to have file location to represent the needed information, as proposed
Summary:� What to do with files already brought online if srmBringOnline is aborted?
Details: should those files be removed automatically or left in the online storage?
Proposed Resolution: to remove them.
Comments: In communications with Maarten, it was agreed that it is safer to have files explicitly removed.
Status: Open 5/12/2006 by Arie
����������� Resolved 5/15/2006
Final Recommendation: not to remove them, and to have clients use srmRemoveFiles explicitly to remove them.