FasTensor  1.0.0
Transform Supercomputing for AI
Public Member Functions | List of all members
EndpointHDF5 Class Reference

#include <src/ft_endpoint_hdf5.h>

Inheritance diagram for EndpointHDF5:
Endpoint

Public Member Functions

 EndpointHDF5 (std::string endpoint_info_p)
 Construct a new EndpointHDF5 object. More...
 
 EndpointHDF5 ()
 Construct a new Endpoint in HDF5 Nothing to do there, can be used as sub-endpoint of directory. More...
 
 EndpointHDF5 (int no_mpi)
 Construct a new EndpointHDF5 object without MPI. More...
 
 ~EndpointHDF5 ()
 
int ExtractMeta () override
 extracts metadata, possbile endpoint_ranks/endpoint_dim_size/data_element_type More...
 
int PrintInfo () override
 print information about the endpoint More...
 
int Create () override
 create the endpoint More...
 
int Open () override
 open the endpoint More...
 
int Read (std::vector< unsigned long long > start, std::vector< unsigned long long > end, void *data) override
 read the data from end-point More...
 
int Write (std::vector< unsigned long long > start, std::vector< unsigned long long > end, void *data) override
 write the data to the end-point More...
 
int Close () override
 close the end-point More...
 
void Map2MyType () override
 call the finalize to close everything (like call Destractor) More...
 
void Map2MyTypeParameters (FTDataType ft_type, hid_t &mem_type, hid_t &disk_type)
 
void EnableCollectiveIO () override
 
void DisableCollectiveIO () override
 
void EnableMPIIO ()
 
void DisableMPIIO ()
 
int ParseEndpointInfo () override
 parse endpoint_info to my own info In HDF5, it map endpoint_info to filename, group name and datasetname More...
 
int Control (int opt_code, std::vector< std::string > &parameter_v) override
 call a special operator on endpoint such as, enable collective I/O for HDF5 dump file from MEMORY to HDF5 More...
 
int WriteAttribute (const std::string &name, const void *data, FTDataType data_type_p, const size_t &data_length_p=0) override
 Set the Attribute object. More...
 
int ReadAllAttributeName (std::vector< std::string > &attr_name)
 Read all attribute name. More...
 
int ReadAttribute (const std::string &name, void *data, FTDataType data_type_p, const size_t &data_length_p=0) override
 Get the Attribute object. More...
 
int GetAttributeSize (const std::string &name, FTDataType data_type_p) override
 
int CreateXDMF ()
 
- Public Member Functions inherited from Endpoint
 Endpoint ()
 
virtual ~Endpoint ()
 
std::vector< unsigned long long > GetDimensions ()
 Get the Dimensions of the data. More...
 
void SetDimensions (std::vector< unsigned long long > endpoint_dim_size_p)
 Set the Dimensions. More...
 
void SetDataElementType (AuEndpointDataType data_element_type_p)
 set the type of data element More...
 
AuEndpointDataType GetDataElementType ()
 Get the Type of Data Element. More...
 
int GetDataElementTypeSize ()
 Get the size of the type for the element. More...
 
void SetEndpointType (AuEndpointType endpoint_type_p)
 Set the Endpoint Type object. More...
 
AuEndpointType GetEndpointType ()
 Get the Endpoint Type object. More...
 
bool GetOpenFlag ()
 
void SetOpenFlag (bool open_flag_p)
 
bool GetCreateFlag ()
 
void SetCreateFlag (bool open_flag_p)
 
void SetRwFlag (unsigned read_write_flag_p)
 
unsigned GetRwFlag ()
 
std::vector< AuEndpointDataTypeUnionVoid2Union (void *vp, size_t n_elements)
 convert my data in (void *) type to Union type More...
 
void * Union2Void (std::vector< AuEndpointDataTypeUnion > &data_vector_in_union_type)
 convert data from union to void type More...
 
void SetEndpointInfo (std::string endpoint_info_p)
 set the endpoint_info string More...
 
std::string GetEndpointInfo ()
 Get the endpoint_info string. More...
 
virtual std::vector< std::string > GetDirFileVector ()
 Get the Dir File Vector object. More...
 
virtual void SetDirFileVector (std::vector< std::string > &file_list)
 Set the Dir File Vector object. More...
 
virtual std::vector< int > GetDirChunkSize ()
 Get the Dir Chunk Size object. More...
 
virtual void SetDirChunkSize (std::vector< int > &dir_chunk_size_p)
 Set the Dir Chunk Size object. More...
 
virtual int MapOpStr2Int (std::string op_cmd_str)
 map a op_cmd_str to int as input of SpecialOperator More...
 
virtual std::string MapOpInt2Str (int op_int)
 map op_int to string More...
 

Additional Inherited Members

- Protected Attributes inherited from Endpoint
AuEndpointType endpoint_type
 
std::string endpoint_info
 
std::vector< unsigned long long > endpoint_size
 
int endpoint_ranks
 
AuEndpointDataType data_element_type
 
std::string data_endpoint_orig
 
bool set_endpoint_dim_size_flag = false
 
bool open_flag = false
 
bool create_flag = false
 
unsigned read_write_flag
 

Constructor & Destructor Documentation

◆ EndpointHDF5() [1/3]

EndpointHDF5::EndpointHDF5 ( std::string  endpoint_info_p)
inline

Construct a new EndpointHDF5 object.

Parameters
data_endpointcontains the info of the endpoint, e.g., file type + file info

◆ EndpointHDF5() [2/3]

EndpointHDF5::EndpointHDF5 ( )
inline

Construct a new Endpoint in HDF5 Nothing to do there, can be used as sub-endpoint of directory.

◆ EndpointHDF5() [3/3]

EndpointHDF5::EndpointHDF5 ( int  no_mpi)
inline

Construct a new EndpointHDF5 object without MPI.

Parameters
no_mpiany number should work

◆ ~EndpointHDF5()

EndpointHDF5::~EndpointHDF5 ( )
inline

Member Function Documentation

◆ Close()

int EndpointHDF5::Close ( )
overridevirtual

close the end-point

Returns
int int < 0 error, >= 0 works

Implements Endpoint.

◆ Control()

int EndpointHDF5::Control ( int  opt_code,
std::vector< std::string > &  parameter_v 
)
overridevirtual

call a special operator on endpoint such as, enable collective I/O for HDF5 dump file from MEMORY to HDF5

Parameters
opt_code,speciallydefined code

call a special operator on endpoint such as, enable collective I/O for HDF5 dump file from MEMORY to HDF5

Parameters
opt_code,speciallydefined code
opt_code,speciallydefined code comment out int EndpointHDF5::SpecialOperator(int opt_code, std::string parameter) { switch (OP_ENABLE_MPI_IO) { case OP_ENABLE_MPI_IO: EnableMPIIO(); break; case OP_DISABLE_MPI_IO: DisableMPIIO(); break; case OP_ENABLE_COLLECTIVE_IO: EnableCollectiveIO(); break; case OP_DISABLE_COLLECTIVE_IO: DisableCollectiveIO(); break; default: break; } }

call a special operator on endpoint such as, enable collective I/O for HDF5 dump file from MEMORY to HDF5

Parameters
opt_code,speciallydefined code

Reimplemented from Endpoint.

◆ Create()

int EndpointHDF5::Create ( )
overridevirtual

create the endpoint

Returns
< 0 error, >= 0 works

Implements Endpoint.

◆ CreateXDMF()

int EndpointHDF5::CreateXDMF ( )

◆ DisableCollectiveIO()

void EndpointHDF5::DisableCollectiveIO ( )
overridevirtual

Reimplemented from Endpoint.

◆ DisableMPIIO()

void EndpointHDF5::DisableMPIIO ( )

◆ EnableCollectiveIO()

void EndpointHDF5::EnableCollectiveIO ( )
overridevirtual

Reimplemented from Endpoint.

◆ EnableMPIIO()

void EndpointHDF5::EnableMPIIO ( )

◆ ExtractMeta()

int EndpointHDF5::ExtractMeta ( )
overridevirtual

extracts metadata, possbile endpoint_ranks/endpoint_dim_size/data_element_type

Returns
int < 0 error, >= 0 works

Author: Bin Dong dbin@.nosp@m.lbl..nosp@m.gov Web: https://crd.lbl.gov/bin-dong Scientific Data Management Research Group Lawrence Berkeley National Laboratory

Implements Endpoint.

◆ GetAttributeSize()

int EndpointHDF5::GetAttributeSize ( const std::string &  name,
FTDataType  data_type_p 
)
overridevirtual

Reimplemented from Endpoint.

◆ Map2MyType()

void EndpointHDF5::Map2MyType ( )
overridevirtual

call the finalize to close everything (like call Destractor)

Returns
int

Implements Endpoint.

◆ Map2MyTypeParameters()

void EndpointHDF5::Map2MyTypeParameters ( FTDataType  ft_type,
hid_t &  mem_type,
hid_t &  disk_type 
)

◆ Open()

int EndpointHDF5::Open ( )
overridevirtual

open the endpoint

Returns
< 0 error, >= 0 works

Implements Endpoint.

◆ ParseEndpointInfo()

int EndpointHDF5::ParseEndpointInfo ( )
overridevirtual

parse endpoint_info to my own info In HDF5, it map endpoint_info to filename, group name and datasetname

Returns
int: 0 works, < 0 error,

Implements Endpoint.

◆ PrintInfo()

int EndpointHDF5::PrintInfo ( )
overridevirtual

print information about the endpoint

Returns
< 0 error, >= 0 works

Implements Endpoint.

◆ Read()

int EndpointHDF5::Read ( std::vector< unsigned long long >  start,
std::vector< unsigned long long >  end,
void *  data 
)
overridevirtual

read the data from end-point

Parameters
start,coordinatesof the cell to start (including)
end,coordinatesof the cell to end (including)
data,storethe result data
Returns
int < 0 error, >= 0 works

Implements Endpoint.

◆ ReadAllAttributeName()

int EndpointHDF5::ReadAllAttributeName ( std::vector< std::string > &  attr_name)

Read all attribute name.

Parameters
attri_name
Returns
int

◆ ReadAttribute()

int EndpointHDF5::ReadAttribute ( const std::string &  name,
void *  data,
FTDataType  data_type_p,
const size_t &  data_length_p = 0 
)
overridevirtual

Get the Attribute object.

Parameters
name
data
Returns
int

Reimplemented from Endpoint.

◆ Write()

int EndpointHDF5::Write ( std::vector< unsigned long long >  start,
std::vector< unsigned long long >  end,
void *  data 
)
overridevirtual

write the data to the end-point

Parameters
start,coordinatesof the cell to start (including)
end,coordinatesof the cell to end (including)
data,storethe result data
Returns
int < 0 error, >= 0 works

Implements Endpoint.

◆ WriteAttribute()

int EndpointHDF5::WriteAttribute ( const std::string &  name,
const void *  data,
FTDataType  data_type_p,
const size_t &  data_length_p = 0 
)
overridevirtual

Set the Attribute object.

Parameters
name
data
Returns
int

Reimplemented from Endpoint.


The documentation for this class was generated from the following files: