pyDNTNK package¶
Submodules¶
pyDNTNK.TN_data_generator module¶
- class pyDNTNK.TN_data_generator.data_generator(args)¶
Bases:
objectGenerates synthetic data for tensor networks. THe factors are generated randomly and then taken product into TT format.
- fit()¶
generates and save factors into Zarr file
- pyDNTNK.TN_data_generator.parser()¶
Reads the input arguments from the user and parses the parameters to the data generator module.
pyDNTNK.pyDNTNK module¶
- class pyDNTNK.pyDNTNK.pyDNTNK(fpath, params, model='tt')¶
Bases:
objectPerforms the distributed Hierrachial Tensor decomposition of given matrix Tensor X into factors/tensor cores
- Parameters
fpath (str) – Path to the tensor file
model (str (optional)) – Tensor networks model (TT/TK) where TT-Tensor train and TK-Tucker
params (class) – Class which comprises following attributes
params.routine (str(optinal)) – Decomposition method (NMF/SVD)
params.init (str) – NMF initialization(rand/nnsvd)
params.err (float) – Error criteria for automating estimating ranks with SVD
params.ranks (list) – List of ranks for decomposition
params.comm (object) – Modified communicator object
params.norm (str) – NMF norm to be minimized
params.method (str) – NMF optimization method
params.eps (float) – Epsilon value
params.verbose (bool) – Flag to enable/disable display results
params.save_factors (bool) – Flag to enable/disable saving computed factors
- compute_svd(data, k)¶
Computes distributed SVD for given data and then returns the factors
- Parameters
data (ndarray) – Data to be decomposed via SVD
k (int) – Rank for decomposition
- Returns
singularValues (list) – A list of singular values
U (ndarray) – U matrix
V (ndarray) – V matrix
rel_error (float) – Relative error of decomposition
- determine_rank(X, err)¶
Automatically estimates the rank for each stage of decomposition with some error criteria
- Parameters
X (ndarray) – Matrix of whose rank is to be estimated
err (float) – Error for estimation
- Returns
rank – Estimated rank of decomposition
- Return type
int
- error_compute()¶
Computes the final decomposition error as a upper bound
- fit()¶
Calls the sub routines to perform distributed Tensor network decomposition with NMF/SVD decomposition for a given TT/TK based method Supports Tensor train and Tucker based Tensor networks decomposition. THe decomposition can be carried out via SVD/NMF. The ranks can be provided by user or can be automatically estimated with SVD.
- lazy_read_file()¶
Reads file from the Zarr file system to each MPI rank and then handles the object to dask array
- Parameters
fpath (str) – File path to the Zarr Tensor File
- return_factors()¶
Returns the factors after decomposition
pyDNTNK.tt_utils module¶
- pyDNTNK.tt_utils.glob_rank_to_ND_rank(p_grid, glob_RANK)¶
Converts global rank to cartesian coordinates i.e 0 -> (0,0,0)
- class pyDNTNK.tt_utils.tn_data_operations(comm, fpath=None, data=None)¶
Bases:
objectPerform various data opetations on TN data
- data_block_idx_range(chunk_size)¶
Computes the block index range of data for each MPI rank
- Parameters
chunk_size (tuple) – Total data size based on which the index range is determined
- Returns
dtr_blk_idx – List of index range for each MPI rank
- Return type
list
- dist_reshape(target_shape)¶
Computes the Distributed reshape operation with the DASK object
- Parameters
data (DASK object) – Data stored into DASK format
target_shape (tuple) – Desired data shape
- Returns
data – Reshapes the dask array into desired shape and then computes the chunk of data for each MPI rank
- Return type
ndarray
- lazy_read_file()¶
Reads file from the Zarr file system to each MPI rank and then handles the object to dask array
- Parameters
fpath (str) – File path to the Zarr Tensor File
- lazy_store_file()¶
Stores the ndarray from each MPI rank into a shared file system
- Parameters
data (ndarray) – Data to be stored from each MPI rank
target_shape (fpath) – Path to which file to be stored