pyCP_APR.torch_backend package#

Submodules#

pyCP_APR.torch_backend.CP_APR_Torch module#

Python implementation of the CP-APR algorithm [1-4] with PyTorch backend.

This backend can be used to factorize sparse tensorsin COO format on GPU or CPU.

References

[1] General software, latest release: Brett W. Bader, Tamara G. Kolda and others, Tensor Toolbox for MATLAB, Version 3.2.1, www.tensortoolbox.org, April 5, 2021.

[2] Dense tensors: B. W. Bader and T. G. Kolda, Algorithm 862: MATLAB Tensor Classes for Fast Algorithm Prototyping, ACM Trans. Mathematical Software, 32(4):635-653, 2006, http://dx.doi.org/10.1145/1186785.1186794.

[3] Sparse, Kruskal, and Tucker tensors: B. W. Bader and T. G. Kolda, Efficient MATLAB Computations with Sparse and Factored Tensors, SIAM J. Scientific Computing, 30(1):205-231, 2007, http://dx.doi.org/10.1137/060676489.

[4] Chi, E.C. and Kolda, T.G., 2012. On tensors, sparsity, and nonnegative factorizations. SIAM Journal on Matrix Analysis and Applications, 33(4), pp.1272-1299.

@author: Maksim Ekin Eren

class pyCP_APR.torch_backend.CP_APR_Torch.CP_APR_MU(epsilon=1e-10, kappa=0.01, kappa_tol=1e-10, max_inner_iters=10, n_iters=1000, print_inner_itn=0, verbose=10, simple_verbose=False, stoptime=1000000.0, tol=0.0001, random_state=42, device='cpu', device_num='0', return_type='numpy', dtype='torch.DoubleTensor', follow_M=False)[source]#

Bases: object

Initilize the CP_APR_MU class. Sets up the class variables and the CUDA for tensors.

Parameters:

  • epsilon (float, optional) -- Prevents zero division. Default is 1e-10.

  • kappa (float, optional) -- Fix slackness level. Default is 1e-2.

  • kappa_tol (float, optional) -- Tolerance on slackness level. Default is 1e-10.

  • max_inner_iters (int, optional) -- Number of inner iterations per epoch. Default is 10.

  • n_iters (int, optional) -- Number of iterations during optimization or epoch. Default is 1000.

  • print_inner_itn (int, optional) -- Print every n inner iterations. Does not print if 0. Default is 0.

  • verbose (int, optional) -- Print every n epoch, or n_iters. Does not print if 0. Default is 10.

  • simple_verbose (bool, optional) -- Turns off details for verbose, such as fit, but instead shows a progress bar.

  • stoptime (float, optional) -- Number of seconds before early stopping. Default is 1e6.

  • tol (float, optional) -- KKT violations tolerance. Default is 1e-4.

  • random_state (int, optional) -- Random seed for initial M. The default is 42.

  • device (string, optional) -- Torch device to be used. 'cpu' to use PyTorch with CPU. 'gpu' to use cuda:0 The default is cpu.

  • device_num (string, optional) -- Which device to to store the tensors.

  • return_type (string, optional) -- Type for the latent factors. 'torch' keep as torch tensors. 'numpy' convert to numpy arrays.

  • dtype (string, optional) -- Type to be used in torch tensors. Default is torch.cuda.DoubleTensor.

  • follow_M (bool, optional) -- Saves M on each iteration. The default is False.

train(tensor=[], coords=[], values=[], rank=2, Minit='random', Type='sptensor')[source]#

Factorize the tensor X (i.e. compute the KRUSKAL tensor M).

Parameters:

  • tensor (PyTorch Sparse Tensor or dense Numpy array as tensor) --

    Original dense or sparse tensor X.

    Can be used when Type = 'sptensor'. Then tensor parameter needs to be a PyTorch Sparse tensor.

    Or use with Type = 'tensor' and pass the tensor parameter as a dense Numpy array.

    Note that PyTorch only supports Type = 'sptensor'.

  • coords (Numpy array (i.e. array that is a list of list)) --

    Array of non-zero coordinates for sparse tensor X. COO format.

    Each entry in this array is a coordinate of a non-zero value in the tensor.

    Used when Type = 'sptensor' and tensor parameter is not passed.

    len(Coords) is number of total entiries in X, and len(coords[0]) should give the number of dimensions.

  • values (Numpy array (i.e. list of non-zero values corresponding to each list of non-zero coordinates)) --

    Array of non-zero tensor entries. COO format.

    Used when Type = 'sptensor' and tensor parameter is not passed.

    Length of values must match the length of coords.

  • rank (int) -- Tensor rank, i.e. number of components to extract. The default is 2.

  • Minit (string or dictionary of latent factors) --

    Initial value of latent factors.

    If Minit = 'random', initial factors are chosen randomly from uniform distribution between 0 and 1.

    Else, pass dictionary where the key is the mode number and value is array size d x r where d is the number of elements on the dimension and r is the rank.

    The default is "random".

  • Type (string) --

    Type of tensor (i.e. sparse or dense).

    Use 'sptensor' for sparse, and 'tensor' for dense tensors.

    'sptensor' can be used with method = 'torch', method = 'numpy'.

    If 'sptensor' used, pass the list of non-zero coordinates using the Coords parameter and the corresponding list of non-zero elements with values.

    'sptensor' can also be used with the PyTorch Sparse format. Pass the torch.sparse format in the tensor parameter.

    'tensor' can be used with method = 'numpy' only. Pass the tensor using tensor parameter in that case.

    The default is 'sptensor'.

Returns:

result -- KRUSKAL tensor M is returned. The latent factors can be found with the key 'Factors'.

The weight of each component can be found with the key 'Weights'.

Return type:

dict

pyCP_APR.torch_backend.ktensor_Torch module#

ktensor_Torch.py contains the K_TENSOR class for KRUSKAL tensor M object representation.

References

[1] General software, latest release: Brett W. Bader, Tamara G. Kolda and others, Tensor Toolbox for MATLAB, Version 3.2.1, www.tensortoolbox.org, April 5, 2021.

[2] Dense tensors: B. W. Bader and T. G. Kolda, Algorithm 862: MATLAB Tensor Classes for Fast Algorithm Prototyping, ACM Trans. Mathematical Software, 32(4):635-653, 2006, http://dx.doi.org/10.1145/1186785.1186794.

[3] Sparse, Kruskal, and Tucker tensors: B. W. Bader and T. G. Kolda, Efficient MATLAB Computations with Sparse and Factored Tensors, SIAM J. Scientific Computing, 30(1):205-231, 2007, http://dx.doi.org/10.1137/060676489.

[4] Chi, E.C. and Kolda, T.G., 2012. On tensors, sparsity, and nonnegative factorizations. SIAM Journal on Matrix Analysis and Applications, 33(4), pp.1272-1299.

@author: Maksim Ekin Eren

class pyCP_APR.torch_backend.ktensor_Torch.K_TENSOR(Rank, Size, Minit='random', random_state=42, device='cpu', dtype='torch.DoubleTensor')[source]#

Bases: object

Initilize the K_TENSOR class.

Creates the object representation of M.

If initial M is not passed, by default, creates M from uniform distribution.

Parameters:

  • Rank (int) -- Tensor rank, i.e. number of components in M.

  • Size (list) -- Shape of the tensor.

  • Minit (string or dictionary of latent factors) --

    Initial value of latent factors.

    If Minit = 'random', initial factors are chosen randomly from uniform distribution between 0 and 1.

    Else, pass dictionary where the key is the mode number and value is array size d x r where d is the number of elements on the dimension and r is the rank.

    The default is "random".

  • random_state (int, optional) -- Random seed for initial M. The default is 42.

  • device (string, optional) -- Torch device to be used. 'cpu' to use PyTorch with CPU. 'gpu' to use cuda:0 The default is cpu.

  • dtype (string, optional) -- Type to be used in torch tensors. Default is torch.cuda.DoubleTensor.

deep_copy_factors()[source]#

Creates a deep copy of the latent factors in M.

Returns:

factors -- Copy of the latent factors of M.

Return type:

dict

pyCP_APR.torch_backend.sptensor_Torch module#

sptensor_Torch.py contains the SP_TENSOR class which is the object representation of the sparse tensor X in COO format.

References

[1] General software, latest release: Brett W. Bader, Tamara G. Kolda and others, Tensor Toolbox for MATLAB, Version 3.2.1, www.tensortoolbox.org, April 5, 2021.

[2] Dense tensors: B. W. Bader and T. G. Kolda, Algorithm 862: MATLAB Tensor Classes for Fast Algorithm Prototyping, ACM Trans. Mathematical Software, 32(4):635-653, 2006, http://dx.doi.org/10.1145/1186785.1186794.

[3] Sparse, Kruskal, and Tucker tensors: B. W. Bader and T. G. Kolda, Efficient MATLAB Computations with Sparse and Factored Tensors, SIAM J. Scientific Computing, 30(1):205-231, 2007, http://dx.doi.org/10.1137/060676489.

[4] Chi, E.C. and Kolda, T.G., 2012. On tensors, sparsity, and nonnegative factorizations. SIAM Journal on Matrix Analysis and Applications, 33(4), pp.1272-1299.

@author: maksimekineren

class pyCP_APR.torch_backend.sptensor_Torch.SP_TENSOR(Tensor, Coords, Values, dtype='torch.DoubleTensor', device='cpu')[source]#

Bases: object

Initilize the SP_TENSOR class.

Sorts the tensor entries.

Parameters:

  • Tensor (PyTorch Sparse Tensor or dense Numpy array as tensor) --

    Original dense or sparse tensor X.

    Can be used when Type = 'sptensor'. Then Tensor needs to be a PyTorch Sparse tensor.

    Or use with Type = 'tensor' and pass Tensor as a dense Numpy array.

    Note that PyTorch only supports Type = 'sptensor'.

  • Coords (Numpy array (i.e. array that is a list of list)) --

    Array of non-zero coordinates for sparse tensor X. COO format.

    Each entry in this array is a coordinate of a non-zero value in the tensor.

    Used when Type = 'sptensor' and tensor parameter is not passed.

    len(Coords) is number of total entiries in X, and len(coords[0]) should give the number of dimensions.

  • Values (Numpy array (i.e. list of non-zero values corresponding to each list of non-zero coordinates)) --

    Array of non-zero tensor entries. COO format.

    Used when Type = 'sptensor' and tensor parameter is not passed.

    Length of values must match the length of coords.

  • dtype (string, optional) -- Type to be used in torch tensors. Default is torch.cuda.DoubleTensor.

  • device (string, optional) -- Torch device to be used. 'cpu' to use PyTorch with CPU. 'gpu' to use cuda:0 The default is cpu.

Module contents#

2021. Triad National Security, LLC. All rights reserved. This program was produced under U.S. Government contract 89233218CNA000001 for Los Alamos National Laboratory (LANL), which is operated by Triad National Security, LLC for the U.S. Department of Energy/National Nuclear Security Administration. All rights in the program are reserved by Triad National Security, LLC, and the U.S. Department of Energy/National Nuclear Security Administration. The Government is granted for itself and others acting on its behalf a nonexclusive, paid-up, irrevocable worldwide license in this material to reproduce, prepare derivative works, distribute copies to the public, perform publicly and display publicly, and to permit others to do so.