Backends

Backends connect users to DSI Core middleware and backends allow DSI middleware data structures to read and write to persistent external storage. Backends are modular to support user contribution. Backend contributors are encouraged to offer custom backend abstract classes and backend implementations. A contributed backend abstract class may extend another backend to inherit the properties of the parent. In order to be compatible with DSI core middleware, backends should create an interface to Python built-in data structures or data structures from the Python collections library. Backend extensions will be accepted conditional to the extention of backends/tests to demonstrate new Backend capability. We can not accept pull requests that are not tested.

Note that any contributed backends or extensions should include unit tests in backends/tests to demonstrate the new Backend capability.

Figure depicts the current DSI backend class hierarchy.

class dsi.backends.sqlite.Artifact

Primary Artifact class that holds database schema in memory. An Artifact is a generic construct that defines the schema for metadata that defines the tables inside of SQL

class dsi.backends.gufi.Gufi(prefix, index, dbfile, table, column, verbose=False)

GUFI Datastore

get_artifacts(query)

Retrieves GUFI’s metadata joined with a dsi database query: an sql query into the dsi_entries table

isVerbose = False

prefix: prefix to GUFI commands index: directory with GUFI indexes dbfile: sqlite db file from DSI table: table name from the DSI db we want to join on column: column name from the DSI db to join on

class dsi.backends.parquet.Parquet(filename, **kwargs)

Support for a Parquet back-end.

Parquet is a convenient format when metadata are larger than SQLite supports.

get_artifacts()

Get Parquet data from filename.

static get_cmd_output(cmd: list) → str

Runs a given command and returns the stdout if successful.

If stderr is not empty, an exception is raised with the stderr text.

put_artifacts(collection)

Put artifacts into file at filename path.