Backends
Backends connect users to DSI Core middleware and backends allow DSI middleware data structures to read and write to persistent external storage. Backends are modular to support user contribution. Backend contributors are encouraged to offer custom backend abstract classes and backend implementations. A contributed backend abstract class may extend another backend to inherit the properties of the parent. In order to be compatible with DSI core middleware, backends should create an interface to Python built-in data structures or data structures from the Python collections
library. Backend extensions will be accepted conditional to the extention of backends/tests
to demonstrate new Backend capability. We can not accept pull requests that are not tested.
Note that any contributed backends or extensions should include unit tests in backends/tests
to demonstrate the new Backend capability.

Figure depicts the current DSI backend class hierarchy.
- class dsi.backends.sqlite.Artifact
Primary Artifact class that holds database schema in memory. An Artifact is a generic construct that defines the schema for metadata that defines the tables inside of SQL
- class dsi.backends.gufi.Gufi(prefix, index, dbfile, table, column, verbose=False)
GUFI Datastore
- get_artifacts(query)
Retrieves GUFI’s metadata joined with a dsi database query: an sql query into the dsi_entries table
- isVerbose = False
prefix: prefix to GUFI commands index: directory with GUFI indexes dbfile: sqlite db file from DSI table: table name from the DSI db we want to join on column: column name from the DSI db to join on
- class dsi.backends.parquet.Parquet(filename, **kwargs)
Support for a Parquet back-end.
Parquet is a convenient format when metadata are larger than SQLite supports.
- get_artifacts()
Get Parquet data from filename.
- static get_cmd_output(cmd: list) str
Runs a given command and returns the stdout if successful.
If stderr is not empty, an exception is raised with the stderr text.
- put_artifacts(collection)
Put artifacts into file at filename path.