bueno.public package

Submodules

bueno.public.container module

Public container activation interfaces.

class bueno.public.container.ImageStager(*args: Any, **kwargs: Any)

Bases: object

Public container image stager singleton meant to provide some programmability to the container image staging process.

install_default_staging_cmd_hook() → None: Installs the default staging command hook.

property staging_cmd_hook: Callable[[], str]: Returns the staging command hook if one is installed. Returns None otherwise.

bueno.public.container.build_information() → List[str]: Returns a list of strings containing captured container and application build information. If build information is not available, an empty list is returned.

bueno.public.container.capture(cmd: str, check_exit_code: bool = True) → str

Executes the provided command and returns a string with the command’s output.

See run() for exceptions.

bueno.public.container.getenv(name: str) → Optional[str]: Get an environment variable, return None if it does not exist.

bueno.public.container.prun(pexec: str, cmd: str, echo: bool = True, check_exit_code: bool = True, preaction: Optional[Any] = None, postaction: Optional[Any] = None, user_data: Optional[Any] = None) → None

Executes the given parallel run command string from within a container. Optionally calls pre- or post-actions if provided.

The primary reason we have a separate command for running serial and parallel workloads is because we need to position image activator commands between the string containing the pexec string (e.g., mpiexec -n 3 -N 1 -mca foo bar) and the cmd string (e.g., nbody –decomp 221). Parsing these strings in a general, reliable way is challenging. This way is much easier.

bueno.public.container.run(cmd: str, echo: bool = True, check_exit_code: bool = True, preaction: Optional[Any] = None, postaction: Optional[Any] = None, user_data: Optional[Any] = None) → None: Runs the given command string from within a container. Optionally calls pre- or post-actions if provided.

bueno.public.data module

Core data types.

class bueno.public.data.BaseAsset

Bases: abc.ABC

Abstract base data asset class.

abstract write(basep: str) → None: Abstract method used to write the contents of a derived class to a location rooted at the specified base path.

class bueno.public.data.FileAsset(srcf: str, subd: Optional[str] = None)

Bases: bueno.public.data.BaseAsset

File asset.

write(basep: str) → None: Abstract method used to write the contents of a derived class to a location rooted at the specified base path.

class bueno.public.data.LoggerAsset

Bases: bueno.public.data.BaseAsset

bueno logger asset.

write(basep: str) → None: Abstract method used to write the contents of a derived class to a location rooted at the specified base path.

class bueno.public.data.PythonModuleAsset(mod: module)

Bases: bueno.public.data.FileAsset

Special file asset used for extra imported modules.

class bueno.public.data.StringIOAsset(srcios: _io.StringIO, fname: str, subd: Optional[str] = None)

Bases: bueno.public.data.BaseAsset

StringIO asset.

write(basep: str) → None: Abstract method used to write the contents of a derived class to a location rooted at the specified base path.

class bueno.public.data.YAMLDictAsset(ydict: Dict[Any, Any], fname: str)

Bases: bueno.public.data.BaseAsset

Convenience YAML (from a dict()) asset.

property fname: str: Returns the output file name.

write(basep: str) → None: Writes the contents of the YAMLDictAsset to a file rooted at the specified path.

bueno.public.data.add_asset(asset: bueno.public.data.BaseAsset) → None: Adds a data asset to the collection of assets to be written.

bueno.public.data.clear() → None: Clears all cached data.

bueno.public.data.write(basep: str) → None: Writes data rooted at basep.

bueno.public.datasink module

Convenience data sinks.

class bueno.public.datasink.InfluxDBMeasurement(measurement: str, values: Dict[str, Union[str, int, float, bool, Dict[str, Union[str, int, float, bool]]]], tags: Optional[Dict[str, str]] = None, verify_data: bool = False)

Bases: bueno.public.datasink.Measurement

InfluxDB measurement type.

data() → str: Returns measurement data as string following InfluxDB line protocol. Raises an exception if data verification is enabled and the data do not adhere to the InfluxDB line protocol.

class bueno.public.datasink.Measurement(verify_data: bool = False)

Bases: abc.ABC

Abstract measurement type.

abstract data() → str: Returns measurement data as string following a given line protocol.

class bueno.public.datasink.RabbitMQBlockingClient(conn_params: bueno.public.datasink.RabbitMQConnectionParams, queue_name: str, exchange: str, routing_key: str, verbose: bool = False)

Bases: object

A straightforward AMQP 0-9-1 blocking client interface that ultimately wraps Pika.

send(measurement: bueno.public.datasink.Measurement, verbose: bool = False) → None: Sends the contexts of measurement to the MQ server.

class bueno.public.datasink.RabbitMQConnectionParams(host: str, port: int, vhost: str = '/', connection_attempts: int = 2, heartbeat: int = 360, blocked_connection_timeout: int = 300, tls_config: Optional[bueno.public.datasink.TLSConfig] = None)

Bases: object

A straightforward RabbitMQ broker configuration container.

class bueno.public.datasink.TLSConfig(certfile: str, keyfile: str)

Bases: object

A straightforward Transport Layer Security (TLS) configuration container.

class bueno.public.datasink.Table

Bases: object

A straightforward class to display formatted tabular data.

class Row(data: List[Any], withrule: bool = False)

Bases: object

Creates a row for use in a table.

addrow(row: List[Any], withrule: bool = False) → None: Adds the contents of row to a table, optionally with a rule.

emit() → None: Emits the contents of the table using logger.log().

bueno.public.experiment module

Public experiment utilities for good.

class bueno.public.experiment.CLIAddArgsAction

Bases: object

Base action class used to add additional arguments to a CLIConfiguration instance.

class bueno.public.experiment.CLIConfiguration(desc: str, argv: List[str])

Bases: object

Command-line interface configuration container and associated utilities.

addargs(action: Type[bueno.public.experiment.CLIAddArgsAction]) → None: Instantiates and then calls provided action class to add additional arguments to argument parser.

property argparser: argparse.ArgumentParser: Returns the internal ArgumentParser instance.

property args: argparse.Namespace: Returns the ArgumentParser Namespace acquired after argument parsing.

property argv: List[str]: Returns the argument list used during instance initialization.

property description: str: Returns the desription used to initialize the encapsulated ArgumentParser.

parseargs() → None: Thin abstraction around argparser.parse_args() that parses and populates internal argparse namespace instance.

property program: str: Returns the basename of argv[0], i.e., the program name.

update(confns: argparse.Namespace) → None: Update the current configuration using the parsedargs provided through confns, a namespace.

class bueno.public.experiment.DefaultCLIConfiguration(desc: str, argv: List[str], defaults: bueno.public.experiment.DefaultCLIConfiguration.Defaults)

Bases: bueno.public.experiment.CLIConfiguration

A ‘canned’ set of parser arguments common to many (but not all) bueno run scripts. This CLI configuration is provided as a convenience for those run scripts that can benefit from the options provided.

class Defaults

Bases: object

Default values for DefaultCLIConfigurations.

csv_output = ''

description = ''

executable = ''

input = ''

name = ''

runcmds = (0, 0, '', '')

class bueno.public.experiment.FOM(name: str, description: str, units: str, value: float)

Bases: object

Figure of Merit data class.

bueno.public.experiment.cli_args_add_runcmds_option(clic: bueno.public.experiment.CLIConfiguration, opt_required: bool = False, opt_default: Tuple[int, int, str, str] = (0, 0, '', '')) → argparse.Action: Adds parser options to the given CLIConfiguration instance for handling runcmds input. Adds –runcmds and a custom action to parse its input.

bueno.public.experiment.factorize(num: int, dim: int) → List[int]: Perform factor calculations

bueno.public.experiment.flush_data(opath: Optional[str] = None) → str: Writes cached data to disk at specified or default path rooted at the output directory determined at run-time. Returns full path of output.

bueno.public.experiment.foutput(fmtstr: Optional[str] = None) → Optional[str]

Formatted experiment data string that determines the directory structure used to store experimental data. If a name string is provided, then it acts as a setter, acting as a getter otherwise.

Accepts a path-like string and returns a path if the decoding was successful. Picture Reference: %d - Date %h - Hostname %i - Unique ID %n - Experiment Name %t - Time %u - User

bueno.public.experiment.generate(spec: str, *args: Any) → List[str]: Given a string containing string.format() replacement fields and a variable number of iterables, attempt to generate an iterable collection of strings generated from the provided specification and corresponding inputs.

bueno.public.experiment.name(ename: Optional[str] = None) → Optional[str]: Experiment name getter/setter. If a name string is provided, then it acts as a setter, acting as a getter otherwise.

bueno.public.experiment.output_path(path: Optional[str] = None) → Optional[str]: Experiment data output path getter/setter. If a path is provided, then it acts as a setter, acting as a getter otherwise.

bueno.public.experiment.parsedargs(argprsr: argparse.ArgumentParser, argv: List[str]) → argparse.Namespace: TODO(skg) add a proper description.

bueno.public.experiment.readgs(gspath: str, config: Optional[bueno.public.experiment.CLIConfiguration] = None) → Iterable[str]

A convenience routine for reading generate specification files.

TODO(skg) Add description of formatting rules, semantics, etc. Don’t forget about yield!

We accept the following forms: # -a/–aarg [ARG_PARAMS] -b/–bargs [ARG PARAMS] # -c/–carg [ARG PARAMS] [positional arguments]

bueno.public.experiment.runcmds(start: int, stop: int, spec: str, nfun: str) → List[str]: TODO(skg) Add proper description. - start: The start value. - stop: The termination value for nfun(nidx) for some value nidx. - spec: The run specification template having the following variables: - %n: The number of processes to run.

bueno.public.host module

Host utilities.

bueno.public.host.capture(cmd: str, check_exit_code: bool = True) → str

Executes the provided command and returns a string with the command’s output.

See run() for exceptions.

bueno.public.host.hostname() → str: Returns the host computer’s name.

bueno.public.host.kernel() → str: Returns the kernel name.

bueno.public.host.kernelrel() → str: Returns the kernel release.

bueno.public.host.os_pretty_name() → str: Returns the host’s pretty name as reported by /etc/os-release.

bueno.public.host.run(cmd: str, verbatim: bool = False, echo: bool = False, capture_output: bool = False, verbose: bool = True, check_exit_code: bool = True) → List[str]

Executes the provided command.

Returns newline-delimited list of output if capture_output if True.

Throws ChildProcessError on error if check_exit_code is True.

bueno.public.host.shostname() → str: Returns the host computer’s short name.

bueno.public.host.tmpdir() → str: Returns tmpdir.

bueno.public.host.which(cmd: str) → Optional[str]

Akin to which(1).

Returns None if cmd is not found.

bueno.public.host.whichl(cmds: List[str]) → Optional[str]

Akin to which(1), but accepts a list of commands to search for. The first command found by which() is returned.

Returns None if none of the provided commands are found.

bueno.public.host.whoami() → str: Akin to whoami(1).

bueno.public.logger module

Logging utilities for good.

bueno.public.logger.emlog(msg: str, *args: Any, **kwargs: Any) → None: Logs the provided message to a central logger with emphasis.

bueno.public.logger.log(msg: str, *args: Any, **kwargs: Any) → None: Logs the provided message to a central logger.

bueno.public.logger.write(topath: str) → None: Writes the current contents of the log to the path provided.

bueno.public.utils module

Utilities for good.

bueno.public.utils.cat(filep: str) → List[str]

Akin to cat(1), but returns a list of strings containing the contents of the provided file.

Raises OSError or IOError on error.

bueno.public.utils.cats(file: str) → str

Akin to cat(1), but returns a string containing the contents of the provided file.

Raises OSError or IOError on error.

bueno.public.utils.chomp(istr: str) → str: Returns a string without trailing newline characters.

bueno.public.utils.dates() → str: Returns a string representation of the current date.

bueno.public.utils.ehorf() → str: Returns header/footer string used for error messages.

bueno.public.utils.emptystr(istr: Optional[str]) → bool: Returns True if the provided string is not empty; False otherwise.

bueno.public.utils.module_imported(modname: str) → bool: Returns whether or not the provided module name has already been imported.

bueno.public.utils.now() → datetime.datetime: Returns the current date and time.

bueno.public.utils.nows() → str: Returns a string representation of the current date and time.

bueno.public.utils.read_logical_lines(fileobj: IO[str]) → Iterable[str]: Reads the contents of fileobj and returns its lines, properly handling line continuations.

bueno.public.utils.yamlp(idict: Any, label: Optional[str] = None) → None: Emits YAML output from the provided dictionary.

bueno.public.utils.yamls(idict: Any) → str: Returns YAML string from the provided dictionary.

bueno.public package

Submodules

bueno.public.container module

bueno.public.data module

bueno.public.datasink module

bueno.public.experiment module

bueno.public.host module

bueno.public.logger module

bueno.public.utils module

Module contents