imars3d.backend.workflow package

iMars3D workflow module.

Submodules

imars3d.backend.workflow.engine module

Workflow engine for imars3d.

class imars3d.backend.workflow.engine.WorkflowEngine[source]

Bases: object

Base class for running data reduction workflows.

property registry: Read only registry.

class imars3d.backend.workflow.engine.WorkflowEngineAuto(config: str | dict | Path)[source]

Bases: WorkflowEngine

Used for running fully specified workflow.

run() → None[source]: Sequential execution of the tasks specified in the JSON configuration file.

exception imars3d.backend.workflow.engine.WorkflowEngineError[source]

Bases: RuntimeError

Base class for workflow engine errors.

exit_code = 1

class imars3d.backend.workflow.engine.WorkflowEngineExitCodes(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)[source]

Bases: Enum

Exit codes to be used with workflow engine errors.

ERROR_GENERAL = 1

ERROR_VALIDATION = 2

SUCCESS = 0

exception imars3d.backend.workflow.engine.WorkflowValidationError[source]

Bases: WorkflowEngineError

Class for workflow validation errors.

exit_code = 1

imars3d.backend.workflow.validate module

The validation happens in 3 steps:

Validate against the schema
Verify that the facility and instrument are supported
Verify that the function for each steps exists

An individual json configuration can be validated from the command line using jsonschema

jsonschema --instance example.json src/imars3d/backend/workflow/schema.json

The schema itself can be viewed in the code, however it suggests a very flat file. An example json configuration from the test suite is linked as another way to understand the format. The top level of the json configuration file is

	Type	Description	Required
facility	`string`	Facility for the measurment	Y
instrument	`string`	Instrument for the measurment	Y
ipts	`string`	The full IPTS identifier for the measurement	Y
name	`string`	Rememberable name for the measurement	Y
workingdir	`string`	Directory to write intermediate results when requested	Y
outputdir	`string`	Directory to write final results	Y
tasks	`array`	Each task is a step in the tomographic reconstruction	Y

Tasks contain the following elements

	Type	Description	Required
name	`string`	Friendly name for the task	Y
function	`string`	Fully qualified python name for the task	Y
inputs	`object`	Dictionary of inputs to the task. These are in the form of key value pairs that are passed into the task.	N
outputs	`array`	List of outputs from the task	N

Since not all tasks require inputs and outputs, they are optional.

iMars3D’s config validation module.

class imars3d.backend.workflow.validate.JSONValid[source]

Bases: object

Descriptor class that validates the json object.

See https://realpython.com/python-descriptors/

required(queryset: Iterable) → bool[source]

Check if a set of input parameters are required by the schema.

Parameters:: queryset – list of input parameters.
Returns:: True if all the input parameters are required by the schema.
Return type:: bool

exception imars3d.backend.workflow.validate.JSONValidationError[source]

Bases: RuntimeError

Custom exception for validation errors independent of what created them.

imars3d.backend.workflow.validate.todict(obj: str | dict | Path) → Dict[source]: Convert the supplied object into a dict. Raise a TypeError if the object is not a type that has a conversion menthod.