Jobs

Jobs represent and control running pipelines. They are returned by shtk.Shell.run() and used by other internal methods of shtk.Shell, such as shtk.Shell.evaluate(). Provided functionality includes starting, killing, and awaiting completion of process pipelines.

The Job.run() method is also responsible for instantiating shtk.PipelineNodeFactory templates to create shtk.PipelineNode instances, as well as instantiating StreamFactory templates to create shtk.Stream instances.

shtk.Job

class shtk.Job(pipeline_factory, cwd=None, env=None, event_loop=None, user=None, group=None, close_fds=True)

Bases: object

Instantiates PipelienNodeFactory instances to run subprocesses.

Job objects instantiate a PipelineNodeFactory template to create PipelineNode’s that run and track the progress of a command pipeline.

Parameters

pipeline_factory (PipelineNodeFactory) – The command pipeline template that will be instantiated by the Job instance.
cwd (str or Pathlib.Path) – The current working directory in which to run the pipeline processes.
env (dict) – The environment variables to pass to processes run within the pipleine
event_loop (None or asyncio.AbstractEventLoop) – The event loop to use for asyncio based processing. If None is passed a new event loop is created from asyncio.new_event_loop() instead.
user (None, int, or str) – The user that will be used to setreuid() any child processes. The behavior is the same as that of the user arg to subprocess.Popen().
group (None, int, or str) – The user that will be used to setregid() any child processes. The behavior is the same as that of the group arg to subprocess.Popen().
close_fds (bool) – If true, close_fds will be passed to the equivalent of subprocess.Popen() (Default value = True).

cwd

The current working directory in which to run the pipeline processes.

Type: str or Pathlib.Path

environment

The environment variables to pass to processes run within the pipleine

Type: dict

event_loop

The event loop to use for asyncio based processing. If None is passed a new event loop is created from asyncio.new_event_loop() instead.

Type: None or asyncio.AbstractEventLoop

pipeline

The running PipelineNode generated by PipelineFactory.build in run().

Type: None, PipelineNode

pipeline_factory

The command pipeline template that will be instantiated by the Job instance.

Type: PipelineNodeFactory

user

The user that will be used to setreuid() any child processes. The behavior is the same as that of the user arg to subprocess.Popen().

Type: None, int, or str

group

The user that will be used to setregid() any child processes. The behavior is the same as that of the group arg to subprocess.Popen().

Type: None, int, or str

close_fds

If true, close_fds will be passed to the equivalent of subprocess.Popen().

Type: bool

property stdin

Returns the pipeline’s stdin_stream.writer(), or None

Returns: Pipeline’s stdin_stream.writer(), or None if no pipeline is running.

property stdout

Returns the pipeline’s stdout_stream.reader(), or None

Returns: Pipeline’s stdout_stream.reader(), or None if no pipeline is running.

property stderr

Returns the pipeline’s stderr_stream.reader(), or None

Returns: Pipeline’s stderr_stream.reader(), or None if no pipeline is running.

async run_async(stdin_factory, stdout_factory, stderr_factory)

Creates and runs a new pipeline

Instantiates and runs a pipeline based on the PipelineNodeFactory provided to the Job’s constructor.

Parameters

stdin_factory (StreamFactory) – the StreamFactory to instantiate to create the job’s default stdin stream.
stdout_factory (StreamFactory) – the StreamFactory to instantiate to create the job’s default stdout stream.
stderr_factory (StreamFactory) – the StreamFactory to instantiate to create the job’s default stderr stream.

run(stdin_factory, stdout_factory, stderr_factory)

Creates and runs a new pipeline

Synchronously wrapper for run_async.

Parameters

stdin_factory (StreamFactory) – the StreamFactory to instantiate to create the job’s default stdin stream.
stdout_factory (StreamFactory) – the StreamFactory to instantiate to create the job’s default stdout stream.
stderr_factory (StreamFactory) – the StreamFactory to instantiate to create the job’s default stderr stream.

async wait_async(pipeline_node=None, exceptions=True)

Waits for all processes in the pipeline to complete

Waits for all processes in the pipleine to complete checks the return codes of each command.

Parameters

pipeline_node (PipelineNode or None) – The pipeline node to wait for
exceptions (Boolean) – When true returns an exception when processes exit with non-zero return codes

Returns

A tuple of exit codes from the completed proceses

Raises

NonzeroExitCodeException – When a process returns a non-zero return code
RuntimeError – When called on a Job that has not invoked Job.run()

wait(pipeline_node=None, exceptions=True)

Synchronous wrapper for the wait_async() method.

Waits for all processes in the pipleine to complete checks the return codes of each command.

Parameters

pipeline_node (PipelineNode or None) – The pipeline node to wait for
exceptions (Boolean) – When true returns an exception when processes exit with non-zero return codes

Returns

A tuple of exit codes from the completed proceses

Raises

NonzeroExitCodeException – When a process returns a non-zero return code

send_signal(signum)

Sends a signal to all child ProcessNode processes.

Parameters: signum (int) – the signal to send.

terminate(): Sends a signal.SIGTERM to all child ProcessNode processes.

kill(): Sends a signal.SIGKILL to all child ProcessNode processes.