As executor in the context of QuemeMe is a controller that will handle parsing of a command line (terminal) command. Any executor must ensure that:

• appropriate metadata is collected
• the command is run
• relevant actions are exposed

This section will give an overall review of executors, including global (base) metadata, and executor-specific details.

The Executor base

All executors should be derived from the ExecutorBase class that will ensure that each one exposes the needed functions. Each executor also has it’s own view under app/templates that renders a page specific to it for the dashboard (under development). You should reference the class to see the functions that are required and conditions for each.

Executors

Base

Each executor must expose the following metadata:

• pwd: the present working directory where the command was run
• command: the command that was run
• user: the user that ran the command
• status: the status of the operation. Since most basic commands save the first time upon completion, the status is usually complete, however this is subject to change. This must be one of “complete” “cancelled” or “running” or None.

The only metadata shown on the table (front) page of the dashboard is these common attributes. For the filesystem database, since we’d need to read many separate files, we just show the executor type and unique id. The user must click on any particular execution to see the full details.

Shell

The “shell” executor is the default that will take any command that doesn’t match a previous regular expression, and the executor will run the command, parse output and error streams, and then provide a result object with the following metadata:

• output: the output stream of running the command
• error: the error stream of running the command
• returncode: the returncode from running the command
• pid: the pid of the child process.

This means that it doesn’t have a specific match string or actions beyond the basic that are provided for any command (delete, view, and re-run). The matching dashboard interface is also optimized to show and search this information, mainly the command and any output or error. An example shell run might look like:

$ qme run echo "hello moto"

$ qme get
Database: sqlite
{
    "executor": "shell",
    "uid": "shell-eab1fcff-d8b8-497a-bb7b-c758b23ff697",
    "data": {
        "pwd": "/home/vanessa/Desktop/Code/qme",
        "user": "vanessa",
        "timestamp": "2020-05-20 16:47:47.955877",
        "output": [
            "hello moto\n"
        ],
        "error": [],
        "returncode": 0,
        "command": [
            "echo",
            "hello moto"
        ],
        "status": "complete",
        "pid": 15048
    },
    "command": "echo hello moto"
}

There are no specific environment variables for shell, beyond the default, nor any actions.

Slurm

The “slurm” executor is intended for using sbatch to run slurm jobs, sacct to check on status, and scancel to cancel. Since the slurm executor is a subclass of shell it exposes the same metadata.

$ qme get
Database: sqlite
{
    "executor": "slurm",
    "uid": "slurm-02eecdcd-a6b2-4055-8fad-c94e846a0f26",
    "data": {
        "pwd": "/home/users/vsochat",
        "user": "vsochat",
        "timestamp": "2020-05-20 16:24:01.216868",
        "output": [
            "Submitted batch job 907484\n"
        ],
        "error": [],
        "returncode": 0,
        "command": [
            "sbatch",
            "--partition",
            "owners",
            "--time",
            "00:00:10",
            "run_job.sh"
        ],
        "status": "complete",
        "pid": 156932
    },
    "command": "sbatch --partition owners --time 00:00:10 run_job.sh"
}

Actions

Actions include the following:

• status: Get a status dictionary, with a default format string set by QueueMe
• output: Get the output file output, if it exists. You should set --out or leave unset.
• error: Get the error file output, if it exists. You should set --err or leave unset.
• cancel: cancel a job that was run with scancel

For output and error files, you can either leave unset (to use a default) or set --out or --err to be read by QueueMe. SBATCH directives are not currently parsed. Here are quick examples of using running executor actions for the job we just ran above. Since it’s the last run task, we don’t need to specify the taskid, akin to qme get without one.

status

$ qme exec status
Database: sqlite
{'jobid': '941170', 'jobname': 'run_job.sh', 'partition': 'owners', 'alloccpus': '1', 'elapsed': '00:00:06', 'state': 'COMPLETED', 'exitcode': '0:0'}

output and error

$ qme exec output
Database: sqlite
HELLO WORLD

cancel

$ qme exec cancel

See the actions guide for how to run actions generally.

You might next want to learn about the interactive dashboard.