pipelib package

Submodules

pipelib.version module

pipelib.logger module

class pipelib.logger.ColorizingStreamHandler(nocolor=False, stream=<_io.TextIOWrapper name='<stderr>' mode='w' encoding='utf-8'>, use_threads=False)[source]

Bases: StreamHandler

BLACK = 0
BLUE = 4
BOLD_SEQ = '\x1b[1m'
COLOR_SEQ = '\x1b[%dm'
CYAN = 6
GREEN = 2
MAGENTA = 5
RED = 1
RESET_SEQ = '\x1b[0m'
WHITE = 7
YELLOW = 3
can_color_tty()[source]
colors = {'CRITICAL': 1, 'DEBUG': 4, 'ERROR': 1, 'INFO': 2, 'WARNING': 3}
decorate(record)[source]
emit(record)[source]

Emit a record.

If a formatter is specified, it is used to format the record. The record is then written to the stream with a trailing newline. If exception information is present, it is formatted using traceback.print_exception and appended to the stream. If the stream has an ‘encoding’ attribute, it is used to determine how to do the output to the stream.

property is_tty
class pipelib.logger.Logger[source]

Bases: object

cleanup()[source]
debug(msg)[source]
error(msg)[source]
exit(msg, return_code=1)[source]
handler(msg)[source]
info(msg)[source]
location(msg)[source]
progress(done=None, total=None)[source]
set_level(level)[source]
set_stream_handler(stream_handler)[source]
shellcmd(msg)[source]
text_handler(msg)[source]

The default snakemake log handler. Prints the output to the console. Args:

msg (dict): the log message dictionary

warning(msg)[source]
pipelib.logger.setup_logger(quiet=False, printshellcmds=False, nocolor=False, stdout=False, debug=False, use_threads=False, wms_monitor=None)[source]

pipelib.pipeline module

class pipelib.pipeline.Pipeline(steps=None)[source]

Bases: object

A pipeline holds or more steps to complete in a comparison process.

add_step(step)[source]

Add a step to the pipeline, validating it first.

We can also add an entire other pipeline to this pipeline, meaning we squash the steps.

run(items, unwrap=True, **kwargs)[source]

Run the pipeline to parse the items.

pipelib.pipelines module

pipelib.wrappers module

pipelib.utils module

pipelib.utils.fileio module

pipelib.utils.fileio.copyfile(source, destination, force=True)[source]

Copy a file from a source to its destination.

pipelib.utils.fileio.creation_date(filename)[source]

Get the creation date, and fallback to modified date.

pipelib.utils.fileio.get_file_hash(image_path, algorithm='sha256')[source]

Return an sha256 hash of the file based on a criteria level.

pipelib.utils.fileio.get_tmpdir(tmpdir=None, prefix='', create=True)[source]

Get a temporary directory for an operation.

pipelib.utils.fileio.get_tmpfile(tmpdir=None, prefix='')[source]

Get a temporary file with an optional prefix.

pipelib.utils.fileio.list_modules(path)[source]

List python modules under a path.

pipelib.utils.fileio.mkdir_p(path)[source]

mkdir_p attempts to get the same functionality as mkdir -p :param path: the path to create.

pipelib.utils.fileio.mkdirp(dirnames)[source]

Create one or more directories

pipelib.utils.fileio.print_json(json_obj)[source]

Print json pretty

pipelib.utils.fileio.read_file(filename, mode='r')[source]

Read a file.

pipelib.utils.fileio.read_json(filename, mode='r')[source]

Read a json file to a dictionary.

pipelib.utils.fileio.recursive_find(base, pattern=None)[source]

Find filenames that match a particular pattern, and yield them.

pipelib.utils.fileio.write_file(filename, content, mode='w', exec=False)[source]

Write content to a filename

pipelib.utils.fileio.write_json(json_obj, filename, mode='w', print_pretty=True)[source]

Write json to a filename

pipelib.utils.terminal module

pipelib.utils.terminal.check_install(software, quiet=True, command='--version')[source]

check_install will attempt to run the singularity command, and return True if installed. The command line utils will not run without this check.

Parameters

software: the software to check if installed quiet: should we be quiet? (default True) command: the command to use to check (defaults to –version)

pipelib.utils.terminal.confirm_action(question, force=False)[source]

confirm if the user wants to perform a certain action

Parameters

question: the question that will be asked force: if the user wants to skip the prompt

pipelib.utils.terminal.confirm_uninstall(filename, force=False)[source]

confirm if the user wants to uninstall a module

Parameters

filename: the file that will be removed force: if the user wants to skip the prompt

pipelib.utils.terminal.get_installdir()[source]

get_installdir returns the installation directory of the application

pipelib.utils.terminal.get_user()[source]

Get the name of the user. We first try to import pwd, but fallback to extraction from the environment.

pipelib.utils.terminal.get_userhome()[source]

get the user home based on the effective uid. If import of pwd fails (not supported for Windows) then fall back to environment variable.

pipelib.utils.terminal.run_command(cmd, sudo=False, stream=False)[source]

run_command uses subprocess to send a command to the terminal.

Parameters

cmd: the command to send, should be a list for subprocess error_message: the error message to give to user if fails, if none specified, will alert that command failed.

pipelib.utils.terminal.which(software=None, strip_newline=True)[source]

get_install will return the path to where Singularity (or another executable) is installed.

pipelib.steps module

pipelib.steps.iter_steps()[source]

Convenince function to iterate over steps.