jobcontrol.interfaces

Interfaces for NEW jobcontrol objects.

Data model:

Build   id SERIAL
-----   job_id TEXT
        start_time TIMESTAMP
        end_time TIMESTAMP
        started BOOLEAN
        finished BOOLEAN
        success BOOLEAN
        skipped BOOLEAN
        job_config TEXT (YAML)
            Copy of the job configuration whan the build was started
        build_config TEXT (YAML)
            Extra configuration, such as dependency build "pinning"
        retval BINARY (Pickled return value)
        exception BINARY
            Pickled exception object (or None)
        exception_tb BINARY
            Pickled TracebackInfo object

Build progress
--------------
        build_id INTEGER (references Build.id)
        group_name VARCHAR(128)
            Name of the "progress group" (separated by '::')
        current INTEGER
            Current progress value
        total INTEGER
            Total progress value
        status_line TEXT
            An optional line of text describing current state
        UNIQUE constraint on (build_id, group_name)

Log     id SERIAL
---     build_id INTEGER (references Build.id)
        created TIMESTAMP
        level INTEGER
        record BINARY
            Pickled LogRecord
        exception_tb BINARY
            Pickled TracebackInfo object

Job configuration:

The job configuration is stored as a YAML-serialized dict.

Recognised keys are:

• function in module:function format, specify the function to be called
• args a list of arguments to be passed to the function
• kwargs a dict of keyword arguments to be passed to the function
• title a descriptive title, to be shown on the interfaces
• notes notes, to be shown in interfaces (in restructured text)
• dependencies list of dependency job names

Additionally, args/kwargs may contain references to return value of dependency builds, by using the !retval <name> syntax.

Exception traceback serialization

To be used both in build records and associated with log messages containing an exception.

We want to include the following information:

• Details about the call stack, as in normal tracebacks: filename, line number, function name, line of code (plus some context)
• Local variables: we are not guaranteed we can safely pickle / unpickle arbitrary values; moreover this might result in huge fields, etc. So our better chance is to just store a dictionary mapping names to repr()s of the values (trimmed to a – large – maximum length, just to be on the safe side).

class jobcontrol.interfaces.StorageBase

classmethod from_url(url)

install()

uninstall()

get_job_builds(job_id, started=None, finished=None, success=None, skipped=None, order='asc', limit=100)

Iterate over all the builds for a job, sorted by date, according to the order specified by order.

Parameters:

• job_id – The job id
• started – If set to a boolean, filter on the “started” field
• finished – If set to a boolean, filter on the “finished” field
• success – If set to a boolean, filter on the “success” field
• skipped – If set to a boolean, filter on the “skipped” field
• order – ‘asc’ (default) or ‘desc’
• limit – only return the first limit builds

Yield:

Dictionaries representing build information

create_build(job_id, job_config, build_config)

Create a build.

Parameters:

• job_id – The job for which a build should be started
• job_config – The job configuration (function, args, kwargs, ..) to be copied inside the object (we will use this from now on).
• build_config –

  Build configuration, containing things like dependency build pinning, etc.

  • dependency_builds: dict mapping job ids to build ids, or None to indicate “create a new build” for this job.

Returns:

the build id

get_build(build_id)

Get information about a build.

Returns:the build information, as a dict

delete_build(build_id)

Delete a build, by id.

start_build(build_id)

Register a build execution start.

finish_build(build_id, success=None, skipped=None, retval=None, exception=None, exception_tb=None)

Register a build execution end.

finish_build_with_exception(build_id)

update_build_progress(build_id, current, total)

report_build_progress(build_id, current, total, group_name='', status_line='')

Report progress for a build.

Parameters:

• build_id – The build id for which to report progress
• current – The current number of “steps” done
• total – The total amount of “steps”
• group_name – Optionally, a name used to nest multiple progress “levels”. A tuple (or string separated by ‘::’ can be used to specify multiple “nesting” levels)
• status_line – Optionally, a line of text indicating the current build status.

get_build_progress_info(build_id)

Return progress information for a build.

Returns:a list of tuples: (name, current, total, status_line)

get_latest_successful_build(job_id)

Helper method to retrieve the latest successful build for a given job. Calls get_job_builds() in the background.

Returns:information about the build, as a dict

log_message(build_id, record)

Store a log record associated with a build.

prune_log_messages(job_id=None, build_id=None, max_age=None, level=None)

Delete (old) log messages.

Parameters:

• job_id – If specified, only delete messages for this job
• build_id – If specified, only delete messages for this build
• max_age – If specified, only delete log messages with an age greater than this one (in seconds)
• level – If specified, only delete log messages with a level equal or minor to this one

iter_log_messages(build_id=None, max_date=None, min_date=None, min_level=None)

Iterate over log messages, applying some filters.

Parameters:

• build_id – If specified, only return messages for this build
• max_date – If specified, only return messages newer than this date
• min_date – If specified, only return messages older than this date
• min_level – If specified, only return messages with a level at least equal to this one

pack(obj, safe=False)

pack_log_record(record)

Pack a log record.

This special-cased function is meant to gracefully handle cases of log messages not being serializable, usually due to some “attr” or the attached exception not being serializable.

pack_exception(exception)

unpack(obj, safe=False)

yaml_pack(obj)

yaml_unpack(obj)