jasmin_arc package

Submodules

jasmin_arc.arc_interface module

class jasmin_arc.arc_interface.ArcInterface(config_path=None, log=<open file '<stdout>', mode 'w'>, log_level=<LogLevels.INFO: <class 'sphinx.ext.autodoc.INFO'>>)[source]

Bases: object

Class to handle interactions with the ARC-CE server

__init__(config_path=None, log=<open file '<stdout>', mode 'w'>, log_level=<LogLevels.INFO: <class 'sphinx.ext.autodoc.INFO'>>)[source]

Create an object to interface with the ARC server.

Parameters:
  • config_path – Path to config JSON file, or None to use the default settings
  • log – File-like object to write log messages to, or None to disable logging. Use sys.stdout or sys.stderr to print messages (default: sys.stdout).
  • log_level – The level of detail logs should show (default: LogLevels.INFO). See LogLevels for the available levels
Raises:

InvalidConfigError – if config is not valid JSON or is otherwise invalid

cancel_job(job_id)[source]

Cancel the given job

Parameters:job_id – ID of the job as returned by submit_job
Raises:JobNotFoundError – if no job with the given ID could be found
create_proxy()[source]

Use arcproxy to create a proxy certificate from private key and certificate, and save it to the path given in the config

Raises:ProxyGenerationError – if the certificate cannot be generated
create_user_config()[source]

Create a user config for use with ARC client

Returns:An instance of arc.UserConfig containing information about the necessary keys, certificates and proxy files
get_job(job_id)[source]

Return an instance of arc.Job representing the job with the given ID

Parameters:job_id – ID of the job as returned by submit_job
Raises:JobNotFoundError – if no job with the given ID could be found
Returns:Instance of arc.Job representing the job
get_job_descriptions(jsdl)[source]

Return an instance of arc.JobDescriptionList containing the job described by the given JSDL

Parameters:jsdl – String containing the job description in JSDL format
get_job_status(job_id)[source]

Return the status of the given job

Parameters:job_id – ID of the job as returned by submit_job
Raises:JobNotFoundError – if no job with the given ID could be found
Returns:The status of the job (see JobStatuses for the available values)
get_user_config()[source]

Return the cached user config, or create a new one. Also check if proxy has expired, and create a new one if so

Returns:An instance of arc.UserConfig (see create_user_config)
save_job_outputs(job_id)[source]

Retrieve output files from a job and save them to a temp directory. The file/directory specified in OUTPUT_FILE will be downloaded, and stdout and stderr outputs are saved as stdout.txt and stderr.txt respectively.

Parameters:job_id – ID of the job as returned by submit_job
Raises:JobNotFoundError – if no job with the given ID could be found
Returns:Path to the directory the output files were saved in, or None if no files were saved
submit_job(executable, args=[], input_files=[])[source]

Submit a job and return the job ID

Parameters:
  • executable – The command to run on the LOTUS cluster
  • args – List of arguments to pass to the executable
  • input_files – A list of paths to local files to copy to the remote session directory (the directory the job will run from on JASMIN)
Raises:
Returns:

Job ID

jasmin_arc.config module

class jasmin_arc.config.ConnectionConfig(config_dict, logger=None)[source]

Bases: object

Class to define available config options and their default values

ARCPROXY_PATH = '/usr/bin/arcproxy'

Path to the arcproxy binary, which is used to generate a proxy certificate from the private key and certificate

ARC_SERVER = 'jasmin-ce.ceda.ac.uk:60000/arex'

URL to the ARC server

CERTS_DIR = '/etc/grid-security/certificates'

Path to directory containing trusted CA certificates

CLIENT_CERT = '~/.arc/usercert.pem'

Path to grid certificate file

CLIENT_KEY = '~/.arc/userkey-nopass.pem'

Path to the private key file associated with your grid certificate

JOBS_INFO_FILE = '~/.arc/jobs.dat'

Path to job information file used by ARC client tools (arcstat, arcget etc) to load information about submitted jobs

OUTPUT_FILE = 'output'

The name of the file/directory to download when retrieving job outputs.

PROXY_FILE = '/tmp/arcproxy_file'

Path to save the generated proxy certificate to

PROXY_RENEWAL_THRESHOLD = 10

The number of seconds the proxy file can have till expiry before a new proxy is automatically generated

PROXY_VALIDITY_PERIOD = 43200

Number of seconds to set the validity period to when generating a proxy file (default: 12 hours)

__init__(config_dict, logger=None)[source]
Parameters:
  • config_dict – A dictionary containing options to override. Each key should be one of the default options listed above
  • logger – An instance of arc.Logger used to log warnings about invalid config options

jasmin_arc.constants module

jasmin_arc.constants.ARC_STATUS_MAPPING = {'Undefined': <JobStatuses.FAILED: 'failed'>, 'Queuing': <JobStatuses.NOT_STARTED: 'not started'>, 'Failed': <JobStatuses.FAILED: 'failed'>, 'Other': <JobStatuses.FAILED: 'failed'>, 'Submitting': <JobStatuses.NOT_SUBMITTED: 'not submitted'>, 'Finishing': <JobStatuses.IN_PROGRESS: 'in progress'>, 'Accepted': <JobStatuses.NOT_SUBMITTED: 'not submitted'>, 'Deleted': <JobStatuses.FAILED: 'failed'>, 'Running': <JobStatuses.IN_PROGRESS: 'in progress'>, 'Finished': <JobStatuses.COMPLETED: 'completed'>, 'Preparing': <JobStatuses.NOT_SUBMITTED: 'not submitted'>, 'Hold': <JobStatuses.IN_PROGRESS: 'in progress'>, 'Killed': <JobStatuses.FAILED: 'failed'>}

Map job statuses as given by ARC to statuses in our simplified model

class jasmin_arc.constants.JobStatuses[source]

Bases: enum.Enum

Possible values for job status

COMPLETED = 'completed'
FAILED = 'failed'
IN_PROGRESS = 'in progress'
NOT_STARTED = 'not started'
NOT_SUBMITTED = 'not submitted'
class jasmin_arc.constants.LogLevels[source]

Bases: enum.Enum

Log levels for specifying the level of details to include in the logs. These levels and descriptions come straight from the ARC library

DEBUG = <class 'sphinx.ext.autodoc.DEBUG'>

DEBUG level designates finer-grained informational events which should only be used for debugging purposes

ERROR = <class 'sphinx.ext.autodoc.ERROR'>

ERROR level designates error events that might still allow the application to continue running

FATAL = <class 'sphinx.ext.autodoc.FATAL'>

FATAL level designates very severe error events that will presumably lead the application to abort

INFO = <class 'sphinx.ext.autodoc.INFO'>

INFO level designates informational messages that highlight the progress of the application at coarse-grained level

VERBOSE = <class 'sphinx.ext.autodoc.VERBOSE'>

VERBOSE level designates fine-grained informational events that will give additional information about the application

WARNING = <class 'sphinx.ext.autodoc.WARNING'>

WARNING level designates potentially harmful situations

jasmin_arc.exceptions module

exception jasmin_arc.exceptions.InputFileError[source]

Bases: exceptions.Exception

Input file does not exist or is not a file

exception jasmin_arc.exceptions.InvalidConfigError[source]

Bases: exceptions.Exception

Exception thrown when config file is invalid

exception jasmin_arc.exceptions.InvalidJobDescription[source]

Bases: exceptions.Exception

Job description XML is invalid

exception jasmin_arc.exceptions.JobNotFoundError[source]

Bases: exceptions.Exception

A job with the given ID could not be found

exception jasmin_arc.exceptions.JobSubmissionError[source]

Bases: exceptions.Exception

Job could not be submitted to any targets

exception jasmin_arc.exceptions.NoTargetsAvailableError[source]

Bases: exceptions.Exception

No execution targets could be found to submit jobs to

exception jasmin_arc.exceptions.ProxyGenerationError[source]

Bases: exceptions.Exception

Proxy could not be created from key and certificate

Module contents