The core module is home to code that is important to both the media_nommer.ec2nommerd and media_nommer.feederd top-level modules. This is best thought of as a lower level API that all of the pieces of the encoding system use to communicate and interact with one another.
Note
There should be nothing that is specific to just ec2nommerd or just feederd here if at all possible.
Storage backends are used to handle download and uploading content to a number of different protocols in an abstracted manner.
media-nommer uses URI strings to represent media locations, whether it be the file to download and encode, or where the output should be uploaded to. A reference to the correct backend for a URI can be found using the get_backend_for_uri() function.
Given a protocol string, return the storage backend that has been tasked with serving said protocol.
Parameters: | protocol (str) – A protocol string like ‘http’, ‘ftp’, or ‘s3’. |
---|---|
Returns: | A storage backend for the specified protocol. |
Given a URI string , return a reference to the storage backend class capable of interacting with the protocol seen in the URI.
Parameters: | uri (str) – The URI to find the appropriate storage backend for. |
---|---|
Return type: | StorageBackend |
Returns: | A reference to the backend class to use with this URI. |
This module contains an S3Backend class for working with URIs that have an s3:// protocol specified.
Abstracts access to S3 via the common set of file storage backend methods.
The job state backend provides a simple API for feederd and ec2nommerd to store and retrieve job state information. This can be everything from the list of currently un-finished jobs, to the down-to-the-minute status of individual jobs.
There are two pieces to the job state backend:
Represents a single encoding job. This class handles the serialization and de-serialization involved when saving and loading encoding jobs to and from SimpleDB.
Tip
You generally won’t be instantiating these objects yourself. To retrieve an existing job, you may use JobStateBackend.get_job_object_from_id().
Parameters: |
|
---|
Returns True if this job is in a finished state.
Return type: | bool |
---|---|
Returns: | True if the job is in a finished state, or False if not. |
Serializes and saves the job to SimpleDB. In the case of a newly instantiated job, also handles queueing the job up into the new job queue.
Return type: | str |
---|---|
Returns: | The unique ID of the job. |
Abstracts storing and retrieving job state information to and from SimpleDB. Jobs are represented through the EncodingJob class, which are instantiated and returned as needed.
Any jobs in the following states are considered “finished” in that we won’t do anything else with them. This is a list of strings.
All possible job states as a list of strings.
Given a job’s unique ID, return an EncodingJob instance.
TODO: Make this raise a more specific exception.
Parameters: | unique_id (str) – An EncodingJob‘s unique ID. |
---|
Queries SimpleDB for a list of pending jobs that have not yet been finished.
Return type: | list |
---|---|
Returns: | A list of unfinished EncodingJob objects. |
Pops any new jobs from the job queue.
Warning
Once jobs are popped from a queue and delete() is ran on the message, they are gone for good. Be careful to handle errors in the methods higher on the call stack that use this method.
Parameters: | num_to_pop (int) – Pop up to this many jobs from the queue at once. This can be up to 10, as per SimpleDB limitations. |
---|---|
Return type: | list |
Returns: | A list of EncodingJob objects. |
Pops any recent state changes from the queue.
Warning
Once jobs are popped from a queue and delete() is ran on the message, they are gone for good. Be careful to handle errors in the methods higher on the call stack that use this method.
Parameters: | num_to_pop (int) – Pop up to this many jobs from the queue at once. This can be up to 10, as per SimpleDB limitations. |
---|---|
Return type: | list |
Returns: | A list of EncodingJob objects. |