Using the Bitmovin Encoder with a Vidispine MAM

Bitmovin provide a cloud-based API for encoding, analytics and HTML5 player for MPEG-DASH and HLS. The premise behind the encoding service is parallel chunk-based transcoding which can scale to meet your needs. 

The Bitmovin encoder is a great solution for very fast encoding of media formats suitable for web delivery. If you have or are considering implementing a Vidispine API based MAM, then this blog shows how to configure the Vidispine API to use the Bitmovin encoding API.

Overview

The aim for this integration was to see if we could add Bitmovin encoding to a Vidispine MAM so the Vidispine MAM could use it as an external transcoder.

For this integration we used a number of Amazon Web Services (AWS) services to integrate the Vidispine APIS with the Bitmovin encoding API, similar results can be achieved with different architectures. (see diagram below 'Integration Architecture')

This integration consists of 3 parts:

1. Setting up the AWS services, this consists of creating two S3 buckets, an SQS queue and a Lambda function.

2. Configuring the Bitmovin encoder. This entails configuring storages and creating encoding profiles.

3. Adding an external transcoder to the Vidispine MAM.

Workflow

The diagram below ('Vidispine Integration with Bitmovin as an External Transcoder') outlines the happy path of this workflow.

1. A user triggers a job.

2. The Vidispine MAM starts a new transcode job, uploads the items source file to the Source in Bucket.

3. Once the file has finished uploading an S3 event is triggered and adds a message to the SQS queue.

4. This SQS message is picked up by a Lambda function which creates the Bitmovin encoding resource, adds the media streams, sets the muxing type and starts the encoding job all via API calls.

5. Bitmovin encoder creates the new file and moves it to our Transcode Out Bucket.

6. The Vidispine API has been polling the Transcode Out Bucket, waiting for a file to appear that meets the regex pattern we configured it to wait for.

7. Once the file from the Bitmovin encoder arrives it cleans up after itself deleting the source file from the Source S3 Bucket.

8. The Vidispine API then completes the transcode job setting its status to complete.

AWS Services

This diagram highlights how AWS’s services form the core of the integration, here is a breakdown of how each service is being used.

S3

We use two S3 buckets, the first to provide the Bitmovin encoder with source files and the second for the Bitmovin encoder to move the transcoded files to.

The S3 bucket which is used for source files also provides an important secondary role, once a file has been successfully uploaded an S3 event is triggered. This bucket has been configured to POST an SQS message to our SQS queue.

Click here for information on how to configure S3 event notifications.

SQS

We are using an SQS queue as our job queue, whenever a file is uploaded to the Source File S3 bucket an SQS message is added to this queue and is consumed by a Lambda function which polls this queue for new jobs.

SQS docs are located here.

Lambda 

Our Lambda function polls the SQS queue for new messages, once it picks up a message,  it will then use the data in that message to make the API calls which are required to create and start a Bitmovin encode job.

Lambda docs are located here.

Configuring Bitmovin

Getting Started

For this integration we need to configure the Bitmovin encoder to use the two S3 storages that we have created in AWS. This can be done via the Bitmovin UI or via IP requests. But first we need to create an account and get a Bitmovin API key. 

Click here for a guide that explains how to do this.

Setting your Input and Output Destinations

Next, we have to configure the input and output destinations that we want to use. The Bitmovin docs explain how to do this.

Inputs
Outputs 

Creating your Codecs

The final step in the Bitmovin encoder config is to create the codecs that you want to use the API docs. Click here to see how this is done.

Adding the Bitmovin Encoder to Vidispine as an External Transcode

Now that we have configured the Bitmovin encoder and our AWS services we can setup the Vidispine external transcoder by making the following API call to Vidispine:

POST /resource/externalTranscoder/
Content-Type: application/xml

<ResourceDocument xmlns="http://xml.vidispine.com/schema/vidispine">
    <externalTranscoder>
        <source>s3:///external-transcoder.aws.amazon/source/</source>
        <destination>s3:///external-transcoder.aws.amazon /destination/</destination>
        <shapeTag>external-format</shapeTag>
        <timeout>60000</timeout>
        <regex>.*demo.*</regex>
   </externalTranscoder>
</ResourceDocument>

More information on how to do this can be found here.

How to Create a Bitmovin Encode Job
To create an encode job in Bitmovin you need to make a series of API calls. These API calls are being made by the AWS Lambda function once it has parsed an SQS message.

For this integration we are doing a simple transcode from XDCAM 422 to the MP4 profile we created in Bitmovin, to do this we need only need to make four API calls:

How to create the encoding package

POST https://api.bitmovin.com/v1/encoding/encodings 

The response from this request will include the encodingId (for this example its 55c7bf89-f46d-4d05-8658-05e974135416 and is used in the subsequent URI’s) 

Docs: 

Add your required streams

POST https://api.bitmovin.com/v1/encoding/encodings/55c7bf89-f46d-4d05-8658-05e974135416/streams/

Docs:

Specify which file format you want to create, in our case it’s an MP4

POST https://api.bitmovin.com/v1/encoding/encodings/55c7bf89-f46d-4d05-8658-05e974135416/muxings/mp4/

Docs: 

We can now trigger a transcode from the Vidispine MAM and the Bitmovin encoder will process the file and create it in our S3 bucket.

Click here for more information about the API calls I use for this integration.

Conclusion

By integrating the Vidispine MAM with Bitmovin’s encoder you are able to make use of a powerful transcoding engine without having the overheads that traditionally come with running a transcoding farm. You also get the flexibility of being able to use the Bitmovin Encoder to transcode items that have been ingested into Vidispine’s MAM.