Ingest HLS Stream into Mediapackage

Elemental Mediaconnect -> Elemental Medialive -> Elemental Mediapackage

We’ve all seen this workflow before. A tried and tested broadcasting workflow on the AWS cloud. And to be honest, the services work very well together. Recently on a project we’re working on needed something between the encoder(Medialive) and the packager(Mediapackage) to process segmented video clips before sending it on its way to mediapackage. A simple enough architecture.

Medialive can put files into an S3 bucket which as you can see here is being used to trigger a Lambda that does the processing and sending it to Mediapackage.

Mediapackage provides you with an HLS ingest endpoint to push your encoded streams to. But we’ve noticed the documentation not mentioning how the stream files need to be sent to the endpoints (more here). Note that the ingest server is a webDav server which works great for storing files.

We tried making PUT requests to the endpoint with the manifest and transport stream as the payload. Strangely enough, we were getting 201 from the server but Mediapackage was logging a 401 for every file that we sent despite responding with a 201 externally. And when we try to fetch the packaged manifest, we keep getting a 404 back from Mediapackage origin endpoint.

We couldn’t find any documentation on the expected file name structure that Mediapackage requires in order to do its Just in Time packaging. Hence this article to help anyone trying to do the same.

We were using Python as the runtime for our Lambda function. The following url is an example Mediapackage ingest endpoint.

https://some\_id.mediapackage.some\_region.amazonaws.com/in/v2/some\_id/some\_id/channel

Here’s the function that rewrites our url:

def __getIngestUrlForMediaPackage(mediaPackageUrl, fileName):
return f"{mediaPackageUrl}.m3u8" if __isRootManifestFile(fileName) else mediaPackageUrl.replace('channel', fileName)

Turns out for root manifest, Mediapackage expects the file to be PUT into the following endpoint.

https://some\_id.mediapackage.some\_region.amazonaws.com/in/v2/some\_id/some\_id/channel.m3u8

And for your segments and other sub manifests, it expects them without the channel path.

https://some\_id.mediapackage.some\_region.amazonaws.com/in/v2/some\_id/some\_id/sub\_manifest.m3u8

https://some\_id.mediapackage.some\_region.amazonaws.com/in/v2/some\_id/some\_id/segment.ts

With that set, we can make that coveted PUT request to Mediapackage.

def sendWebDavRequest(webDavRequestVars, fileName, data, headers = {}):
response = requests.put(
    webDavRequestVars["ingestUrl"],
    data=data,
    auth=HTTPDigestAuth(webDavRequestVars["username"], webDavRequestVars["password"]),
    headers=__getHeaderValues(headers)
)

The header values are not important here. Keep in mind that Mediapackage expects the authentication mechanism to be HTTPDigestAuth which Python requests library already exports.

And Voila. We get a working stream.

We’ve tried asking the community at stackoverflow and aws repost.