Getting Down and Dirty with XMP and AS3

One of the best ways to get started using XMP is to take a look at how Adobe is using it to embed speech-to-text data inside video files using Adobe Premiere.

As Flash Platform devs, the easiest way to get access to the XMP data in a video file created by Adobe Premiere is to make sure you handle the onXMPData callback that NetStream fires (Flash Player 10 only). One way to do this is by extending NetStream and adding a callback handler method that looks like this:

// callback handler inside a NetStream subclass dispatches a custom event that  
// forwards the xmp data found in the callback handler
public function onXMPData( info:Object ):void
{
	dispatchEvent( new XMPEvent( XMPEvent.XMP_STRING, info.data, true ) ); 
}

Next you need to make sure you’re listening for the XMPEvent, then load the MPEG-4 AVC / F4V / FLV media:

// pseudo code
var video:NetStream = new NetStream( connection );
video.addEventListener( XMPEvent.XMP_STRING, xmpHandler, false, 0, true );
video.play( "h264_captions.f4v" );

Now its time to see what XMP looks like. We’ll use the AS3 XMP library that Adobe has up on labs:

private function xmpHandler( e:XMPEvent ):void
{
	// XMPMeta is root data structure for xmp data. Pass in the xmp data
	// we received from the NetStream callback
	var xmp:XMPMeta = new XMPMeta( e.xmp );
 
	 var xml:XML = xmp.serializeToXML();
	 trace( xml.toXMLString() );
}

The output of the previous trace should look something like this (full example xml available here):

As you can see in the example above the much of the XMP information is stored in the Adobe Dynamic Media schema that uses the xmpDM namespace.

XMP itself builds on top of another metadata framework called RDF (Rich Description Framework). You can think of RDF as the raw building blocks or data structures necessary to generically describe data.

The gigantic mess of RDF and XMP namespace elements that need to be mixed and matched can be overwhelming, but keep yer cool–Adobe’s AS3 XMP library makes it relatively simple to parse, modify or create XMP metadata.

First off let’s come to a basic understanding of the core RDF elements used in XMP:

• Description – the basic data structure of RDF, this sucka holds just about anything.
• Bag – an array structure. I just think of these as unordered all purposes arrays.
• Seq – an array structure that’s ordinal by nature.
• li – an item. If yer a Bag or Seq and aren’t just an empty vessel then plenty of these will be present.

Luckily, Adobe’s AS3 XMP library has a similar ActionScript object data model:

• XMPStruct – the core data structure of XMP.
• XMPArray – there are bag and seq flavors, but for the most part these are implicitly derived.

Let’s take a look at the speech-to-text data again and see if we can piece together some of the details.

Now lets see how we could use this information to parse the XMP data and extract values:

private function xmpHandler( e:XMPEvent ):void
{
    var xmp:XMPMeta = new XMPMeta( e.xmp );
    // define a namespace that will allow us to "dot down" into the data
    // the XMP lib provides constants for many of the common XMP schemas
    var xmpDM:Namespace = XMPConst.xmpDM; 
 
    // "Tracks" is a Bag array containing XMPStruct types.
    // Let's take a look at how we can parse out the values in
    // first XMPStruct. NOTE: All XMP arrays start at 1 rather than 0.
    var track1:XMPStruct = xmp.xmpDM::Tracks[1];
    trace( track1.xmpDM::trackName );
    trace( track1.xmpDM::frameRate );
 
    // the second struct contains the array of speech-to-text
    // markers that we're interested in.
    var track2:XMPStruct = xmp.xmpDM::Tracks[2];
    var markers:XMPArray = track2.xmpDM::markers;
    var item:XMPStruct;
 
    for ( var i:int = 1; i < markers.length + 1; i++ )
    {
        item = markers[i]; //  a reference to the current marker of type XMPStruct
 
        // now we can simply look at all of the property values for each 
        // item in the array
        trace( "startTime: " + item.xmpDM::startTime );
        trace( "duration: " + item.xmpDM::duration );
        trace( "name: " + item.xmpDM::name );
        trace( "speaker: " + item.xmpDM::speaker );
        trace( "probability: " + item.xmpDM::probability + "\n");
    }
}

Ok, so that wasn’t too bad was it? Armed with some basic RDF knowledge, the Adobe AS3 XMP lib actually makes it pretty easy to parse out any XMP metadata you’re likely to run into. But we can do better than that, right? Let’s use the XMP lib and our reverse engineered knowledge of the Adobe XMP Dynamic Media schema to create some XMP marker data from scratch.

private function createXMPMarkers():void
{
    // create an xmp object and create a namespace for the Adobe DynamicMedia schema
    var xmp:XMPMeta = new XMPMeta();
    var xmpDM:Namespace = XMPConst.xmpDM;
 
    // create some markers and set some properties
    var item1:XMPStruct = new XMPStruct();
    item1.xmpDM::name = "XMP Standard Schemas";
    item1.xmpDM::startTime = 0;
    item1.xmpDM::duration = 15000;
    item1.xmpDM::type = "Chapter";
 
    var item2:XMPStruct = new XMPStruct();
    item2.xmpDM::name = "Dublin Core schema";
    item2.xmpDM::startTime = 0;
    item2.xmpDM::duration = 9999;
    item2.xmpDM::type = "Index";
 
    var item3:XMPStruct = new XMPStruct();
    item3.xmpDM::name = "XMP Dynamic Media schema";
    item3.xmpDM::startTime = 10000;
    item3.xmpDM::duration = 5000;
    item3.xmpDM::type = "Index";
 
    var item4:XMPStruct = new XMPStruct();
    item4.xmpDM::name = "Specialized Schema";
    item4.xmpDM::startTime = 15001;
    item4.xmpDM::duration = 50000;
    item4.xmpDM::type = "Chapter";
 
    // create an sequenced array to hold all of the markers
    var markers:XMPArray = XMPArray.newSeq();
    markers.append( item1 );
    markers.append( item2 );
    markers.append( item3 );
    markers.append( item4 );
 
    //Create Tracks Struct
    var tracks:XMPStruct = new XMPStruct();
    tracks.xmpDM::trackName = "TOC Markers";
    tracks.xmpDM::markers = markers;
 
    // Add the "Tracks" property to the xmp object and set its value equal to the track variable.
    xmp.xmpDM::Tracks[1] = tracks;
 
    var xml:XML = xmp.serializeToXML();
    trace( xml.toXMLString() );
}

That’s it for now kids – y’all should be XMP parsing and generating machines by now. Look for an upcoming tutorial on creating and using custom XMP schemas / namespaces to store application specific metadata in your media files.

ThumbGenie 1.3.0 Update = Designerfied

There’s an updated version of ThumbGenie in the wild that sports a new skin designed by Nick Gorsline (aka Gorz). It was the first Flex component skin for Nick and first Illustrator-to-Flex workflow for moi. I think after after the experience we’re both looking forward to some Catalyst.

In addition to the new look, videos are now scaled in the preview window to match the output dimensions. This should make it easier to get a handle on what thumbnails of various sizes will actually look like.

Creating Professional Screencasts With Jing Pro and iMovie

In the past I’ve shown examples of the type of compelling screencast content you can create with the combination of Jing Pro and iMovie. Today, I’ve created a quick tutorial (7:30) that illustrates how to do the following in iMovie:

• Import Jing video.
• Create and select clips.
• Add transitions.
• Use lower thirds.
• Introduce your screencast with a title clip.
• Create credits.
• Export an HD MPEG-4 video.

Go fullscreen to see the video in full 1:1 pixel clarity.

Flex Skins with Illustrator CS4

Looking for a quick intro to building Flex components with Illustrator? Check out this short video.

Go fullscreen to see the video in full 1:1 pixel clarity.

Embracing Unwanted Change – Video Quotes From Michael Rosenblum

Michael Rosenblum is one of my favs–the man has deep insights into the decline of old media and the ascendency of video on web. I caught a link to a speech he gave to news execs in the UK and just wanted to share a couple of my favorite quotes.

No one wants change–it screws up your business, but its inevitable and how you react defines you (1:10):

Don’t just have contigency plans for the Internet–embrace it (1:17):