Pixel Bender + Video = Killer Runtime Effects

I’ve been playing with Pixel Bender a bit and noticed that there weren’t really any examples (I could find) that illustrated effects being applied to video. So I gathered together a whole slew of kernels from the interweb, downloaded a copy of Dancing Matt and shoehorned them into a media player that allows you to select and apply the effects during video playback.

Instructions: Press the “play” button, then select the effect you’d like to see applied to the video.

Get Adobe Flash player

The ten thousand dollar question is what’s the value outside of gratuitous eye candy? It seems that in some cases you might want to apply a particular stylistic treatment (wouldn’t it be great if the cartoon effect that ships with After Effects would compile for Flash Player). There’s also the opportunity to animate the filters and create some really killer transition effects–anyone with a media player component or video editing / mashup RIA ought to see the potential there. And that’s just the tip of the iceberg, I’m sure there are far grander ideas being born as I write this.

FYI – All of the filters were gathered up out on the interweb (mesh, exchange, pixelero).

21 thoughts on “Pixel Bender + Video = Killer Runtime Effects

  1. Brooks, nice demo showing off all the current effects.

    One of the challenges with the real-time BixelBender effects and video is the overall processor load. For me, it was no issue because I am sitting on a 8-core box, but even then I watched the load jump to nearly 18% across all cores. Imagine trying to deploy an app to a large audience and how the user experience would be. Probably not pretty.

    What I do see is some interesting prospects for media creation and real-time editing solutions. One could be an AIR app that allows people to edit video, apply filters and transitions and then export the FLV (or AfterEffects.com). Another example would be for VJs, who want to apply real-time mixing for shows and/or installations. For them, they understand that the system will put a load on their box… much smaller and controlled market.

    As machines get more powerful, I can see this in more populace based applications but for now I feel the cost may outweigh the benefit of just rendering the effect and distributing the results.

  2. @James -God point. Performance is definitely an issue and I wouldn’t recommend just applying wholesale effects during the entirety of video. I could see, short transitions (pre-roll, post-roll) with filters that have been optimized. It also seems like you could do some very fun stuff with interactions (roll overs, etc.) and thumbnails.

    Hopefully round 2 of pixel bender will see the processing offloaded to the GPU whenever possible.

    Brooks

  3. Awesome stuff. I particularly like the tube vision and focusing linear blur, since they have definite useful functionality imo.

    I’m sure someone will answer that 10k question. There’s always creative uses for things that seem gratuitous at first. And when the tech demos are shown, people groan because they think it’s gonna be 1995 with the scrolling marquee and flashing text all over again. But then someone strolls in months later and creates something absolutely badass with the effects everyone laughed about, then people start to copy that brilliant person, then standards are created.

    People raised eyebrows at all the pixel shaders and 3d effects that WPF can do, for instance, but then some people created absolutely badass and unquestionably useful UI effects with them and the laughter subsided.

  4. Very interesting question about how we are going to use it. One thing I know is the scale line effect can definitely be used for playing full screen video. It greatly increases the video quality, in a cheating way.

    Let’s wait for some genius to find out how to use some other in a genius way, then we just need to kick ourselves “why didn’t I think of it in that way?”

  5. @Jey – the lightbeam filter was built by Petri Leskinen. He’s posted a number of filters on the exchange and on hist flickr stream:

    http://www.flickr.com/photos/pixelero/sets/72157603676008625/

    If you chase the links on his flickr stream you’ll eventually find this link to the source:

    http://labs.adobe.com/wiki/index.php/Lightbeam_hydra

    He’s got some killer filters, but quite a few need to be tweaked to get them to compile with the release build of Pixel Bender.

  6. can you please tell me we the TV pixelator filter is from. I had a look at all the sites you listed but didn’t see it. Sorry if I missed it.

  7. It would indeed be great if you’d share your code, as it’s a hassle wading through pixel benders only to find that they’re broken, no preview is available, they don’t compile, or whatever. Having a bunch of them that are labeled, preview-able and known to work would be really useful.

    So we don’t really need the source for your media player, but a zip of the filters you used would prevent a lot of hunting around on the net. Hope you’ll do it! Thanks…

  8. @Scuty / Howdy – I definitely don’t have a problem sharing my code, but in this case the Pixel Bender kernels are not my code and I just don’t have the time to gather permissions from all of the various authors. When I get a chance I’ll gather up the compiled kernels and drop a link here–hopefully that doesn’t put me at odds with the authors of the kernels).

  9. Where can I find the random pixelation script? I tried googling “random pixelation” and pixel bender but to no avail.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>