By Jeremiah Morrill on 8/30/2008 11:48 AM

In my last post about the WPF MediaKit library, I mentioned a little about the design goals and architecture.  I have two distinct, but complimentary class hierarchies.  The "MediaPlayerBase" class tree wraps DirectShow COM functionality into a DispatcherObject, running in it's own Dispatcher.  The "MediaElementBase" wraps the "MediaPlayerBase" into a nice XAML friendly, data-bindable FrameworkElement.

Today the class inheritance looks like this:

On the DirectShow centric side

DispatcherObject->MediaPlayerBase->MediaSeekingPlayer

...

Read More »

By Jeremiah Morrill on 8/27/2008 2:13 PM

A few of you have emailed, wondering if I'm still alive, or if I'm working more D3DImage video player.  The answer is "YES!".

I have been very busy with work, but I found some time here and there to work on what I am calling (for the lack of a better name), the "WPF Multimedia Library".  Sounds pretty self explanatory, no?  The quick synopsis, it's a lib to quickly and easily create custom video/audio players in WPF.  And hopefully some XAML enabled sound processing controls, though I have to check to see if I can release that part(it may be part of a closed source app).

So, yeah, the multimedia library, I mentioned quick and easy?  I suppose those are both subjective terms, but I have been spending a lot of time refactoring and massaging the architecture to make it as easy as possible.  Let me explain a little of where I'm at.

COM and DispatcherObject sitting in a tree...

The first thing I wanted to tackled was a tidy way of communicating with our DirectShow COM objects.  See, COM is subject to threading apartment rules of Single-Threaded-Apartment or Multi-Threaded-Apartment.  To be said simply an object created in STA can only be accessed via the same thread that created it.  An object created in MTA can be called from any *MTA* thread.  An STA thread can not talk to an MTA object and vise-versa.

So this MTA/STA ish sounds like a problem in WPF because it uses an STA UI thread and you don't want to create the DirectShow COM objects on the UI thread for threat of blocking the UI thread (bad!) when you make blocking calls to the COM objects.  But couldn't we create all the COM objects under an MTA thread?  Yes, but the WPF STA thread won't be able to access them (see previous paragraph).  The problem is solved by sub-classing DispatcherObject and running the Dispatcher on it's own thread.  To access your COM safely, you use the familiar WPF Dispatcher->Invoke/BeginInoke.  So now all COM functionality (DirectShow in this case) is neatly wrapped up in it's own DispatcherObject.  Here is an example factory method:

/// ...

Read More »

By Jeremiah Morrill on 8/17/2008 6:34 AM

I have been toiling more with the XNA libs, trying to get the...

Read More »

By Jeremiah Morrill on 8/12/2008 2:28 AM

The new D3DImage in WPF opens up worlds for WPF.  Direct3D content rendered with WPF...it doesn't...

Read More »

By Jeremiah Morrill on 8/11/2008 11:52 AM

Everyone grab a copy of .NET 3.5 SP1, its RTM.  For this RTM release I have a...

Read More »

By Jeremiah Morrill on 5/21/2008 2:41 PM

I've been playing around with the new WPF 3.5 SP1 beta lately and I wanted to test the performance of the new WriteableBitmap in comparison to the VRE.

The test simply copies the same pixel buffer to the WriteableBitmap or the VRE at a specific interval.  This way we can see the raw CPU it takes to just update pixels.

For this test, I use a 1280x720 (720p) pixel buffer at around 30FPS.  On my quad-core system,...

Read More »

By Jeremiah Morrill on 4/28/2008 8:24 AM

Pavan released his now popular ElementFlow control about a month ago.  Dissecting his code was both a pleasure and very educational (It was written much nicer than my version).  The coolest part was how he sub-classed a Panel and overlayed a Viewport3D on top.  Being a sub-class of Panel, you have all the nice built in WPF functionality that goes along with it, such as using ItemsControl class...

Read More »

By Jeremiah Morrill on 4/19/2008 8:33 AM

I've spent a lot of time tuning up the VRE to be as flexible as possible as to support advanced media scenarios, such as DVDs.  Things are not easy given the nature of DirectShow, WPF and the MediaElement (the 3 essential ingredients of VRE).  For weeks I have been pondering on the new features Microsoft is releasing for WPF, such as the pixel shaders, new WriteableBitmap and the Direct3D interop.

The Direct3D interop...

Read More »

By Jeremiah Morrill on 4/16/2008 6:11 AM

So I have pretty much finished the DirectShow Renderer, which is a glorified SampleGrabber and I started testing the DVDs with it.  For some reason Microsoft's DVD Navigator filter will only play the FBI warning and stop.  I swapped out MS's DVD navigator with CyberLink's and it played fine!  I did some tests with the "Dump" filter from the DirectShow SDK and it only did the FBI warning also.

Could this be some anti-piracy feature of Microsoft's DVD Navigator when the graph has a 3rd party...

Read More »

By Jeremiah Morrill on 4/15/2008 5:00 AM

In my last post I talked a little about how I was modifying the VRE to support it's own DirectShow Renderer.  This is to get around how crappy/inflexible the SampleGrabber really is.  The DirectShow renderer I have made is really just a glorified SampleGrabber which is very simple and probably won't be more than 20 LoC when it is finished.  This should, in theory make the VRE compatible with...

Read More »

By Jeremiah Morrill on 4/13/2008 4:02 AM

If you've been reading this blog this week, you probably noticed am attempting to make a WPF control that allows a developer to play DVDs rendered WPF.  I had some reasonable success, but the project had to be hard-coded way to much.  For instance I required 3rd party DVD filters and two (count them, TWO!) FFDShow filters to even make the DirectShow graph work.  The problem lies in the design of the VideoRendererElement.  I expected to always be able to use the SampleGrabber filter to get samples...

Read More »

By Jeremiah Morrill on 4/10/2008 9:22 AM

Here's the video of Interactive DVD menus working.  I made it out to be hard, but really it was so easy I don't want to say.  Should work fine on 3D too!

BTW, If it looks choppy, that's the encoding.  Local its smooth and I have it down to 5% CPU now :)

 

By Jeremiah Morrill on 4/10/2008 7:09 AM

I was asked by some peeps about how the multi-media pipeline (DirectShow) works and also how it relates to WPF.  I've decided to write a little post to cover all this.  I've tried to keep it as high level has possible.

DirectShow

DirectShow is the multimedia framework in Windows.  Most media players that run on windows use DirectShow, including Windows Media Player.  Ever wonder how you can install a DiVX codec and all of a sudden you can play those (legal I hope) DiVX AVI files?  How...

Read More »

By Jeremiah Morrill on 4/10/2008 3:11 AM

Here is a video of what I was able to do so far with the VideoRendererElement and a DVD.  I'm showing the DVD title screen so you know it truely is the DVD and not some rip.

*UPDATE*  Woo, got it down to 8% CPU usage, but I still have some unnecessary overhead I can get rid of.

 

By Jeremiah Morrill on 4/9/2008 3:53 AM

I've read quite a few posts (and lots of bitching) about how the MediaElement does not support DVD playback.  With the VideoRendererElement, this should be possible...but not without its hurdles.

It seems as the authors of the DVD decoder filters don't like you creating a sample sink (ie SampleGrabber filter), probably for obvious reasons, such as pirating.  I need this sample sink so I can feed the VideoRendererElement some samples to render.

I have come across some commercial DVD filters...

Read More »

By Jeremiah Morrill on 4/5/2008 2:18 AM

I've spent some time this weekend working on the VideoRendererElement, cleaning up the source, commenting, bug fixing, etc.  The source-code is now correctly in CodePlex so you can now browse it.  I also made a few breaking changes to the API to support flagging a sample as a keyframe, so compressed...

Read More »

By Jeremiah Morrill on 3/31/2008 4:23 PM

I keep going on and on about performance of the VideoRendererElement...well because I think it's important that peeps know.

As you can see in this video that the webcam is indeed in WPF space (reflection).  It is doing 640x480x30FPS live from the camera.  I show the Task Manager running about about 4 - 6% CPU.  Playing the webcam in regular graphedt.exe (not in WPF) yields about 4% CPU usage.  So most of the CPU tax is on the webcam driver itself, making VideoRendererElement almost nothing...

Read More »

By Jeremiah Morrill on 3/30/2008 11:12 PM

The VideoRendererElement..well I'm sure some of ya'll are tired of hearing about it.  But the beta is on Codeplex with a sample that shows you how to get a capture source (webcam) in WPF along with the GDI and compressed sample example.

There's some bugs left it in..and definitely a little refectoring needed.  I don't think the VideoRendererElement should inherit from Control, but maybe FrameworkElement.  I need...

Read More »

By Jeremiah Morrill on 3/28/2008 5:14 PM

I think most people know what VLC is.  I figured VLC API could be a good example of a use case example for the VideoRendererElement. 

So I remember coming across a gentleman that made a WPF player that used VLC to decode the video.  The problem is the frames were updated with this code:

videoOutput.Source = BitmapSource.Create(            width,            height,           ...

Read More »

By Jeremiah Morrill on 3/28/2008 3:46 AM

I posted up a preview build of the VideoRendererElement on codeplex here.  I also posted a notification on the WPF forums here.  I have posted warnings that it is very much a preview and very messy right now. 

Feel free to contact me about any questions/jokes on it.  My email is jeremiah[dot]morrill[at]gmail[dot]com

...

Read More »

By Jeremiah Morrill on 3/27/2008 12:06 PM

Last night, I changed the API of the VideoRendererElement to be synchronous, which makes things easier on the consumer developer, but I'm not entirely happy with how its implemented.  Though it should work just fine for production.  Here's the story:

There is a DirectShow filter I have made that works behind the scenes, that allows all the "magic" to happen.  This filter inherits from CSource and the filter's pin inherits from CSourceStream.  Really, inheriting from just CBaseFilter/CBaseOutputPin...

Read More »

By Jeremiah Morrill on 3/26/2008 1:02 AM

Here is a screen-shot of the VideoRendererElement I posted about yesterday.  What you are seeing is a basic System.Drawing.Bitmap at 640x480, with Graphics class running FillRectange(), then FillEllipse() on it.  The Bitmap's buffer is then sent to the VideoRendererElement at 30 FPS...all around 1 - 2% CPU usage.



...

Read More »

By Jeremiah Morrill on 3/25/2008 4:24 PM

When I first started using WPF, the first thing I messed with was the MediaElement, after all, I am a nut for video.  I found the MediaElement only allowed you to supply a source Uri to your media.  It sadly did not allow any API access to write your own custom video.  WriteableBitmap was more of a convenience API and marked certain death for any high-performance application.  What if somebody wished to add a web camera to their application, in which they wished to encode and display the video at...

Read More »

By Jeremiah Morrill on 3/18/2008 12:57 AM

I was pretty impressed when I saw the head tracking with the Wii remote hack.  I was also very impressed by Rick Barraza's Minority Report-like gloves used with WPF.  I wanted to see what was possible with a little computer vision.  Below is a video of my attempt at the head tracking.  As you see, the algorithm needs a LOT of fine tuning. ...

Read More »

By Jeremiah Morrill on 3/18/2008 12:35 AM

When I heard about the new, high-performance WriteableBitmap coming in .NET 3.5 my ears perked a bit.  Could this be what us video nuts need to dump the WPF MediaBridge once and for all?  Maybe...but probably not.  The new WriteableBitmap lets you write directly to the pixel buffer and tell WPF which areas are...

Read More »

By Jeremiah Morrill on 3/14/2008 4:00 PM

Someone requested to see the video for the 3D Item flow control I had.  Here it is :)

 

By Jeremiah Morrill on 3/13/2008 8:26 AM

At work, we have a small team working on a WPF application to supplement our web-based UI.  Of course, us being software developers, we can be really lame when it comes to design and I want our application to look decent until we bring a real designer on board.  I have told people that I'm good enough at design to know when something looks bad.  So I naturally scour the Internet's series of tubes, searching for other...

Read More »

By Jeremiah Morrill on 3/8/2008 2:01 AM

A long while back, I was oogling over Pavan Podila's ElementFlow.  He is unable to release I became very impatient and started to attempt to make my own.  I had these glorious ambitions to support templating, binding...Just a seamless WPF control.  It turns out that the brilliant folks at IdentityMine have already beat me to the punch with...

Read More »

By Jeremiah Morrill on 2/29/2008 5:08 PM

The Flotzam design winner was announced today and I was excited to find an email stating that I was the winner!  I was also happy to see the other submissions as there were a lot of great ones.  I hope next year they have the same/similar contest and that it gets a little more airtime because it seems as if there were only a handful of submissions this year.

Thanks go to the Mix team...

Read More »

By Jeremiah Morrill on 2/16/2008 5:48 PM

I do not think patents are very relevant to software.  Anyone remember the EOLAS vs Microsoft case that made us rewrite some of our web pages if we didn't want the "Click to activate"?  What about MS making claims against Linux?  Or the case against RIM?  Amazon has it's click-once patent.  What about click-none?  Can I patent that?  How many billions are wasted on blood sucking lawyers and payouts?  The one sure loser in patent cases is the customer.



...

Read More »