Login
 
Archive
Links
Search
Blog List
There are no categories in this blog.
 
May 19

Written by: Jeremiah Morrill
5/19/2007 6:40 PM

This code has now been replaced by v2, called the VideoRendererElement.  You can find it here:

http://www.codeplex.com/VideoRendererElement

 

Get Reference to MediaElement's DirectShow Graph and Manipulate it

For the past six or seven months, I have been gorging myself with the Windows Presentation Foundation framework.  It's embarrassing, but I have dreams and nightmares about Dependency Properties and one time I swear the dream was vector based...

This WPF technology has me smittin, but it's not without it's downsides.  One specific drawback I want to cover is the WPF's lack of low-level video support.  Even more specifically, direct access to a bitmap image buffers.  Now before you tar and feather me and say "Wait!  What about CopyPixels() and BitmapSource.Create()?", I just want to say that this is a terribly inefficient way to do image manipulation.  For example, a 640x480x3bpp equals 921,600 bytes.  Multiply that by 30fps and you have yourself one cpu hog of a video player.

WPF has a video player that sits on top of WMP.  This player brings video into the WPF universe, so you can animate it, transform it or even use it as a texture on 3D geometry.  It does a pretty good job with the CPU, but unfortunately the API is so high-level, there is not much customization possible.  But because it uses WMP on the backend (and eventually uses DirectShow), it can playback any media on your computer that has a supported CODEC installed.

So what about people that need to build their OWN directshow graph.  For example:  A TV Tuner application or a web camera viewer.  What if you wanted to do image adjustments on the video using your own custom filter?  It really is a near impossibility in WPF.

This drove me to make a solution..even if it was a form of hackery.  So I submitted a demo project to the the WPF community forums on how to get reference to the underlying DirectShow graph within that runs "behind the scenes" in the WPF MediaElement and MediaPlayer classes.

The idea is pretty simple.  I register a custom "Dummy" filter with the system.  Then I tell WPF to load up a custom media URL scheme, "MediaBridge://Something".  This causes WPF to load up my dummy filter.  I then make a callback to the main application using a static method/delegates, passing reference to the DirectShow graph.  Then the filter-graph can be constructed by the programmer and WPF/MediaPlayer doesn't know the difference.

 This webcam makes me look greasy!

This is a live webcam brought into WPF using my MediaBridge hack.  I swear I don't look that greasy IRL.

WPF Modified Graph

The modified MediaElement's Graph.  You can see my dummy filter is pretty much a bastard.

I still was unhappy with the solution as it felt a little hokey.  The 'holy-grail' would be to figure out how to get low-level, high-performance access to a bitmap in WPF and to be able to make our own video players, out side of Microsoft's MediaElement sandbox.  I hope to get you guys a little excited, because I found out HOW!  I'll post full source when it's cleaned up a little.

-Jer

 MediaBridge Demo Project - Use at your own risk

Tags:

30 comments so far...

Re: WPF - Hacked! Part I

Man, I really hope I can spend a little more time with you on this WPF stuff. It is so interesting and the stuff you have already developed is just so far ahead of the curve right now. Keep up the great work. -- Oh, and I just wanted to be the first comment on your blog :)

By TWhidden on   5/19/2007 8:03 PM

Re: WPF - Hackery! Part I

amazing work - the dog is barkin

By hv3 on   5/21/2007 7:12 PM

Re: WPF - Hackery! Part I

You're genius! :)

By Rodrigo Ratan on   7/3/2007 10:04 PM

Re: WPF - Hackery! Part I

This is great, I've downloaded the demo project and run it, but the callback never gets called, on either xp or vista, any ideas at what could be going wrong.

By pedfx on   8/23/2007 10:18 AM

Re: WPF - Hackery! Part I

this is great stuff, I have downloaded the demo but for some reason the callback never gets called, I have tried on xp and vista but still no go, any ideas as to what could be going wrong

By pedfx on   8/23/2007 10:19 AM

Re: WPF - Hackery! Part I

Run "regsvr32 MediaBridgeSourceFilter.ax" on a release build. That filter must be registered to work.

By Jeremiah Morrill on   8/24/2007 3:38 AM

Re: WPF - Hackery! Part I

Hi Jeremiah!

Great job! I have one little question though. I can somehow compile your sample, and it runs fine, at least release configuration of it. If I put your code into new projects, and then run it, it runs fine, but upon exit I am gettng exception 0x8000003 (The exception Breakpoint. A breakpoint has been reached.) I am running debug configuration, and am getting this exception on exit both in IDE and in command line runs. Do you have any idea what could it be?

Thanks.

By Roman on   9/5/2007 10:44 AM

Re: WPF - Hackery! Part I

Update: managed to recompile your code in debug mode. Guess what!? Same exception. It seems to be somehow related to incomplete release of COM objects (guess in the graph), but I was not yet able to figure out what is causing it.

By Roman on   9/5/2007 2:04 PM

Re: WPF - Hackery! Part I

I had a similar issue, but only when I compiled the media bridge filter in debug mode. After I built it in release mode, all the problems went away. I haven't had time to figure out what the problem was...probably the debug libs that are being used or the debug build configuration.

By Jeremiah Morrill on   9/5/2007 3:09 PM

Re: WPF - Hackery! Part I

This is an amazing pice of reverse engineering. Cudos from here. I have a question though. Do you know if it posible to load a previous generated graph file (*.grf) and the use the enhaced video render to render the video in such a graph? In a project I have today I use the DirectShowLib.Utils.FilterGraphTools.LoadGraphFile(IGraphBuilder,path) to build my graph. I can't render to the EVR after i've added it with GraphBuilder.AddFilter(renderer, "Enhanced Video Renderer"). I get a error as follows:
Context 0x1a1b50' is disconnected. Releasing the interfaces from the current context (context 0x1a17b8).This may cause corruption or data loss. To avoid this problem, please ensure that all contexts/apartments stay alive until the applicationis completely done with the RuntimeCallableWrappers that represent COM components that liveinside them.

My graphs must get rendered since I hear the sound(it is rendering a TV caputure card). Perhaps you have some input?

By Kristian Schneider on   10/2/2007 3:37 AM

Re: WPF - Hackery! Part I

The MediaElement already create and adds the EVR to the graph. You simply just have to render your video pins to the input pin of the EVR. I do not suggest ever using *.grf files, as these are really made for debugging purposes. For your solution, just get the tv tuner filter, add it to the graph and render the output pin of the tv tuner filter and DShow should connect it to the input of the EVR.

By Jeremiah Morrill on   10/2/2007 3:01 PM

Re: WPF - Hackery! Part I

would this work with more than one mediaelement at the same time? I have a multiple monitor setup with each display having a different window playing its own mediaelement (simultaneous). the thing is I need to direct the sound of each mediaelement to a specific USB sound card. would both mediaelements use the same EVR? if not, how would I get a seperate reference to each individual graph?

By jeff on   10/10/2007 11:30 PM

Re: WPF - Hackery! Part I

would this work with more than one mediaelement at the same time? I have a multiple monitor setup with each display having a different window playing its own mediaelement (simultaneous). the thing is I need to direct the sound of each mediaelement to a specific USB sound card. would both mediaelements use the same EVR? if not, how would I get a seperate reference to each individual graph?

By jeff on   10/10/2007 11:30 PM

Re: WPF - Hackery! Part I

would this work with more than one mediaelement at the same time? I have a multiple monitor setup with each display having a different window playing its own mediaelement (simultaneous). the thing is I need to direct the sound of each mediaelement to a specific USB sound card. would both mediaelements use the same EVR? if not, how would I get a seperate reference to each individual graph?

By jeff on   10/10/2007 11:31 PM

Re: WPF - Hackery! Part I

would this work with more than one mediaelement at the same time? I have a multiple monitor setup with each display having a different window playing its own mediaelement (simultaneous). the thing is I need to direct the sound of each mediaelement to a specific USB sound card. would both mediaelements use the same EVR? if not, how would I get a seperate reference to each individual graph?

By jeff on   10/10/2007 11:31 PM

Re: WPF - Hackery! Part I

would this work with more than one mediaelement at the same time? I have a multiple monitor setup with each display having a different window playing its own mediaelement (simultaneous). the thing is I need to direct the sound of each mediaelement to a specific USB sound card. would both mediaelements use the same EVR? if not, how would I get a seperate reference to each individual graph?

By jeff on   10/10/2007 11:31 PM

Re: WPF - Hackery! Part I

would this work with more than one mediaelement at the same time? I have a multiple monitor setup with each display having a different window playing its own mediaelement (simultaneous). the thing is I need to direct the sound of each mediaelement to a specific USB sound card. would both mediaelements use the same EVR? if not, how would I get a seperate reference to each individual graph?

By jeff on   10/10/2007 11:32 PM

grf in mediaelement

I managed to render my custom graph programmaticaly, I guess I'll have to do it that way insted off using a .grf file as basis. Thanks for you input

By Kristian Schneider on   10/11/2007 3:23 AM

Re: WPF - Hackery! Part I

Hi Jeremiah, this is an very nice improvement but there is one important handicap. If you have not any active internet connection the windows media player by default can not open any url (error code: 0xC00D10B3). Otherwise this works fine.

By dan on   10/12/2007 12:02 PM

error code: 0xC00D10B3

>> Hi Jeremiah, this is an very nice improvement but there is one important handicap. If you have not any active internet connection the windows media player by default can not open any url (error code: 0xC00D10B3). Otherwise this works fine.
-----------------------------------

Solve this with following setting:
HKEY_CURRENT_USER\Software\Microsoft\MediaPlayer\Preferences

Change ForceOnline to 0x1.

By dan on   10/12/2007 1:56 PM

Re: WPF - Hackery! Part I

Wow, good find dan! Thanks for the fix. Maybe I'll add it to the register code of the filter.

-Jer

By Jeremiah Morrill on   10/12/2007 2:18 PM

Re: WPF - Hackery! Part I

I'm trying the demo and it works well except that when I close the app, the webcam is still on. I made a stop button that calls BridgedElement.Stop() and MediaBridgeManager.UnregisterCallback(...) but that doesn't do the trick. Any ideas on how I could fix this?

By Mark on   10/14/2007 6:23 PM

Re: WPF - Hackery! Part I

Hi Jeremiah

As I've said before, I love what you've done here. I do have a question about clean up really. I've been working with the bridge to work with video files rather than the live source. I get MDA's thrown when I go to load new file. This doesn't appear to affect the app when run out of the debugger and memory use does not continually increase. Just wondering if you've got some guidelines in this respect.

By steve on   11/14/2007 5:48 PM

Re: WPF - Hackery! Part I

Hi Jeremiah

As I've said before, I love what you've done here. I do have a question about clean up really. I've been working with the bridge to work with video files rather than the live source. I get MDA's thrown when I go to load new file. This doesn't appear to affect the app when run out of the debugger and memory use does not continually increase. Just wondering if you've got some guidelines in this respect.

By steve on   11/14/2007 5:49 PM

Re: WPF - Hackery! Part I

Hi Jeremiah

As I've said before, I love what you've done here. I do have a question about clean up really. I've been working with the bridge to work with video files rather than the live source. I get MDA's thrown when I go to load new file. This doesn't appear to affect the app when run out of the debugger and memory use does not continually increase. Just wondering if you've got some guidelines in this respect.

By steve on   11/14/2007 5:49 PM

The 'holy-grail'

>>>>>The 'holy-grail' would be to figure out how to get low-level, high-performance access to a bitmap in WPF and to be able to make our own video players, out side of Microsoft's MediaElement sandbox. I hope to get you guys a little excited, because I found out HOW! I'll post full source when it's cleaned up a little<<<<<

That would be great!!!
Did you have the chance to clean up the code and post it somewhere?

By Simone on   11/30/2007 7:07 PM

Re: WPF - Hackery! Part I

Hi Jeremiah! I living in Ukraine. Sorry in my bad INGLISH :-).
I make video control in Expression Blend on basis of your example.
Control run automatically.
Cоde XAML:
___________________________
xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
x:Class="UntitledProject2.UserControl1"
x:Name="UserControl"
Width="320" Height="240">




_____________________________
Code C#:
using System;
using System.Collections.Generic;
using System.Text;
using System.Windows;
using System.Windows.Controls;
using System.Windows.Data;
using System.Windows.Documents;
using System.Windows.Input;
using System.Windows.Media;
using System.Windows.Media.Imaging;
using System.Windows.Shapes;
using MediaBridge;
using DirectShowLib;
using System.Runtime.InteropServices.ComTypes;
using System.Runtime.InteropServices;

namespace UntitledProject2
{
public partial class UserControl1

{
IBaseFilter filter;
IFilterGraph graph;
IGraphBuilder graphBuilder;
ICaptureGraphBuilder2 captureGB;


public UserControl1()
{
this.InitializeComponent();

//// Insert code required on object creation below this point.
//btnStart.Click += new RoutedEventHandler(btnStart_Click);
//}
// void btnStart_Click(object sender, RoutedEventArgs e)
//{
string url = "MediaBridge://MyDataString";

MediaBridge.MediaBridgeManager.RegisterCallback(url, BridgeCallback);

BridgedElement.Source = new Uri(url);

BridgedElement.Play();
}
private void BridgeCallback(MediaBridgeGraphInfo GraphInfo)
{
int hr = 0;

//Find a capture filter
filter = FindCaptureDevice();

//Convert pointer of filter graph to an object we can use
graph = (IFilterGraph)Marshal.GetObjectForIUnknown(GraphInfo.FilterGraph);

graphBuilder = (IGraphBuilder)graph;

captureGB = (ICaptureGraphBuilder2)new CaptureGraphBuilder2();

hr = captureGB.SetFiltergraph(graphBuilder);

//Add our capture filter to the graph
hr = graph.AddFilter(filter, "CAPTURE");

IBaseFilter renderer;

//Find WPF renderer. It's always named the same thing
hr = graphBuilder.FindFilterByName("Enhanced Video Renderer", out renderer);
//hr = graphBuilder.FindFilterByName("Video Mixing Renderer (VR)", out renderer);

//Render our filters outpin to the renderer
hr = captureGB.RenderStream(PinCategory.Capture, MediaType.Video, filter, null, renderer);

DsROTEntry entry = new DsROTEntry(graph);
}
public IBaseFilter FindCaptureDevice()
{
int hr = 0;

IEnumMoniker classEnum = null;
IMoniker[] moniker = new IMoniker[1];

object source = null;

// Create the system device enumerator
ICreateDevEnum devEnum = (ICreateDevEnum) new CreateDevEnum();

// Create an enumerator for the video capture devices
hr = devEnum.CreateClassEnumerator(FilterCategory.VideoInputDevice, out classEnum, 0);
DsError.ThrowExceptionForHR(hr);

// The device enumerator is no more needed
Marshal.ReleaseComObject(devEnum);

// If there are no enumerators for the requested type, then
// CreateClassEnumerator will succeed, but classEnum will be NULL.
if (classEnum == null)
{
throw new ApplicationException("No video capture device was detected.\r\n\r\n" +
"This sample requires a video capture device, such as a USB WebCam,\r\n" +
"to be installed and working properly. The sample will now close.");
}

// Use the first video capture device on the device list.
// Note that if the Next() call succeeds but there are no monikers,
// it will return 1 (S_FALSE) (which is not a failure). Therefore, we
// check that the return code is 0 (S_OK).

if (classEnum.Next (moniker.Length, moniker, IntPtr.Zero) == 0)
{
// Bind Moniker to a filter object
Guid iid = typeof(IBaseFilter).GUID;
moniker[0].BindToObject(null, null, ref iid, out source);
}
else
{
throw new ApplicationException("Unable to access video capture device!");
}

// Release COM objects
Marshal.ReleaseComObject(moniker[0]);
Marshal.ReleaseComObject(classEnum);

// An exception is thrown if cast fail
return (IBaseFilter) source;

}
}
}
_______________________________
WEB camera working excellent!!!!
I`m want use TV-tuner. How did do it????????????????
I`m novice in programming.
Help me code! Please! Already "roof depart" :-)

By Andrey on   2/21/2008 11:13 PM

Re: WPF - Hackery! Part I

XAML code :
___________________________________________________________________
xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
x:Class="UntitledProject2.UserControl1"
x:Name="UserControl"
Width="320" Height="240">







_____________________________________

By Andrey on   2/21/2008 11:16 PM

Re: WPF - Hackery! Part I

Hello Andrey,

Feel free to email me, jeremiah.morrill[ at ]gmail.com and I think I can give you some pointers on how to do a TV Tuner.

-Jer

By Jeremiah Morrill on   2/21/2008 11:26 PM

Re: WPF - Hackery! Part I

On the way to get this running I had problems that the MediaBridgeSourceFilter.ax did not load.

LDR: LdrRelocateImageWithBias() failed 0xc0000018
LDR: OldBase : 10000000
LDR: NewBase : 019F0000
LDR: Diff : 0x7c91dec202aee404
LDR: NextOffset : 00000000
LDR: *NextOffset : 0x0
LDR: SizeOfBlock : 0x19f0000

Then I removed the compiler switch /FORCE and the filter (.ax) could be loaded. I hope that switch change has no other implications !?

By chr on   3/7/2008 6:31 AM

Your name:
Title:
Comment:
Security Code
Enter the code shown above in the box below
Add Comment    Cancel  
  Minimize
Text/HTML Minimize
Copyright 2007
Downloaded from DNNSkins.com