Sunday, December 27, 2009

Silverlight 4 : WebCam, Microphone and much more…

After a long time finally got some time to pen down few things here, Bad time from past few weeks since weekends also got occupied and saga may continue for coming week or two.

Well, don’t want to waste this free time today, so started looking at Silverlight 4 again, Last time I broke my head over Bidi-RTL feature of silverlight where I shown demo(s) on how you can do arabic language integration etc. I hope that post was useful for you. Today I am going to discuss in short and to specific point on the WebCam and Microphone integration inside Silverlight application and possibilities because of that. All you need is good quality WebCam (maybe external or integrated inside your laptop) along with Microphone.

Code is very simple and you can implement it directly, there is no great learning as far as code is concern, but there are few things to be understood so as to use this functionality at best. Today we have following UI with us :

InitialScreen

Today, I was playing with friend’s iPhone, I saw iPhone first time, It is good, and overall feel is also good, So I decided to use this as UI of our today’s POC, well frankly the code and discussion we are doing today has nothing to do with iPhone, so don’t get confuse,This iPhone UI is just to do something different that’s it,It has nothing to do with iPhone. we are just using this is as UI (Don’t even spend your valuable time in thinking :) ), Observe carefully that I have 4 buttons (Image Control) on bottom side as :

ScanDevice For Scan Devices

StartCam For Starting Camera

StopCam For Stopping Camera

TakePic For Capturing Snap/ Take a Snap

(All Images I got from internet while surfing, they carries their respective copyrights and owners,I am just using them for demos)

Scan for Devices :

It is important to detect the Audio and Video devices with the system, so as to use them further for capture. One simple way to identify is just right click and hit Silverlight option and then navigate to “WebCam/Mic” tab, you will get following screen :

Capture

We are doing this job via C# code and making use of Child window. I got one child window with me where I am showing Available Devices inside listbox as follows :

ScanDevices

To display such list, I have written following piece of C# code :

void Devices_Loaded(object sender, RoutedEventArgs e)
        {
            ddlVoice.ItemsSource = CaptureDeviceConfiguration.GetAvailableAudioCaptureDevices();
            ddlVideo.ItemsSource = CaptureDeviceConfiguration.GetAvailableVideoCaptureDevices();           
        }

Where ddlVoice and ddlVideo are my list for showing Collection of Audio and Video devices, Point to note here is that, GetAvailableAudioCaptureDevices() and GetAvailableVideoCaptureDevices() will return collection which can be bind to specific control. I can very well send values from Child Window to main page, but right now I am not giving stress on that functionality.

To call this popup, I wrote following C# inside my Image MouseLeftButton down event :

private void btnScan_MouseLeftButtonDown(object sender, System.Windows.Input.MouseButtonEventArgs e)
          {
            Devices dv = new Devices();
            dv.Show();
          }

Start Camera :

Once I hit start camera, My default Video device should be in game. Well I can pick if I got many from collection which we talked about in above section of scanning devices. Right now I am not connecting my child window here, so I will go for default one like this :

private void btnStart_MouseLeftButtonDown(object sender, System.Windows.Input.MouseButtonEventArgs e)
          {
              if (m_Camera != null)
            {
                m_Camera.Stop();

                VideoCaptureDevice vdo = CaptureDeviceConfiguration.GetDefaultVideoCaptureDevice();
                m_Camera.VideoCaptureDevice = vdo;
                VideoBrush vidBrush = new VideoBrush();
                vidBrush.SetSource(m_Camera);
                WebCamViewer.Fill = vidBrush;

                if (CaptureDeviceConfiguration.AllowedDeviceAccess || CaptureDeviceConfiguration.RequestDeviceAccess())
                {
                    m_Camera.Start();
                }
            }
          }

So what is this m_Camera?? , Well this is a member declared as :

CaptureSource m_Camera;

CaptureSource Class provides methods for specific Audio/Video capture devices (Refer more in Silverlight documentation)

WebCamViewer is simple rectangle declared in XAML like this :

<Rectangle x:Name="WebCamViewer" Height="214" VerticalAlignment="Top" Width="200" />

GetDefaultVideoCaptureDevice() will allow you to capture default WebCam with System, Make note that, in case of multiple,its your job to choose correct one from collection (Do I need to tell how? :) ), Like other brushes, we have VideoBrush here which will help us to fill the captured output inside rectangle.

  if (CaptureDeviceConfiguration.AllowedDeviceAccess || CaptureDeviceConfiguration.RequestDeviceAccess())
             { //Some Code..}

Many smart people will raise this question here, what is this and/or what if I ignore this? , Well this will allow access to use the specific devices which you seek via RequestDeviceAccess() and if at all you ignore this block thinking like you are using this one as “Spy Cam”..well your attempts will fail and you will get following screen :)

deny

When all goes well, It will prompt you with typical Yes/No box like this :

Permission

Once you press “Yes” (Take the Red Pill and enter..Alice in Wonderland..Matrix movie ;-) ), you can see output like this :

StartVideo

Done !! :)

Stopping Camera :

To stop camera, you don’t need to do much beside this one liner C# :

private void btnStop_MouseLeftButtonDown(object sender, System.Windows.Input.MouseButtonEventArgs e)
          {
               if (m_Camera != null)
              {
                  m_Camera.Stop();
              }
          }

Take a Snap ! :

It will be useless application if it does not allow you to take snap, so let’s take snap ! ( Folks, There are few damm good articles written by wonderful people like you and few of my MVP friends who have done nice things like Video recording and Applying Effects to images etc. Do check those articles too), Now you need a lambda as :

private void btnTakePic_MouseLeftButtonDown(object sender, System.Windows.Input.MouseButtonEventArgs e)
          {
                if (m_Camera != null)
              {
                   m_Camera.AsyncCaptureImage((snapImage) =>
                  {
                      PhotoCam.Visibility = System.Windows.Visibility.Visible;
                      PhotoCam.Source = snapImage;
                      WebCamViewer.Visibility = System.Windows.Visibility.Collapsed;                     
                  });
              }
          }

Note that, AsycnCaptureImage() takes WritableBitmap as param, we are then assigning that to our Image Control PhotoCam declared in XAML as :

<Image x:Name="PhotoCam" Margin="191,115,199,196" Width="200" Height="214"/>

and we are just doing normal on/off of WebCamViewer which is our rectangle where we first started rendering, is now replaced by our Image, You can do twists and tricks here by your own imagination and get that output image in some viewbox etc. Well finally it will look like this :

StopandCapture

So this is how you can integrate your Audio/Video devices smoothly inside silverlight which is new feature with Version 4 Beta. Possibilities with this feature are endless and many applications can be build on top of this, All you need is Imagination ! :) , Sayonara ! I will hit back here very soon with Printing and HTML hosting features and much more..with much more out of the box UI like we have today ..Muhaa haaa ! :D

Vikram.

6 comments:

jayesh sidhwani said...

Hi,

Nice Article Vikram and exciting addition as well to SL!

Well, If we capture video directly from the camera, we get a raw uncoded stream. Is there any way to programmatically use codecs like h.264 (or is Microsoft planning to release any codec functionality built in with further SL's?) etc and encode my raw stream before transporting it?

Thanks.

Rahul said...

hi,

First of all. Thanks very much for your useful post.

I just came across your blog and wanted to drop you a note telling you how impressed I

was with the information you have posted here.

Please let me introduce you some info related to this post and I hope that it is useful

for .Net community.

There is a good C# resource site, Have alook

http://www.csharptalk.com

simi

Helen said...

Hi Vikram, nice blog. I was looking for ways to explore the latest Windows OS, and came across your blog.


Keep it up.

Helen

Vikram Pendse said...

Thanks Helen,Thanks Rahul :)

Anil Kumar Pandey said...

Nice article Vikram sir, I was searching to integrate Audio/Video in my white board and i came across your post..

Thanks...
Anil Kumar Pandey

Dhruval Dave said...

How to Strean Live Video Over MMS Protocol .?