Difference between revisions of "ECE497 Project Remote Web Cam Viewer"

From eLinux.org
Jump to: navigation, search
(User Instructions)
m (Removed grade. Moved to Fall2012)
 
(30 intermediate revisions by 3 users not shown)
Line 1: Line 1:
[[Category:ECE497]]
+
[[Category:ECE497Fall2012 |Project]]
 
{{YoderHead}}
 
{{YoderHead}}
  
Team members: [[user:Draneaw|Alexander W. Drane]], [[user:Lobdeljt|John Lobdel]]
+
Team members: [[user:Draneaw|Alexander W. Drane]], [[user:Lobdeljt|John Lobdell]]
  
== Grading Template ==
+
== Executive Summary ==
I'm using the following template to grade. Each slot is 10 points.
+
The goal of this project is to create a simple webcam based security system that allows a user to remotely view the device's web cam by navigating to a webpage. Then using buttons the user can change what camera they are viewing.
0 = Missing, 5=OK, 10=Wow!
+
 
 +
Currently we have the ability to select a camera and view the stream in a remote webpage. This view uses the browser's VLC Plugin. We have tested this with 4 webcams on the xM board.
 +
 
 +
Operating with four webcams hooked to the board causes many failures of the system to stream back the video information. We determined that this was due to the physical cameras and the overhead of the devices by testing the system with only two we cams where the issues experienced regularly with the four camera setup greatly reduced in frequency.
  
<pre style="color:red">
+
Investigation of how to accomplish this task took up the bulk of our time and effort largely due to the infancy of HTML5 Related standards(sockets.io and MediaStream). In the end we decided to utilize the vlc plugin when the HTML5 proved unfeasible at its current level of development. Implementing the four cameras did seem to cause some device issues that caused the webpage to not sync with the incoming feed, but this appears to be hardware related only and not from the implementation (a further explanation is later on).
00 Executive Summary
 
00 Installation Instructions
 
00 User Instructions
 
00 Highlights
 
00 Theory of Operation
 
00 Work Breakdown
 
00 Future Work
 
00 Conclusions
 
00 Demo
 
00 Late
 
Comments: I'm looking forward to seeing this.
 
  
Score:  10/100
+
== Installation Instructions ==
</pre>
 
  
<span style="color:red">(Inline Comment)</span>
+
Extra Hardware Required:
 +
*Webcam 1+ (or Playstation Eye's) Max 4 without other supplementary hardware. Best performing build currently is 2 web cams on the board. 4 makes the XM Slow down too much.
  
== Executive Summary ==
+
*Connected to the internet.
The goal of this project is to create a simple webcam based security system that allows a user to remotely view the devices web cam by navigating to a webpage. Then using buttons the user can change what camera they are viewing.
+
 
 +
First Download the git Repository, it is located here:[https://github.com/jtlobdell/bbWebcamFeed.git WebcamFeed]
 +
 
 +
xm$ '''mkdir WebcamFeed'''
 +
xm$ '''git clone git://github.com/jtlobdell/bbWebcamFeed.git WebcamFeed'''
 +
xm$ '''cd WebcamFeed'''
  
//Give two sentences telling what works.
+
Install Node.js
Currently we have the ability to select a camera and view the stream in a remote webpage. This view uses the VLC Plugin.
+
xm$ '''opkg update'''
 +
xm$ '''opkg install nodejs'''
  
Give two sentences telling what isn't working.
+
Verify gstreamer is available and up to date
Currently we do not have the system switching between camera's via user inputs. We have not tested with multiple cameras.
 
  
End with a two sentence conclusion.
+
xm$ '''opkg update'''
Investigation of how to accomplish this task took up the bulk of our time and effort due to the infancy of HTML5 Related standards(sockets.io and MediaStream). We decided to utilize the vlc plugin when solving the problem with HTML5 proved unfeasible at its current level of development.
+
xm$ '''opkg install gstreamer'''
  
== Installation Instructions ==
+
One last thing must be done on the user computer. VLC Media Player Plugin must be available in your browser of choice. This can be done by downloading the software here: [[http://www.videolan.org/vlc/index.html]]  and making sure during installation to install the browser plugin. Many modern browsers will guide you through this if you run into a page that requires a missing plugin.
  
Give step by step instructions on how to install your project on the SPEd2 image.
+
Now for reference type the following to get the xM's IP address. It is needed quite soon
  
* Include your [https://github.com/ github] path as a link like this: [https://github.com/MarkAYoder/gitLearn https://github.com/MarkAYoder/gitLearn]. 
+
  xm$ '''ifconfig'''
* Include any additional packages installed via '''opkg'''.
 
* Include kernel mods.
 
* If there is extra hardware needed, include links to where it can be obtained.
 
  
 
== User Instructions ==
 
== User Instructions ==
 
Move to the source directory
 
Move to the source directory
xm$ '''cd ./WebcamFeeds/'''
+
<source lang="text">
 +
xm$ cd ./WebcamFeeds/
 +
</source>
  
Once the instillation is done the program can be ran
+
The current code does not automatically direct the VLC plugin to play from the correct IP address; it relies instead on passing a hard-coded IP address to the client in the WebcamFeed.html file. Modify the following code in the file to use your xM's IP address:
 
<source lang="text">
 
<source lang="text">
xm$ '''node WebcamFeed.js'''
+
<embed
 +
        type="application/x-vlc-plugin"
 +
        name="videoplayer"
 +
        autoplay="yes" loop="no" hidden="no"
 +
        width="320" height="240"
 +
        target="tcp://[YOUR_BEAGLE_IP]:8080" />
 +
</source>
 +
 
 +
Once the installation is done, the program is ready. Run it using:
 +
<source lang="text">
 +
xm$ node WebcamFeed.js
 
</source>
 
</source>
 
Now navigate on the user computer using a web browser to your computers IP Address. Once there click on the link to the video feed page. The program will start and video should play.
 
Now navigate on the user computer using a web browser to your computers IP Address. Once there click on the link to the video feed page. The program will start and video should play.
Line 59: Line 65:
 
NOTE: When changing video feeds the XM will randomly not load and play the feed. This we believe is due to the physical limitations of the hardware. We tested this by attaching 4 webcams to the board and found that the failure rate went up drastically. When the extra 2 webcams were removed the failure rate returned to where it was before. Since the program only ever calls one feed at a time(it changes which device it targets) this has to be a memory or hardware related issue and not due to the software.
 
NOTE: When changing video feeds the XM will randomly not load and play the feed. This we believe is due to the physical limitations of the hardware. We tested this by attaching 4 webcams to the board and found that the failure rate went up drastically. When the extra 2 webcams were removed the failure rate returned to where it was before. Since the program only ever calls one feed at a time(it changes which device it targets) this has to be a memory or hardware related issue and not due to the software.
  
At the top of the screen is a text box which can be entered the desired video device. Each camera is added sequentially starting at 0. Below the Set Cam button are two buttons for convenience. These function identically as entering the number's 1 or 0 into the text box and clicking Set Cam.
+
At the top of the screen is a text box which can be entered the desired video device. Each camera is added sequentially starting at 0. Below the Set Cam button are four buttons for convenience. These function identically as entering the number's 0 though 3 into the text box and clicking Set Cam for each number.
  
 
== Highlights ==
 
== Highlights ==
  
Here is where you brag about what your project can do.
+
A single video is captured from a web cam and streamed to a web page. The video capture device can be selected from the same page.
  
Include a [http://www.youtube.com/ YouTube] demo.
+
This program implements node.js and GStreamer working together to accomplish the video output and control over which video device to be viewed.
 +
 
 +
The software allows for as many video capture devices (webcams) as can be connected to the device. In the case of the Beagle Board xM, up to four cameras can be used. We only access one webcam at any given time and so the only increasing overhead on the system is that which is associated with the individual USB webcam's.
 +
 
 +
[http://youtu.be/kPK35JGzazY Here is a YouTube video demo of the four camera operation]. A picture of the test setup is shown below:
 +
 
 +
[[File:BbWebFeed_demo_setup.jpg|400px]]
 +
 
 +
This video starts out very slow and illustrates some of the hardware related issues that were encountered. While the most effective implementation currently is the two camera version we wished to show that four camera feeds could be accessed on the xM.
  
 
== Theory of Operation ==
 
== Theory of Operation ==
  
Give a high level overview of the structure of your software. Are you using GStreamer?  Show a diagram of the pipeline.  Are you running multiple tasks?  Show what they do and how they interact.
+
The project uses [http://gstreamer.freedesktop.org/ GStreamer] to capture video from a webcam, convert it to video/ogg format, and stream it through Transmission Control Protocol (TCP). A diagram of the GStreamer pipeline is shown below.
  
== Work Breakdown ==
+
[[File:BbWebcamFeed_Gstreamer_pipeline.png|400px|Web Cam Feed - GStreamer Pipeline]]
 +
 
 +
The client system connects to a node.js server running on the xM. The server embeds the video stream into the web page and also contains controls for selecting a webcam for the stream.
 +
 
 +
When the server begins running the node.js script, it launches the GStreamer pipeline discussed above. When the client connects, they receive a web page containing controls for the video source and the embedded video that gets played using the [http://www.videolan.org/vlc/index.html VLC] web plugin.
 +
 
 +
When the client uses the controls to change the video source, the server kills the current GStreamer process and launches a new one using the new video source. It then makes the client's page refresh to make the embedded video player use the new stream.
 +
 
 +
A block diagram of the system is shown below.
  
List the major tasks in your project and who did what.
+
[[File:BbWebcamFeed_Block_Diagram.png|400px|Web Cam Feed Block Diagram]]
  
Also list here what doesn't work yet and when you think it will be finished and who is finishing it.
+
== Work Breakdown ==
  
 
*Research- Research took up the bulk of our time and effort on this project due to the obscure nature of video streaming on the web using HTML5 and related tools. Most sources online advise you to use a plugin or third party software and treat the idea of using Javascript of HTML5 for this task as a taboo subject.  
 
*Research- Research took up the bulk of our time and effort on this project due to the obscure nature of video streaming on the web using HTML5 and related tools. Most sources online advise you to use a plugin or third party software and treat the idea of using Javascript of HTML5 for this task as a taboo subject.  
**GStreamer Implementation- John Lobdel investigated using GStreamer to acomplish this project. This implementation in the end proved effective and the easiest to implement.  
+
**GStreamer Implementation- John Lobdell investigated using GStreamer to accomplish this project. This implementation in the end proved effective and the easiest to implement.  
 
**HTML5/JavaScript Implementation - Alex Drane investigated using a pure HTML5 implementation in order to not require anything to be done by the user to view the video streams. From initial research this appeared to be something that could be done quite easily. After considerable research wading through the draft standards that have been developed I found very little sample code or examples that pertained to remote video viewing. The easiest implementation I found was using it to create a 1-to-1 chat application. This is not suitable for our needs.
 
**HTML5/JavaScript Implementation - Alex Drane investigated using a pure HTML5 implementation in order to not require anything to be done by the user to view the video streams. From initial research this appeared to be something that could be done quite easily. After considerable research wading through the draft standards that have been developed I found very little sample code or examples that pertained to remote video viewing. The easiest implementation I found was using it to create a 1-to-1 chat application. This is not suitable for our needs.
  
*Program Creation- John Lobdel used his research to get gstreamer to stream a video or camera input onto port 8081 that could then be read by the VLC Media Player on a remote computer. The further integration of this into a program was done with the introduction of the VLC-plugin in an html document. The final program was jointly compiled by both John and Alex.
+
*Program Creation- John Lobdell used his research to get GStreamer to stream a video or camera input onto port 8080 that could then be read by the VLC Media Player on a remote computer. The further integration of this into a program was done with the introduction of the VLC-plugin in an HTML document. The final program was jointly compiled by both John and Alex.
  
The program creation went quite fast after a working method was discovered by John. Then it was a matter of putting the pieces together. The program consists of a node.js client server using the example provided in the examples/node.js folder, which provides the HTML server and controls for which video to stream. We embedded the vlc viewer into the page and pointed it to the location where the video would be streamed by an external function call in the .js file. This allows for a simplified approach to running the needed utilities without having to port code into the java script language.
+
The program creation went quite fast after a working method was identified by John. Then it was a matter of putting the pieces together. The program consists of a node.js client server using the example provided in the examples/node.js folder, which provides the HTML server and controls for which video to stream. We embedded the VLC viewer into the page and pointed it to the location where the video would be streamed by an external function call in the .js file. This allowed for a simplified approach to running the needed utilities without having to port code into the Javascript language.
  
 
== Future Work ==
 
== Future Work ==
  
Adding video capture would add true security camera capabilities to this system. In addition adding motion detection into the video selection stream would be interesting. Though this might be too much for the XM's hardware.  
+
Adding video capture would add true security camera capabilities to this system. In addition adding motion detection into the video selection stream would be interesting. Though this might be too much for the xM's hardware.  
  
 
Having a webpage that displayed all of the video feeds at once would also be an interesting feature but this could over tax the XM's hardware or internet connection.
 
Having a webpage that displayed all of the video feeds at once would also be an interesting feature but this could over tax the XM's hardware or internet connection.
  
Figuring out how to implement this project with only HTML5 and make it so that no plugin is required by the end user was Alex's original plan but the maturity of the knowledge base for HTML5 is not adequate to accomplish this task at this time. But in the future when many of the draft standards are finished this project could be implemented without the vlc-plugin.
+
Figuring out how to implement this project with only HTML5 and make it so that no plugin is required by the end user was Alex's original plan but the maturity of the knowledge base for HTML5 is not adequate to accomplish this task at this time. But in the future when many of the draft standards are finished this project could be implemented without the VLC plugin.
 +
 
 +
Solving the interesting problem with the random stream failure. This issue arises when switching between streams. Occasionally the video receiver does not receive the stream even though the stream is playing across the network. From experimentation we found that as cameras were added the performance of the entire program greatly decreased. Since we only access one camera at a time the only explanation we could find was that the OS was devoting resources to the devices when they were connected thus creating a much greater overhead as cameras were added. When the cameras were disconnected the performance returned to their previous levels. With four cameras operating a noticeable increase in stream failures was observed. With only two cameras operating about 1/5 of attempts would be unsuccessful. With four cameras the failure rate was significantly higher (as much as 1/3 of attempts would fail). One possible explanation of this is the overhead causing a slow down in setup time for the video out thread compared to the load up time of the browser. While this is set up to go first if the system hanged due to lack of resources this could cause the video stream to start after the HTML file stopped looking for it. Occasionally the camera just won't turn on. We have found that signaling it again usually fixes the problem without much effort. After trying to figure out what was causing the problem we could not definitively pin down the root cause of the problem and we found no simple or effective solution for it but would like to see it fixed.
 +
 
 +
The server should automatically detect its external IP and then use it to direct the client's VLC plugin to the video stream.
 +
 
 +
The video should stream through UDP rather than TCP. UDP would have better performance, and data loss is not a major concern for low-resolution video.
 +
 
 +
Audio can be added to the video stream.
  
 
== Conclusions ==
 
== Conclusions ==

Latest revision as of 09:18, 18 May 2013

thumb‎ Embedded Linux Class by Mark A. Yoder


Team members: Alexander W. Drane, John Lobdell

Executive Summary

The goal of this project is to create a simple webcam based security system that allows a user to remotely view the device's web cam by navigating to a webpage. Then using buttons the user can change what camera they are viewing.

Currently we have the ability to select a camera and view the stream in a remote webpage. This view uses the browser's VLC Plugin. We have tested this with 4 webcams on the xM board.

Operating with four webcams hooked to the board causes many failures of the system to stream back the video information. We determined that this was due to the physical cameras and the overhead of the devices by testing the system with only two we cams where the issues experienced regularly with the four camera setup greatly reduced in frequency.

Investigation of how to accomplish this task took up the bulk of our time and effort largely due to the infancy of HTML5 Related standards(sockets.io and MediaStream). In the end we decided to utilize the vlc plugin when the HTML5 proved unfeasible at its current level of development. Implementing the four cameras did seem to cause some device issues that caused the webpage to not sync with the incoming feed, but this appears to be hardware related only and not from the implementation (a further explanation is later on).

Installation Instructions

Extra Hardware Required:

  • Webcam 1+ (or Playstation Eye's) Max 4 without other supplementary hardware. Best performing build currently is 2 web cams on the board. 4 makes the XM Slow down too much.
  • Connected to the internet.

First Download the git Repository, it is located here:WebcamFeed

xm$ mkdir WebcamFeed
xm$ git clone git://github.com/jtlobdell/bbWebcamFeed.git WebcamFeed
xm$ cd WebcamFeed

Install Node.js

xm$ opkg update
xm$ opkg install nodejs

Verify gstreamer is available and up to date

xm$ opkg update
xm$ opkg install gstreamer

One last thing must be done on the user computer. VLC Media Player Plugin must be available in your browser of choice. This can be done by downloading the software here: [[1]] and making sure during installation to install the browser plugin. Many modern browsers will guide you through this if you run into a page that requires a missing plugin.

Now for reference type the following to get the xM's IP address. It is needed quite soon

xm$ ifconfig

User Instructions

Move to the source directory

xm$ cd ./WebcamFeeds/

The current code does not automatically direct the VLC plugin to play from the correct IP address; it relies instead on passing a hard-coded IP address to the client in the WebcamFeed.html file. Modify the following code in the file to use your xM's IP address:

<embed
         type="application/x-vlc-plugin"
         name="videoplayer"
         autoplay="yes" loop="no" hidden="no"
         width="320" height="240"
         target="tcp://[YOUR_BEAGLE_IP]:8080" />

Once the installation is done, the program is ready. Run it using:

xm$ node WebcamFeed.js

Now navigate on the user computer using a web browser to your computers IP Address. Once there click on the link to the video feed page. The program will start and video should play.

NOTE: When changing video feeds the XM will randomly not load and play the feed. This we believe is due to the physical limitations of the hardware. We tested this by attaching 4 webcams to the board and found that the failure rate went up drastically. When the extra 2 webcams were removed the failure rate returned to where it was before. Since the program only ever calls one feed at a time(it changes which device it targets) this has to be a memory or hardware related issue and not due to the software.

At the top of the screen is a text box which can be entered the desired video device. Each camera is added sequentially starting at 0. Below the Set Cam button are four buttons for convenience. These function identically as entering the number's 0 though 3 into the text box and clicking Set Cam for each number.

Highlights

A single video is captured from a web cam and streamed to a web page. The video capture device can be selected from the same page.

This program implements node.js and GStreamer working together to accomplish the video output and control over which video device to be viewed.

The software allows for as many video capture devices (webcams) as can be connected to the device. In the case of the Beagle Board xM, up to four cameras can be used. We only access one webcam at any given time and so the only increasing overhead on the system is that which is associated with the individual USB webcam's.

Here is a YouTube video demo of the four camera operation. A picture of the test setup is shown below:

BbWebFeed demo setup.jpg

This video starts out very slow and illustrates some of the hardware related issues that were encountered. While the most effective implementation currently is the two camera version we wished to show that four camera feeds could be accessed on the xM.

Theory of Operation

The project uses GStreamer to capture video from a webcam, convert it to video/ogg format, and stream it through Transmission Control Protocol (TCP). A diagram of the GStreamer pipeline is shown below.

Web Cam Feed - GStreamer Pipeline

The client system connects to a node.js server running on the xM. The server embeds the video stream into the web page and also contains controls for selecting a webcam for the stream.

When the server begins running the node.js script, it launches the GStreamer pipeline discussed above. When the client connects, they receive a web page containing controls for the video source and the embedded video that gets played using the VLC web plugin.

When the client uses the controls to change the video source, the server kills the current GStreamer process and launches a new one using the new video source. It then makes the client's page refresh to make the embedded video player use the new stream.

A block diagram of the system is shown below.

Web Cam Feed Block Diagram

Work Breakdown

  • Research- Research took up the bulk of our time and effort on this project due to the obscure nature of video streaming on the web using HTML5 and related tools. Most sources online advise you to use a plugin or third party software and treat the idea of using Javascript of HTML5 for this task as a taboo subject.
    • GStreamer Implementation- John Lobdell investigated using GStreamer to accomplish this project. This implementation in the end proved effective and the easiest to implement.
    • HTML5/JavaScript Implementation - Alex Drane investigated using a pure HTML5 implementation in order to not require anything to be done by the user to view the video streams. From initial research this appeared to be something that could be done quite easily. After considerable research wading through the draft standards that have been developed I found very little sample code or examples that pertained to remote video viewing. The easiest implementation I found was using it to create a 1-to-1 chat application. This is not suitable for our needs.
  • Program Creation- John Lobdell used his research to get GStreamer to stream a video or camera input onto port 8080 that could then be read by the VLC Media Player on a remote computer. The further integration of this into a program was done with the introduction of the VLC-plugin in an HTML document. The final program was jointly compiled by both John and Alex.

The program creation went quite fast after a working method was identified by John. Then it was a matter of putting the pieces together. The program consists of a node.js client server using the example provided in the examples/node.js folder, which provides the HTML server and controls for which video to stream. We embedded the VLC viewer into the page and pointed it to the location where the video would be streamed by an external function call in the .js file. This allowed for a simplified approach to running the needed utilities without having to port code into the Javascript language.

Future Work

Adding video capture would add true security camera capabilities to this system. In addition adding motion detection into the video selection stream would be interesting. Though this might be too much for the xM's hardware.

Having a webpage that displayed all of the video feeds at once would also be an interesting feature but this could over tax the XM's hardware or internet connection.

Figuring out how to implement this project with only HTML5 and make it so that no plugin is required by the end user was Alex's original plan but the maturity of the knowledge base for HTML5 is not adequate to accomplish this task at this time. But in the future when many of the draft standards are finished this project could be implemented without the VLC plugin.

Solving the interesting problem with the random stream failure. This issue arises when switching between streams. Occasionally the video receiver does not receive the stream even though the stream is playing across the network. From experimentation we found that as cameras were added the performance of the entire program greatly decreased. Since we only access one camera at a time the only explanation we could find was that the OS was devoting resources to the devices when they were connected thus creating a much greater overhead as cameras were added. When the cameras were disconnected the performance returned to their previous levels. With four cameras operating a noticeable increase in stream failures was observed. With only two cameras operating about 1/5 of attempts would be unsuccessful. With four cameras the failure rate was significantly higher (as much as 1/3 of attempts would fail). One possible explanation of this is the overhead causing a slow down in setup time for the video out thread compared to the load up time of the browser. While this is set up to go first if the system hanged due to lack of resources this could cause the video stream to start after the HTML file stopped looking for it. Occasionally the camera just won't turn on. We have found that signaling it again usually fixes the problem without much effort. After trying to figure out what was causing the problem we could not definitively pin down the root cause of the problem and we found no simple or effective solution for it but would like to see it fixed.

The server should automatically detect its external IP and then use it to direct the client's VLC plugin to the video stream.

The video should stream through UDP rather than TCP. UDP would have better performance, and data loss is not a major concern for low-resolution video.

Audio can be added to the video stream.

Conclusions

This project was quite surprising by how difficult it is to do using the new HTML5 standards. Through the research we learned that there were currently no simple implementations of what we wished to do with this project. We had originally assumed that something as common as video streaming would have a direct implementation in the new standard but in fact it is far more restricted then we expected. By splitting each of our focuses during the research stage we were able to research the use of the bleeding edge of development and the use of the common implementation and then whoever found an implementation that worked first would allow the project to progress. In addition our parallel research gave us greater insight into the potential ways of accomplishing this task then if we had both focused on the HTML5 implementation and did not allow the project to reach a dead end as it would have if we had both focused on the HTML5 method.

In the end we did settle on using the vlc-plugin to capture the outgoing stream that John setup using gstreamer. This worked exactly as we wished it to without all of the syntax fighting that took place when trying to port the HTML5 sample code into code we had the codex's for(i.e. websocket to socket.io syntax Note: websocket wanted a Microsoft related XML file which after searching for could not be found when we tried to just use websocket).

Many of the ideas listed in Future Work would be amazing addon's to the project that would greatly expand and enhance this projects implementation.




thumb‎ Embedded Linux Class by Mark A. Yoder