Difference between revisions of "ECE497 Project Remote Web Cam Viewer"

From eLinux.org
Jump to: navigation, search
(Future Work)
(Theory of Operation)
Line 100: Line 100:
 
The project uses [http://gstreamer.freedesktop.org/ GStreamer] to capture video from a webcam, convert it to video/ogg format, and stream it through Transmission Control Protocol (TCP). A diagram of the GStreamer pipeline is shown below.
 
The project uses [http://gstreamer.freedesktop.org/ GStreamer] to capture video from a webcam, convert it to video/ogg format, and stream it through Transmission Control Protocol (TCP). A diagram of the GStreamer pipeline is shown below.
  
TODO add picture of GStreamer pipeline
+
[[File:BbWebcamFeed_Gstreamer_pipeline.png|400px|Web Cam Feed - GStreamer Pipeline]]
  
 
The client system connects to a node.js server running on the xM. The server embeds the video stream into the web page and also contains controls for selecting a webcam for the stream.
 
The client system connects to a node.js server running on the xM. The server embeds the video stream into the web page and also contains controls for selecting a webcam for the stream.
Line 110: Line 110:
 
A block diagram of the system is shown below.
 
A block diagram of the system is shown below.
  
TODO add picture of block diagram
+
[[File:BbWebcamFeed_Block_Diagram.png|400px|Web Cam Feed Block Diagram]]
  
 
== Work Breakdown ==
 
== Work Breakdown ==

Revision as of 23:00, 13 November 2012

thumb‎ Embedded Linux Class by Mark A. Yoder


Team members: Alexander W. Drane, John Lobdel

Grading Template

I'm using the following template to grade. Each slot is 10 points. 0 = Missing, 5=OK, 10=Wow!

00 Executive Summary
00 Installation Instructions 
00 User Instructions
00 Highlights
00 Theory of Operation
00 Work Breakdown
00 Future Work
00 Conclusions
00 Demo
00 Late
Comments: I'm looking forward to seeing this.

Score:  10/100

(Inline Comment)

Executive Summary

The goal of this project is to create a simple webcam based security system that allows a user to remotely view the devices web cam by navigating to a webpage. Then using buttons the user can change what camera they are viewing.

Currently we have the ability to select a camera and view the stream in a remote webpage. This view uses the browser's VLC Plugin. We have tested this with 4 webcams on the xM board.

Operating with four webcams hooked to the board causes many failures of the system to stream back the video information. We determined that this was due to the physical cameras and the overhead of the devices by testing the system with only two we cams where the issues experienced regularly with the four camera setup greatly reduced in frequency.

Investigation of how to accomplish this task took up the bulk of our time and effort largely due to the infancy of HTML5 Related standards(sockets.io and MediaStream). In the end we decided to utilize the vlc plugin when the HTML5 proved unfeasible at its current level of development. Implementing the four cameras did seem to cause some device issues that caused the webpage to not sync with the incoming feed, but this appears to be hardware related only and not from the implementation (a further explanation is later on).

Installation Instructions

Extra Hardware Required:

  • Webcam 1+ (or Playstation Eye's) Max 4 without other supplementary hardware. Best performing build currently is 2 web cams on the board. 4 makes the XM Slow down too much.
  • Connected to the internet.

First Download the git Repository, it is located here:WebcamFeed

xm$ mkdir WebcamFeed
xm$ git clone https://github.com/jtlobdell/bbWebcamFeed.git WebcamFeed
xm$ cd WebcamFeed

Install Node.js

xm$ opkg update
xm$ opkg install nodejs

Verify gstreamer is available and up to date

xm$ opkg update
xm$ opkg install gstreamer

One last thing must be done on the user computer. VLC Media Player Plugin must be available in your browser of choice. This can be done by downloading the software here: [[1]] and making sure during installation to install the browser plugin. Many modern browsers will guide you through this if you run into a page that requires a missing plugin.

Now for reference type the following you will need it quite soon

xm$ ifconfig

Take note of the IP Address as you will need to enter it in your user's web browser and to align to the video feed. (This is due to the address being in the client HTML file and not able to be easily informed of its source IP Address.)

User Instructions

Move to the source directory

xm$ cd ./WebcamFeeds/

Once the instillation is done the program can be ran

xm$ node WebcamFeed.js

Now navigate on the user computer using a web browser to your computers IP Address. Once there click on the link to the video feed page. The program will start and video should play.

NOTE: When changing video feeds the XM will randomly not load and play the feed. This we believe is due to the physical limitations of the hardware. We tested this by attaching 4 webcams to the board and found that the failure rate went up drastically. When the extra 2 webcams were removed the failure rate returned to where it was before. Since the program only ever calls one feed at a time(it changes which device it targets) this has to be a memory or hardware related issue and not due to the software.

At the top of the screen is a text box which can be entered the desired video device. Each camera is added sequentially starting at 0. Below the Set Cam button are four buttons for convenience. These function identically as entering the number's 0 though 3 into the text box and clicking Set Cam for each number.

Highlights

A single video is captured from a web cam and streamed to a web page. The video capture device can be selected from the same page.

This program implements node.js and GStreamer working together to accomplish the video output and control over which video device to be viewed.

The software allows for as many video capture devices (webcams) as can be connected to the device. In the case of the Beagle Board xM, up to four cameras can be used. We only access one webcam at any given time and so the only increasing overhead on the system is that which is associated with the individual USB webcam's.

Here is a YouTube video demo of the four camera operation.

This video starts out very slow and illustrates some of the hardware related issues that were encountered. While the most effective implementation currently is the two camera version we wished to show that four camera feeds could be accessed on the xM.

Theory of Operation

The project uses GStreamer to capture video from a webcam, convert it to video/ogg format, and stream it through Transmission Control Protocol (TCP). A diagram of the GStreamer pipeline is shown below.

Web Cam Feed - GStreamer Pipeline

The client system connects to a node.js server running on the xM. The server embeds the video stream into the web page and also contains controls for selecting a webcam for the stream.

When the server begins running the node.js script, it launches the GStreamer pipeline discussed above. When the client connects, they receive a web page containing controls for the video source and the embedded video that gets played using the VLC web plugin.

When the client uses the controls to change the video source, the server kills the current GStreamer process and launches a new one using the new video source. It then makes the client's page refresh to make the embedded video player use the new stream.

A block diagram of the system is shown below.

Web Cam Feed Block Diagram

Work Breakdown

List the major tasks in your project and who did what.

Also list here what doesn't work yet and when you think it will be finished and who is finishing it.

  • Research- Research took up the bulk of our time and effort on this project due to the obscure nature of video streaming on the web using HTML5 and related tools. Most sources online advise you to use a plugin or third party software and treat the idea of using Javascript of HTML5 for this task as a taboo subject.
    • GStreamer Implementation- John Lobdel investigated using GStreamer to acomplish this project. This implementation in the end proved effective and the easiest to implement.
    • HTML5/JavaScript Implementation - Alex Drane investigated using a pure HTML5 implementation in order to not require anything to be done by the user to view the video streams. From initial research this appeared to be something that could be done quite easily. After considerable research wading through the draft standards that have been developed I found very little sample code or examples that pertained to remote video viewing. The easiest implementation I found was using it to create a 1-to-1 chat application. This is not suitable for our needs.
  • Program Creation- John Lobdel used his research to get gstreamer to stream a video or camera input onto port 8081 that could then be read by the VLC Media Player on a remote computer. The further integration of this into a program was done with the introduction of the VLC-plugin in an html document. The final program was jointly compiled by both John and Alex.

The program creation went quite fast after a working method was discovered by John. Then it was a matter of putting the pieces together. The program consists of a node.js client server using the example provided in the examples/node.js folder, which provides the HTML server and controls for which video to stream. We embedded the vlc viewer into the page and pointed it to the location where the video would be streamed by an external function call in the .js file. This allows for a simplified approach to running the needed utilities without having to port code into the java script language.

Future Work

Adding video capture would add true security camera capabilities to this system. In addition adding motion detection into the video selection stream would be interesting. Though this might be too much for the XM's hardware.

Having a webpage that displayed all of the video feeds at once would also be an interesting feature but this could over tax the XM's hardware or internet connection.

Figuring out how to implement this project with only HTML5 and make it so that no plugin is required by the end user was Alex's original plan but the maturity of the knowledge base for HTML5 is not adequate to accomplish this task at this time. But in the future when many of the draft standards are finished this project could be implemented without the vlc-plugin.

Solving the interesting problem with the random stream failure. This issue arises when switching between streams. Occasionally the video receiver does not receive the stream even though the stream is playing across the network. From experimentation we found that as cameras were added the performance of the entire program greatly decreased. Since we only access one camera at a time the only explanation we could find was that the OS was devoting resources to the devices when they were connected thus creating a much greater overhead as cameras were added. When the cameras were disconnected the performance returned to their previous levels. With four cameras operating a noticeable increase in stream failures was observed. With only two cameras operating about 1/5 of attempts would be unsuccessful. With four cameras the failure rate was significantly higher (as much as 1/3 of attempts would fail). One possible explanation of this is the overhead causing a slow down in setup time for the video out thread compared to the load up time of the browser. While this is set up to go first if the system hanged due to lack of resources this could cause the video stream to start after the html file stopped looking for it. Occasionally the camera just won't turn on. We have found that signaling it again usually fixes the problem without much effort. After trying to figure out what was causing the problem we could not definitively pin down the root cause of the problem and we found no simple or effective solution for it but would like to see it fixed.

Conclusions

This project was quite surprising by how difficult it is to do using the new HTML5 standards. Through the research we learned that there were currently no simple implementations of what we wished to do with this project. We had originally assumed that something as common as video streaming would have a direct implementation in the new standard but in fact it is far more restricted then we expected. By splitting each of our focuses during the research stage we were able to research the use of the bleeding edge of development and the use of the common implementation and then whoever found an implementation that worked first would allow the project to progress. In addition our parallel research gave us greater insight into the potential ways of accomplishing this task then if we had both focused on the HTML5 implementation and did not allow the project to reach a dead end as it would have if we had both focused on the HTML5 method.

In the end we did settle on using the vlc-plugin to capture the outgoing stream that John setup using gstreamer. This worked exactly as we wished it to without all of the syntax fighting that took place when trying to port the HTML5 sample code into code we had the codex's for(i.e. websocket to socket.io syntax Note: websocket wanted a Microsoft related XML file which after searching for could not be found when we tried to just use websocket).

Many of the ideas listed in Future Work would be amazing addon's to the project that would greatly expand and enhance this projects implementation.




thumb‎ Embedded Linux Class by Mark A. Yoder