EBC Exercise 21 Running Audio and Video

From eLinux.org
Jump to: navigation, search

thumb‎ Embedded Linux Class by Mark A. Yoder

Welcome to Audio and Video labs Parts a and b. In Part a, you will combine the audio loop-thru application with the video loop-thru application by running them from the Linux Bash terminal.

In Part b, you’ll programmatically combine these two applications into a single (multi-threaded) application that handles both data streams – audio and video.

Part a – Run Audio and Video in Separate Processes

Build Audio Executable

  • Change to the lab06c_audio_loopthru directory from before.
  • Build the application using “make debug”.

This will build the debug version. Note, if you have problems building at this step, try cleaning, then building:

beagle$ make clean
beagle$ make
beagle$ mv app_DEBUG.Beagle ../../videoThru/lab07d_video_loopthru/

The last command moved app_DEBUG.Beagle to app_AUDIO in the video directory.

Build Video Executable

  • Change to the lab07d_video_loopthru directory then build the application.
beagle$ cd ../../videoThru/lab07d_video_loopthru/
beagle$ make clean
beagle$ make

Run Audio and Video in Separate Processes

  • Execute the app_AUDIO application using the following command:
beagle$ ./app_AUDIO &

Note, the trailing ampersand (&) in this command indicates that the application is to be run as a separate process. (In this case, our audio app will run in the terminal background, meaning that the terminal will remain open to new commands even while the application is executing.)

  • Execute the videoThru_DEBUG.Beagle application (the video loop-thru application) using the following command:
beagle$ ./videoThru_DEBUG.Beagle

You should now have both audio loop-thru and video loop-thru running concurrently on the board. They are running as concurrent, but separate, processes. In Part b we will use pthreads to run the audio and video loop-thru in parallel threads within the same process or application.

  • Halt the video loop-thru (running in the terminal foreground) by pressing Ctrl-C.
  • Use the following command to determine the process jobs number of the audio loop-thru, which is running in the terminal background:
beagle$ jobs
  • Halt the audio loop-thru using the kill command.
beagle$ kill %1

Part a Question

  • Which scheduling policy is being used by each of the audio and video program processes (i.e. how is the thread within each process being scheduled)?

Part b - Audio and Video - same process

In this lab, we will combine lab06c_audio_loopthru, and lab07d_video_loopthru into a single, multi-threaded application.

File Management

  • Change to the VideoThru directory.
beagle$ cd ..
  • List the lab08b_audio_video directory.

Two previously unused files have been provided for you (make sure they’re #included):

beagle$ ls lab08b_audio_video
thread.c  thread.h
  • Examine thread.c in gedit or similar editor.

This file contains one function, launch_pthread(), which is a “wrapper” function that implements thread creation procedure which was reviewed in class. (That is, rather than making you write all this code from scratch, we chose to wrap it up for you into a single call to launch a pthread.)

Our pthread launch function takes five parameters:

pthread_t *hThread_byref
This is a pointer to a handle (i.e. a handle passed by reference) which is used as a return value. The memory location pointed to by hThread_byref will be updated with a pthread handle.
int type
Integer specifying that the created thread will either be REALTIME or TIMESLICE as per #define in thread.h
int priority
The priority assigned to the thread. (It’s only used for thread type = REALTIME)
void *(*thread_fxn)(void *env)
A pointer to the function that is the entry point for the created thread. This function takes a single pointer as an argument (although the pointer may be a pointer to a structure, effectively allowing multiple arguments.)
void *env
The pointer which will be passed as the argument to thread_fxn() as per the above

Hint: It is important to do the following three steps in this exact order. Otherwise, some of the following directions (i.e. editing main.c) will be incorrect!

Don’t worry about overwriting any files.

  • Copy the full contents of lab07a_osd_setup into lab08b_audio_video.
beagle$ cp –rf lab07a_osd_setup/* lab08b_audio_video
  • Copy the full contents of lab06c_audio_loop-thru into lab08b_audio_video.
beagle$ cp –rf ../audioThru/lab06c_audio_loopthru/* lab08b_audio_video
  • Copy the full contents of lab07d_video_loopthru into lab08b_audio_video.
beagle$ cp –rf lab07d_video_loopthru/* lab08b_audio_video

Edit main.c

  • Open lab08b_audio_video/main.c in a text editor.
  • Fill in the missing .h files, as well as the missing _env variables for the audio and osd threads to main.c.

Your main.c file should contain the following:

video_env (which is the video_thread_env variable)

You need to add the following to main.c. (Refer to lab06c for this code.)

audio_env (which is the audio_thread_env variable)
  • Make sure that video and audio while loops exit when Ctrl-C is pressed.

Recall that the signal_handler function is run whenever Ctrl-C is pressed. This signal handler sets the quit field in both of these global structures to true, signaling to the thread that it should proceed to its cleanup phase and then exit.

How do the threads know where to look for these variables? These environment structures are passed as the argument to the thread. Within the thread function, the main while loop tests on the appropriate quit variable. When the quit variable becomes true, execution drops out of the while loop and into the final (cleanup) phase of the function.

Currently the signal_handler() function sets video_env.quit to one (true). We need to add similar statements for the audio_env.quit to the signal handler below.

void signal_handler( int sig ) {
    DBG( "Ctrl-C pressed, cleaning up and exiting..\n" );
    video_env.quit = 1;
  • To make debugging easier, put a one-second delay in between *.quit=1 statements. Since we’re exiting three pthreads back-to-back, you might find that their debug messages become interleaved – which can make debugging more difficult. To this end, when building with our “debug” profile, we could delay the start of the second quit by using a Linux time function.

Insert the following code between each *.quit=1 call to cause Linux to sleep for one second. This should make debugging easier.

#ifdef _DEBUG_

Don’t forget to include the proper header file for the sleep() function: unistd.h

  • Include the <pthread.h> header file that prototypes the launch_pthread() function.

This is the header file for the code you examined a few steps above.

  • Declare two pthread handles and two return pointers needed to manage our new threads (audio, video).

At the top of main(), add two pthread handles (of type pthread_t) named audioThread and videoThread. Handles are used to refer to instantiated objects; the handle value will be set during pthread_create() below, then used again later to refer to that pthread instance.

Also, we want to add two void pointers: one named audioThreadReturn and one named videoThreadReturn. The return pointers will be used when “joining” (i.e. exiting) each thread below; they will allow you to interrogate the status after an exit.

It should end up looking like:

pthread_t audioThread, videoThread;
void *audioThreadReturn, *videoThreadReturn;
  • Replace the direct call of video_thread_fxn() with a call of launch_pthread() to create a new thread that has video_thread_fxn() as its entry point.

Currently main.c calls video_thread_fxn. Replace this direct function call with a call to launch_pthread()

Replace the following:

 /* Call video thread function */
 videoThreadReturn = video_thread_fxn((void *) &video_env);

With this starting code … you have to fill in a few details:

/* Create a thread for video loopthru */
if(launch_pthread(&thread, (type), (priority), video_function, argument) != thread_SUCCESS){
    ERR("pthread create failed for video thread\n");
    status = EXIT_FAILURE;
    goto cleanup;

We’ve given you hints, but you need to figure out the actual arguments to launch_pthread(). Don’t forget to add #defines for audio & video thread created masks.

Either REALTIME or TIMESLICE as per thread.h. As per the lecture, we want to set this field to TIMESLICE for video_thread_fxn().
This value will be ignored for TIMESLICE type, so you can set it to zero.
  • Add launch_pthread() calls to launch the audio_thread_fxn() entry points as threads with the following characteristics:
type: REALTIME, priority: 99

At this point, you should have 2 sets of launch_pthread calls (one for video and audio). Be sure to record successful launching of the audio thread in the initMask.

  • To make debugging easier, put a one-second delay in between pthread_create() calls.

Since we’re creating two pthread’s back-to-back, you might find that their debug messages could become interleaved – which can make debugging more difficult. To this end, when building with our “debug” profile, we could delay the start of the second pthread_create() by using a Linux time function. Insert the following code between your two launch_pthread() calls to cause Linux to sleep for one second. This should make debugging easier.

#ifdef _DEBUG_
  • Add “cleanup” section using pthread_join() for both audio and video threads.

First, let’s create a “cleanup” section in our main.c file. After the audio and video threads have been created, use pthread_join on both threads to pause execution of the main thread until all threads have exited.

The prototype for pthread_join is:

int pthread_join(pthread_t thread, void **value_ptr);

The first parameter is the handle to the thread to join to (the variable we created above, then filled-in with pthread_create() above).

We use audioThreadReturn and videoThreadReturn pointers (by reference) to store the return status (pass/fail) from the join function. Since we want to return a value via this argument, we want to pass the (void pointer) argument by reference. To avoid getting an incompatible pointer type warning, we want to recast this argument – since this recasting can be a bit tricky for some of us, rather than have you figure it out by trial-and-error, here are the values to use for the second argument of each join function:

(void **) &videoThreadReturn
(void **) &audioThreadReturn

So, as an example, to join the video thread, you can write the following after “cleanup:” :

pthread_join(videoThread, (void **) &videoThreadReturn);
  • Ensure all initMask #defines are completed for video and audio.
  • Save and close main.c.

Edit makefile_profile.mak

  • Modify makefile_profile.mak to use the ALSA sound libraries by adding -lasound to CFLAGS.
  • Build and run the application
beagle$ make 
beagle$ ./videoThru_DEBUG.Beagle 

You should have simultaneous audio and video playing through the board.

  • Press Ctrl-C to exit the application.

Demo your program

  1. Demo you program.
  2. Run htop in another window. Why does videoThru_DEBUG.Beagle show up twice? How much of the CPU does it use?
  3. Run ps aux | grep vid. What shows up? Try ps aux -L | grep vid. What can you tell about the threads?

thumb‎ Embedded Linux Class by Mark A. Yoder