EBC Exercise 21 Running Audio and Video

Welcome to Audio and Video labs Parts a and b. In Part a, you will combine the audio loop-thru application with the video loop-thru application by running them from the Linux Bash terminal.

In Part b, you’ll programmatically combine these two applications into a single (multi-threaded) application that handles both data streams – audio and video.

Build Audio Executable

 * Change to the lab06c_audio_loopthru directory from before.

This will build the debug version. Note, if you have problems building at this step, try cleaning, then building: beagle$ make clean beagle$ make beagle$ mv app_DEBUG.Beagle ../../videoThru/lab07d_video_loopthru/
 * Build the application using “make debug”.

The last command moved app_DEBUG.Beagle to app_AUDIO in the video directory.

Build Video Executable
beagle$ cd ../../videoThru/lab07d_video_loopthru/ beagle$ make clean beagle$ make
 * Change to the lab07d_video_loopthru directory then build the application.

Run Audio and Video in Separate Processes
beagle$ ./app_AUDIO & Note, the trailing ampersand (&) in this command indicates that the application is to be run as a separate process. (In this case, our audio app will run in the terminal background, meaning that the terminal will remain open to new commands even while the application is executing.)
 * Execute the app_AUDIO application using the following command:

beagle$ ./videoThru_DEBUG.Beagle You should now have both audio loop-thru and video loop-thru running concurrently on the board. They are running as concurrent, but separate, processes. In Part b we will use pthreads to run the audio and video loop-thru in parallel threads within the same process or application.
 * Execute the videoThru_DEBUG.Beagle application (the video loop-thru application) using the following command:

beagle$ jobs
 * Halt the video loop-thru (running in the terminal foreground) by pressing Ctrl-C.
 * Use the following command to determine the process jobs number of the audio loop-thru, which is running in the terminal background:

beagle$ kill %1
 * Halt the audio loop-thru using the kill command.

Part a Question

 * Which scheduling policy is being used by each of the audio and video program processes (i.e. how is the thread within each process being scheduled)?

Part b - Audio and Video - same process
In this lab, we will combine lab06c_audio_loopthru, and lab07d_video_loopthru into a single, multi-threaded application.

File Management
beagle$ cd ..
 * Change to the VideoThru directory.

Two previously unused files have been provided for you (make sure they’re #included): beagle$ ls lab08b_audio_video thread.c thread.h This file contains one function, launch_pthread, which is a “wrapper” function that implements thread creation procedure which was reviewed in class. (That is, rather than making you write all this code from scratch, we chose to wrap it up for you into a single call to launch a pthread.)
 * List the lab08b_audio_video directory.
 * Examine thread.c in gedit or similar editor.

Our pthread launch function takes five parameters:
 * pthread_t *hThread_byref
 * This is a pointer to a handle (i.e. a handle passed by reference) which is used as a return value. The memory location pointed to by hThread_byref will be updated with a pthread handle.


 * int type
 * Integer specifying that the created thread will either be REALTIME or TIMESLICE as per #define in thread.h


 * int priority
 * The priority assigned to the thread. (It’s only used for thread type = REALTIME)


 * void *(*thread_fxn)(void *env)
 * A pointer to the function that is the entry point for the created thread. This function takes a single pointer as an argument (although the pointer may be a pointer to a structure, effectively allowing multiple arguments.)


 * void *env
 * The pointer which will be passed as the argument to thread_fxn as per the above

Hint: It is important to do the following three steps in this exact order. Otherwise, some of the following directions (i.e. editing main.c) will be incorrect!

Don’t worry about overwriting any files.

beagle$ cp –rf lab07a_osd_setup/* lab08b_audio_video
 * Copy the full contents of lab07a_osd_setup into lab08b_audio_video.

beagle$ cp –rf ../audioThru/lab06c_audio_loopthru/* lab08b_audio_video
 * Copy the full contents of lab06c_audio_loop-thru into lab08b_audio_video.

beagle$ cp –rf lab07d_video_loopthru/* lab08b_audio_video
 * Copy the full contents of lab07d_video_loopthru into lab08b_audio_video.

Edit main.c

 * Open lab08b_audio_video/main.c in a text editor.

Your main.c file should contain the following: video_thread.h video_env (which is the video_thread_env variable) You need to add the following to main.c. (Refer to lab06c for this code.) audio_thread.h audio_env (which is the audio_thread_env variable)
 * Fill in the missing .h files, as well as the missing _env variables for the audio and osd threads to main.c.

Recall that the signal_handler function is run whenever Ctrl-C is pressed. This signal handler sets the quit field in both of these global structures to true, signaling to the thread that it should proceed to its cleanup phase and then exit.
 * Make sure that video and audio while loops exit when Ctrl-C is pressed.

How do the threads know where to look for these variables? These environment structures are passed as the argument to the thread. Within the thread function, the main while loop tests on the appropriate quit variable. When the quit variable becomes true, execution drops out of the while loop and into the final (cleanup) phase of the function.

Currently the signal_handler function sets video_env.quit to one (true). We need to add similar statements for the audio_env.quit to the signal handler below. void signal_handler( int sig ) { DBG( "Ctrl-C pressed, cleaning up and exiting..\n" ); _______________________________   video_env.quit = 1; }


 * To make debugging easier, put a one-second delay in between *.quit=1 statements. Since we’re exiting three pthreads back-to-back, you might find that their debug messages become interleaved – which can make debugging more difficult. To this end, when building with our “debug” profile, we could delay the start of the second quit by using a Linux time function.

Insert the following code between each *.quit=1 call to cause Linux to sleep for one second. This should make debugging easier. sleep(1); Don’t forget to include the proper header file for the sleep function: unistd.h
 * 1) ifdef _DEBUG_
 * 1) endif

This is the header file for the code you examined a few steps above.
 * Include the  header file that prototypes the launch_pthread function.


 * Declare two pthread handles and two return pointers needed to manage our new threads (audio, video).

At the top of main, add two pthread handles (of type pthread_t) named audioThread and videoThread. Handles are used to refer to instantiated objects; the handle value will be set during pthread_create below, then used again later to refer to that pthread instance.

Also, we want to add two void pointers: one named audioThreadReturn and one named videoThreadReturn. The return pointers will be used when “joining” (i.e. exiting) each thread below; they will allow you to interrogate the status after an exit.

It should end up looking like: pthread_t audioThread, videoThread; void *audioThreadReturn, *videoThreadReturn;

Currently main.c calls video_thread_fxn. Replace this direct function call with a call to launch_pthread
 * Replace the direct call of video_thread_fxn with a call of launch_pthread to create a new thread that has video_thread_fxn as its entry point.

Replace the following: /* Call video thread function */ videoThreadReturn = video_thread_fxn((void *) &video_env);

With this starting code … you have to fill in a few details: /* Create a thread for video loopthru */ if(launch_pthread(&thread, (type), (priority), video_function, argument) != thread_SUCCESS){ ERR("pthread create failed for video thread\n"); status = EXIT_FAILURE; goto cleanup; } initMask |= VIDEOTHREADCREATED; We’ve given you hints, but you need to figure out the actual arguments to launch_pthread. Don’t forget to add #defines for audio & video thread created masks.


 * (type)
 * Either REALTIME or TIMESLICE as per thread.h. As per the lecture, we want to set this field to TIMESLICE for video_thread_fxn.


 * (priority)
 * This value will be ignored for TIMESLICE type, so you can set it to zero.


 * Add launch_pthread calls to launch the audio_thread_fxn entry points as threads with the following characteristics:
 * audio_thread_fxn
 * type: REALTIME, priority: 99

At this point, you should have 2 sets of launch_pthread calls (one for video and audio). Be sure to record successful launching of the audio thread in the initMask.

Since we’re creating two pthread’s back-to-back, you might find that their debug messages could become interleaved – which can make debugging more difficult. To this end, when building with our “debug” profile, we could delay the start of the second pthread_create by using a Linux time function. Insert the following code between your two launch_pthread calls to cause Linux to sleep for one second. This should make debugging easier. sleep(1);
 * To make debugging easier, put a one-second delay in between pthread_create calls.
 * 1) ifdef _DEBUG_
 * 1) endif

First, let’s create a “cleanup” section in our main.c file. After the audio and video threads have been created, use pthread_join on both threads to pause execution of the main thread until all threads have exited.
 * Add “cleanup” section using pthread_join for both audio and video threads.

The prototype for pthread_join is: int pthread_join(pthread_t thread, void **value_ptr); The first parameter is the handle to the thread to join to (the variable we created above, then filled-in with pthread_create above).

We use audioThreadReturn and videoThreadReturn pointers (by reference) to store the return status (pass/fail) from the join function. Since we want to return a value via this argument, we want to pass the (void pointer) argument by reference. To avoid getting an incompatible pointer type warning, we want to recast this argument – since this recasting can be a bit tricky for some of us, rather than have you figure it out by trial-and-error, here are the values to use for the second argument of each join function: (void **) &videoThreadReturn (void **) &audioThreadReturn

So, as an example, to join the video thread, you can write the following after “cleanup:” : pthread_join(videoThread, (void **) &videoThreadReturn);
 * Ensure all initMask #defines are completed for video and audio.
 * Save and close main.c.

Edit makefile_profile.mak
beagle$ make beagle$ ./videoThru_DEBUG.Beagle You should have simultaneous audio and video playing through the board.
 * Modify makefile_profile.mak to use the ALSA sound libraries by adding -lasound to CFLAGS.
 * Build and run the application
 * Press Ctrl-C to exit the application.

Demo your program

 * 1) Demo you program.
 * 2) Run htop in another window.  Why does videoThru_DEBUG.Beagle show up twice? How much of the CPU does it use?
 * 3) Run ps aux | grep vid.  What shows up?  Try ps aux -L | grep vid.  What can you tell about the threads?