EBC Exercise 17 Using ALSA for Audio Processing

From eLinux.org
Revision as of 09:34, 12 September 2012 by Yoder (talk | contribs) (Part a - Audio Record)
Jump to: navigation, search

thumb‎ Embedded Linux Class by Mark A. Yoder


This lab demonstrate the Linux ALSA driver as well as basic file I/O. Parts a and b are inspection labs. While c requires you to combine the previous two parts.

  • Part a analyzes the function calls necessary to record audio from line input to a file.
  • Part b examines the function calls necessary to playback audio from a recorded audio file.
  • Part c combines a and b into a single application that loops the audio from input to output (i.e. audio loop-thru) without recording to a file. For an extra challenge, advanced users may want to try to build c without referring to the procedure.

Much of this lab was adapted for the Beagle from OMAP™/DaVinci™ System Integration using Linux Workshop.

This looks like a nice ALSA reference.

Part a - Audio Record

The goal of this part is to analyze the function calls necessary to record audio from the Beagle's a line input to a file.


Note: The 1/8 inch audio in jack on the Beagle is a line in jack, not a microphone in. It doesn't not have the extra amplification needed for a microphone.


beagle$ cd exercises
beagle$ git pull
beagle$ cd audioThru
beagle$ ls

(See EBC Exercise 05 Getting Exercise Support Materials if this doesn't work.)

Make sure the ALSA utils are installed

beagle$ opkg update
beagle$ opkg install alsa-utils-aplay alsa-utils-amixer
  • Change directories to lab06a_audio_record
  • List the files used to build this application:
beagle$ cd lab06a_audio_record
beagle$ ls

File Inspection

  • Use a text editor to examine the new files in this application.
beagle$ gedit *.c *.h


main.c

This is the entry point for the application. main() does the following:

  1. Creates a signal handler to trap the Ctrl-C signal (also called SIGINT, the interrupt signal). When this signal is sent to the application, the audioEnv.quit global variable is set to true to signal the audio thread to exit its main loop and begin cleanup.
  2. Calls the audio_thread_fxn() function to enter into the audio function.
  3. Upon completion of this function, the main routine checks – and reports – success or failure returned from the audio function.

audio_thread.c

audio_thread_fxn( ) encapsulates the code required to run the audio recorder. The lab06a_audio_recorder application is single-threaded, so the motivation for encapsulation in this manner may not be initially obvious. We will see in later labs – when combining audio and video in a multi-threaded program – why declaring this function (as opposed to running everything from main) is useful.

audio_thread_fxn utilizes the following:

audio_io_setup
Opens and configures the audio input driver. See audio_input_output.c for details.
while()
will execute until the envPtr->quit global variable is set to true.
    • Inside the while() loop, snd_pcm_readi() is used to read data from the audio input driver (ALSA) and fwrite() is used to write the data into a file. The i in snd_pcm_readi() means the left and right channels of the stereo sign are interleaved. At is, one 16-bit value is the left channel, the next 16-bit value is the right channel.
    • When the envPtr->quit variable is set to true (occurs when the user presses Ctrl-C in the terminal) this capture (record) process exits and the application proceeds to the cleanup phase before exiting.
initMask

It goes without saying, writing robust code – and debugging it – can be a tedious chore; it is further exasperated when using printf() statements as the primary means of providing debug information back to the programmer. To this end, we have employed an initMask to help keep track of resources opened (and closed) during the program.

The audio_thread_fxn() uses an initialization mask (initMask) to keep track of how many resources have been opened and initialized. Each bit in the mask corresponds to a resource; the bit positions in the initMask variable are #defined towards the top of the file.

/* The levels of initialization for initMask */
#define ALSA_INITIALIZED 0x1
#define INPUT_BUFFER_ALLOCATED 0x2
#define OUTPUT_FILE_OPENED 0x4
/* Only used to cleanup items that were initialized */
unsigned int initMask = 0x0;
    • When you OR the initMask with a #define'd value, the associated bit will get set in the initMask variable. For example,

InitMask.png

This is useful so that if an error occurs, the application will not attempt to close or free resources that were never opened or allocated. If you look down at the cleanup part of our audio_thread.c, you’ll see how we used the initMask variable to accomplish this.

Build and Run the application

  • Build the application using make, i.e. “make all”.
  • Test your audio connection:
beagle$ arecord –f cd | aplay –f cd

This command uses the arecord (“ALSA recorder”) utility to capture audio and, instead of sending to a file, uses a Linux process pipe to send the data to the aplay (“ALSA player”) application. This will loop audio through the board. If you have a working audio input and the board is connected to a speaker, you should hear the audio play over the speakers. arecord defaults to using the Beagle's on-board audio. To use the mikes on the PS EYE use the following to switch input devices.

beagle$ arecord -D plughw:1,0 | aplay

Press ctrl-c to quit.

On arecord and aplay, the –f option lets you change the format:

Quality # Channels Bits Rate
default mono 8-bit 8 KHZ
cd stereo 16-bit 44.1 KHz
dat stereo 16-bit 48 KHz
  • Execute the ./app_DEBUG.Beagle application.

The application is hard-coded (using a #define statement in audio_thread.c) to save the audio data to the file /tmp/audio.raw. Execute the application.

  • Press Ctrl-C to exit the application.

After a suitable amount of time press Ctrl-C in the terminal to exit from the application. You can list the /tmp/audio.raw file with the –lsa options to see the size of the file and verify that it has recorded properly:

beagle$ ls –lsa /tmp/audio.raw

Recall that a signal handler was placed in main.c to trap the SIGINT (Ctrl-C) signal. When Ctrl-C is placed, this signal handler will execute, signaling the audio thread to exit its main loop, proceed to cleanup, and then exit.

  • Use the Linux aplay utility to confirm successful recording.

Because this application saves the audio as a raw stream you may also check that the record has operated properly using the aplay utility.

beagle$ aplay –f dat /tmp/audio.raw

DBG vs ERR

Let’s explore the debugging features we’re using in our lab files. We are using two macros defined in the file debug.h. They are DBG() and ERR() – essentially, they are wrapper functions around an fprintf() function.

  • Add a new debug statement to your file.

In main.c, immediately after the signal handler function, add a DBG() statement:

// Set the signal callback for Ctrl-C
pSigPrev = signal( SIGINT, signal_handler );
DBG( "Registered SIGINT signal handler.\n" );
  • Build both debug and release profiles on the Beagle, run then and compare their outputs.
beagle$ make                  // Defaults to making the DEBUG version
beagle$ PROFILE=RELEASE make  // Make the RELEASE version
beagle$ ./app_DEBUG.Beagle    // Runs the DEBUG version
beagle$ ./app_RELEASE.BEAGLE  // Runs the RELEASE version

Does your new statement show up in the terminal when you execute the program app_DEBUG.Beagle? Does it show up when app_RELEASE.Beagle?

  • Switch from DBG() to ERR(), then once again, build, run and compare both profiles. What is the difference between DBG and ERR?
  • Either Delete the new ERR() statement, or switch it back to DBG().

We don’t really need this statement, so feel free to remove it. On the other hand, if you want to leave the new debugging statement, we recommend that you, at the very least, change it back to a DBG() statement.

Part b - Audio Playback

The goal of this part is to analyze the function calls necessary to play back audio from a recorded file to the driver.

  • Copy the AudioThru files from here to your Beagle.
  • Change directories to AudioThru/lab06b_audio_playback
  • Use a text editor to examine audio_thread.c.

Only audio_thread.c has changed from the lab06a_audio_record application. Let’s look at some of the differences:

    • The ‘create’ part of the audio_thread_fxn():
      • Uses the fopen() function call to open a file for playback.
      • Uses audio_io_setup() and configure the audio output driver.
      • The malloc() function allocates a RAM buffer to store the audio data from the input file before it is written to the audio driver.
    • Inside the while() loop:
      • fread() method is used to read audio data from the input file (/tmp/audio.raw)
      • snd_pcm_writei() method is used to write the data to the ALSA driver. The i, like in the snd_pcm_readi(), mean the data is interleaved.
      • When the envPtr->quit variable is set to true, the loop exits. (This occurs when the user presses Ctrl-C in the terminal.)
    • Finally, review the “cleanup” phase, which runs right before exiting. (This basically undo’s the steps in the create/setup phase).

Build and Run the Application

  • Build the application using make.
  • Make sure that audio.raw was created properly.

List the contents of /tmp with the “-lsa” flags setting to verify that /tmp/audio.raw exists and has a greater than zero filesize.

The application is hard coded (using a #define statement in audio_thread.c) to read data from the file /tmp/audio.raw.


Note: If the Beagle is rebooted after running the lab6a_audio_record application, the /tmp/audio.raw file is erased. If this happens, run it again in either DEBUG or RELEASE mode to re-record the /tmp/audio.raw file

Finally, you can return to lab06b_audio_playback and run the playback utility.


  • Execute the ./app_DEBUG.Beagle application.

The application should play back the audio that was recorded in lab06a_audio_record and then exit. If you do not wish to hear all of the audio, press Ctrl-C to exit.

Questions about: audio_thread.c

  1. Which function call is used in the while() loop to read audio data from the line input via the ALSA driver?
  2. In the while( ) loop there is an fread() function (similar to fwrite() function, lab06a). For the file read (or write) function, a FILE pointer is the last parameter passed. What is the purpose of the FILE pointer, and where does it come from? (In other words, what function is used to generate valid FILE pointers from which read and write operations can be made?)

Part c - Audio Loopthru

In this part, you will combine parts a and b into a single loopthru application.

What do we need to change?

Before we start copying, cutting, and pasting files and code, let’s think about what must be done to get the loop-thru lab to work.

  • In our Lab06a_audio_record application, we used fwrite() to PUT (i.e. write) the audio data to audio.raw. Which function was used to GET (read) the audio data from the ALSA driver?
GET audio data: ________________________________________________
PUT audio data: fwrite() inputBuffer -> audio.raw
  • Similarly, in the Lab06b_audio_plaback application we used the snd_pcm_writei() function to PUT the data to the ALSA driver. But, how did we GET (i.e. read) the audio data?
GET audio data: ________________________________________________
PUT audio data: snd_pcm_writei() outputBuffer -> hSound
  • Now, in Lab06c_audio_loopthru we want to create an audio pass-thru application. Which two functions should be used to read and write data to/from ALSA driver?
GET audio data: ________________________________________________
PUT audio data: ________________________________________________

File Management

  • Begin by copying all files from lab06b_audio_playback into lab06c_audio_loopthru with the following:
beagle$ cd ..
beagle$ mkdir –p lab06c_audio_loopthru
beagle$ cp –R –f lab06b_audio_playback/* lab06c_audio_loopthru

The mkdir “–p” option prevents an error if the directory already exists.

The cp “-R” options says to recurse directories, while the “-f” option forces over write if the file already exists.


Note: Since lab06c is a combination of lab06a and lab06b - and only file that differs between them is audio_thread.c. Sure, you could have copied the files in reverse order, but we highly recommend copy them in the order above so that the following steps are consistent with your files/directories.

Modify the files

Modify the audio_thread.c and the other files as needed to make it work.

Build and Test

  • Build and run the application.
beagle$ make
beagle$ app_DEBUG.Beagle

You should hear audio playing.

Assignment

There are many interesting things you can do now that you have the framework for bringing live audio into and out of the Beagle. It's easy to get buffer overrun or underrun errors (called xrun errors). Overrun errors are caused by not reading the input buffer often enough. Underrun errors are caused by not sending data to the output fast enough. It's already sent everything and is waiting for more. I was able to eliminate these errors by doing doing some dummy reads and writes before the main processing loop.

Required

  1. Add a snd_pcm_readi and some snd_pcm_writeis of 0 values before the main processing loop to eliminate the xrun errors. Report on you success. How many writeis did you need?
  2. What percentage of the CPU is used to pass audio through? Use htop to esitmate the total % of CPU time.
  3. With ALSA you request a certain buffer size (BLOCKSIZE) the drivers tell you what size you get. These two values may not be the same. What buffer size (both input and output) do you get when using the Beagle's built in audio (plughw:0,0)?
  4. What buffer size do you get for the PS EYE (plughw:1,0)? Increase your requested buffer size. What' the largest buffer the PS EYE will return?
  5. In the variable period in audio_input_output.c controls how many times per buffer the CPU is interrupted. A larger value would mean more interruptions, but less data transferred per interrupt. What's the smallest value of period that works?
  6. Does making period large increase the CPU usage?
  7. Try changing the sampling rate. How does the % CPU change when switching from 48K to 8K sample per second?
  8. Verify that the left input is going to the left speaker and the right input to the right. Replace the memcpy() with code to switch the left and right channels.
  9. Write code the independently change the gain on the left and right channels.

Optional

  1. What is the delay, in ms, from the input to the output? The best way to measure this is to use the oscilliscope to display the signal going into the microphone and the signal coming out of the speakers.
  2. How does the buffer size (initially 48000) impact this delay. Try changing it and keep detailed records of your experiments.
  3. What's the minimum delay you can get an not drop samples? Report values for 8k and 48k sampling rates.
  4. Write code to delay one channel by N samples. Be careful, You'll need to carry some samples over from one buffer to the next.
  5. Apply a filter to the audio.

Reference

Here's a good article on ALSA.




thumb‎ Embedded Linux Class by Mark A. Yoder