Difference between revisions of "ECE434 Project - Audio Spectrum Display"

From eLinux.org
Jump to: navigation, search
(more content)
(2 intermediate revisions by the same user not shown)
Line 29: Line 29:
 
== Executive Summary ==
 
== Executive Summary ==
  
Picture that summarizes the project.
 
  
 +
[[File:Audio spectrum display 8x8.jpeg|thumb|The completed project in use on a breadboard]]
 
This project uses a microphone via an analog input, performs a FFT calculation on it and displays the output to a RGB LED matrix.  
 
This project uses a microphone via an analog input, performs a FFT calculation on it and displays the output to a RGB LED matrix.  
  
 
The portions of the project which work well include the microphone and the FFT. Using continuous analog input on the iio system, the microphone produces an array which is perfect for FFT work.  
 
The portions of the project which work well include the microphone and the FFT. Using continuous analog input on the iio system, the microphone produces an array which is perfect for FFT work.  
  
The RGB LED matrix refuses to work, at least with the PRU input. I have not tried with mmap in python yet but am working towards it if I cannot get it working quickly. As a last resort, I may end up using an ESP32 to drive the matrix as it has a supported arduino library.  
+
The RGB LED matrix refuses to work, at least with the PRU input. With mmap in python, it works better but not well enough to integrate with the audio processing. Instead, I used the 8x8 bi-color LED matrix found in the ECE434 kit, which required some code modification to accommodate the different display size.
 +
After some fine tuning of the scaling of magnitude and width of FFT buckets, the project is very good at displaying a rudimentary audio spectrum.  
  
End with a two sentence conclusion.
+
== Packaging ==
 
 
The sentence count is approximate and only to give an idea of the expected length.
 
  
== Packaging ==
+
I will put a fritzing diagram here when I get that worked out. It currently sits on a breadboard but could easily be compressed to a tiny box for portable operation, or attached to the back of the larger matrix display.  
If you have hardware, consider [http://cpprojects.blogspot.com/2013/07/small-build-big-execuition.html Small Build, Big Execuition] for ideas on the final packaging.
 
  
 
== Installation Instructions ==
 
== Installation Instructions ==
  
Give step by step instructions on how to install your project.
+
This project requires no more than the basic systems on board the Bone for the working version with the 8x8 matrix. an install.sh script is included for the python libraries used.
 
 
 
The Github link is here, it includes much more detail than is listed here. [https://github.com/dfirestorm/bone-audio-display https://github.com/dfirestorm/bone-audio-display].  
 
The Github link is here, it includes much more detail than is listed here. [https://github.com/dfirestorm/bone-audio-display https://github.com/dfirestorm/bone-audio-display].  
 
  
 
== User Instructions ==
 
== User Instructions ==
  
Once everything is installed, how do you use the program?  Give details here, so if you have a long user manual, link to it here.
+
to run, just run ./main.py , and use ctrl-c to stop. This program runs continuously and takes up about 8% CPU so autostart was not the greatest option.  
 
 
Consider making it autostart for full credit.
 
  
 
== Highlights ==
 
== Highlights ==
  
Here is where you brag about what your project can do.
+
This project is very capable of real-time audio spectrum display, and could be used to determine the acoustic characteristics of a room, voice, or instrument quite quickly though somewhat inelegantly. The code is incredibly modular and can easily be changed to output the raw values to the console, or to change the matrix size. There is a lot of variability in the tuning of the output as well, from the minimum and maximum dB scaling to the amount of low end frequency to ignore.  
  
 +
The FFT process is highly efficient and modular, and is coded in such a way that if one wished to change the display size or microphone, it is easily tunable to new parameters with just variables at the top of the code.
  
 
== Theory of Operation ==
 
== Theory of Operation ==
  
This program uses Python to interact with the iio analog system, which is convenient as it allows access to numpy for FFT. To output to the display, I scaled the FFT values to height of a column in the array and set thresholds for different colors.  
+
The way it works is by using the industrial io system on the beaglebone to generate a 96kHz audio signal, making a 481-point FFT of it with numpy, then splitting that FFT into a number of buckets according to the display width. The relative magnitude of the highest point in each bucket to the minimum and maximum determines its height, and that height relative to the display maximum height determines its color. A series of if-else statements produce a matrix to output to the display, with all of the code functions fully modular and variable to different displays so long as the format is compatible.
 +
 
 +
the code is set up to output an array with hex values for each column, one bit for each LED in the column and two hex values per column. The first hex value is green and the second is red, combining to make yellow between 50 and 75% of the maximum volume. The integration would need to send this array to the display to be output in the correct order.  It assumes the 0,0 is in the bottom right of the display so that may be a desired change for a future user. As well, the setup of the display would need to be updated if the array size would be different as the initial array is hard coded and it only gets modified as the code runs.  
  
 
== Work Breakdown ==
 
== Work Breakdown ==
  
I did this project solo, but the tasks are as follows:
+
I did this project solo, so I got all of the code and compiled it myself. The major tasks involved were making all of the functions modular, making the FFT to bucket to display code, and fine tuning all of the thresholds for the variables. I also wrote all of the documentation and wiring diagrams, as well as a lot of work on the 64x32 matrix to get that somewhat working even though in the end it did not work.
* Analog in: used the analogInContinuous function found in /opt/vsx-examples/examples/BeagleBone/Black
 
  
 
== Future Work ==
 
== Future Work ==
 
+
This project is incredibly modular so long as the display output is in a similar format to the one used by the Adafruit 8x8 matrix. The obvious continuation is to convert it to the 64x32 matrix, but another important addition would be tuning it to the standard 20Hz-20kHz human hearing and marking the audio peaks. The audio peaks would not be difficult but on an 8x8 display are a little awkward.  
Suggest addition things that could be done with this project.
 
  
 
== Conclusions ==
 
== Conclusions ==
  
Give some concluding thoughts about the project. Suggest some future additions that could make it even more interesting.
+
This was an enjoyable project, though working with the 64x32 matrix was much more difficult than expected. I would love to see someone get it working but it proved too much for myself. As well, this project could be interesting to port to other devices.  
  
 
{{YoderFoot}}
 
{{YoderFoot}}

Revision as of 09:22, 22 February 2023

thumb‎ Embedded Linux Class by Mark A. Yoder


Team members: Donald Hau

Grading Template

I'm using the following template to grade. Each slot is 10 points. 0 = Missing, 5=OK, 10=Wow!

Add Extras

09 Executive Summary
09 Packaging
09 Installation Instructions 
09 User Instructions
09 Highlights
09 Theory of Operation
09 Work Breakdown
09 Future Work/Conclusions
09 Hackster.io
09 Demo/Poster
00 Not Late

Score:  90/100

Executive Summary

The completed project in use on a breadboard

This project uses a microphone via an analog input, performs a FFT calculation on it and displays the output to a RGB LED matrix.

The portions of the project which work well include the microphone and the FFT. Using continuous analog input on the iio system, the microphone produces an array which is perfect for FFT work.

The RGB LED matrix refuses to work, at least with the PRU input. With mmap in python, it works better but not well enough to integrate with the audio processing. Instead, I used the 8x8 bi-color LED matrix found in the ECE434 kit, which required some code modification to accommodate the different display size. After some fine tuning of the scaling of magnitude and width of FFT buckets, the project is very good at displaying a rudimentary audio spectrum.

Packaging

I will put a fritzing diagram here when I get that worked out. It currently sits on a breadboard but could easily be compressed to a tiny box for portable operation, or attached to the back of the larger matrix display.

Installation Instructions

This project requires no more than the basic systems on board the Bone for the working version with the 8x8 matrix. an install.sh script is included for the python libraries used. The Github link is here, it includes much more detail than is listed here. https://github.com/dfirestorm/bone-audio-display.

User Instructions

to run, just run ./main.py , and use ctrl-c to stop. This program runs continuously and takes up about 8% CPU so autostart was not the greatest option.

Highlights

This project is very capable of real-time audio spectrum display, and could be used to determine the acoustic characteristics of a room, voice, or instrument quite quickly though somewhat inelegantly. The code is incredibly modular and can easily be changed to output the raw values to the console, or to change the matrix size. There is a lot of variability in the tuning of the output as well, from the minimum and maximum dB scaling to the amount of low end frequency to ignore.

The FFT process is highly efficient and modular, and is coded in such a way that if one wished to change the display size or microphone, it is easily tunable to new parameters with just variables at the top of the code.

Theory of Operation

The way it works is by using the industrial io system on the beaglebone to generate a 96kHz audio signal, making a 481-point FFT of it with numpy, then splitting that FFT into a number of buckets according to the display width. The relative magnitude of the highest point in each bucket to the minimum and maximum determines its height, and that height relative to the display maximum height determines its color. A series of if-else statements produce a matrix to output to the display, with all of the code functions fully modular and variable to different displays so long as the format is compatible.

the code is set up to output an array with hex values for each column, one bit for each LED in the column and two hex values per column. The first hex value is green and the second is red, combining to make yellow between 50 and 75% of the maximum volume. The integration would need to send this array to the display to be output in the correct order. It assumes the 0,0 is in the bottom right of the display so that may be a desired change for a future user. As well, the setup of the display would need to be updated if the array size would be different as the initial array is hard coded and it only gets modified as the code runs.

Work Breakdown

I did this project solo, so I got all of the code and compiled it myself. The major tasks involved were making all of the functions modular, making the FFT to bucket to display code, and fine tuning all of the thresholds for the variables. I also wrote all of the documentation and wiring diagrams, as well as a lot of work on the 64x32 matrix to get that somewhat working even though in the end it did not work.

Future Work

This project is incredibly modular so long as the display output is in a similar format to the one used by the Adafruit 8x8 matrix. The obvious continuation is to convert it to the 64x32 matrix, but another important addition would be tuning it to the standard 20Hz-20kHz human hearing and marking the audio peaks. The audio peaks would not be difficult but on an 8x8 display are a little awkward.

Conclusions

This was an enjoyable project, though working with the 64x32 matrix was much more difficult than expected. I would love to see someone get it working but it proved too much for myself. As well, this project could be interesting to port to other devices.




thumb‎ Embedded Linux Class by Mark A. Yoder