Difference between revisions of "ECE434 Project: Dancing Fish"

From eLinux.org
Jump to: navigation, search
(Conclusions)
 
(67 intermediate revisions by the same user not shown)
Line 3: Line 3:
 
{{YoderHead}}
 
{{YoderHead}}
  
Team members: [[user:Romeueh|Eliza Romeu]], Melina Ferner, Kurtis Vonbargen
+
Team members: [[user:Romeueh|Eliza Romeu]], [[user:fernerme|Melina Ferner]], [[user:vonbarke|Kurtis VonBargen]]
  
 
== Grading Template ==
 
== Grading Template ==
Line 10: Line 10:
  
 
<pre style="color:red">
 
<pre style="color:red">
09 Executive Summary
+
00 Executive Summary
09 Packaging
+
00 Packaging
09 Installation Instructions  
+
00 Installation Instructions  
09 User Instructions
+
00 User Instructions
09 Highlights
+
00 Highlights
09 Theory of Operation
+
00 Theory of Operation
09 Work Breakdown
+
00 Work Breakdown
09 Future Work/Conclusions
+
00 Future Work/Conclusions
09 Hackster.io
+
00 Hackster.io
09 Demo/Poster
+
00 Demo/Poster
 
00 Not Late
 
00 Not Late
  
Score:  90/100
+
Score:  00/100 (not graded yet)
 
</pre>
 
</pre>
  
 
== Executive Summary ==
 
== Executive Summary ==
  
This project is making a device using the Beaglebone which can stream music. In order to make the music being played more interactive, the device will feature a fish that moves on beat with the music and a sound-activated LED Music spectrum to reflect the different tones that are being played.
+
This project is making a device using the Beaglebone which can stream any chosen audio. The music will come from a specified youtube video and will be played on a wired speaker. In order to make the music being played more interactive, the device will feature a fish that moves on beat with the music controlled by a servo. Additionally, we will have a sound-activated LED Music spectrum (on two different 8x8 LED Matrices) to reflect the different tones that are being played.
 +
 
 +
Attached is a picture of the final device:
 +
 
 +
[[File:final_project.jpg|Flow Chart|500px]]
 +
 
 +
== Timeline ==
 +
 
 +
{| border="1"
 +
|10/29
 +
|Order Parts
 +
|-
 +
|11/4
 +
|Install necessary packages on Bone
 +
|-
 +
|11/6
 +
|Create an interface for led Matrices
 +
|-
 +
|11/7
 +
|Create an interface for servo
 +
|-
 +
|11/9
 +
|Play music from youtube on speaker
 +
|-
 +
|11/12
 +
|Create a python file for FFT of music
 +
|-
 +
|11/13
 +
|Combine the interfaces for the completed project
 +
|-
 +
|11/14
 +
|Debugging
 +
|-
 +
|11/17
 +
|Finish documentation
 +
|}
  
 
== Packaging ==
 
== Packaging ==
If you have hardware, consider [http://cpprojects.blogspot.com/2013/07/small-build-big-execuition.html Small Build, Big Execuition] for ideas on the final packaging.
+
 
 +
This project uses:
 +
* [https://www.sparkfun.com/products/12857 Beaglebone Black]
 +
* [https://www.adafruit.com/product/1632 Two 8x8 LED Matrices]
 +
* [https://www.adafruit.com/product/169 RC Servo Motor]
 +
* A wired Speaker or Headphones (We used [https://www.amazon.com/DOSS-Wireless-Bluetooth-Portable-Speaker/dp/B01CQOV3YO this speaker])
 +
* Any cords needed to attach the speaker to a USB 2 port
 +
* Movable Fish and any materials needed to attach to the servo
 +
 
 +
 
 +
Attached is a picture of the packaging that we made for the device:
 +
 
 +
[[File:fish_circuit_diagram.jpg|Flow Chart|500px]]
  
 
== Installation Instructions ==
 
== Installation Instructions ==
  
Give step by step instructions on how to install your project.   
+
To install the project, ssh into your Beaglebone and follow the instructions below:
 +
 
 +
1.) Clone this [https://github.com/rhit-vonbarke/ece434_BigFish github] repository
 +
 
 +
  bone$ git clone https://github.com/rhit-vonbarke/ece434_BigFish
  
* Include your [https://github.com/ github] path as a link like this to the read-only git site:  [https://github.com/MarkAYoder/gitLearn https://github.com/MarkAYoder/gitLearn].  
+
2.) Navigate to the project directory and run install.sh. This only needs to be run the first time that you are running the program as it installs all of the needed libraries.
* Be sure your README.md is includes an up-to-date and clear description of your project so that someone who comes across you git repository can quickly learn what you did and how they can reproduce it.
+
 
* Include a Makefile for your code if using C.
+
bone$ bash install.sh
* Include any additional packages installed via '''apt'''. Include '''install.sh''' and '''setup.sh''' files.
+
 
* Include kernel mods.
+
3.) Run setup.sh. This needs to be run every time after connecting to the bone before you want to run the program.
* If there is extra hardware needed, include links to where it can be obtained.
+
 
 +
bone$ sudo bash setup.sh
 +
 
 +
4.) Find a youtube video you would like to listen to and copy the youtube ID within the link. For example, for the link https://youtu.be/I4ZDWBBBsJ4, the youtube id is I4ZDWBBBsJ4.
 +
 
 +
5.) Run the program using the previously copied youtube id (this needs to be rerun every time you want to play a new song). Additionally, the -d specifies that the video needs to be downloaded before playing so it can be discluded if you have already listened to the song once.
 +
 
 +
  bone$ ./runvisualizer [video-id] [-d]
 +
 
 +
Once you've done the above three steps, you can proceed to the user instructions.
  
 
== User Instructions ==
 
== User Instructions ==
  
Once everything is installed, how do you use the program?  Give details here, so if you have a long user manual, link to it here.
+
Once the program is running, there will be a waiting period for the music to start so that the audio processing can be synced and if a new song file needs to be downloaded. However, all the user needs to do at this point in time is sit back and enjoy the music. After the song is finished, if the user wants to listen to a new song, all they have to do is rerun the main program and enter in another Youtube link or the same Youtube link to listen to.
  
Consider making it autostart for full credit.
+
== Highlights ==
  
== Highlights ==
+
We've included a [https://youtu.be/I4ZDWBBBsJ4 video] demoing our project in action.
  
Here is where you brag about what your project can do.
+
We've also included our [https://www.hackster.io/littlestpetcat/dancing-fish-caf63e/ hackster.io] page.
  
Include a [http://www.youtube.com/ YouTube] demo the audio description.
+
Additionally, our project includes a lot of behind-the-scenes optimizations. The LED Matrix display is logarithmic so a very large range of frequencies can be displayed, meaning the spectrum is more adaptable to different types of audio. Also, even though the fish's movements might sometimes seem sporadic, its movements are based on the magnitude of the frequencies in the human speaking range. This means that the fish is able to more or less detect when a person is singing or talking and dance accordingly. Further, both the LED Matrix and the servo have a 'moving' largest value within the code so if parts of the song are very quiet (or even the whole song), the spectrum and servo will still be able to use their full range of values.
  
 
== Theory of Operation ==
 
== Theory of Operation ==
  
Give a high level overview of the structure of your software. Are you using GStreamer?  Show a diagram of the pipeline. Are you running multiple tasks?  Show what they do and how they interact.
+
The user interacts with the computer terminal and selects a song from Youtube. A script handles the input and uses the YoutubeDL library to download a .wav file into the audio downloads folder if the -d tag was added, if not it pulls from the existing file in the audio downloads directory. Once the file is downloaded, the script runs two processes in parallel: a python file for the audio processing and a script to play the music. Both the python file and the script are given the starting time of the execution so they can both start playing 10 seconds after, this is to allow the processing to sync up with the actual audio. Within the python script, we compute the Fourier transform of the audio with the help of various libraries, and some mathematical specifications are added based on the optimizations that were described earlier in the highlights section. Once the Fourier transform is computed on periodic intervals of the audio, the python file calls two different interfaces: one to handle the LED Matrices output and the other for the servo motor output.
 +
 
 +
[[File:Fish.jpg|Flow Chart|500px]]
  
 
== Work Breakdown ==
 
== Work Breakdown ==
  
List the major tasks in your project and who did what.
+
All three partners worked together on most of the major tasks but below is the breakdown of team members main focuses:
  
Also list here what doesn't work yet and when you think it will be finished and who is finishing it.
+
==== Melina ====
 +
* LED Matrix Interface
 +
* Servo Interface
 +
* Speaker Interface
 +
* Music playing to work with Audio Processing
 +
* Setup and Installation Files
 +
 
 +
==== Eliza ====
 +
* Speaker Interface
 +
* Documentation
 +
* Music playing to work with Audio Processing
 +
* Setup and Installation Files
 +
 
 +
==== Kurtis ====
 +
* Youtube-DL Setup
 +
* Fourier Transform Handling
 +
* Script files (download, parallel playing, delay)
 +
* Music playing to work with Audio Processing
 +
* Setup and Installation Files
  
 
== Future Work ==
 
== Future Work ==
  
Suggest addition things that could be done with this project.
+
Some additional things that could be done with this project in the future:
 +
* Work to interface the program with streaming services other than Youtube (i.e. Spotify, Apple Music, Pandora, etc.)
 +
* Find ways to play music remotely, unfortunately, the beagle bone black doesn't support Bluetooth without extra handware but Flask or Blynk would be options
 +
* Add more capabilities related to playing the music like pausing, playing skipping, etc.
 +
* Display the cover of the music video on an LCD or any other device with a screen
  
 
== Conclusions ==
 
== Conclusions ==
  
Give some concluding thoughts about the project. Suggest some future additions that could make it even more interesting.
+
Our initial goals for this project were loosely set since all of our inspiration came from a viral video of a singing fish. Over time, we realized that some project goals were simply unrealistic given equipment and time. However, our overall goal was to produce a device that plays music, makes a fish dance, and makes an LED matrix react to the music in real-time. We did successfully meet our goals, but there is still some room for improvement. As mentioned in the future work section, it would be really cool to interface the program with streaming services other than Youtube and work on our current timing delay issues.
 +
 
 +
Through the course of this project, we learned a lot about the Beagle Bone, Linux capabilities, scriptwriting, and even how to redirect group work when something goes wrong. At one point during the project, we realized that the Beagle Bone Black does not have Bluetooth capabilities. At this point in time, it was too close to the deadline to buy new parts so we had to undergo a last-minute scramble trying to find cords that would allow for functionality with the speaker and had to scratch our entire plan of interfacing with the speaker. This taught us to be adaptable and how to work quickly when learning an unfamiliar library. One of the biggest areas of learning was running processes in parallel and dealing with delays. After, getting the speaker to function we realized that it paused the whole terminal as the song was playing so we had to research how to run processes in parallel in order for our audio processor to function at the same time. After we were able to get this to function, we underwent the long struggle of learning how to sync up code in a python script and in a shell script. Throughout this process, we generated many different processes to manage to delay (some of which were simply hard coding), we underestimated how long processing (specifically array coping) could take, and learned how to feed a timestamp input to both python and shell scripts. In the end, we were able to sync up the processing after much trial and error, learning a lot along the way.
  
 
{{YoderFoot}}
 
{{YoderFoot}}

Latest revision as of 21:22, 16 November 2021

thumb‎ Embedded Linux Class by Mark A. Yoder


Team members: Eliza Romeu, Melina Ferner, Kurtis VonBargen

Grading Template

I'm using the following template to grade. Each slot is 10 points. 0 = Missing, 5=OK, 10=Wow!

00 Executive Summary
00 Packaging
00 Installation Instructions 
00 User Instructions
00 Highlights
00 Theory of Operation
00 Work Breakdown
00 Future Work/Conclusions
00 Hackster.io
00 Demo/Poster
00 Not Late

Score:  00/100 (not graded yet)

Executive Summary

This project is making a device using the Beaglebone which can stream any chosen audio. The music will come from a specified youtube video and will be played on a wired speaker. In order to make the music being played more interactive, the device will feature a fish that moves on beat with the music controlled by a servo. Additionally, we will have a sound-activated LED Music spectrum (on two different 8x8 LED Matrices) to reflect the different tones that are being played.

Attached is a picture of the final device:

Flow Chart

Timeline

10/29 Order Parts
11/4 Install necessary packages on Bone
11/6 Create an interface for led Matrices
11/7 Create an interface for servo
11/9 Play music from youtube on speaker
11/12 Create a python file for FFT of music
11/13 Combine the interfaces for the completed project
11/14 Debugging
11/17 Finish documentation

Packaging

This project uses:


Attached is a picture of the packaging that we made for the device:

Flow Chart

Installation Instructions

To install the project, ssh into your Beaglebone and follow the instructions below:

1.) Clone this github repository

bone$ git clone https://github.com/rhit-vonbarke/ece434_BigFish

2.) Navigate to the project directory and run install.sh. This only needs to be run the first time that you are running the program as it installs all of the needed libraries.

bone$ bash install.sh

3.) Run setup.sh. This needs to be run every time after connecting to the bone before you want to run the program.

bone$ sudo bash setup.sh

4.) Find a youtube video you would like to listen to and copy the youtube ID within the link. For example, for the link https://youtu.be/I4ZDWBBBsJ4, the youtube id is I4ZDWBBBsJ4.

5.) Run the program using the previously copied youtube id (this needs to be rerun every time you want to play a new song). Additionally, the -d specifies that the video needs to be downloaded before playing so it can be discluded if you have already listened to the song once.

bone$ ./runvisualizer [video-id] [-d]

Once you've done the above three steps, you can proceed to the user instructions.

User Instructions

Once the program is running, there will be a waiting period for the music to start so that the audio processing can be synced and if a new song file needs to be downloaded. However, all the user needs to do at this point in time is sit back and enjoy the music. After the song is finished, if the user wants to listen to a new song, all they have to do is rerun the main program and enter in another Youtube link or the same Youtube link to listen to.

Highlights

We've included a video demoing our project in action.

We've also included our hackster.io page.

Additionally, our project includes a lot of behind-the-scenes optimizations. The LED Matrix display is logarithmic so a very large range of frequencies can be displayed, meaning the spectrum is more adaptable to different types of audio. Also, even though the fish's movements might sometimes seem sporadic, its movements are based on the magnitude of the frequencies in the human speaking range. This means that the fish is able to more or less detect when a person is singing or talking and dance accordingly. Further, both the LED Matrix and the servo have a 'moving' largest value within the code so if parts of the song are very quiet (or even the whole song), the spectrum and servo will still be able to use their full range of values.

Theory of Operation

The user interacts with the computer terminal and selects a song from Youtube. A script handles the input and uses the YoutubeDL library to download a .wav file into the audio downloads folder if the -d tag was added, if not it pulls from the existing file in the audio downloads directory. Once the file is downloaded, the script runs two processes in parallel: a python file for the audio processing and a script to play the music. Both the python file and the script are given the starting time of the execution so they can both start playing 10 seconds after, this is to allow the processing to sync up with the actual audio. Within the python script, we compute the Fourier transform of the audio with the help of various libraries, and some mathematical specifications are added based on the optimizations that were described earlier in the highlights section. Once the Fourier transform is computed on periodic intervals of the audio, the python file calls two different interfaces: one to handle the LED Matrices output and the other for the servo motor output.

Flow Chart

Work Breakdown

All three partners worked together on most of the major tasks but below is the breakdown of team members main focuses:

Melina

  • LED Matrix Interface
  • Servo Interface
  • Speaker Interface
  • Music playing to work with Audio Processing
  • Setup and Installation Files

Eliza

  • Speaker Interface
  • Documentation
  • Music playing to work with Audio Processing
  • Setup and Installation Files

Kurtis

  • Youtube-DL Setup
  • Fourier Transform Handling
  • Script files (download, parallel playing, delay)
  • Music playing to work with Audio Processing
  • Setup and Installation Files

Future Work

Some additional things that could be done with this project in the future:

  • Work to interface the program with streaming services other than Youtube (i.e. Spotify, Apple Music, Pandora, etc.)
  • Find ways to play music remotely, unfortunately, the beagle bone black doesn't support Bluetooth without extra handware but Flask or Blynk would be options
  • Add more capabilities related to playing the music like pausing, playing skipping, etc.
  • Display the cover of the music video on an LCD or any other device with a screen

Conclusions

Our initial goals for this project were loosely set since all of our inspiration came from a viral video of a singing fish. Over time, we realized that some project goals were simply unrealistic given equipment and time. However, our overall goal was to produce a device that plays music, makes a fish dance, and makes an LED matrix react to the music in real-time. We did successfully meet our goals, but there is still some room for improvement. As mentioned in the future work section, it would be really cool to interface the program with streaming services other than Youtube and work on our current timing delay issues.

Through the course of this project, we learned a lot about the Beagle Bone, Linux capabilities, scriptwriting, and even how to redirect group work when something goes wrong. At one point during the project, we realized that the Beagle Bone Black does not have Bluetooth capabilities. At this point in time, it was too close to the deadline to buy new parts so we had to undergo a last-minute scramble trying to find cords that would allow for functionality with the speaker and had to scratch our entire plan of interfacing with the speaker. This taught us to be adaptable and how to work quickly when learning an unfamiliar library. One of the biggest areas of learning was running processes in parallel and dealing with delays. After, getting the speaker to function we realized that it paused the whole terminal as the song was playing so we had to research how to run processes in parallel in order for our audio processor to function at the same time. After we were able to get this to function, we underwent the long struggle of learning how to sync up code in a python script and in a shell script. Throughout this process, we generated many different processes to manage to delay (some of which were simply hard coding), we underestimated how long processing (specifically array coping) could take, and learned how to feed a timestamp input to both python and shell scripts. In the end, we were able to sync up the processing after much trial and error, learning a lot along the way.




thumb‎ Embedded Linux Class by Mark A. Yoder