Difference between revisions of "ECE597 Project NodeRed Voice"
| Line 98: | Line 98: | ||
== Theory of Operation == | == Theory of Operation == | ||
| − | + | Our project uses the opensource Jasper project as it's backbone. The system is constantly listening to it's surroundings, and when it finds a discrete sound block sends it to the google voice API for analysis. Google voice returns text blocks of what it thought the sound click was of. Jasper then runs it across modules looking for a hit, and then executes a command if it finds one. | |
==Completed Tasks== | ==Completed Tasks== | ||
Revision as of 11:54, 13 November 2014
Embedded Linux Class by Mark A. Yoder
Team members: Dmitry , Matt Olson
Contents
Grading Template
I'm using the following template to grade. Each slot is 10 points. 0 = Missing, 5=OK, 10=Wow!
00 Executive Summary 00 Installation Instructions 00 User Instructions 00 Highlights 00 Theory of Operation 00 Work Breakdown 00 Future Work 00 Conclusions 00 Demo 00 Late Comments: I'm looking forward to seeing this. Score: 10/100
(Inline Comment)
Executive Summary
Our project has branched out along several paths since it's inception. The goal was to create a personal assistant that could listen to commands, talk back, and perform them. For modularity, our original goal was to use IBM's Node Red to accomplish this. However, Node Red is not designed for the complicated and workflows that we needed, and extensions were not scale-able. In this document we are going to describe three things.
- A simple node red speech recognition and synthesis system
- Using the open source jasper project on Raspberry Pi
- Using the open source Jasper project on Beaglebone Black
The simple node red speech synthesis will essentially listen to the user, and the repeat back what is said. Currently the speech synthesis is working, and we are trying to integrate pocket sphinx for the speech recognition.
Jasper was developed with the Raspberry Pi, and as a result has a very simple set up procedure. Our goal is too use this platform to begin developing modules. We currently have working versions of jasper installed and have 2 modules developed.
There are no reliable installation instructions for setting up Jasper on the Beaglebone black. We have done the research, and are hoping to be able to provide concise installation instructions and port over our work.
Packaging
Jerry Talyor of Mansfield, IN is going to make a fantastic wooden laser cut and laser engraved box for us.
Installation Instructions
Installing Node Red on Beaglebone Black
Setting up Wifi "=In Progress="
Installing Node Red
host$ git clone https://github.com/node-red/node-red.git host$ cd node-red host$ npm install --production
Installing extra Nodes "=In Progress="
Installing PocketSphinx
Download latest realizes from CMU Repo of both sphinxbase and pocketsphinx. The version currently being used is 0.8 Go to sphinxbase root directory and run
./autoconf.sh ./configure.sh make make install
Installing Jasper on raspberry Pi
Follow the instructions provided on the documentation page to install JASPER http://jasperproject.github.io/documentation/software/
The modules we have implemented use the python RPI.gpio library, run the following commands to install it
sudo apt-get update sudo apt-get install python-dev sudo apt-get install python-rpi.gpio
Installing our custom Modules to demonstrate Jasper
git clone https://github.com/dmitryvv/woodoo cd woodoo chmod +x install_modules_RPi.sh ./install_modules_RPi.sh
Installing Jaser on Beaglebone Black
See above details for setting up wifi and installing PocketSphinx.
User Instructions
Jasper on Raspberry Pi Instructions
To start jasper, move to the jasper directory and run
python jasper.py
Highlights
Here is where you brag about what your project can do.
Include a YouTube demo.
Theory of Operation
Our project uses the opensource Jasper project as it's backbone. The system is constantly listening to it's surroundings, and when it finds a discrete sound block sends it to the google voice API for analysis. Google voice returns text blocks of what it thought the sound click was of. Jasper then runs it across modules looking for a hit, and then executes a command if it finds one.
Completed Tasks
Matt Olson
- Node Red Installed - 10/20/2014
- Wi-fi dongle working - 10/22/2014
- Created Module for Jasper - 10/28/2014
- Created install script for installing modules on RPi - 11/12/2014
Dmitry Votinstev
- Speech recognition software installed - 10/20/2014
- Sound recording/payback with arecord aplay works 10/22/2013
Tasks Remaining
- Pass input text into Node Red - Matt
- Produce text from speech synethsizer - Dmitry
Future Work
Suggest addition things that could be done with this project.
Conclusions
Node Red does not provide a very good architecture for acting as a speech recognition system and dispatcher, which resulted in our choice to investigate other options.
Embedded Linux Class by Mark A. Yoder
