Difference between revisions of "BeagleBoard/GSoC/2019ProposalTemplate"

From eLinux.org
Jump to: navigation, search
(Modern "Speak & Spell" using Pocketbeagle)
(Proposal)
Line 19: Line 19:
  
 
=Proposal=
 
=Proposal=
 
+
Please complete the requirements listed on the [http://bbb.io/gsocideas| ideas page] and fill out this template.
Task Completion:<br>
 
A "Hello World" application has been created,cross-compiled using GCC-ARM tool chain and executed using QEMU. A Pull request has been generated for the same.Please find the link here.<br />
 
  
  
  
 
==About you==
 
==About you==
''IRC'': jm_donthineni,jyothi<br>
+
''IRC'': IRC Username Goes Here<br>
''Github'':https://github.com/jyothidonthineni <br>
+
''Github'': Github Account<br>
''School'': National Institute of Technology,Karnataka,Surathkal<br>
+
''School'': School name <br>
''Country'': India<br>
+
''Country'': Country<br>
''Primary language'' (We have mentors who speak multiple languages): English<br>
+
''Primary language'' (We have mentors who speak multiple languages): Language<br>
''Typical work hours'' (We have mentors in various time zones): 2:30 AM to 11.30 AM UTC<br>
+
''Typical work hours'' (We have mentors in various time zones): 8AM-5PM US Eastern<br>
''Previous GSoC participation'': No previous experience but excited to join the open source community.
+
''Previous GSoC participation'': Provide list of URLs for previous participation or tell us why you want to participate here. No previous experience required.
  
  
 
==About your project==
 
==About your project==
''Project name'':  Modern "Speak & Spell" using PocketBeagle<br>
+
''Project name'':  Super Awesome Project <br>
  
 
===Description===
 
===Description===
This project motto is to implement an updated "Speak&Spell" using PocketBeagle for modern day preschoolers with improved games/puzzles and better hardware interfaces like I2C gpio expander.Previously used VF display will be replaced with an OLED display and instead of using keypad everytime,speech recognition will be implemented.<br>
+
In 10-20 sentences, what are you making, for whom, why and with what technologies (programming languages, etc.)? (We are looking for open source SOFTWARE submissions.)
Current Proposal for the project aims at building an open source reproducible Linux application that can be implemented by downloading the code anywhere for its real time implementation so that it can be promoted for usage in commercial purposes. Since the current Speak&Spell's basic games are not sophisticated enough for present day preschoolers,the updated puzzles will help improving their Spelling skills.Puzzles will be implemented using python in 2 levels(PyGame library will be used).For text to voice generation,a light open source program engine called 'CMU Flite' will be used. Flite is specially built for embedded systems and has an added advantage of changing to multiple accents.For adding the speech recognition feature,we use TIesr libraries.To acquire good recognition of single words(to respond to the voice commands from outside--Chatbot feature),we create word based acoustic models rather than the default TIesr phonetic based models.Phonetic based models will be used when the beagle has to recognise
 
the spelling of a word as spelt by the user.
 
 
 
Level-1 game: This level basically compares the word spelt out by the player with the original spelling and gives feedback accordingly.
 
 
 
Level-2 game: A rough image of how this game is implemented is shown in the below picture.
 
 
 
[[File:Game level2.png|400px|center|frameless]]
 
 
 
 
 
This project will be implemented in 3 phases.<br>
 
Phase 1:<br>
 
* Install the Flite program for text to speech generation.
 
* Install TI Embedded speech recognition libraries for speech recognition.
 
* Build Level 1 puzzle: Implement the basic 'spell the word' puzzle.
 
Phase 2:<br>
 
* Build Level 2 puzzle: Fill the missing letters in a word within specified time (This will be implemented as a dynamic game with graphics)
 
* Improving the Chatbot functionality in terms of giving feedback after every puzzle.The puzzles are invoked by giving commands through voice.It will also be able to give suggestions to improve user's performance.This way it will be developed as a teaching aid.
 
Phase 3:<br>
 
* Implement the complete model by adding a keyboard,audio in/out and an OLED display.
 
* Bug fixing and Documentation of the project.
 
 
 
===Timeline===
 
 
 
Community bonding period :
 
* Refine the weekly plan with suggestions from the mentor and community.<br>
 
* Familiarise myself with Pocketbeagle.<br>
 
 
 
2017-06-06:
 
* Install flite for text to speech generation.<br>
 
* Start developing an efficient voice recognition system using TIesr libraries.<br>
 
2017-06-13: Implementing the voice recognition system<br>
 
2017-06-20: Implement basic "spell the word" game in python<br>
 
2017-06-27: Implement level 2 game and documenting the finished code for '''Phase-1 evaluation'''<br>
 
2017-07-04:
 
* Improve the code based on feedback after phase-1 evaluation.<br>
 
* Implement level 2 game. <br>
 
2017-07-11: Develop level 2 game<br>
 
2017-07-18: Build Chatbot with feedback feature<br>
 
2017-07-25: Develop the Chatbot into a Teaching aid and document the finished work for '''Phase-2 evaluation'''<br>
 
2017-08-01: Improve the Chatbot features based on feedback after phase-2 evaluation <br>
 
2017-08-08: Make a demo adding a keyboard,OLED display and an audio IN/OUT<br>
 
2017-08-15:
 
* BUG fixing and documentation for '''final evaluation(Phase-3)'''<br>
 
* Prepare final presentation slides and video.<br>
 
 
 
'''Stretch goals''': Add a level-3 game in which  misspelt words are to be detected  and corrected from a set of words given .<br>
 
  
After GSoC,I will make sure that I remain in touch with the community and keep contributing to the organisation the best I can.
 
  
 
===Experience and approach===
 
===Experience and approach===
I have done projects on Raspberry Pi before and therefore will be comfortable enough to implement this project on a Pocketbeagle.I will get my hands-on with Pocketbeagle before the project starts since I have free time after my semester exams.I am quite new to Speech recognition,so I have been doing required background work to implement these functionalities on Pocketbeagle and therefore can assure you that I will be able to complete the Phase-1 in planned time. I had only done some basic projects on python before and therefore planning to spend most of Phase-2 to build the games.
+
In 5-15 sentences, convince us you will be able to successfully complete your project in the timeline you have described.
  
  
 
===Contingency===
 
===Contingency===
I will ensure that probability of this happening is less by keeping in regular touch with my mentor and make sure I know about his unavailability beforehand .I will also check with the community if they have a backup mentor provision.I will communicate my coding issues with the community or students working on similar projects to see if someone can help. Else I will contact the organisation administrator to talk about the issue and make sure the work is not delayed in the mean time by documenting the finished work.
+
What will you do if you get stuck on your project and your mentor isn’t around?
  
 
===Benefit===
 
===Benefit===
"Speak&Spell" was more than just a popular kids toy,it should be better described as a blueprint for the devices we use today.Until its invention, real time speech synthesis was defined to be impossible.Implementing and updating its functionality for the present day kids is a perfect way to celebrate its importance in the development of  current Signal processing technology.
+
If successfully completed, what will its impact be on the BeagleBoard.org community? Include quotes from BeagleBoard.org community members who can be found on [[http://beagleboard.org/discuss]] and [[http://bbb.io/gsocchat]]
  “<erik.welsh> Basically, the ideas was to re-brain a speak-and-spell with a PocketBeagle to celebrate the 40th anniversary.<br> Speak-and-Spell was a great educational tool and bringing it back into the public.<br>
 
  Generate open-source code around the Speak-and-Spell functionality.Plus we can get one of the creators (Gene Frantz) to then promote it.
 
  
 
===Suggestions===
 
===Suggestions===
I am currently pursuing my Bachelor's degree in Electronics and Communication Engineering and I am good in C and Python programming.I have no other commitments this summer and can dedicate my entire time for the project.
+
Is there anything else we should have asked you?

Revision as of 22:06, 23 March 2018


ProposalTemplate

Student: New Student
Mentors: Jason Kridner
Code: https://github.com/BeaglePilot
Wiki: https://elinux.org/BeagleBoard/GSoC/ProposalTemplate
GSoC: GSoC Entry

Status

This project is currently just a proposal.

Proposal

Please complete the requirements listed on the ideas page and fill out this template.


About you

IRC: IRC Username Goes Here
Github: Github Account
School: School name
Country: Country
Primary language (We have mentors who speak multiple languages): Language
Typical work hours (We have mentors in various time zones): 8AM-5PM US Eastern
Previous GSoC participation: Provide list of URLs for previous participation or tell us why you want to participate here. No previous experience required.


About your project

Project name: Super Awesome Project

Description

In 10-20 sentences, what are you making, for whom, why and with what technologies (programming languages, etc.)? (We are looking for open source SOFTWARE submissions.)


Experience and approach

In 5-15 sentences, convince us you will be able to successfully complete your project in the timeline you have described.


Contingency

What will you do if you get stuck on your project and your mentor isn’t around?

Benefit

If successfully completed, what will its impact be on the BeagleBoard.org community? Include quotes from BeagleBoard.org community members who can be found on [[1]] and [[2]]

Suggestions

Is there anything else we should have asked you?