Difference between revisions of "BeagleBoard/GSoC/2021 Proposal/TensorFlow Lite Compatibility with BeagleBone AI"

From eLinux.org
Jump to: navigation, search
Line 10: Line 10:
 
''Code'': none yet, lots to follow! <br>
 
''Code'': none yet, lots to follow! <br>
 
''Wiki'': http://elinux.org/BeagleBoard/GSoC/2021 Proposal/TensorFlow Lite Compatibility with BeagleBone AI<br>
 
''Wiki'': http://elinux.org/BeagleBoard/GSoC/2021 Proposal/TensorFlow Lite Compatibility with BeagleBone AI<br>
''GSoC'': [https://summerofcode.withgoogle.com/archive/2016/projects/4885400476712960/ GSoC entry]<br>
+
''GSoC'': N/A<br>
 
<div style="clear:both;"></div>
 
<div style="clear:both;"></div>
  
Line 24: Line 24:
 
''Other languages'' : Spanish, Swahili, Hebrew, some Hindi/Urdu, Telugu, Bengali<br>
 
''Other languages'' : Spanish, Swahili, Hebrew, some Hindi/Urdu, Telugu, Bengali<br>
 
''Typical work hours'' : 9AM-5:30PM US Pacific, though I may be on Eastern time part of the summer<br>
 
''Typical work hours'' : 9AM-5:30PM US Pacific, though I may be on Eastern time part of the summer<br>
''Previous GSoC participation'': I've never participated in GSOC; I am interested to do so to learn more about embedded software, improve my coding skills, and participate in the community.  I use open source tools all the time, and it is really exciting to make something useful that other people would use too.
+
''Previous GSoC participation'': I haven't ever participated in GSOC; I am interested to do so to learn more about embedded software, improve my coding skills, and participate in the community.  I use open source tools all the time, and it is really exciting to make something useful that other people would use too.
  
 
==About your project==
 
==About your project==
Line 82: Line 82:
  
 
===Benefit===
 
===Benefit===
If successfully completed, what will its impact be on the BeagleBoard.org community? Include quotes from BeagleBoard.org community members who can be found on http://beagleboard.org/discuss and http://bbb.io/gsocchat.
+
BeagleBoard options are strong contenders with Raspberry Pi and Arduino as tools that are easy to use to do IoT and industrial projects. While BeagleBones are geared more towards engineers and have some special features that actually make them quite different than the others (PRUs, ability to do low level and high level control simultaneously, power usage etc), they often get compared with the other frameworks.  Given that Raspberry Pi and Arduino have TensorFlow Lite compatibility, I think it is important for BeagleBones to do the same. Also, there are algorithms that are easier to access with TensorFlow Lite than in other settings.
 +
 
 +
For example,
 +
<blockquote>
 +
jduchniewicz said:
 +
lpillsbury: I tried tackling YOLO deployment on BBAI but it is currently impossible because of TIDL library restrictions. TFLite is the way to go with DL on BBAI and it uses TIDL underneath
 +
<blockquote>
 +
 
 +
and many frustrated users have posted on the internet:
 +
 
  
 
==Misc==
 
==Misc==

Revision as of 17:52, 11 April 2021


ProposalTemplate

About Student: Leah Pillsbury
Mentors: Who else is interested in this?
Code: none yet, lots to follow!
Wiki: http://elinux.org/BeagleBoard/GSoC/2021 Proposal/TensorFlow Lite Compatibility with BeagleBone AI
GSoC: N/A

Status

This project is currently just a proposal.

About you

IRC: lpillsbury
Github: lpillsbury
School: Pasadena City College and Boston University before that
Country: United States
Primary language : English
Other languages : Spanish, Swahili, Hebrew, some Hindi/Urdu, Telugu, Bengali
Typical work hours : 9AM-5:30PM US Pacific, though I may be on Eastern time part of the summer
Previous GSoC participation: I haven't ever participated in GSOC; I am interested to do so to learn more about embedded software, improve my coding skills, and participate in the community. I use open source tools all the time, and it is really exciting to make something useful that other people would use too.

About your project

Project name: TensorFlow Lite Compatability with BeagleBone AI

Description

While the specs of BeagleBone AI are awesome, it seems limiting for an AI board for the general public to not be able to access TensorFlow Lite. I would like to change that by choosing a current release of TensorFlow, and writing the code necessary to run it on the Arm M4 processors that the BeagleBone AI board has, and that should work with TensorFlow Lite. This project will be a combination of C coding, dealing with the Linux kernel, and doing some example cases with TensorFlow Lite in python. Obviously, given the diversion between OpenCL and TensorFlow, there is no perfect way to seamlessly integrate TensorFlow Lite, but there seem to be enough people in the community who want it, that it is worth at least making a gesture towards a compatibility solution. I first realized this was a problem when I decided I wanted to learn more about AI on the edge. One of the first things that comes up when I search for information on this topic is using TensorFlow Lite and the wide variety of other boards that it plays well with.

Ideally, I'd spend the first few weeks getting TensorFlow Lite working on a BBAI, document my hacking process, and then create a smooth stable way to make it work more out of the box. Then I'd spend the rest of the time doing examples that would be highlighted both on the BeagleBoard website, and also on the TensorFlow Lite Examples page by making a pull request to the owners of the github repo. Given that there are already examples of using TensorFlow Lite on Raspberry Pi, with picamera, the starting point would be to have equivalent BeagleBone examples, most likely doing image capture through OpenCV (OpenCV with BeagleBone Black). I'm also very interested in sound recognition, and if time allows I'd also like to do some examples in this area.

Timeline

Provide a development timeline with a milestone each of the 11 weeks and any pre-work. (A realistic timeline is critical to our selection process.)

Apr 8 Found out about exciting projects at Beagle Board, joined the community!
Apr 13 Proposal complete, Submitted to https://summerofcode.withgoogle.com
May 17 Proposal accepted or rejected
All of May Prework: * Learn more about Linux Kernels
  • Try out TensorFlow Lite on Raspberry Pi that I already have so that I know how it is supposed to work
  • Brainstorm ideas for cool examples
  • Learn more about how TensorFlow Lite works on microcontrollers through the TensorFlow Lite google-group (that I've already joined) and other sources
Jun 07 Pre-work complete, Coding officially begins!
Jun 17 Milestone #1, Introductory YouTube video
June 24 Milestone #2, Working version of TensorFlow Lite on a BeagleBone AI
June 30 Milestone #3, Smooth(er) process and documentation for others to use TensorFlow Lite on BeagleBone AI
July 12 18:00 UTC Milestone #4, Is Milestone #3 as user friendly as possible? If not, improve it
July 23 Milestone #5, Example Cases on BeagleBone AI (vision)
July 30 Milestone #6, Example Cases on BeagleBone AI (sound)
Aug 06 Milestone #7, explore whether TensorFlow Lite can also be integrated with the BeagleBoard-X-15, and/or whether

what I've learned about TensorFlow Lite can be extended to PyTorch or mlpack

August 10 Milestone #8, Completion YouTube video
August 16 - 26 18:00 UTC Final week: Students submit their final work product and their final mentor evaluation
August 23 - 30 18:00 UTC Mentors submit final student evaluations

Experience and approach

In 5-15 sentences, convince us you will be able to successfully complete your project in the timeline you have described.

Contingency

What will you do if you get stuck on your project and your mentor isn’t around?

Benefit

BeagleBoard options are strong contenders with Raspberry Pi and Arduino as tools that are easy to use to do IoT and industrial projects. While BeagleBones are geared more towards engineers and have some special features that actually make them quite different than the others (PRUs, ability to do low level and high level control simultaneously, power usage etc), they often get compared with the other frameworks. Given that Raspberry Pi and Arduino have TensorFlow Lite compatibility, I think it is important for BeagleBones to do the same. Also, there are algorithms that are easier to access with TensorFlow Lite than in other settings.

For example,

jduchniewicz said: lpillsbury: I tried tackling YOLO deployment on BBAI but it is currently impossible because of TIDL library restrictions. TFLite is the way to go with DL on BBAI and it uses TIDL underneath

and many frustrated users have posted on the internet:


Misc

Please complete the requirements listed on the ideas page. Provide link to pull request.

Suggestions

Is there anything else we should have asked you?