BeagleBoard/GSoC/2020Proposal/HydraulicSheep

From eLinux.org
Jump to: navigation, search

Implementing a covert UI and Stepwise mathematical solver on the PocketBeagle in order to act as a compact utility device.

Student: HydraulicSheep
Mentors: Pac23 (Contacted him and discussed)
Code: https://github.com/HydraulicSheep
GSoC: GSoC entry

Status

This project is currently just a proposal.

Proposal

About me

IRC: HydraulicSheep
Github: HydraulicSheep
Primary language: English
Previous GSoC participation: Tell us why you want to participate in GSoC here. I have not participated in GSoC previously but I did partake in 'Google Code In' with Zulip.

About my project

Project name: PocketBeagle Covert Mathematical Solver

Description

The project involves implementing a Stepwise mathematical solver on the PocketBeagle in order to act as a covert calculator modification. The key value of the project is twofold:
1. Implementing a mathematical solver on the PocketBeagle. This makes inroads into more symbolic algebra and perhaps SAT-related computation on the small board.
2. Working within covert constraints (e.g. an unassuming UI without compromising UX). This may aid future efforts in other similar ventures (such as networking on such a device) or even security-related fields (an unassuming device for war walking).

Goals beyond the suggested ideas:

3. Add high-level scripting functionality (e.g. in lua or something similar). This would allow users with little low-level experience to increase the device's functionality. For example, a user could implement a solver for geometric problems which cannot be written symbolically.
4. Extend the device's UI capability. This has a great software value as it lends this device far more utility. This interface could allow web-browsing capability (but such an extension is likely out of the scope of this proposal) or pentesting/security modules as a pocket-calculator is a relatively unassuming device to carry around.

Preliminary UI Design Ideas attached below...

Implementation Details

Hardware

For initial development, a PocketBeagle+GamePup and USB wireless keyboard. The correct components for an identical form-factor to specific calculators can be realised later.

Regarding calculator choice:

As per my own teardown and on on Youtube (at https://www.youtube.com/watch?v=n8Y8gPi6QfE due to my poor photo quality), it is not feasible to cannibalise the existing board for the project. This is because there is no easy way to access the keyboard or interface with the PCB at all.

This is true of other mass market calculators (as per similar teardowns):

   TI-30XS (https://www.youtube.com/watch?v=YbYUMBorMx0) 
TI-30X (https://www.youtube.com/watch?v=8bKEO3VnAvE)

As such, it seems to me that the only way of adding functionality to the calculator is completely reimplementing its functionality on the Pocket Beagle. This shouldn’t be too hard due to its sufficient processing power and ability to handle math. However, considering time constraints, I hope to use a mass-market calculator with a smaller featureset (e.g. TI-30XB) which should be easier to implement. (Other calculators have multiple modes and table capabilities).

The timeline has been adjusted to accommodate this reimplementation.

Basic schematic for IO and hardware interfacing
Pin allocation:

A maximum of 16 GPIO pins for a multiplexed keyboard (allowing for a max of 61 buttons).

The calculator will use a gray TN LCD and requires a minimum screen resolution in order to imitate the original calculators.

For the TI 30XB, this is around 60 x 130 from some basic measurements. Presumably, this is actually a 64 x 128 screen.

LCD Modules are easy to source. This example seems suitable (requiring 28 pins): https://www.digikey.com/products/en?mpart=64128K%20FC%20BW-3&v=1756

These pins total within the GPIO capabilities of the board. If extra pins are required, the keyboard can easily be abstracted away with a small circuit to convert it to binary.


Mathematical Solver

The approach would be implementing a tree-based math expression parser and step-by-step solver.
Relevant details and papers exist:
https://www.cs.utexas.edu/users/novak/algebra.pdf
https://www.codeproject.com/Articles/10316/Binary-Tree-Expression-Solver

Implementations (on other platforms) exist as a reference:
https://github.com/socraticorg/mathsteps


Preliminary UI Design Ideas

Two lines of simple stepwise working. Scroll marker in top right. Page count in bottom right.
Potential program viewer. These icons would be higher-resolution and the bottom bar would display system information (Battery %, Available Memory, Location in file tree if viewing files).

(These are a Work In Progress that I am continuing to update on the elinux site).
The resolution on these mock-ups is too low and will be better in practice.
See images and captions on right...


On board scripting language

I aim to integrate an existing scripting framework into the design. The requirements of such a language are that it be lightweight, user-friendly and well-documented.
The two candidates are:
eLua which is well-documented and has considered supporting the Beaglebone Black previously (https://github.com/elua/elua)
Some form of Javascript with node.js. This has existing support on Linux and has been utilised in a lightweight non-blocking form on other Beaglebone hardware by Node-Red (https://nodered.org/docs/getting-started/beaglebone).


Timeline

Provide a development timeline with a milestone each of the 11 weeks and any pre-work. (A realistic timeline is critical to our selection process.)

Mar 30 Proposal complete, Submitted to https://summerofcode.withgoogle.com
Apr 27 Proposal accepted or rejected
May 18 Pre-work complete, Coding officially begins!
May 25 Milestone #1, Introductory YouTube video
June 1 Milestone #2, Setup calculator environment. This includes interfacing with the screen and capacitive keyboard (as well as physically fitting in the case). Configuring I/O ports. Start work on reimplementation of featureset.
June 8 Milestone #3, Continue mimicking the original device’s feature-set on PocketBeagle hardware. This involve getting the UI correct and all the relevant symbols abstracted away to functions on the board.
June 15 18:00 UTC Milestone #4, Mentors and students can begin submitting Phase 1 evaluations. Continue implementing the basic tree-based expression system which will be recycled for the stepwise solver.
June 19 18:00 UTC Phase 1 Evaluation deadline.
June 22 Milestone #5 Finish making trees and expression parsing. By this point, step-by-step output will be fully functional for these basic algebra operations. The week will be spent turning these trees into steps and cleaning up code to optimise usability. The time spent on this is justified by the importance of this to the board's entire functionality.
June 29 Milestone #6 Start the UI and control systems that hide the secret features of the device while allowing easy interaction.
July 6 Milestone #7 The featureset will be extended to include basic Calculus (integration/derivation) and matrix-related operations.
July 13 18:00 UTC Milestone #8, The board file system will be completed. This will be simple and segmented, allowing only storage of user files (rather than access to the underlying OS).
July 17 18:00 UTC Phase 2 Evaluation deadline
July 20 Milestone #9 The scripting language will be implemented, sample programs will be written to demonstrate its functionality, and the permitted featureset of the chosen language will be documented.
July 27 Milestone #10 Conclude by tidying up any documentation and expanding the UI so it is accessible and adaptable to new features. This would include a sort of ‘app viewer’ as referenced in the above screenshots.
August 3 Milestone #11, Completion YouTube video
August 10 - 17 18:00 UTC Final week: Students submit their final work product and their final mentor evaluation
August 17 - 24 18:00 UTC Mentors submit final student evaluations

Experience and approach

I have a wealth of experience working with electronics and small boards like Arduino. Further, I am comfortable with the Linux environment and low-level toolchains (assemblers, cross-compilers, linkers etc.) For this project, I think my previous attempts at implementing math solvers will guide me. Although these involved many trials and tribulations, I believe my experience overcoming similar issues will guide me in making solid design choices. I have many years of programming experience in Python, Javascript, C# and Java. Importantly, I have been steadily working on my low-level knowledge over the past few months, learning C and some workable Intel assembly. I have worked on Open Source projects before (Zulip and Unitystation on GitHub) and I have coded remotely in a team environment through competitions like BattleCode and Halite. I am an independent worker who is more than willing to ask for feedback/help when I get stuck. On the whole, I believe myself more than capable to carry out the 4 goals of this proposal.

Contingency

What will you do if you get stuck on your project and your mentor isn’t around?

I think, particularly for a project such as this, there are three alternate avenues to overcome obstacles: Firstly, the internet provides a great wealth of guides, videos and tools that should be able to aid me, no matter the problem. It is highly likely someone will have encountered my problem before and I should be able to learn from their experience. Secondly, I am not afraid to turn to a reference manual or datasheet in order to get things done. Although often dense and challenging, given the time, I am sure I can find the answers I need. Thirdly, if all else fails, there is always the option of applying strategies to diagnose the problem and tinkering to resolve it. Although this may take longer, it will ultimately give me a greater understanding of the project and help with future dilemmas.

Benefit

If successfully completed, what will its impact be on the BeagleBoard.org community?

From this proposal, the BeagleBoard community will receive a platform for covert and compact systems. Such a system has value both for security and for compact integrated applications. A valuable component of this system is its UI which will hopefully provide inspiration to potential creators and guide them in creating compact, covert interfaces.

Further, this calculator project is likely to bring more people into the BeagleBoard community as it has utility, is an entertaining concept and is made from easily accessible components.

(Quotes from community members coming soon)

Misc

Here is a link to my Pull Request, demonstrating my cross-compilation: https://github.com/jadonk/gsoc-application/pull/144