This page has notes and an outline for Tim Bird's Linux Auto-Reduction research.

== Talk info ==
=== Title ===
Advanced size optimization of the Linux kernel
=== Abstract ===
This presentation will cover recent research by Tim on aggressive size reduction of the Linux kernel. This will include results from using gcc link-time optimization (LTO) with the ARM architecture (using Andi Kleen's out-of-tree patches), as well as results and discussion of other optimization techniques (including whole-system optimization for embedded devices).

This talk is directed at kernel developers interested in reducing the size of their Linux systems (and possible improving their performance in the process). The talk will be highly technical.

== Research Areas ==
=== LTO ===
* What is it?
* what was required to get it to work?
* Andi Kleen's patch set
** what do they do?
** how big are they?
** mainline status?
* what is the size gain (see ELC poster)
* what can be done with it?
* long-term possibilities for LTO

=== global constraints ===
* overall idea: create constraints external to code, and use for optimization
* rationale: can't maintain in-tree - too many config items
* make the application of constraints automatic
* use existing constraints to generate new constraints
* constraints can flow between user-space and kernel

* example: uid=0
* constraint language
* application by commenting out references (replace with 0 constant)
** use compiler to find code references (via error messages)
*** eliminates problem with duplicate names (uid in different structure)
* constant propagation (by, e.g. LTO) reduces code

=== syscall elimination ===
* scan file system
* create report of used and unused system calls
* mark syscalls unused in kernel
** arch/arm/kernel/calls.S (and arch/arm/kernel/entry-common.S
* make sure unused syscalls are not __attribute__(externally_visible)
** technique of asmlinkage_<syscall>
* use LTO to eliminate calls
* results: 50K-90K

=== ARM stack reduction ===
* 4k stacks
* stack extensions

=== link-time rewriting ===

=== cold code compression ===
* D. Chanet did cold code compression
* consists of:
** profiling the kernel
** marking code regions as cold or frozen
** replacing them with stubs
** compressing them
* At execution time:
** if a stub is called, it decompresses the code and calls it
** stub is fixed up to directly call decompressed code in future
** code is left decompressed forever

=== cold code compression ===
Results:
** MUST see paper for details (it's quite complicated)
* on 2.4.25 kernel
** cold code compression resulted in 7% reduction for i386 kernel and
** 11.7% reduction for ARM kernel

== Talk outline ==
This talk will be presented at LinuxCon Japan 2013:

=== Title ===
* Advanced size optimization of the Linux kernel
** by Tim Bird, Sony Mobile Communication

=== Self-Introduction ===
* I am Tim Bird
* Now working at Sony Mobile
* Researching system size for many years
* Long background in extremely small systems
** pre-professional: first program on TRS-80, in basic, 8K ram
** NetWare Lite - file and print server in 50K (in 1991)

=== The problem of Bloat ===
* Software bloat occurs because systems are built with more software than is really needed for a given task
* Open Source software meets the needs of thousands of different systems
** Linux scales from tiny sensors to supercomputers (extreme SMP and high-end clusters)
** Linux supports many, many features, only some of which are configurable
* Software must be generalized for many use cases
* bloat problem is:
** How to re-specialize the software, eliminating unused features and dead code?

=== Bloat (cont.) ===
* Software gets more generalized over time
* Can't use strategy of manual tuning (config options)
** It gets harder and harder to remove things over time
** About 13,000 config items now (2.6.12 had 4700)
** You have to be an expert in too many things to reduce the kernel
* Must rely on automated methods of reduction
* Should use an additive, rather than subtractive method of building a system
** ultimate vision: indicate what you want/need, and build up system to support it

=== Bloat (cont. 2) ===
* In desktop or server, virtual memory makes bloat issue less important for user-space programs
** Only working set of program is loaded - pages are loaded on demand
** For kernel, all pages are always loaded

=== Automatic reduction (intro)===
The problem with automatic reduction is that "the system" doesn't know what
software is needed and what is not. there needs to be a way to tell it
about things that are not going to be used.

=== auto-reduce - story of 8 bytes of bloat ===
Story of the conditional check in kdb:
* I found a bug in kdb, when a particular option was using in the configuration file
* not everyone uses the configuration file
* not everyone uses the particular option
* bug only triggered in those circumstances
* I wrote a small patch, to guard against use of a variable prematurely
* problem: all users of KDB now have this check, and suffer this overhead
** it wasn't much, just a single compare
** but this is how bloat builds up over time
** It bothered me because I knew most people didn't need the check
* "correct" solution would be to parse the config file, and make the code compile-time configurable
** this adds more complexity than it is worth.

=== generalizing the problem of bloat ===
System doesn't know inputs:

* It's very easy to configure the kernel to omit the driver for missing hardware.
* It's very difficult to configure the kernel to omit error handling for bugs that
will never occur due to fixed use cases.

=== An example of fixed input (uid in kernel) ===
* throughout the kernel, there are references to uid
** comparisons, storing, referencing
* it turns out this is set by setuid(), by the 'login' program.
** login does a lookup and validates user account name in /etc/passwd
* what if /etc/passwd only has 'root' and no others?
* setuid() could only be called with a value of 0
* can I encode this constraint on the system.

=== types of constraints ===
There are numerous other examples of constraints:
* kernel command line arguments never used
* syscalls never called by any program
* parameters that are never used, or parameter values that are never passed in
** e.g. ioctl value that is not possible
** this only works in a fixed
* /proc values never referenced
* /sys values never referenced

=== also goes back up to user-space ===
* return values that are not possible

=== kernel command line args ===
* Documented in Documentation/kernel-parameters.txt
* defined with __setup() and early_param from include/linux/init.h
** approximately 480 __setup routines in kernel
** about 200 __setup_* in System.map on ARM kernel build (98 __setup_str_*)
** about 230 early_param routines in kernel
* points to function
* almost always sets a variable, which would default to 0

* on ARM, with only console_setup and early_mem marked as 'used', there was a 19K difference in size:
(non-LTO kernel)
{{{
vmlinux.baseline-setup-used => vmlinux-param-used
baseline other change percent
text: 7680084 7663472 -16612 0%
data: 362868 360516 -2352 0%
bss: 745312 745184 -128 0%
total: 8788264 8769172 -19092 0%
}}}

* on ARM, with only console_setup and early_mem marked as 'used', there was a 19K difference in size:
(LTO kernel)
{{{
vmlinux.lto-param => vmlinux.param-used
baseline other change percent
text: 1653672 1648920 -4752 0%
data: 131636 130244 -1392 -1%
bss: 50688 50528 -160 0%
total: 1835996 1829692 -6304 0%
}}}

System.map from kernel with console_setup and early_mem as only routines marked 'used':
{{{
$ grep __setup System.map
c00ea4bc T __setup_irq.153323
c00f1adc t __setup_per_zone_wmarks.172539.15755
c019d570 t __setup_str_early_mem.21664.160821
c019d884 t __setup_str_console_setup.61958.160201
c019ef00 t __setup_early_mem.21659.160819
c019ef00 T __setup_start
c019ef0c t __setup_console_setup.61953.160195
c019ef18 T __setup_end
}}}

=== /proc values ===
* Includes sysctl values
* there are approximately 1200, NOT related specifically to a process
* about 120 per process
** about 80 related to networking (on my desktop box)
* 40 others

=== Tiny Distribution ===
* poky-tiny distribution (yocto project)
* see https://wiki.yoctoproject.org/wiki/Poky-Tiny
* Good for testing and further research

== References ==
* Chanet D. ... "Automated reduction of the memory footprint of the linux kernel"
* Haifen He. ..."Code Compaction of an Operating System Kernel"

== Materials ==
* [[File:0001-ARM-LTO-avoid-errors-on-unified-assembly-macros.patch]]

System Size Auto-Reduction

2013-04-19T21:23:37Z

Tim Bird: /* kernel command line args */

This page has notes and an outline for Tim Bird's Linux Auto-Reduction research.

== Talk info ==
=== Title ===
Advanced size optimization of the Linux kernel
=== Abstract ===
This presentation will cover recent research by Tim on aggressive size reduction of the Linux kernel. This will include results from using gcc link-time optimization (LTO) with the ARM architecture (using Andi Kleen's out-of-tree patches), as well as results and discussion of other optimization techniques (including whole-system optimization for embedded devices).

This talk is directed at kernel developers interested in reducing the size of their Linux systems (and possible improving their performance in the process). The talk will be highly technical.

== Research Areas ==
=== LTO ===
* What is it?
* what was required to get it to work?
* Andi Kleen's patch set
** what do they do?
** how big are they?
** mainline status?
* what is the size gain (see ELC poster)
* what can be done with it?
* long-term possibilities for LTO

=== global constraints ===
* overall idea: create constraints external to code, and use for optimization
* rationale: can't maintain in-tree - too many config items
* make the application of constraints automatic
* use existing constraints to generate new constraints
* constraints can flow between user-space and kernel

* example: uid=0
* constraint language
* application by commenting out references (replace with 0 constant)
** use compiler to find code references (via error messages)
*** eliminates problem with duplicate names (uid in different structure)
* constant propagation (by, e.g. LTO) reduces code

=== syscall elimination ===
* scan file system
* create report of used and unused system calls
* mark syscalls unused in kernel
** arch/arm/kernel/calls.S (and arch/arm/kernel/entry-common.S
* make sure unused syscalls are not __attribute__(externally_visible)
** technique of asmlinkage_<syscall>
* use LTO to eliminate calls
* results: 50K-90K

=== ARM stack reduction ===
* 4k stacks
* stack extensions

=== link-time rewriting ===

=== cold code compression ===
* D. Chanet did cold code compression
* consists of:
** profiling the kernel
** marking code regions as cold or frozen
** replacing them with stubs
** compressing them
* At execution time:
** if a stub is called, it decompresses the code and calls it
** stub is fixed up to directly call decompressed code in future
** code is left decompressed forever

=== cold code compression ===
Results:
** MUST see paper for details (it's quite complicated)
* on 2.4.25 kernel
** cold code compression resulted in 7% reduction for i386 kernel and
** 11.7% reduction for ARM kernel

== Talk outline ==
This talk will be presented at LinuxCon Japan 2013:

=== Title ===
* Advanced size optimization of the Linux kernel
** by Tim Bird, Sony Mobile Communication

=== Self-Introduction ===
* I am Tim Bird
* Now working at Sony Mobile
* Researching system size for many years
* Long background in extremely small systems
** pre-professional: first program on TRS-80, in basic, 8K ram
** NetWare Lite - file and print server in 50K (in 1991)

=== The problem of Bloat ===
* Software bloat occurs because systems are built with more software than is really needed for a given task
* Open Source software meets the needs of thousands of different systems
** Linux scales from tiny sensors to supercomputers (extreme SMP and high-end clusters)
** Linux supports many, many features, only some of which are configurable
* Software must be generalized for many use cases
* bloat problem is:
** How to re-specialize the software, eliminating unused features and dead code?

=== Bloat (cont.) ===
* Software gets more generalized over time
* Can't use strategy of manual tuning (config options)
** It gets harder and harder to remove things over time
** About 13,000 config items now (2.6.12 had 4700)
** You have to be an expert in too many things to reduce the kernel
* Must rely on automated methods of reduction
* Should use an additive, rather than subtractive method of building a system
** ultimate vision: indicate what you want/need, and build up system to support it

=== Bloat (cont. 2) ===
* In desktop or server, virtual memory makes bloat issue less important for user-space programs
** Only working set of program is loaded - pages are loaded on demand
** For kernel, all pages are always loaded

=== Automatic reduction (intro)===
The problem with automatic reduction is that "the system" doesn't know what
software is needed and what is not. there needs to be a way to tell it
about things that are not going to be used.

=== auto-reduce - story of 8 bytes of bloat ===
Story of the conditional check in kdb:
* I found a bug in kdb, when a particular option was using in the configuration file
* not everyone uses the configuration file
* not everyone uses the particular option
* bug only triggered in those circumstances
* I wrote a small patch, to guard against use of a variable prematurely
* problem: all users of KDB now have this check, and suffer this overhead
** it wasn't much, just a single compare
** but this is how bloat builds up over time
** It bothered me because I knew most people didn't need the check
* "correct" solution would be to parse the config file, and make the code compile-time configurable
** this adds more complexity than it is worth.

=== generalizing the problem of bloat ===
System doesn't know inputs:

* It's very easy to configure the kernel to omit the driver for missing hardware.
* It's very difficult to configure the kernel to omit error handling for bugs that
will never occur due to fixed use cases.

=== An example of fixed input (uid in kernel) ===
* throughout the kernel, there are references to uid
** comparisons, storing, referencing
* it turns out this is set by setuid(), by the 'login' program.
** login does a lookup and validates user account name in /etc/passwd
* what if /etc/passwd only has 'root' and no others?
* setuid() could only be called with a value of 0
* can I encode this constraint on the system.

=== types of constraints ===
There are numerous other examples of constraints:
* kernel command line arguments never used
* syscalls never called by any program
* parameters that are never used, or parameter values that are never passed in
** e.g. ioctl value that is not possible
** this only works in a fixed
* /proc values never referenced
* /sys values never referenced

=== also goes back up to user-space ===
* return values that are not possible

=== kernel command line args ===
* Documented in Documentation/kernel-parameters.txt
* defined with __setup() and early_param from include/linux/init.h
** approximately 480 __setup routines in kernel
** about 200 __setup_* in System.map on ARM kernel build (98 __setup_str_*)
** about 230 early_param routines in kernel
* points to function
* almost always sets a variable, which would default to 0

=== /proc values ===
* Includes sysctl values
* there are approximately 1200, NOT related specifically to a process
* about 120 per process
** about 80 related to networking (on my desktop box)
* 40 others

=== Tiny Distribution ===
* poky-tiny distribution (yocto project)
* see https://wiki.yoctoproject.org/wiki/Poky-Tiny
* Good for testing and further research

== References ==
* Chanet D. ... "Automated reduction of the memory footprint of the linux kernel"
* Haifen He. ..."Code Compaction of an Operating System Kernel"

== Materials ==
* [[File:0001-ARM-LTO-avoid-errors-on-unified-assembly-macros.patch]]

System Size Auto-Reduction

2013-04-19T19:01:51Z

Tim Bird: /* kernel command line args */ add some info about ARM kernel

This page has notes and an outline for Tim Bird's Linux Auto-Reduction research.

== Talk info ==
=== Title ===
Advanced size optimization of the Linux kernel
=== Abstract ===
This presentation will cover recent research by Tim on aggressive size reduction of the Linux kernel. This will include results from using gcc link-time optimization (LTO) with the ARM architecture (using Andi Kleen's out-of-tree patches), as well as results and discussion of other optimization techniques (including whole-system optimization for embedded devices).

This talk is directed at kernel developers interested in reducing the size of their Linux systems (and possible improving their performance in the process). The talk will be highly technical.

== Research Areas ==
=== LTO ===
* What is it?
* what was required to get it to work?
* Andi Kleen's patch set
** what do they do?
** how big are they?
** mainline status?
* what is the size gain (see ELC poster)
* what can be done with it?
* long-term possibilities for LTO

=== global constraints ===
* overall idea: create constraints external to code, and use for optimization
* rationale: can't maintain in-tree - too many config items
* make the application of constraints automatic
* use existing constraints to generate new constraints
* constraints can flow between user-space and kernel

* example: uid=0
* constraint language
* application by commenting out references (replace with 0 constant)
** use compiler to find code references (via error messages)
*** eliminates problem with duplicate names (uid in different structure)
* constant propagation (by, e.g. LTO) reduces code

=== syscall elimination ===
* scan file system
* create report of used and unused system calls
* mark syscalls unused in kernel
** arch/arm/kernel/calls.S (and arch/arm/kernel/entry-common.S
* make sure unused syscalls are not __attribute__(externally_visible)
** technique of asmlinkage_<syscall>
* use LTO to eliminate calls
* results: 50K-90K

=== ARM stack reduction ===
* 4k stacks
* stack extensions

=== link-time rewriting ===

=== cold code compression ===
* D. Chanet did cold code compression
* consists of:
** profiling the kernel
** marking code regions as cold or frozen
** replacing them with stubs
** compressing them
* At execution time:
** if a stub is called, it decompresses the code and calls it
** stub is fixed up to directly call decompressed code in future
** code is left decompressed forever

=== cold code compression ===
Results:
** MUST see paper for details (it's quite complicated)
* on 2.4.25 kernel
** cold code compression resulted in 7% reduction for i386 kernel and
** 11.7% reduction for ARM kernel

== Talk outline ==
This talk will be presented at LinuxCon Japan 2013:

=== Title ===
* Advanced size optimization of the Linux kernel
** by Tim Bird, Sony Mobile Communication

=== Self-Introduction ===
* I am Tim Bird
* Now working at Sony Mobile
* Researching system size for many years
* Long background in extremely small systems
** pre-professional: first program on TRS-80, in basic, 8K ram
** NetWare Lite - file and print server in 50K (in 1991)

=== The problem of Bloat ===
* Software bloat occurs because systems are built with more software than is really needed for a given task
* Open Source software meets the needs of thousands of different systems
** Linux scales from tiny sensors to supercomputers (extreme SMP and high-end clusters)
** Linux supports many, many features, only some of which are configurable
* Software must be generalized for many use cases
* bloat problem is:
** How to re-specialize the software, eliminating unused features and dead code?

=== Bloat (cont.) ===
* Software gets more generalized over time
* Can't use strategy of manual tuning (config options)
** It gets harder and harder to remove things over time
** About 13,000 config items now (2.6.12 had 4700)
** You have to be an expert in too many things to reduce the kernel
* Must rely on automated methods of reduction
* Should use an additive, rather than subtractive method of building a system
** ultimate vision: indicate what you want/need, and build up system to support it

=== Bloat (cont. 2) ===
* In desktop or server, virtual memory makes bloat issue less important for user-space programs
** Only working set of program is loaded - pages are loaded on demand
** For kernel, all pages are always loaded

=== Automatic reduction (intro)===
The problem with automatic reduction is that "the system" doesn't know what
software is needed and what is not. there needs to be a way to tell it
about things that are not going to be used.

=== auto-reduce - story of 8 bytes of bloat ===
Story of the conditional check in kdb:
* I found a bug in kdb, when a particular option was using in the configuration file
* not everyone uses the configuration file
* not everyone uses the particular option
* bug only triggered in those circumstances
* I wrote a small patch, to guard against use of a variable prematurely
* problem: all users of KDB now have this check, and suffer this overhead
** it wasn't much, just a single compare
** but this is how bloat builds up over time
** It bothered me because I knew most people didn't need the check
* "correct" solution would be to parse the config file, and make the code compile-time configurable
** this adds more complexity than it is worth.

=== generalizing the problem of bloat ===
System doesn't know inputs:

* It's very easy to configure the kernel to omit the driver for missing hardware.
* It's very difficult to configure the kernel to omit error handling for bugs that
will never occur due to fixed use cases.

=== An example of fixed input (uid in kernel) ===
* throughout the kernel, there are references to uid
** comparisons, storing, referencing
* it turns out this is set by setuid(), by the 'login' program.
** login does a lookup and validates user account name in /etc/passwd
* what if /etc/passwd only has 'root' and no others?
* setuid() could only be called with a value of 0
* can I encode this constraint on the system.

=== types of constraints ===
There are numerous other examples of constraints:
* kernel command line arguments never used
* syscalls never called by any program
* parameters that are never used, or parameter values that are never passed in
** e.g. ioctl value that is not possible
** this only works in a fixed
* /proc values never referenced
* /sys values never referenced

=== also goes back up to user-space ===
* return values that are not possible

=== kernel command line args ===
* Documented in Documentation/kernel-parameters.txt
* defined with __setup(), from include/linux/init.h
** approximately 480 __setup routines in kernel
** about 200 __setup_* in System.map on ARM kernel build (98 __setup_str_*)
* points to function
* almost always sets a variable, which would default to 0

=== /proc values ===
* Includes sysctl values
* there are approximately 1200, NOT related specifically to a process
* about 120 per process
** about 80 related to networking (on my desktop box)
* 40 others

=== Tiny Distribution ===
* poky-tiny distribution (yocto project)
* see https://wiki.yoctoproject.org/wiki/Poky-Tiny
* Good for testing and further research

== References ==
* Chanet D. ... "Automated reduction of the memory footprint of the linux kernel"
* Haifen He. ..."Code Compaction of an Operating System Kernel"

== Materials ==
* [[File:0001-ARM-LTO-avoid-errors-on-unified-assembly-macros.patch]]

Reasons to participate in open source

2013-04-19T17:46:14Z

Tim Bird: add some more info

Tim Bird's notes on participating in open source.

== Other people's talks ==
Many people have written papers or given presentations on the value of participating in open source.
Over time, I may collect the ones I know about, and list them here.

(For starters, there are the excellent presentations by Andrew Morton and James Bottomly)
Andrew Morton gave one of the best talks I have heard, at Embedded Linux Conference 2008
See: [[Session:The_Relationship_Between_kernel.org_Development_and_the_Use_of_Linux_for_Embedded_Applications_ELC_2008]]

James Bottomley gave a great talk at LinuxCon Japan, 2011 (I think), where he described the
reasons that it is valuable for a company to contribute to open source. I'll try to get a link
to the talk and any video that was recorded.

I'm sure that Greg KH has given talks of this nature as well.

== my old talks ==
[Note to self: Look up my old talks.] In the early days, I presented on this topic a lot, but
I haven't recently. However, many companies still don't participate in open source like they should.
I should put some links to my own talks here.

== Metaphors ==
=== Pioneers ===
Some of my ancestors crossed the plains and mountains of the western United States as pioneers.
One interesting story from this era, was that earlier parties would perform work for parties that
were to come subsequently. In one extreme example, one group of pioneers planted seeds, another
group of pioneers tended crops, and a later group of pioneers harvested the crops, all along a route
that none of them expected to visit again. The first two groups put in labor that benefited the third
group.

One question that arises, is how could the third group contribute to the first group's work?
They couldn't plant the seeds, because they weren't physically there.
They couldn't break the trail, because they came later. Even though they benefited from the work,
they couldn't help with it initially.

Now, software development is not nearly as linear a pursuit as is crossing (what to them was) a wilderness.
However, there are parallels.

Companies often don't have the expertise to contribute upstream. Even long-term users of open source
software may be several versions behind the upstream releases of software they are using. This makes
it difficult for them to contribute in a meaningful way.

However, companies can hire experts, and have them work directly on the upstream source. This would
be similar to a pioneer company sending a few of their individuals in the advance parties, going ahead
of them to help guide the initial group, and help break the trail for their passage.

One difficulty is that it's very difficult to measure the value of such people to the group that sends
them ahead.

=== standards organizations ===
Companies used to put a lot of effort into standards organizations, because by controlling
what standards were adopted they could help themselves and hurt their competitors.
When standards are needed for interoperability, if a company invests in a particular technology
by developing it for market and integrating it into their products, if that technology becomes a
standard, they have leveraged their investment. Network effects make that technology more valuable.
If something does not become a standard, then the opposite happens. Network effects tend to
discourage adoption of the technology, and the investment can be a total loss.

For this reasons, for particular standards there were huge, costly battles to promote one
technology or another.

This is tree to some degree with open source software as well. A first-mover in providing
an upstream implementation has an advantage against their competitors. Admittedly, the
advantage is less, since competitors can use the open source code sooner than they would
if the implementations were proprietary and unavailable. However, there is still an advantage
to the entity that has their code accepted upstream.

== List of reasons ==
* control of direction
* first-move advantage
* reduced maintenance cost (vs. maintaining out of tree)

Reasons to participate in open source

2013-04-19T17:38:47Z

Tim Bird: add pioneer metaphor

User:Tim Bird

2013-04-19T17:17:25Z

Tim Bird: /* Bookmarks for things I'm working on in this wiki */

Tim Bird is a Linux developer working for Sony Network Entertainment - a subsidiary of Sony Corporation.

Tim is Chair of the Architecture Group of the [http://www.linuxfoundation.org/collaborate/workgroups/celf CE Workgroup of the Linux Foundation] (formerly the [[CE Linux Forum]]). In the past, Tim worked at Lineo, Caldera, and Novell.

== Current contact information ==
You can contact Tim at the following address, which has been made spam-resistant:

* e-mail: tim (dot) bird (at) am (dot) sony (dot) com.

== Bookmarks for things I'm working on in this wiki ==
* [[System Size Auto-Reduction]] - presentation for LCJ 2013 - wrapping up my auto-reduce work
* [[Tim's Tips and Tricks]] - Miscelaneous development tips gleaned over the years
* [[Ftrace]] - with my knowledge of ftrace, this page really needs a fixup
* [[Embedded linux status]] - list of things for embedded linux status reports (which I do frequently)
* [[elinux issues]] - list of things that need fixing
* [[Embedded Wiki wanted features]]
* [[Kernel Timer Systems]] - needs a major overhaul to bring it up to date
* [[Reasons to participate in open source]] - my own thoughts on open source contributions

* [[Android Boot-time Readahead]]
* [[Beginning Programming]]

=== Pages needing LOTS of HELP!! ===
Here are some high priority candidates for work...
* [[Android Kernel Build]]

=== Samples and help ===
* See [[Table Examples]]

== Areas of interest ==
Here are some things Tim is interested in on this site:
* [[Other wikis]] - a list of other wikis to examine the design of
* [http://www.linuxworld.com/news/2006/120606-closed-modules1.html?page=1 Good article on US case law related to closed-source kernel modules]

* [http://en.wikibooks.org/wiki/MediaWiki_Administrator's_Handbook MediaWiki Administrator's Handbook]

== Things I want to add to the wiki ==
* diffinfo, qdiff, reisolate - see [[Diff And Patch Tricks]], [[Source Management Tools]], and
[[Tim's patch management tools]]
* review each portal page and try to hook up material I know is orphaned

== News ==
[http://www.edn.com/article/CA6582852.html Free software encircles embedded design] Warren Web, EDN 8/7/2008

=== Videos, Articles, Presentations of, by or about me ===
* http://video.linuxfoundation.org/video/1654 - JLS 2009 - Status of Embedded Linux presentation
* http://www.vimeo.com/8042326 - Measuring Function Duration with Ftrace

Tim's Tips and Tricks

2013-04-19T16:53:59Z

Tim Bird: /* Git tips */

== Sources ==
* Hugh Blemings - Learning Kernel Hacking from clever people - [[Media:Elc-us-2008-slides-final.pdf|elc-us-2008-slides-final.pdf]]

== Ideas for individual tips ==

=== Git tips ===
* Show when a commit was entered in the tree: "git describe <commit>"
* Show the version of software where a change first appeared: "git describe --contains <commit>"
* git log <file> # narrow scope of log
** git log <dir>
** git lg [<dir_or_file>] # nice summary
*** alias.lg=log --pretty=format:'%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%cr %an)%Creset' --abbrev-commit --date=relative
*** how to do a git alias, with colors!

* find the commit for a specific line in a file: "git annotate <file> | grep <regex_of_line>"
** Then do: "git show <commit_of_interest>"
** This is good to find out: why is it here? who did this?, when did it get changed to this?

* git bisect

* git proxy (finally!!)
** See [[Using git with a proxy]]

* git grep (grep only git-managed files) (credit to Thomas Petazzoni)

Note also that you can use 'git grep', that will only grep the source
files that are under version control, skipping all object files and
other generated files you may have in your tree, if you're not doing
out of tree builds. I use 'git grep' routinely.

* Reflog: Git reflog is great for finding commits that you have 'lost' but how do you see where they went? I use reflog piped into gitk....
** git reflog | awk '{ print $1 }' | xargs gitk

----

* several people said: "git rebase -i" as a replacement for quilt
** git add -p
** git commit --amend

For fixup commits,
* git commit --fixup <commit-id>
followed by
* git rebase origin/master --autosquash
- Automatically updates an existing patch

----
More miscellaneous items
* git add -i <files> - add interactive (select individual hunks)
* git log -G<search_string> --all
** show log entries for the entire git history (including deleted files) for the search string on any added or removed lines
* git log -- file1
** show log for deleted file 'file1'

=== patch management ===
* quilt - patches as first class objects
** git for development work, then convert to quilt patches
** git rebase --interactive
* precommit - quilt pop -a, basediff, quilt push -a, scripts/fix-filemodes.sh
* diffinfo - my own tool for managing patches
** splitting a patch: splitpatch - tool to break patches apart

=== source finding ===
* cgrep, mgrep, confgrep, kgrep, armcgrep, jgrep - scan C code, Makefiles, .config, Kconfig, arm C files, and java code, respectively for a search pattern
** thin wrappers on 'find . [some conditions] -exec egrep $1'
* finds - find a string in a set of files (Credit to Todd Fischer of RidgeRun)

I use a tool, finds, I got from someone about 15 years ago. It simply
does a recursive look for a string in all files of interest in a
directory tree. When I get a big tarball of code that I don't intend to
build, but want to grab some logic, I use finds. I have created a
variety of variations of finds (which I usually later throw away) as my
search needs change. Essentially finds is

find . -exec fgrep -H "$1" {} \;

but with lots and lots of other qualifiers. You can see a usable
version at

https://www.ridgerun.com/developer/wiki/index.php/Tip_of_the_day#finds_-_find_a_string_in_common_text_files

=== kernel debugging ===
* use of fs/sync.c:SYSCALL_DEFINE0(sync) for user-space triggered printks
* use of qemu for kernel debugging

=== testing ===
* tbtorture.sh - for stress testing

=== board handling ===
* consistent board setup (including remote access)
** connections: serial, network, power (web-based control) [images]
*** power port controller [image]
** Usually use tftp boot kernel, with NFS rootfs
*** but can do other setups: feed kernel through serial line, put kernel on SD card using known-good kernel, etc.

* [[Ttc]] - command line tool for board control
** abstracts differences between boards
** allows for board sharing (ttc reserve)
** everything is scritable, and remotable
** object-oriented: inheritance to quickly modify attributes of a board "class"
** items abstracted:
*** kernel source, toolchain, architecture, defconfig, kernel image name, tool paths
*** kernel build, install
*** board console access, reboot, run arbitrary commands, reset
*** filesystem access (copy from/to)

Finding the ttyUSB* for a board connected to a multi-port hub:
(Credit to Todd Fischer of RidgeRun):
A recurring problem I had was finding the right /dev/ttyUSB? device. I
have a nice USB to 4 serial port adaptor in a metal box I have screwed
to my bench. I also use several USB serial dongles. For reasons I
don't understand, they change association (at least under Ubuntu). I
created two scripts lsuart and uart. The output of lsuart gives me
information about the serial ports, like

Available USB UARTs
-------------------
a:/dev/ttyUSB0 FTDI USB Serial Device 0000:00:04.1-4.1 (4 port jack A)
b:/dev/ttyUSB1 FTDI USB Serial Device 0000:00:04.1-4.2 (4 port jack B)
c:/dev/ttyUSB2 FTDI USB Serial Device 0000:00:04.1-4.3 (4 port jack C)
:/dev/ttyUSB3 pl2303 0000:00:02.1-4.2 ()
d:/dev/ttyUSB4 FTDI USB Serial Device 0000:00:04.1-4.4 (4 port jack D)

In use
------
/dev/ttyUSB4: 9705: picocom -b 115200 -r -l /dev/ttyUSB4

I gave names (a, b, c, d) to the 4 fixed uarts. Now I look at which
port my serial cable is plugged into - say the 3rd one, which I call c,
and I can fire up picocom with the standard setting using

uart c

Sometimes I can't find a terminal session (too many windows open), so I
use the PID from the In use section of the lsuart output to kill it.

If others have the "where is my USB serial port" problem, I can love up
my shell script (lsuart is a symbolic link to uart), which is around 200
lines of ugly sysfs parsing.
----

* ser2net (Credit to Jean-Christophe PLAGNIOL-VILLARD)

* ptxdist and barebox (Credit to Robert Schwebel)
** can load kernel via nfs server (does not require root privileges on host)

---
How to copy data over serial line to Linux: (credit to Alan Carvalho de Assis)
http://acassis.wordpress.com/2012/10/21/how-to-transfer-files-to-a-linux-embedded-system-over-serial/

=== personal productivity ===
* my workflow:
** different day for each topic area
** today script
* multiple todo's, NOTES all over the place (several in each work directory)
** must do a context save in each directory, before moving on
** git branch is essential - commit changes (even if flaky) before moving on

* my favoriate mailing lists:
** sorry, but linux-embedded seems to be dead
** lkml, linux-arm, yocto-devel
** celinux-dev

* attend conferences, talk to smart people
* conferences I attend:
** ELC, ELCE, linuxcon US, Linuxcon Japan, Linuxcon Europe (by serendipity)
** what about plumbers?? other regional events??

Tim's Tips and Tricks

2013-04-19T16:52:42Z

Tim Bird: /* Git tips */ add a few more tips

== Sources ==
* Hugh Blemings - Learning Kernel Hacking from clever people - [[Media:Elc-us-2008-slides-final.pdf|elc-us-2008-slides-final.pdf]]

== Ideas for individual tips ==

=== Git tips ===
* Show when a commit was entered in the tree: "git describe <commit>"
* Show the version of software where a change first appeared: "git describe --contains <commit>"
* git log <file> # narrow scope of log
** git log <dir>
** git lg [<dir_or_file>] # nice summary
*** alias.lg=log --pretty=format:'%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%cr %an)%Creset' --abbrev-commit --date=relative
*** how to do a git alias, with colors!

* find the commit for a specific line in a file: "git annotate <file> | grep <regex_of_line>"
** Then do: "git show <commit_of_interest>"
** This is good to find out: why is it here? who did this?, when did it get changed to this?

* git bisect

* git proxy (finally!!)
** See [[Using git with a proxy]]

* git grep (grep only git-managed files) (credit to Thomas Petazzoni)

Note also that you can use 'git grep', that will only grep the source
files that are under version control, skipping all object files and
other generated files you may have in your tree, if you're not doing
out of tree builds. I use 'git grep' routinely.

* Reflog: Git reflog is great for finding commits that you have 'lost' but how do you see where they went? I use reflog piped into gitk....
** git reflog | awk '{ print $1 }' | xargs gitk

----

* several people said: "git rebase -i" as a replacement for quilt
** git add -p
** git commit --amend

For fixup commits,
* git commit --fixup <commit-id>
followed by
* git rebase origin/master --autosquash
- Automatically updates an existing patch

----
More miscellaneous items
* git add -i <files> - add interactive (select individual hunks)
* git log -G<search_string> --all
** show log entries for the entire git history (including deleted files) for the search string on any added or removed lines

=== patch management ===
* quilt - patches as first class objects
** git for development work, then convert to quilt patches
** git rebase --interactive
* precommit - quilt pop -a, basediff, quilt push -a, scripts/fix-filemodes.sh
* diffinfo - my own tool for managing patches
** splitting a patch: splitpatch - tool to break patches apart

=== source finding ===
* cgrep, mgrep, confgrep, kgrep, armcgrep, jgrep - scan C code, Makefiles, .config, Kconfig, arm C files, and java code, respectively for a search pattern
** thin wrappers on 'find . [some conditions] -exec egrep $1'
* finds - find a string in a set of files (Credit to Todd Fischer of RidgeRun)

I use a tool, finds, I got from someone about 15 years ago. It simply
does a recursive look for a string in all files of interest in a
directory tree. When I get a big tarball of code that I don't intend to
build, but want to grab some logic, I use finds. I have created a
variety of variations of finds (which I usually later throw away) as my
search needs change. Essentially finds is

find . -exec fgrep -H "$1" {} \;

but with lots and lots of other qualifiers. You can see a usable
version at

https://www.ridgerun.com/developer/wiki/index.php/Tip_of_the_day#finds_-_find_a_string_in_common_text_files

=== kernel debugging ===
* use of fs/sync.c:SYSCALL_DEFINE0(sync) for user-space triggered printks
* use of qemu for kernel debugging

=== testing ===
* tbtorture.sh - for stress testing

=== board handling ===
* consistent board setup (including remote access)
** connections: serial, network, power (web-based control) [images]
*** power port controller [image]
** Usually use tftp boot kernel, with NFS rootfs
*** but can do other setups: feed kernel through serial line, put kernel on SD card using known-good kernel, etc.

* [[Ttc]] - command line tool for board control
** abstracts differences between boards
** allows for board sharing (ttc reserve)
** everything is scritable, and remotable
** object-oriented: inheritance to quickly modify attributes of a board "class"
** items abstracted:
*** kernel source, toolchain, architecture, defconfig, kernel image name, tool paths
*** kernel build, install
*** board console access, reboot, run arbitrary commands, reset
*** filesystem access (copy from/to)

Finding the ttyUSB* for a board connected to a multi-port hub:
(Credit to Todd Fischer of RidgeRun):
A recurring problem I had was finding the right /dev/ttyUSB? device. I
have a nice USB to 4 serial port adaptor in a metal box I have screwed
to my bench. I also use several USB serial dongles. For reasons I
don't understand, they change association (at least under Ubuntu). I
created two scripts lsuart and uart. The output of lsuart gives me
information about the serial ports, like

Available USB UARTs
-------------------
a:/dev/ttyUSB0 FTDI USB Serial Device 0000:00:04.1-4.1 (4 port jack A)
b:/dev/ttyUSB1 FTDI USB Serial Device 0000:00:04.1-4.2 (4 port jack B)
c:/dev/ttyUSB2 FTDI USB Serial Device 0000:00:04.1-4.3 (4 port jack C)
:/dev/ttyUSB3 pl2303 0000:00:02.1-4.2 ()
d:/dev/ttyUSB4 FTDI USB Serial Device 0000:00:04.1-4.4 (4 port jack D)

In use
------
/dev/ttyUSB4: 9705: picocom -b 115200 -r -l /dev/ttyUSB4

I gave names (a, b, c, d) to the 4 fixed uarts. Now I look at which
port my serial cable is plugged into - say the 3rd one, which I call c,
and I can fire up picocom with the standard setting using

uart c

Sometimes I can't find a terminal session (too many windows open), so I
use the PID from the In use section of the lsuart output to kill it.

If others have the "where is my USB serial port" problem, I can love up
my shell script (lsuart is a symbolic link to uart), which is around 200
lines of ugly sysfs parsing.
----

* ser2net (Credit to Jean-Christophe PLAGNIOL-VILLARD)

* ptxdist and barebox (Credit to Robert Schwebel)
** can load kernel via nfs server (does not require root privileges on host)

---
How to copy data over serial line to Linux: (credit to Alan Carvalho de Assis)
http://acassis.wordpress.com/2012/10/21/how-to-transfer-files-to-a-linux-embedded-system-over-serial/

=== personal productivity ===
* my workflow:
** different day for each topic area
** today script
* multiple todo's, NOTES all over the place (several in each work directory)
** must do a context save in each directory, before moving on
** git branch is essential - commit changes (even if flaky) before moving on

* my favoriate mailing lists:
** sorry, but linux-embedded seems to be dead
** lkml, linux-arm, yocto-devel
** celinux-dev

* attend conferences, talk to smart people
* conferences I attend:
** ELC, ELCE, linuxcon US, Linuxcon Japan, Linuxcon Europe (by serendipity)
** what about plumbers?? other regional events??

System Size Auto-Reduction

2013-04-18T22:32:07Z

Tim Bird: add some stuff about kernel command line args and /proc entries

This page has notes and an outline for Tim Bird's Linux Auto-Reduction research.

== Talk info ==
=== Title ===
Advanced size optimization of the Linux kernel
=== Abstract ===
This presentation will cover recent research by Tim on aggressive size reduction of the Linux kernel. This will include results from using gcc link-time optimization (LTO) with the ARM architecture (using Andi Kleen's out-of-tree patches), as well as results and discussion of other optimization techniques (including whole-system optimization for embedded devices).

This talk is directed at kernel developers interested in reducing the size of their Linux systems (and possible improving their performance in the process). The talk will be highly technical.

== Research Areas ==
=== LTO ===
* What is it?
* what was required to get it to work?
* Andi Kleen's patch set
** what do they do?
** how big are they?
** mainline status?
* what is the size gain (see ELC poster)
* what can be done with it?
* long-term possibilities for LTO

=== global constraints ===
* overall idea: create constraints external to code, and use for optimization
* rationale: can't maintain in-tree - too many config items
* make the application of constraints automatic
* use existing constraints to generate new constraints
* constraints can flow between user-space and kernel

* example: uid=0
* constraint language
* application by commenting out references (replace with 0 constant)
** use compiler to find code references (via error messages)
*** eliminates problem with duplicate names (uid in different structure)
* constant propagation (by, e.g. LTO) reduces code

=== syscall elimination ===
* scan file system
* create report of used and unused system calls
* mark syscalls unused in kernel
** arch/arm/kernel/calls.S (and arch/arm/kernel/entry-common.S
* make sure unused syscalls are not __attribute__(externally_visible)
** technique of asmlinkage_<syscall>
* use LTO to eliminate calls
* results: 50K-90K

=== ARM stack reduction ===
* 4k stacks
* stack extensions

=== link-time rewriting ===

=== cold code compression ===
* D. Chanet did cold code compression
* consists of:
** profiling the kernel
** marking code regions as cold or frozen
** replacing them with stubs
** compressing them
* At execution time:
** if a stub is called, it decompresses the code and calls it
** stub is fixed up to directly call decompressed code in future
** code is left decompressed forever

=== cold code compression ===
Results:
** MUST see paper for details (it's quite complicated)
* on 2.4.25 kernel
** cold code compression resulted in 7% reduction for i386 kernel and
** 11.7% reduction for ARM kernel

== Talk outline ==
This talk will be presented at LinuxCon Japan 2013:

=== Title ===
* Advanced size optimization of the Linux kernel
** by Tim Bird, Sony Mobile Communication

=== Self-Introduction ===
* I am Tim Bird
* Now working at Sony Mobile
* Researching system size for many years
* Long background in extremely small systems
** pre-professional: first program on TRS-80, in basic, 8K ram
** NetWare Lite - file and print server in 50K (in 1991)

=== The problem of Bloat ===
* Software bloat occurs because systems are built with more software than is really needed for a given task
* Open Source software meets the needs of thousands of different systems
** Linux scales from tiny sensors to supercomputers (extreme SMP and high-end clusters)
** Linux supports many, many features, only some of which are configurable
* Software must be generalized for many use cases
* bloat problem is:
** How to re-specialize the software, eliminating unused features and dead code?

=== Bloat (cont.) ===
* Software gets more generalized over time
* Can't use strategy of manual tuning (config options)
** It gets harder and harder to remove things over time
** About 13,000 config items now (2.6.12 had 4700)
** You have to be an expert in too many things to reduce the kernel
* Must rely on automated methods of reduction
* Should use an additive, rather than subtractive method of building a system
** ultimate vision: indicate what you want/need, and build up system to support it

=== Bloat (cont. 2) ===
* In desktop or server, virtual memory makes bloat issue less important for user-space programs
** Only working set of program is loaded - pages are loaded on demand
** For kernel, all pages are always loaded

=== Automatic reduction (intro)===
The problem with automatic reduction is that "the system" doesn't know what
software is needed and what is not. there needs to be a way to tell it
about things that are not going to be used.

=== auto-reduce - story of 8 bytes of bloat ===
Story of the conditional check in kdb:
* I found a bug in kdb, when a particular option was using in the configuration file
* not everyone uses the configuration file
* not everyone uses the particular option
* bug only triggered in those circumstances
* I wrote a small patch, to guard against use of a variable prematurely
* problem: all users of KDB now have this check, and suffer this overhead
** it wasn't much, just a single compare
** but this is how bloat builds up over time
** It bothered me because I knew most people didn't need the check
* "correct" solution would be to parse the config file, and make the code compile-time configurable
** this adds more complexity than it is worth.

=== generalizing the problem of bloat ===
System doesn't know inputs:

* It's very easy to configure the kernel to omit the driver for missing hardware.
* It's very difficult to configure the kernel to omit error handling for bugs that
will never occur due to fixed use cases.

=== An example of fixed input (uid in kernel) ===
* throughout the kernel, there are references to uid
** comparisons, storing, referencing
* it turns out this is set by setuid(), by the 'login' program.
** login does a lookup and validates user account name in /etc/passwd
* what if /etc/passwd only has 'root' and no others?
* setuid() could only be called with a value of 0
* can I encode this constraint on the system.

=== types of constraints ===
There are numerous other examples of constraints:
* kernel command line arguments never used
* syscalls never called by any program
* parameters that are never used, or parameter values that are never passed in
** e.g. ioctl value that is not possible
** this only works in a fixed
* /proc values never referenced
* /sys values never referenced

=== also goes back up to user-space ===
* return values that are not possible

=== kernel command line args ===
* Documented in Documentation/kernel-parameters.txt
* defined with __setup()
** approximately 480 __setup routines in kernel
* points to function
* almost always sets a variable, which would default to 0

=== /proc values ===
* Includes sysctl values
* there are approximately 1200, NOT related specifically to a process
* about 120 per process
** about 80 related to networking (on my desktop box)
* 40 others

=== Tiny Distribution ===
* poky-tiny distribution (yocto project)
* see https://wiki.yoctoproject.org/wiki/Poky-Tiny
* Good for testing and further research

== References ==
* Chanet D. ... "Automated reduction of the memory footprint of the linux kernel"
* Haifen He. ..."Code Compaction of an Operating System Kernel"

== Materials ==
* [[File:0001-ARM-LTO-avoid-errors-on-unified-assembly-macros.patch]]

System Size Auto-Reduction

2013-04-17T18:39:11Z

Tim Bird:

This page has notes and an outline for Tim Bird's Linux Auto-Reduction research.

== Talk info ==
=== Title ===
Advanced size optimization of the Linux kernel
=== Abstract ===
This presentation will cover recent research by Tim on aggressive size reduction of the Linux kernel. This will include results from using gcc link-time optimization (LTO) with the ARM architecture (using Andi Kleen's out-of-tree patches), as well as results and discussion of other optimization techniques (including whole-system optimization for embedded devices).

This talk is directed at kernel developers interested in reducing the size of their Linux systems (and possible improving their performance in the process). The talk will be highly technical.

== Research Areas ==
=== LTO ===
* What is it?
* what was required to get it to work?
* Andi Kleen's patch set
** what do they do?
** how big are they?
** mainline status?
* what is the size gain (see ELC poster)
* what can be done with it?
* long-term possibilities for LTO

=== global constraints ===
* overall idea: create constraints external to code, and use for optimization
* rationale: can't maintain in-tree - too many config items
* make the application of constraints automatic
* use existing constraints to generate new constraints
* constraints can flow between user-space and kernel

* example: uid=0
* constraint language
* application by commenting out references (replace with 0 constant)
** use compiler to find code references (via error messages)
*** eliminates problem with duplicate names (uid in different structure)
* constant propagation (by, e.g. LTO) reduces code

=== syscall elimination ===
* scan file system
* create report of used and unused system calls
* mark syscalls unused in kernel
** arch/arm/kernel/calls.S (and arch/arm/kernel/entry-common.S
* make sure unused syscalls are not __attribute__(externally_visible)
** technique of asmlinkage_<syscall>
* use LTO to eliminate calls
* results: 50K-90K

=== ARM stack reduction ===
* 4k stacks
* stack extensions

=== link-time rewriting ===

=== cold code compression ===
* D. Chanet did cold code compression
* consists of:
** profiling the kernel
** marking code regions as cold or frozen
** replacing them with stubs
** compressing them
* At execution time:
** if a stub is called, it decompresses the code and calls it
** stub is fixed up to directly call decompressed code in future
** code is left decompressed forever

=== cold code compression ===
Results:
** MUST see paper for details (it's quite complicated)
* on 2.4.25 kernel
** cold code compression resulted in 7% reduction for i386 kernel and
** 11.7% reduction for ARM kernel

== Talk outline ==
This talk will be presented at LinuxCon Japan 2013:

=== Title ===
* Advanced size optimization of the Linux kernel
** by Tim Bird, Sony Mobile Communication

=== Self-Introduction ===
* I am Tim Bird
* Now working at Sony Mobile
* Researching system size for many years
* Long background in extremely small systems
** pre-professional: first program on TRS-80, in basic, 8K ram
** NetWare Lite - file and print server in 50K (in 1991)

=== The problem of Bloat ===
* Software bloat occurs because systems are built with more software than is really needed for a given task
* Open Source software meets the needs of thousands of different systems
** Linux scales from tiny sensors to supercomputers (extreme SMP and high-end clusters)
** Linux supports many, many features, only some of which are configurable
* Software must be generalized for many use cases
* bloat problem is:
** How to re-specialize the software, eliminating unused features and dead code?

=== Bloat (cont.) ===
* Software gets more generalized over time
* Can't use strategy of manual tuning (config options)
** It gets harder and harder to remove things over time
** About 13,000 config items now (2.6.12 had 4700)
** You have to be an expert in too many things to reduce the kernel
* Must rely on automated methods of reduction
* Should use an additive, rather than subtractive method of building a system
** ultimate vision: indicate what you want/need, and build up system to support it

=== Bloat (cont. 2) ===
* In desktop or server, virtual memory makes bloat issue less important for user-space programs
** Only working set of program is loaded - pages are loaded on demand
** For kernel, all pages are always loaded

=== Automatic reduction (intro)===
The problem with automatic reduction is that "the system" doesn't know what
software is needed and what is not. there needs to be a way to tell it
about things that are not going to be used.

=== auto-reduce - story of 8 bytes of bloat ===
Story of the conditional check in kdb:
* I found a bug in kdb, when a particular option was using in the configuration file
* not everyone uses the configuration file
* not everyone uses the particular option
* bug only triggered in those circumstances
* I wrote a small patch, to guard against use of a variable prematurely
* problem: all users of KDB now have this check, and suffer this overhead
** it wasn't much, just a single compare
** but this is how bloat builds up over time
** It bothered me because I knew most people didn't need the check
* "correct" solution would be to parse the config file, and make the code compile-time configurable
** this adds more complexity than it is worth.

=== generalizing the problem of bloat ===
System doesn't know inputs:

* It's very easy to configure the kernel to omit the driver for missing hardware.
* It's very difficult to configure the kernel to omit error handling for bugs that
will never occur due to fixed use cases.

=== An example of fixed input (uid in kernel) ===
* throughout the kernel, there are references to uid
** comparisons, storing, referencing
* it turns out this is set by setuid(), by the 'login' program.
** login does a lookup and validates user account name in /etc/passwd
* what if /etc/passwd only has 'root' and no others?
* setuid() could only be called with a value of 0
* can I encode this constraint on the system.

=== types of constraints ===
There are numerous other examples of constraints:
* kernel command line arguments never used
* syscalls never called by any program
* parameters that are never used, or parameter values that are never passed in
** e.g. ioctl value that is not possible
** this only works in a fixed
* /proc values never referenced
* /sys values never referenced

=== also goes back up to user-space ===
* return values that are not possible

=== Tiny Distribution ===
* poky-tiny distribution (yocto project)
* see https://wiki.yoctoproject.org/wiki/Poky-Tiny
* Good for testing and further research

== References ==
* Chanet D. ... "Automated reduction of the memory footprint of the linux kernel"
* Haifen He. ..."Code Compaction of an Operating System Kernel"

== Materials ==
* [[File:0001-ARM-LTO-avoid-errors-on-unified-assembly-macros.patch]]

File:0001-ARM-LTO-avoid-errors-on-unified-assembly-macros.patch

2013-04-11T19:00:32Z

Tim Bird:

System Size Auto-Reduction

2013-04-11T19:00:07Z

Tim Bird: add link to LTO-for-arm patch

2013-03-27T20:34:27Z

2013-03-08T23:59:41Z

Tim Bird:

Japan Technical Jamboree 44

2013-03-08T00:40:00Z

Tim Bird: /* Agenda */

{| cellpadding="0" cellspacing="0" border="0"
|-
|[[Image:HeadTitle.gif]]
|-
| style="text-align:right" |<span style="font-size:large">Date: March 8th/ 日付: 3月8日（金）</span>
* At Nakano Sunplaza / 於、中野サンプラザ
* Please join "celinux-dev" mailinglist to get the latest announcement.
* "celinux-dev" メーリングリストにこのイベント関係のアナウンスが流れます。ぜひ参加してください。
** [http://lists.celinuxforum.org/mailman/listinfo/celinux-dev How to join.]

|-
|[[Image:J_Jam_SessionProp.gif]]
|-
|style="text-align:right" |[[Japan TJ Session Proposal|<span style="color:red">Session proposal how-to. / 提案の方法</span>]]
|}
= Introduction / はじめに =
* The Japan Technical Jamboree is a forum-wide technical meeting of the CE Workgroup of the Linux Foundation. This meeting will be located conveniently in Japan and use Japanese as the native language of the event.
* A general guidance page is available. / 初めての方はこちらもお読みください。
** [http://tree.celinuxforum.org/CelfPubWiki/JapanTechnicalJamboreeGuidance JapanTechnicalJamboreeGuidance] (Japanese/English)
== Special remarks for non Japanese speakers ==
* This page is Japanese/English bilingual. Please allow some contents on this page are not translated into English because of this event is Japan regional one, though we try to place English translation.
* If you would like to perform your presentation in English, we also welcome you to join! We hope you to speak slowly without any complicated expressions. Most Japanese developers are capable to understand plain English.
== Previous Jamboree ==
* Please look into the wiki page. / 下記のWikiページをご覧ください。
** [[Japan Technical Jamboree 43]]

= Date and venue... / 日付・場所... =
{|
|+
|-
| style="vertical-align:top" |
[[Image:NakanoSunplaza2.jpg]]
| style="vertical-align:top; background-color:#F0F0F0" width=90% |
* Date '''March 8th, 2013'''
** '''Starting at 10 am'''
* At '''''Nakano Sunplaza / 8F Training Room 2''''' / 会場 '''''中野サンプラザ / 8階・研修室２'''''
** Nakano Sunplaza is located just close to '''Nakano''' station (JR/Tokyo Metoro ).
** http://www.sunplaza.jp/ (Japanese)
* Admission: '''''Free of charge''''' / 参加費用: '''''無料'''''
* Not limited for CELF members. / CELF会員以外も参加・セッション持ち込み共に可能
* Coordinators / 世話役 (Your inquiries in English welcome)
** Hisao Munakata / 宗像尚郎 (hisao_dot_munakata_dot_vt(a)renesas_dot_com)
** Satoru Ueda / 上田理 (Satoru_dot_Ueda(a)jp_dot_sony_dot_com)
|}
== Registration / 参加登録 ==
* No registration required / 参加登録は要りません。

= Main Topics =
* It is just after the Kernel Summit and final Jamboree before ELC Europe.

= Agenda / 進行 =
== Agenda ==

{|
|+
! style="text-align:left; background-color:#B0FFB0; width:50" | Time
! style="text-align:left; background-color:#B0FFB0; width:70%" | Title and presenter
! style="text-align:left; background-color:#B0FFB0" | Notes
Presentation Materials
|-
| style="vertical-align:top; background-color:#F0F0F0" | 10:00..
| style="vertical-align:top; background-color:#F0F0F0" |
* Opening / 連絡事項
| style="vertical-align:top; background-color:#F0F0F0" |

|-
| style="vertical-align:top; background-color:#F0F0F0" | 10:15am..
| style="vertical-align:top; background-color:#F0F0F0" |
'''Embedded Linux status update'''
* '''Tim Bird''' (Connect from USA)
** Latest Community topics update - quick update.
** Latest CE WG project updates
| style="vertical-align:top; background-color:#F0F0F0" |
(In English / 英語のセッションです)
* [[Media:Status-of-Embedded-Linux-2013-03-JJ44.pdf]]
* [[Media:Status-of-Embedded-Linux-2013-030JJ44.pptx]]
|-
| style="vertical-align:top; background-color:#B0B0FF" | 11:30..
| style="vertical-align:top; background-color:#B0B0FF" | Lunch
| style="vertical-align:top; background-color:#B0B0FF" |
|-
| style="vertical-align:top; background-color:#F0F0F0" | 12:30pm..
| style="vertical-align:top; background-color:#F0F0F0" |
'''Evalution of Linux Container (LXC) on Embedded Linux'''
* '''Yuki Machida, FUJITSU COMPUTER TECHNOLOGIES LIMITED'''
| style="vertical-align:top; background-color:#F0F0F0" |
|-
| style="vertical-align:top; background-color:#F0F0F0" | 1:30pm..
| style="vertical-align:top; background-color:#F0F0F0" |
'''barebox''' with GUI and the up comming Application support
* '''Jean-Christophe PLAGNIOL-VILLARD'''
| style="vertical-align:top; background-color:#F0F0F0" |
(In English / 英語のセッションです)
* Connecting from Hong Kong
* http://www.jcrosoft.com/
* [[Media:barebox.pdf]]
|-
| style="vertical-align:top; background-color:#F0F0F0" | 2:00pm..
| style="vertical-align:top; background-color:#F0F0F0" |
'''The latest status of LTSI project'''
* '''Hisao Munakata'''
| style="vertical-align:top; background-color:#F0F0F0" |
|-
| style="vertical-align:top; background-color:#F0F0F0" | 2:30pm..
| style="vertical-align:top; background-color:#F0F0F0" |
'''Tips of Malloc & Free'''
* '''Tetsuyuki Kobayashi'''
| style="vertical-align:top; background-color:#F0F0F0" |
http://d.hatena.ne.jp/embedded/20130226/p1
|-
| style="vertical-align:top; background-color:#F0F0F0" | 3:30pm..
| style="vertical-align:top; background-color:#F0F0F0" |
'''Applying the Linux Cgroups Mechanism to AV appliances'''
* '''Yukinori Endo / Mitsubishi Electric Corp.'''
| style="vertical-align:top; background-color:#F0F0F0" |
|-
| style="vertical-align:top; background-color:#F0F0F0" | 4:00pm..
| style="vertical-align:top; background-color:#F0F0F0" |
'''Android Builders Summit 2013 / Embedded Linux Conference 2013 Report'''
* '''Kazuomi Kato, Panasonic'''
* '''Shinsuke Kato, Panasonic'''
** ABSとELCの参加レポートを予定しています。セッションは終わりの方でOKです。個別の話題を前に追記下さい。
* Togetter
** Android Builders Summit 2013 #abs2013 http://togetter.com/li/458582
** Yocto Project Developer Day #yocto http://togetter.com/li/459273
** Embedded Linux Conference 2013 #lfelc http://togetter.com/li/459672
* Links
** ABS https://events.linuxfoundation.org/events/android-builders-summit
** ELC https://events.linuxfoundation.org/events/embedded-linux-conference
* Links after archived
** ABS https://events.linuxfoundation.org/archive/2013/android-builders-summit
** ELC https://events.linuxfoundation.org/archive/2013/embedded-linux-conference
| style="vertical-align:top; background-color:#F0F0F0" |
|-
| style="vertical-align:top; background-color:#F0F0F0" | 4:40pm..
| style="vertical-align:top; background-color:#F0F0F0" |
* '''''Calling for your session proposals'''''
| style="vertical-align:top; background-color:#F0F0F0" |
* [[Japan TJ Session Proposal|<span style="color:red">Session proposal how-to. / 提案の方法</span>]]
|-
|}

* Please be noted above time table is just a guideline and may be shifted. / 上記の時間割は目安です。かなり前後する可能性がありますので、あらかじめご承知おきください。

== Special Remarks ==
* Please place some detail descriptions about each presentation topics.
: こちらに各プレゼンテーションの詳細などを記載してください。

* togetter

= Ask for your help / お願い =
== Presentation Materials ==
* We wish you to prepare the materials in English. / 出来るだけプレゼンテーション資料は英語で表記してください。絶対ではありませんが、日本語が理解できない方に対しての配慮が出来ればと思います。
* Please leave your material in this wiki site after the event. / ジャンボリー終了後、プレゼンテーション資料はこのWikiに残してください。
== English Translation Volunteer ==
* If you can help the translation volunteer from Japanese to English, we would be very much appreciated! / 日本語を英訳していただくボランティアを大歓迎します！

[[Category:Japan Technical Jamboree]]
[[Category:Events]]