Linux Tiny Notes
Here are some miscellaneous notes on using Linux tiny. I wrote this up really quickly while I was integrating Linux-tiny into a 2.6.11 kernel and doing some size testing.
I'm providing these raw notes so that others might benefit from my experience.
Contents
- 1 Patch Notes
- 2 devlist.patch
- 3 core-small.patch - CONFIG_CORE_SMALL
- 4 config-net-small.patch
- 5 Miscellanoues patches
- 6 *inline patches
- 7 06-crypto-sleep.patch
- 8 # measurement bits
- 9 kill-printk.patch
- 10 nobug.patch
- 11 nopanic.patch
- 12 no-elf-core.patch
- 13 no-kcore.patch
- 14 no-aio.patch
- 15 no-xattr.patch
- 16 fslock.patch
- 17 direct-io-core.patch
- 18 Changes in number of in-kernel items
- 19 change-hz.patch
- 20 Notes on building and size reductions from various configs
Patch Notes
Here are some notes on individual linux-tiny sub-patches:
devlist.patch
- replaces device allocation code, don't know size impact, is NOT conditional
- maybe should add, but don't know - looks dangerous
core-small.patch - CONFIG_CORE_SMALL
Add CONFIG_CORE_SMALL Kconfig option
CONFIG_CORE_SMALL patches
These are hash table and data structure static size adjustments
- pid-max.patch
- user-hash.patch
- futex-queues.patch
- tvec_bases.patch
- con_buf.patch
config-net-small.patch
Add CONFIG_NET_SMALL Kconfig option
CONFIG_NET_SMALL patches
These tune some net-related data sizes
- cache_defer_hash.patch
- unix_socket_table.patch
- inet_protos.patch
- flow-cache-small.patch
Miscellanoues patches
- small-8023.patch - what does this do?
- tg3-oops.patch - what does this do?
*inline patches
- just make big routines not inline (unconditionally)
- Seems pretty safe to me.
06-crypto-sleep.patch
- looks scary - not immediately obvious from patch how it saves size.
# measurement bits
Looks like very harmless stuff to instrument kernel for mem debugging, and give tools for parsing stuff.
- kmalloc-accounting.patch - keep track of kmallocs
- audit-bootmem - print out all bootmem allocations
- bloat-o-meter.patch - provides bloat-o-meter program to compare two kernel binary images
- This script shows the size difference on a symbol-by-symbol basis between two binaries (sorted by the amount of difference)
- deprecate-inline.patch - mark inlines as deprecated so compiler emits warnings
- Note that it provides count-inlines program to parse compiler output and report size of inlines.
- func-size.patch - use information about inlines to calculate function sizes (script is very similar to count-inlines program)
kill-printk.patch
- is single biggest reducer of kernel size, doesn't seem to work with serial console!!
- CONFIG_PRINTK
nobug.patch
Allow elimination of BUG code, which expands with file and line strings. There are 1776 BUG() references in the 2.6.11 kernel. There are 1930 BUG_ON(xxx) references in the kernel.
- uses CONFIG_BUG
nopanic.patch
too dangerous to turn off panic, not used.
no-elf-core.patch
Allow disabling of ELF core dumps
- CONFIG_ELF_CORE
no-kcore.patch
Allow disabling of /proc/kcore, is x86-specific
- uses CONFIG_PROC_KCORE
no-aio.patch
Allow disabling of Posix async IO
- uses CONFIG_AIO
no-xattr.patch
Allow disabling of xattr syscalls
- uses CONFIG_XATTR
fslock.patch
Allow disabling of file locking syscalls
- uses CONFIG_FILE_LOCKING
direct-io-core.patch
Allow disabling of direct IO support
- uses CONFIG_DIRECTIO
Changes in number of in-kernel items
max-swapfiles.patch
Make number of swapfiles configurable (default=32) (shift=5 => 32 files, shift=0 => 1 file)
- uses CONFIG_MAX_SWAPFILES_SHIFT
ldiscs.patch
Make number of tty line disciplines configurable (default=16)
- uses CONFIG_NR_LDISCS
max_user_rt_prio.patch
Make number of schedule slots configurable (default=100)
- uses CONFIG_MAX_USER_RT_PRIO
change-hz.patch
It's not obvious how this changes the size of the kernel, and it appears to be x86 only
Notes on building and size reductions from various configs
Notes on OSK builds:
- first, did 'quilt pop pm/deferred-resume-test.patch', then built to get vmlinux.baseline
text data bss dec filename 2107356 333344 113680 2554380 vmlinux.baseline
- did 'quilt push -a', then built to get vmlinux.patched
bloat-o-meter vmlinux.baseline vmlinux.patched = +2920/-8884 text data bss dec filename 2101372 333888 113648 2548908 vmlinux.patched
- configured all parameters at minimums, then built
- couldn't finish linking, and got the following errors:
- undefined reference to add_preferred_console
- undefined reference to generic_setxattr and generic_getxattr
- undefined reference to printk
- couldn't finish linking, and got the following errors:
- configured printk and xattrs back on, then built to get vmlinux.try1
bloat-o-meter vmlinux.patches vmlinux.try1 = +4520/-350490 text data bss dec filename 1734504 90616 89036 1914156 vmlinux.try1
Panic trying to mount rootfs via nfs
- tried to isolate nfs problem:
- configured with AIO, file locks and direct IO on - same problem
- configured with 5 swap files, 8 tty line disciplines, and 100 mat rt levels - same problem
- went back to working config - diffed and noticed NFS_FS was off
- NFS_FS requires FILE_LOCKING
Table of Config options
Here is a table showing default value and value recommended for size savings.
Note that I couldn't use CONFIG_PRINTK, CONFIG_XATTR or CONFIG_FILE_LOCKING in my setup.
CONFIG option | description | default | small |
---|---|---|---|
CONFIG_CORE_SMALL | tune some kernel data sizes | N | Y |
CONFIG_NET_SMALL | tune some net-related data sizes | N | Y |
CONFIG_KMALLOC_ACCOUNTING | turn on kmalloc accounting | N | Y - for measurement |
CONFIG_AUDIT_BOOTMEM | print out all bootmem allocations | N | Y - for measurement |
CONFIG_DEPRECATE_INLINES | cause compiler to emit info about inlines | N | Y - for measurement |
CONFIG_PRINTK | allow disable of printk code and message data | Y | N |
CONFIG_BUG | allow elimination of BUG (and BUG_ON??) code | Y | N |
CONFIG_ELF_CORE | allow disabling of ELF core dumps | Y | N |
CONFIG_PROC_KCORE | allow disabling of /proc/kcore | Y | N |
CONFIG_AIO | allow disabling of async IO syscalls | Y | N |
CONFIG_XATTR | allow disabling of xattr syscalls | Y | N |
CONFIG_FILE_LOCKING | allow disabling of file locking syscalls | Y | N |
CONFIG_DIRECTIO | allow disabling of direct IO support | Y | N |
CONFIG_MAX_SWAPFILES_SHIFT | number of swapfiles | 5 | 0 |
CONFIG_NR_LDISCS | number of tty line disciplines | 16 | 2? |
CONFIG_MAX_USER_RT_PRIO | number of RT priority levels (schedule slots) | 100 | 5? |
Other config options | These are not in Linux-tiny, but help with size | default | small |
CONFIG_KALLSYMS | load all symbols for debugging/kksymoops | Y | N |
CONFIG_SHMEM | allow use of shmem filesystem | Y | N |
Some results
This was for a quick check of Linux-tiny on an OSK board.
section | vmlinux.baseline | vmlinux.try3 | delta | percent |
---|---|---|---|---|
text | 2107356 | 2057276 | -50080 | -2% |
data | 333344 | 100988 | -232356 | -69% |
bss | 113680 | 92784 | -20896 | -18% |
total | 2554380 | 2251048 | -303332 | -11% |
Program for producing report (Download)
Miscellaneous test notes
- kernel would not link with CONFIG_PRINTK=n
- undefined reference to add_preferred_console
- undefined reference to printk
- kernel would not link with CONFIG_XATTR=n
- undefined reference to generic_setxattr and generic_getxattr
- kernel would not mount root fs (NFS) with CONFIG_FILE_LOCKING=n
Config changes
[tbird@crest osk]$ diff config.baseline config.try3 --- config.baseline 2005-05-23 17:31:57.000000000 -0700 +++ config.try3 2005-05-23 17:31:40.000000000 -0700 @@ -1,7 +1,7 @@ # # Automatically generated make config: don't edit # Linux kernel version: 2.6.11.10-alp -# Mon May 23 16:49:43 2005 +# Mon May 23 17:27:14 2005 # CONFIG_ARM=y CONFIG_MMU=y @@ -34,34 +34,33 @@ CONFIG_FASTBOOT=y # CONFIG_RTC_NO_SYNC is not set CONFIG_PRESET_LPJ=0 -# CONFIG_EMBEDDED is not set -CONFIG_KALLSYMS=y -CONFIG_KALLSYMS_ALL=y -CONFIG_KALLSYMS_EXTRA_PASS=y +CONFIG_EMBEDDED=y +# CONFIG_KALLSYMS is not set # CONFIG_KMALLOC_ACCOUNTING is not set # CONFIG_AUDIT_BOOTMEM is not set CONFIG_PRINTK=y -CONFIG_ELF_CORE=y -# CONFIG_CORE_SMALL is not set -# CONFIG_NET_SMALL is not set -CONFIG_FUTEX=y -CONFIG_EPOLL=y -CONFIG_AIO=y +# CONFIG_ELF_CORE is not set +CONFIG_CORE_SMALL=y +CONFIG_NET_SMALL=y +# CONFIG_FUTEX is not set +# CONFIG_EPOLL is not set +# CONFIG_AIO is not set CONFIG_XATTR=y CONFIG_FILE_LOCKING=y -CONFIG_DIRECTIO=y +# CONFIG_DIRECTIO is not set +# CONFIG_UID16 is not set # CONFIG_MEASURE_INLINES is not set CONFIG_CC_OPTIMIZE_FOR_SIZE=y -CONFIG_SHMEM=y +# CONFIG_SHMEM is not set CONFIG_CC_ALIGN_FUNCTIONS=0 CONFIG_CC_ALIGN_LABELS=0 CONFIG_CC_ALIGN_LOOPS=0 CONFIG_CC_ALIGN_JUMPS=0 # CONFIG_TINY_CFLAGS is not set -CONFIG_MAX_SWAPFILES_SHIFT=5 -CONFIG_NR_LDISCS=16 -CONFIG_MAX_USER_RT_PRIO=100 -# CONFIG_TINY_SHMEM is not set +CONFIG_MAX_SWAPFILES_SHIFT=0 +CONFIG_NR_LDISCS=2 +CONFIG_MAX_USER_RT_PRIO=5 +CONFIG_TINY_SHMEM=y # # Loadable module support @@ -435,6 +434,7 @@ CONFIG_SERIO=y CONFIG_SERIO_SERPORT=y # CONFIG_SERIO_CT82C710 is not set +# CONFIG_SERIO_LIBPS2 is not set # CONFIG_SERIO_RAW is not set # @@ -912,7 +912,7 @@ # CONFIG_DEBUG_SPINLOCK is not set # CONFIG_DEBUG_KOBJECT is not set CONFIG_BUG=y -CONFIG_DEBUG_BUGVERBOSE=y +# CONFIG_DEBUG_BUGVERBOSE is not set CONFIG_DEBUG_INFO=y # CONFIG_DEBUG_FS is not set CONFIG_FRAME_POINTER=y
Series file
Please note that I didn't try all the patches. I only integrated and used 37 of the broken-out patches. You can see this from the series file I used, which follows:
#################################### # start of linux tiny patches # key: # ## = patches Tim commented out # # = patches commented out in original 2.6.11-tiny broken out series file ##tiny/tiny-extraversion.patch # # stuff already in -mm # ##tiny/kgdb-ga.patch ##tiny/kgdboe-netpoll.patch tiny/use-funit-at-a-time.patch ##tiny/devlist.patch # # tiny stuff # # core tiny/core-small.patch tiny/pid-max.patch tiny/user-hash.patch tiny/futex-queues.patch tiny/tvec_bases.patch tiny/con_buf.patch # net tiny/config-net-small.patch tiny/cache_defer_hash.patch tiny/unix_socket_table.patch tiny/inet_protos.patch tiny/flow-cache-small.patch # # Unconditional bits # ##tiny/small-8023.patch ##tiny/tg3-oops.patch tiny/namei-inlines.patch tiny/buffer-inlines.patch tiny/fs_open-inlines.patch tiny/inode-inlines.patch tiny/ext2namei-inlines.patch tiny/fslocks-inline.patch ##tiny/06-crypto-sleep.patch # # build options # tiny/tiny-cflags.patch # # measurement bits # tiny/kmalloc-accounting.patch tiny/audit-bootmem.patch tiny/bloat-o-meter.patch tiny/deprecate-inline.patch tiny/func-size.patch # # trimmed debugging support # ##tiny/no-doublefault.patch tiny/kill-printk.patch ##tiny/tiny-panic.patch tiny/nobug.patch ##tiny/nopanic.patch tiny/no-elf-core.patch tiny/no-kcore.patch # # trimmed code # # TRB - 5/19 - haven't looked at these yet ##tiny/tiny-crc.patch ##tiny/threadinfo-ool.patch ##tiny/semaphore-inline.patch ##tiny/slob.patch ##tiny/slob-accounting.patch ##tiny/mempool-shrink.patch #tiny/sysfs-backingstore.patch ##tiny/no-translations.patch #tiny/tinyvt.patch # # trimmed apis # # TRB - 5/19 - haven't looked at ## ones yet ##tiny/ptrace.patch ##tiny/remove-vm86.patch ##tiny/sysenter.patch tiny/no-aio.patch tiny/no-xattr.patch tiny/fslock.patch tiny/direct-io-core.patch tiny/uid16.patch #tiny/posix-timers.patch ##tiny/ethtool.patch ##tiny/inetpeer.patch ##tiny/net-filter.patch ##tiny/dev_mcast.patch #tiny/rtnetlink.patch ##tiny/igmp.patch ##tiny/binfmt-script.patch ##tiny/elf-no-aout.patch # # trimmed constants # tiny/max-swapfiles.patch tiny/ldiscs.patch tiny/max_user_rt_prio.patch ##tiny/change-hz.patch # # trimmed hw support # # TRB - 5/19 - haven't looked at ## ones yet #tiny/small-nomodedb.patch ##tiny/ide-hwif.patch ##tiny/sbf.patch ##tiny/serial-pci.patch ##tiny/dmi_blacklist.patch ##tiny/pci-quirks.patch ##tiny/tsc.patch ##tiny/cpu-support.patch ##tiny/mtrr.patch # # inflate cleanups # # TRB - 5/19 - haven't looked at ## ones yet ##tiny/inflate-noinline.patch ##tiny/inflate-formatting.patch ##tiny/inflate-legacy.patch ##tiny/inflate-input.patch ##tiny/inflate-iostate.patch ##tiny/inflate-huft.patch ##tiny/inflate-crc.patch ##tiny/inflate-killglobals.patch ##tiny/inflate-initramfs.patch ##tiny/inflate-initrd.patch ##tiny/inflate-i386.patch ##tiny/inflate-arm.patch ##tiny/inflate-x86_64.patch # # unsorted # # TRB - 5/19 - haven't looked at ## ones yet ##tiny/inflate-roll-crc.patch ##tiny/netpoll-timeout.patch ##tiny/movsl-mask.patch #tiny/rolled-md4.patch #tiny/kill-ext3-md4.patch # end of linux tiny patches ####################################