Difference between revisions of "Busybox replacement"

From eLinux.org
Jump to: navigation, search
(Prioritization: add list of files for toolbox complement)
 
(12 intermediate revisions by 3 users not shown)
Line 1: Line 1:
This page has information about the commands and features desired for a busybox replacement program.
+
== Toybox ==
 +
The leading contender for a multi-function program that could be
 +
used instead of busybox is toybox.  
  
See [[Busybox replacement project]] for the project proposal for this work.
+
The Toybox material previously on this page
 
+
has been moved to http://landley.net/toybox/roadmap.html
== Command List ==
 
 
 
We have several potential use cases for a busybox replacement, and are using those to determine a consensus on which commands to target for a 1.0 release of an unencumbered busybox replacement package.
 
 
 
Our current candidate list combines the commands toybox already implements, the development environment command list, the toolbox standard commands, the Sony configuration of busybox, and the selected subset of the SUSv4 standard.
 
 
 
Thus the first release of a busybox replacement should include the following commands:
 
 
 
  at awk basename bc bunzip2 bzcat bzip2 cal cat catv chgrp chmod
 
  chown chroot chvt cksum cmp comm count cp cpio csplit cut date dd df
 
  diff dirname dmesg dnsdomainname du echo egrep env expand expr
 
  false fgrep file find fold freeramdisk ftpd ftpget ftpput fuser
 
  getconf getty grep gunzip gzip halt hd head help hostname id
 
  ifconfig iftop init insmod install ioctl ionice join kill
 
  killall klogd less link ln logger login logname losetup ls lsmod
 
  lsof man md5sum mdev mkdir mkfifo mknod mkswap mktemp modprobe
 
  more mount mountpoint mv nc netcat netstat newgrp newfs_msdos
 
  nice nl nohup notify od oneit paste patch pathchk pgrep pidof
 
  ping ping6 pivot_root pkill poweroff printenv printf ps pwd
 
  readlink reboot renice rm rmdir rmmod route sed seq setsid
 
  sfdisk sh sha1sum sleep sort split start stat stop stty swapoff
 
  swapon switch_root sync syslogd tabs tac tail tar taskset tee
 
  telnet telnetd test time top touch tput tr true truncate tty
 
  umount uname unexpand uniq unlink unshare uptime usleep uudecode
 
  uuencode vconfig vi vmstat wc wget which who whoami xargs yes
 
  zcat
 
 
 
Plus additional to-be-determined shell functionality.
 
 
 
== Toybox currently implements ==
 
Here's a link to the toybox project: http://www.landley.net/toybox/about.html
 
which is a prime candidate for replacing busybox.
 
 
 
The triaged toybox (2 clause BSD license) is currently implementing usable versions of:
 
 
 
  bzcat cal cat catv chroot chvt cksum count cp df dirname dmesg
 
  echo false help link mkswap nc netcat nice nohup oneit patch pwd
 
  rmdir seq setsid sha1sum sleep sort sync tee true truncate tty
 
  uname unlink unshare wc which yes
 
 
 
And contains partial implementations of:
 
  mdev sh
 
 
 
See [http://landley.net/code/toybox/todo.txt the Toybox todo list] for the most up to date status.
 
 
 
== Use case: provide a self-hosting development environment ==
 
 
 
The following commands are enough to build the Aboriginal Linux development environment, boot it to a shell prompt, and build Linux From Scratch 6.8 under it.  (Aboriginal Linux currently uses BusyBox for this, thus provides a corresponding test environment for toybox.)
 
 
 
This use case includes running init scripts and other shell scripts, running configure, make, and install in each package, and providing basic command line facilities such as a text editor.  (It does not include a compiler toolchain or C library, those are outside the scope of this project.)
 
 
 
=== Already in toybox ===
 
  bzcat cat cp dirname echo patch rmdir sha1sum sleep sort sync
 
  true uname wc which yes env
 
 
 
=== Additional commands for development environment ===
 
  zcat awk basename bzip2 chmod chown cmp cut date dd diff
 
  egrep expr find grep gzip head hostname id install ln ls
 
  mkdir mktemp mv od readlink rm sed sh tail tar touch tr uniq
 
  wget whoami xargs chgrp comm gunzip less logname man split
 
  tee test time bunzip2 chgrp chroot comm cpio dmesg
 
  dnsdomainname ftpd ftpget ftpput gunzip ifconfig init less
 
  logname losetup man mdev mount mountpoint nc pgrep pkill
 
  pwd route split stat switch_root tac tee test time umount vi
 
 
 
Note: Aboriginal Linux installs bash 2.05b as #!/bin/sh and its scripts require bash extensions not present in busybox ash.
 
 
 
== Use case: replacing Toolbox commands ==
 
 
 
=== Standard commands ===
 
Toolbox in Android (what version?) implements the following standard commands:
 
 
 
cat chmod chown cmp date dd dmesg df hd
 
id ifconfig iftop insmod ioctl ionice
 
kill ls netstat ln lsmod lsof
 
mkdir mount mv newfs_msdos notify
 
ps printenv reboot renice rm rmdir rmmod route
 
sleep start stop sync top umount uptime vmstat
 
 
 
=== Android-specific commands ===
 
Toolbox also provides the following nonstandard commands, which are unique to Android (or at least do not appear in Ubuntu or
 
SUSv4):
 
 
 
  alarm schedtop getprop log setprop watchprops notify wipe
 
  getevent sendevent nandread smd setconsole r ioctl
 
 
 
== Busybox commands ==
 
=== default 1.10 build ===
 
A default build of Busybox 1.10 has the following commands:
 
 
 
<pre>
 
[, [[, addgroup, adduser, adjtimex, ar, arp, arping, ash,
 
awk, basename, bunzip2, bzcat, bzip2, cal, cat, catv,
 
chattr, chgrp, chmod, chown, chpasswd, chpst, chroot,
 
chrt, chvt, cksum, clear, cmp, comm, cp, cpio, crond,
 
crontab, cryptpw, cut, date, dc, dd, deallocvt, delgroup,
 
deluser, df, dhcprelay, diff, dirname, dmesg, dos2unix,
 
du, dumpkmap, dumpleases, echo, ed, egrep, eject, env,
 
envdir, envuidgid, expand, expr, false, fbset, fdflush,
 
fdformat, fdisk, fgrep, find, fold, free, freeramdisk,
 
fsck, fsck.minix, fuser, getopt, getty, grep, gunzip,
 
gzip, halt, hdparm, head, hexdump, hostid, hostname, hwclock,
 
id, ifconfig, ifdown, ifup, inetd, init, insmod, install,
 
ip, ipaddr, ipcalc, ipcrm, ipcs, iplink, iproute, iprule,
 
iptunnel, kbd_mode, kill, killall, killall5, klogd, last,
 
length, less, linux32, linux64, linuxrc, ln, loadfont,
 
loadkmap, logger, login, logname, logread, losetup, lpd,
 
lpq, lpr, ls, lsattr, lsmod, lzmacat, makedevs, md5sum,
 
mdev, mesg, microcom, mkdir, mkfifo, mkfs.minix, mknod,
 
mkswap, mktemp, modprobe, more, mount, mountpoint, mt,
 
mv, nameif, netstat, nice, nmeter, nohup, nslookup, od,
 
openvt, passwd, patch, pgrep, pidof, ping, pipe_progress,
 
pivot_root, pkill, poweroff, printenv, printf, ps, pscan,
 
pwd, raidautorun, rdate, readlink, readprofile, realpath,
 
reboot, renice, reset, resize, rm, rmdir, rmmod, route,
 
rpm, rpm2cpio, run-parts, runlevel, runsv, runsvdir, rx,
 
script, sed, sendmail, seq, setarch, setconsole, setkeycodes,
 
setlogcons, setsid, setuidgid, sh, sha1sum, sleep, softlimit,
 
sort, split, start-stop-daemon, stat, strings, stty, su,
 
sulogin, sum, sv, svlogd, swapoff, swapon, switch_root,
 
sync, sysctl, syslogd, tac, tail, tar, taskset, tee, telnetd,
 
test, time, top, touch, tr, true, tty, ttysize, udhcpc,
 
udhcpd, umount, uname, uncompress, unexpand, uniq, unix2dos,
 
unlzma, unzip, uptime, usleep, uudecode, uuencode, vi,
 
vlock, watch, watchdog, wc, wget, which, who, whoami,
 
xargs, yes, zcat
 
</pre>
 
  
 
=== Sony CE Linux (internal distribution) 1.13.4 build ===
 
=== Sony CE Linux (internal distribution) 1.13.4 build ===
Line 146: Line 21:
 
         umount, uname, uniq, usleep, vconfig, wc, xargs, zcat
 
         umount, uname, uniq, usleep, vconfig, wc, xargs, zcat
 
</pre>
 
</pre>
 
== POSIX-2008/SUSv4 ==
 
 
The best standards are the kind that describe reality, rather than attempting to impose a new one.  (I.E. a good standard should document, not legislate.)
 
 
The kind of standards which describe existing reality tend to be approved by more than one standards body, such ANSI and ISO both approving C.  That's why the IEEE POSIX committee's 2008 standard, the Single Unix Specification version 4, and the Open Group Base Specification edition 7 are all the same standard from three sources.
 
 
The [http://opengroup.org/onlinepubs/9699919799/idx/utilities.html
 
"utilities" section] these standards is devoted to the unix command line, and are the best such standard for our purposes.  (BusyBox was implemented with regard to SUSv3, an earlier version of this standard.)
 
 
=== Problems with the standard ===
 
 
Unfortunately, these standards describe a subset of reality, lacking any mention of commands such as init, login, or mount required to actually boot a system. It provides ipcrm and ipcs, but not ipcmk, so you can use System V IPC resources but not create them.
 
 
These standards also contain a large number of commands that are inappropriate for toybox to implement in its 1.0 release.  (Perhaps some of these could be reintroduced in later releases, but not now.)
 
 
Starting with the full "utilities" list, we first remove generally obsolete commands (compess ed ex pr uncompress uccp uustat uux), commands for the pre-CVS "SCCS" source control system (admin delta get prs rmdel sact sccs unget val what), fortran support (asa fort77), and batch processing support (batch qalter qdel qhold qmove qmsg qrerun qrls qselect qsig qstat qsub).
 
 
Some commands are for a compiler toolchain (ar c99 cflow ctags cxref getcat iconv lex m4 make nm strings strip tsort yacc), which is outside of toybox's mandate and should be supplied externally.  (Again, some of these may be revisited later, but not for toybox 1.0.)
 
 
Some commands are part of a command shell, and cannot be implemented as separate executables (alias bg cd command fc fg getopts hash jobs kill read type ulimit umask unalias wait).  These may be revisited as part of a built-in toybox shell, but are not exported into $PATH via symlinks.  (If you fork a child process and have it "cd" then exit, you've accomplished nothing.)
 
 
A few other commands are judgement calls, providing command-line internationalization support (iconv locale localedef), System V inter-process communication (ipcrm ipcs), and cross-tty communication from the minicomputer days (talk mesg write).  The "pax" utility was supplanted by tar, "mailx" is a command line email client, and "lp" submits files for printing to... what exactly?  (cups?)  The standard defines crontab but not crond.
 
 
Removing all of that leaves the following commands, which toybox should implement:
 
 
  at awk basename bc cal cat chgrp chmod chown cksum cmp comm cp
 
  csplit cut date dd df diff dirname du echo env expand expr false file find
 
  fold fuser getconf grep head id join kill link ln logger logname ls man
 
  mkdir mkfifo more mv newgrp nice nl nohup od paste patch pathchk printf ps
 
  pwd renice rm rmdir sed sh sleep sort split stty tabs tail tee test time
 
  touch tput tr true tty uname unexpand uniq unlink uudecode uuencode vi wc
 
  who xargs zcat
 
  
 
=== Prioritization ===
 
=== Prioritization ===
Line 201: Line 43:
 
  nohup od paste patch pathchk renice split stty tabs
 
  nohup od paste patch pathchk renice split stty tabs
 
  tput tr tty unexpand unlink uudecode uuencode
 
  tput tr tty unexpand unlink uudecode uuencode
 
  
 
Must-have unimplemented: The following list has Tim's must-have commands,
 
Must-have unimplemented: The following list has Tim's must-have commands,
which are currently not implemented in toybox:
+
which are currently (as of Jan, 2012) not implemented in toybox:
 
  chgrp chmod chown date dd
 
  chgrp chmod chown date dd
 
  expr find grep head id kill ln
 
  expr find grep head id kill ln
Line 210: Line 51:
 
  tail test touch uniq xargs
 
  tail test touch uniq xargs
  
 +
==== prioritized, must-have, unimplemented list ====
 
Prioritized top 13:
 
Prioritized top 13:
 
  1. ls
 
  1. ls
Line 230: Line 72:
 
IMHO - ls needs to support -ladR at first
 
IMHO - ls needs to support -ladR at first
  
 +
==== toolbox complement ====
 
If we omit toolbox supported commands from the must-have, unimplemented list, we get:
 
If we omit toolbox supported commands from the must-have, unimplemented list, we get:
  chgrp expr false find grep head more nice
+
  chgrp expr false find grep head more
  sh sort tail test touch true uname xargs
+
  nice tail test touch true uname xargs
 +
 
 +
I would add tab completion and globbing for toysh to the list.
  
 
This would be for a toolbox complement.
 
This would be for a toolbox complement.
 +
 +
== Other multi-tool programs ==
 +
=== Beastiebox currently implements ===
 +
Here’s a link to the Beastiebox project: http://beastiebox.sourceforge.net/
 +
which has been proven to be capable to replace busybox, in general.
 +
It mostly consists of BSD sources and contains, according to the homepage:
 +
 +
  [ cat chmod cp date df disklabel dmesg ex fdisk fsck fsck_ffs
 +
  getty halt hostname ifconfig init kill less ln login ls [https://www.mirbsd.org/mksh.htm mksh] more
 +
  mount mount_ffs mv pfctl ping poweroff ps reboot rm route sed sh
 +
  stty sysctl tar test traceroute umount vi wiconfig
 +
 +
Of course, every command is optional; for example, pfctl is currently BSD specific, and fsck_ffs/mount_ffs do not make much sense on Linux either, so these would be disabled in a Linux build of Beastiebox.
 +
 +
On the TODO side, the Linux specific commands would need to be implemented
 +
or taken from one of the others (Toybox, Android Toolbox, etc).
 +
 +
It does contain a complete shell (mksh, MirBSD Korn Shell) under a BSD-ish
 +
licence, which can be used as /bin/sh (e.g. on Debian) and is the default
 +
/system/bin/sh on Android ICS and later, even. This is not much bigger than
 +
the sh included with Busybox, Beastiebox (can be disabled in favour of mksh)
 +
and others, but has functions such as command line editing, UTF-8 support,
 +
Tab completion, Korn Shell scripting features ('''[[''', arrays, etc). and is
 +
actively developed on its own.
  
 
== Random Notes ==
 
== Random Notes ==
Line 240: Line 109:
 
Rob wrote:
 
Rob wrote:
  
One nice thing about busybox/toybox/toolbox is you can install multiple
+
One nice thing about busybox/toolbox is you can install multiple
 
implementations side by side, and have what symlinks you create (or what
 
implementations side by side, and have what symlinks you create (or what
 
comes first in the $PATH) determine who is implementing what.
 
comes first in the $PATH) determine who is implementing what.

Latest revision as of 12:23, 1 March 2012

Toybox

The leading contender for a multi-function program that could be used instead of busybox is toybox.

The Toybox material previously on this page has been moved to http://landley.net/toybox/roadmap.html

Sony CE Linux (internal distribution) 1.13.4 build

This is from a version of busybox used in lots of Sony products: Currently defined functions:

        [, [[, ash, basename, bunzip2, bzcat, cat, chgrp, chmod, chown,
        chroot, cmp, cp, cpio, cut, date, dd, dirname, dmesg, echo, egrep,
        env, expr, false, fgrep, freeramdisk, ftpget, ftpput, getty, grep,
        gunzip, gzip, halt, ifconfig, insmod, kill, killall, klogd, ln,
        logger, login, ls, lsmod, md5sum, mkdir, mkfifo, mknod, mktemp,
        modprobe, mount, mv, pidof, ping, ping6, pivot_root, poweroff,
        pwd, readlink, reboot, rm, rmdir, rmmod, route, sed, sfdisk, sh,
        sha1sum, sleep, sort, stty, swapoff, swapon, switch_root, sync,
        syslogd, tail, tar, taskset, telnet, telnetd, test, touch, true,
        umount, uname, uniq, usleep, vconfig, wc, xargs, zcat

Prioritization

The following is Tim's prioritization of the above list, based on his own experience with embedded systems. I will divide the above list into 3 groups: must-have, nice-to-have, not-that-important.

Must-have:

cat chgrp chmod chown cp date df
echo env expr false find grep head id kill ln
ls mkdir more mv nice ps pwd rm rmdir sh sleep
sort tail test touch true uname uniq wc xargs

(must have, not in the above list: chroot, dmesg, sync)

Nice-to-have:

basename cut dd diff dirname du mkfifo printf sed tee time vi who zcat

Not-that-important:

at awk bc cal cksum cmp comm csplit expand file
fold fuser getconf link logger logname man newgrp nl
nohup od paste patch pathchk renice split stty tabs
tput tr tty unexpand unlink uudecode uuencode

Must-have unimplemented: The following list has Tim's must-have commands, which are currently (as of Jan, 2012) not implemented in toybox:

chgrp chmod chown date dd
expr find grep head id kill ln
ls mkdir more mv nice ps rm
tail test touch uniq xargs

prioritized, must-have, unimplemented list

Prioritized top 13:

1. ls
2. mkdir
3. rm
4. ln
5. mv
6. ps
7. kill
8. chmod
9. chown
10. date
11. grep
12. find
13. xargs

Second prioritized group:

chgrp id nice expr head tail test touch uniq dd more

IMHO - ls needs to support -ladR at first

toolbox complement

If we omit toolbox supported commands from the must-have, unimplemented list, we get:

chgrp expr false find grep head more
nice tail test touch true uname xargs

I would add tab completion and globbing for toysh to the list.

This would be for a toolbox complement.

Other multi-tool programs

Beastiebox currently implements

Here’s a link to the Beastiebox project: http://beastiebox.sourceforge.net/ which has been proven to be capable to replace busybox, in general. It mostly consists of BSD sources and contains, according to the homepage:

 [ cat chmod cp date df disklabel dmesg ex fdisk fsck fsck_ffs
 getty halt hostname ifconfig init kill less ln login ls mksh more
 mount mount_ffs mv pfctl ping poweroff ps reboot rm route sed sh
 stty sysctl tar test traceroute umount vi wiconfig

Of course, every command is optional; for example, pfctl is currently BSD specific, and fsck_ffs/mount_ffs do not make much sense on Linux either, so these would be disabled in a Linux build of Beastiebox.

On the TODO side, the Linux specific commands would need to be implemented or taken from one of the others (Toybox, Android Toolbox, etc).

It does contain a complete shell (mksh, MirBSD Korn Shell) under a BSD-ish licence, which can be used as /bin/sh (e.g. on Debian) and is the default /system/bin/sh on Android ICS and later, even. This is not much bigger than the sh included with Busybox, Beastiebox (can be disabled in favour of mksh) and others, but has functions such as command line editing, UTF-8 support, Tab completion, Korn Shell scripting features ([[, arrays, etc). and is actively developed on its own.

Random Notes

Can implement incrementally

Rob wrote:

One nice thing about busybox/toolbox is you can install multiple implementations side by side, and have what symlinks you create (or what comes first in the $PATH) determine who is implementing what.

This allows gradual transitions. Each release, we replace a couple more commands from the old one, until the old one finally isn't being used for anything anymore and we can uninstall it...