Subject: Re: replacement for bc(1), dc(1), diff(1), and diff(3)
To: None <tech-userlevel@NetBSD.org>
From: Phil Nelson <phil@cs.wwu.edu>
List: tech-userlevel
Date: 03/26/2005 21:33:09
=2D----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Note:  It appears that my first copy of this e-mail was trashed by a spam=20
filter.  I've edited it a bit to see if I can get it by the spam filter. =20
I've removed most of the bc output.  --Phil


On Friday 25 March 2005 08:54, Igor Sobrado wrote:
> My suggestion is replacing bc(1), dc(1), and the diffutils commands
> diff(1) and diff3(1) with the OpenBSD ones. =A0sdiff(1) has no
> replacement yet, though. =A0I do not know if this command is "required",
> if it can be easily replaced, or if it must stay as is.

I'll speak to bc(1) and dc(1).   (In case you don't get to the end of this
e-mail, I don't think OpenBSD bc and dc are good replacements for GNU bc and
dc.)

> It is not a
> licensing issue; in my humble opinion, there are good technical reasons

Good.  I like technical discussions.

> (i.e., better end-user documentation, clean and well written source code,
> and better integration with the operating system source tree) for
> importing these commands from OpenBSD:
>
> =A0 - Man pages are better written in the OpenBSD versions of those
> =A0 =A0 commands. =A0NetBSD currently only offers a list of options
> =A0 =A0 available for these commands:
>...
> =A0 =A0 These man pages are much better and describe the behaviour of the
> =A0 =A0 commands, instead of providing a list of options without even a
> =A0 =A0 brief description of the commands.
>...
> =A0 - Source code is well written and can be easily integrated in the
> =A0 =A0 NetBSD source tree (e.g., src/gnu/dist/diffutils/diffutils2netbsd
> =A0 =A0 is not required).

Since I haven't investigated OpenBSD's bc(1) and dc(1) for a while, I thoug=
ht
I'd check them out again.  So I checked out the current versions from an
anoncvs site and got them running on NetBSD-2.0 (i386, 3.2Ghz machine).

Let me look at the points one by one:

a) better end-user documentation:

=2D - ---->~/OpenBSD/src/usr.bin/bc
nooksack[247]$ uname -a
NetBSD nooksack 2.0 NetBSD 2.0 (NOOKSACK.MP) #1: Mon Dec 20 17:16:15 PST 20=
04
phil@nooksack:/home/phil/nb-2.0/src/sys/arch/i386/compile/NOOKSACK.MP i386

=2D - ---->~/OpenBSD/src/usr.bin/bc
nooksack[248]$ wc -l bc.cat1 /usr/share/man/cat1/bc.0
     237 bc.cat1
     799 /usr/share/man/cat1/bc.0
    1036 total

=2D - ---->~/OpenBSD/src/usr.bin/bc
nooksack[249]$ wc -l ../dc/dc.cat1 /usr/share/man/cat1/dc.0
     333 ../dc/dc.cat1
     294 /usr/share/man/cat1/dc.0
     627 total

Well, for the bc manual, the current (GNU) one has 799 lines versus 237 for
OpenBSD.  The GNU version is a much more comprehensive description of the
POSIX bc language with the GNU extensions.   All command line switches are
documented in full although they appear a little further down in the manual
than the OpenBSD version.  In fact, most current bug reports for GNU bc are
features documented in the manual page.

=46or the dc manual, the OpenBSD version has a few more lines than the GNU
version.  Again, all command line switches are documented.

b) well written source code

This is a hard thing to quantify.  What specifically about the GNU version =
is
not "well written source code"?

If you are talking about the methodology .... in that GNU bc is a complete
product not requiring GNU dc as a backend, that was a design decision.  I
wouldn't necessarily call one version superior to the other.  (OpenBSD bc
uses the original design of having bc being a preprocessor for dc.)

c) better integration with the operating system source tree

I don't see how it is "better".   With OpenBSD bc, dc you have the following
trees to compile bc and dc.
  src/usr.bin/bc
  src/usr.bin/dc
  src/crypto/dist/openssl
  src/lib/libcrypto  (reachover makefiles)

=46or GNU bc:
   src/gnu/dist/bc
   src/gnu/usr.bin/bc  (reachover makefiles)
   src/gnu/usr.bin/dc  (reachover makefiles)

d) Although not mentioned in the original post, for me the real technical
issue is how does the product perform in use. The following are time tests =
on
a variety of operations.  It is targetting the most expensive operations in
bc(1).   First, it checks out the math library and then looks at things like
multiplication, division, raising and square root.   As you will see, GNU bc
outperforms OpenBSD in many of the tests and some cases, OpenBSD can't even
do what GNU bc does.   In 2 cases below, I killed OpenBSD dc after it ran f=
or
10 or more minutes of CPU time. (time reports ' Command terminated
abnormally.' in these cases.)

Each of the tests listed below are quite simple.  For example, the "exp.b"
test is:
     for (a=3D0; a<180; a+=3D.4) x=3De(a)
     x
     quit

The one for fact.b is:
    define f (x) {

      if (x<=3D1) return(1)
      return (f(x-1)*x)
    }

    for (a=3D1; a<600; a++) b=3Df(a)
    "
    "
    "b=3D";b
    quit

If anyone wants the full source code for these tests, just let me know and
I'll send them.

The quick analysis:

   GNU bc is faster on all tested operations except raise and square root.
There are 4 tests in which I killed OpenBSD dc because of excessive time use
(10 or more cpu minutes) or OpenBSD dc reported an error.  Here is the test
data taken on NetBSD-2.0/i386, 3.2 Ghz processor.

Timing exp.b with /usr/bin/bc
99836408603283573660347145562829683495827909199408566065153345558783\
9427595471.89114392327665123852
        1.38 real         1.38 user         0.00 sys
Timing exp.b with openbc
time: Command terminated abnormally.
      713.47 real       713.15 user         0.01 sys
Timing ln.b with /usr/bin/bc
87.336544750553108986571247303730247577513016929392162019205681
        0.83 real         0.83 user         0.00 sys
Timing ln.b with openbc
87.336544750553108986571247303730247577513016929392162019205681
        1.62 real         1.61 user         0.00 sys
Timing sine.b with /usr/bin/bc
=2D - -.00318530179313823899
=2E11677468864383377454
        0.47 real         0.47 user         0.00 sys
Timing sine.b with openbc
dc: allocation failure 306a041: Cannot allocate memory
      153.50 real       152.37 user         1.03 sys
Timing atan.b with /usr/bin/bc
1.56979432312222174523
1.10514068836449430089
        0.39 real         0.39 user         0.00 sys
Timing atan.b with openbc
time: Command terminated abnormally.
     1208.93 real      1207.27 user         1.29 sys
Timing jn.b with /usr/bin/bc
=2D - -.03329855487630566800748309439984550659781158351287
=2D - -.00648115168876276895858580295786734310987910950934
=2E04431532933245365649911092813736180550142494873120
=2D - -.04084340882273173036919689349257616993289838931589
=2D - -.02243471758440192810060521803721671199059663161876
=2D - -.01449122706478569886133684583938086724345975237695
        2.60 real         2.57 user         0.00 sys
Timing jn.b with openbc
=2D - -.03329855487630566800748309439984550659781158351287
=2D - -.00648115168876276895858580295786734310987910950934
=2E04431532933245365649911092813736180550142494873120
=2D - -.04084340882273173036919689349257616993289838931589
=2D - -.02243471758440192810060521803721671199059663161876
=2D - -.01449122706478569886133684583938086724345975237695
        6.49 real         6.45 user         0.00 sys
Timing mul.b with /usr/bin/bc
989901
99990000000000000000000000000000000000000000000000000000000000009899\
01
       16.57 real        16.55 user         0.00 sys
Timing mul.b with openbc
989901
999900000000000000000000000000000000000000000000000000000000000098990\
1
       65.31 real        65.28 user         0.01 sys
Timing div.b with /usr/bin/bc
=2E00673400673400673400
        0.94 real         0.94 user         0.00 sys
Timing div.b with openbc
=2E00673400673400673400
        7.02 real         7.00 user         0.00 sys
Timing raise.b with /usr/bin/bc
5357 ...
3969 ...
1299 ...
        3.01 real         2.99 user         0.01 sys
Timing raise.b with openbc
5357 ...
3969 ...
1299 ...
        0.75 real         0.73 user         0.00 sys
Timing sqrt.b with /usr/bin/bc
22.33830
22.3383079036
22.3383079036886766608314514
22.3383079036886766608314514388942891544207
        0.32 real         0.31 user         0.00 sys
Timing sqrt.b with openbc
22.33830
22.3383079036
22.3383079036886766608314514
22.3383079036886766608314514388942891544207
        0.15 real         0.14 user         0.00 sys

Timing fact.b with /usr/bin/bc

b=3D2109 ...  (note factorial of 600)
        5.33 real         5.32 user         0.00 sys
Timing fact.b with openbc
dc: recursion too deep
        0.13 real         0.11 user         0.01 sys

My conclusion is that OpenBSD bc and dc are not a good replacements for GNU
versions.   This is based on runtime performance.   We haven't even begun to
talk about POSIX compliance.   The above tests show that OpenBSD doesn't
comply with the POSIX output requirement that the \ in continuation numbers
is also to be counted in the line length.  So a line of a maximum of 70
characters, the default output line size, should have 69 digits and one \.
OpenBSD bc has 70 digits and the \.

=2D --Phil

=2D --
Phil Nelson                       NetBSD: http://www.netbsd.org
e-mail: phil@cs.wwu.edu           Coda: http://www.coda.cs.cmu.edu
http://www.cs.wwu.edu/nelson
=2D----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.3 (NetBSD)

iD8DBQFCRkWZzbodwsP3RI0RAkiwAKCtV5361qJAGXhlmkNRuMg8+ujcLwCggO+k
wPFX/qNs4h4upHefCccS+m0=3D
=3DshUp
=2D----END PGP SIGNATURE-----