Re: clisp vs. SBCL

[Chavdar Ivanov <ci4ic4%gmail.com@localhost>]

On Sun, 8 Aug 2021 at 19:21, Yorick Hardy <yorickhardy%gmail.com@localhost> wrote:
>
> Dear Chavdar,
>
> On 2021-08-08, Chavdar Ivanov wrote:
> > Hi,
> >
> > If one wants to use xMaxima for some maths, it is strongly advisable
> > to make sure it is using SBCL, as opposed to the default clisp. Here
> > are the results of the tests performed on the same machine, same
> > version of xMaxima, first with clisp, then with SBCL:
> > .....
> >
> > xMaxima test, clisp
> >
> > Error summary:
> > Error(s) found:
> >   /usr/pkg/share/maxima/5.44.0/tests/rtest_rules.mac problems:
> >     (11 13)
> >   /usr/pkg/share/maxima/5.44.0/tests/rtest9a.mac problems:
> >     (24 27 30 31 35 36 47 48 51 52 55 56 59 60 63 64 67 68 71 72)
> >   /usr/pkg/share/maxima/5.44.0/tests/rtest11.mac problems:
> >     (136 137)
> >   /usr/pkg/share/maxima/5.44.0/tests/rtest16.mac problem:
> >     (50)
> >   /usr/pkg/share/maxima/5.44.0/tests/rtest3.mac problems:
> >     (161 162 163 164 165 166 167 168 169 170)
> > 35 tests failed out of 11,711 total tests.
> > Real time: 2261.4172f0 sec.
> > Run time: 2258.7747f0 sec.
> > Space: 38127589208 Bytes
> > GC: 9020, GC time: 736.08234f0 sec.
> >
> > xMaxima test, SBCL:
> >
> > No unexpected errors found out of 11,711 tests.
> > Evaluation took:
> >   131.133 seconds of real time
> >   131.023201 seconds of total run time (127.417664 user, 3.605537 system)
> >   [ Run times consist of 4.800 seconds GC time, and 126.224 seconds
> > non-GC time. ]
> >   99.92% CPU
> >   21,598 forms interpreted
> >   15,783 lambdas converted
> >   339,898,359,702 processor cycles
> >   30,489,795,840 bytes consed
> >
> > ...
> >
> > That's about 17 times faster and with no unexpected errors...
> >
> >
> > --
> > ----
>
> Interesting. I thought a comparison between clisp and ecl might
> also be useful. So here are the numbers on my (much slower!) PC.
>
> Xmaxima, clisp:
>
> Error summary:
> Error(s) found:
>   /usr/pkg/share/maxima/5.44.0/tests/rtest_rules.mac problems:
>     (11 13)
>   /usr/pkg/share/maxima/5.44.0/tests/rtest9a.mac problems:
>     (24 27 30 31 35 36 47 48 51 52 55 56 59 60 63 64 67 68 71 72)
>   /usr/pkg/share/maxima/5.44.0/tests/rtest11.mac problems:
>     (136 137)
>   /usr/pkg/share/maxima/5.44.0/tests/rtest16.mac problem:
>     (50)
>   /usr/pkg/share/maxima/5.44.0/tests/rtest3.mac problems:
>     (161 162 163 164 165 166 167 168 169 170)
> 35 tests failed out of 11,711 total tests.
> Real time: 5683.2637f0 sec.
> Run time: 5575.9287f0 sec.
> Space: 37989447000 Bytes
> GC: 9113, GC time: 2085.8396f0 sec.
>
>
> Xmaxima, ecl (maxima+ecl is correctly passing tests which were
> expected to fail due to bugs):
>
> No unexpected errors found out of 11,711 tests.
> Tests that were expected to fail but passed:
>   /usr/pkg/share/maxima/5.44.0/tests/rtest1.mac problem:
>     (115)
>   /usr/pkg/share/maxima/5.44.0/tests/rtest6.mac problems:
>     (43 45)
>   /usr/pkg/share/maxima/5.44.0/tests/rtest14.mac problems:
>     (145 201 233 234 249 250 251 252 267 297 298 307 310 312 315 319)
>   /usr/pkg/share/maxima/5.44.0/tests/rtest16.mac problems:
>     (50 561)
>   /usr/pkg/share/maxima/5.44.0/tests/rtest8.mac problem:
>     (104)
> real time : 1809.874 secs
> run time  : 1912.675 secs
> gc count  : 4325 times
> consed    : 51671573456 bytes

I haven't tried this with ecl (I use it only as a bootstrap compiler
for SBCL, subsequently it rebuilds itself); I ran the test on the same
machine and got:

xMaxima test, ECL:

No unexpected errors found out of 11,711 tests.
Tests that were expected to fail but passed:
  /usr/pkg/share/maxima/5.44.0/tests/rtest1.mac problem:
    (115)
  /usr/pkg/share/maxima/5.44.0/tests/rtest6.mac problems:
    (43 45)
  /usr/pkg/share/maxima/5.44.0/tests/rtest14.mac problems:
    (145 201 233 234 249 250 251 252 267 297 298 307 310 312 315 319)
  /usr/pkg/share/maxima/5.44.0/tests/rtest16.mac problems:
    (50 561)
  /usr/pkg/share/maxima/5.44.0/tests/rtest8.mac problem:
    (104)
real time : 1030.576 secs
run time  : 1081.703 secs
gc count  : 3494 times
consed    : 51897350160 bytes

(Interesting that the test expects errors when using ecl, but in fact
all the results are as expected).

The considerable difference in speed is perhaps due to SBCL being a
compiler essentially; on top of that, recently threading was enabled -
only SBCL's *features* contain THREAD:

SBCL:
* *features*
(:X86-64 :GENCGC :64-BIT :ANSI-CL :BSD :COMMON-LISP :ELF :IEEE-FLOATING-POINT
 :LITTLE-ENDIAN :NETBSD :PACKAGE-LOCAL-NICKNAMES :SB-CORE-COMPRESSION :SB-LDB
 :SB-PACKAGE-LOCKS :SB-THREAD :SB-UNICODE :SBCL :UNIX)

ECL:
> *features*

(:WALKER :CDR-1 :CDR-5 :NETBSD :FORMATTER :CDR-7 :ECL-WEAK-HASH :LITTLE-ENDIAN
 :LONG-LONG :UINT64-T :UINT32-T :UINT16-T :LONG-FLOAT :UNICODE :DFFI
 :CLOS-STREAMS :CMU-FORMAT :UNIX :ECL-PDE :DLOPEN :CLOS :BOEHM-GC :ANSI-CL
 :COMMON-LISP :FLOATING-POINT-EXCEPTIONS :IEEE-FLOATING-POINT
 :PACKAGE-LOCAL-NICKNAMES :CDR-14 :PREFIXED-API :FFI :X86_64 :COMMON :ECL)

CLISP:
[1]> *features*
(:READLINE :REGEXP :SYSCALLS :I18N :LOOP :COMPILER :CLOS :MOP :CLISP
:ANSI-CL :COMMON-LISP :LISP=CL :INTERPRETER :SOCKETS :GENERIC-STREAMS
:LOGICAL-PATHNAMES :SCREEN :FFI :GETTEXT
 :UNICODE :BASE-CHAR=CHARACTER :WORD-SIZE=64 :PC386 :UNIX)


>
> --
> Kind regards,
>
> Yorick Hardy

Best regards,

Chavdar Ivanov

-- 
----