At Sun, 7 Jun 2020 12:46:51 +0200, Johnny Billquist <bqt%update.uu.se@localhost> wrote: Subject: Re: Postfix and local mail delivery - still relevant in 2020? > > On 2020-06-07 07:32, Greg A. Woods wrote: > > > > At Sun, 7 Jun 2020 01:53:34 +0200, Johnny Billquist <bqt%update.uu.se@localhost> wrote: > > Is the slow startup you observe happening with just the default > > configuration (for local delivery)? > > It's happening with any kind of configuration. > > Basically, anything written in C++ is almost a lost cause even before > starting. Do you mean you're under the impression Postifix is written in C++? Perhaps you should look at the Postfix code -- it's entirely pure C. > postfix alone takes close to a minute to get going at boot time on my > VAXen... > And that is just counting before rc continues to the next item. I > haven't checked if postfix are then still being busy actually getting > things sorted in the background... That's very interesting. The slowest machine I have running at the moment is a little old Soekris box, with a FLASH disk as root: # time /etc/rc.d/postfix restart postfix/postfix-script: stopping the Postfix mail system postfix/postfix-script: waiting for the Postfix mail system to terminate postfix/postfix-script: starting the Postfix mail system 3.31s real 1.21s user 0.76s system Now that's a little unfair for two reasons. First off the programs were just running so the good sized parts of their text segments were already in memory. So I stopped Postfix, flushed it from RAM by reading a bunch of big files from an attached cache disk and started it again: # time /etc/rc.d/postfix start postfix/postfix-script: starting the Postfix mail system 3.29s real 1.08s user 0.77s system So, that's still pretty fast. Now the second unfairness is this: # file /usr/libexec/postfix/master /usr/libexec/postfix/master: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), statically linked, for NetBSD 8.99.32, stripped I.e. my whole build is static-linked. This is perhaps the big problem since the two processes that have to start are like this on stock builds: $ ldd /usr/libexec/postfix/master /usr/libexec/postfix/master: -lsaslc.0 => /usr/lib/libsaslc.so.0 -lcrypto.14 => /usr/lib/libcrypto.so.14 -lcrypt.1 => /lib/libcrypt.so.1 -lc.12 => /usr/lib/libc.so.12 -lgcc_s.1 => /usr/lib/libgcc_s.so.1 -lssl.14 => /usr/lib/libssl.so.14 -lgssapi.11 => /usr/lib/libgssapi.so.11 -lkrb5.27 => /usr/lib/libkrb5.so.27 -lhx509.6 => /usr/lib/libhx509.so.6 -lasn1.10 => /usr/lib/libasn1.so.10 -lcom_err.8 => /usr/lib/libcom_err.so.8 -lroken.20 => /usr/lib/libroken.so.20 -lutil.7 => /usr/lib/libutil.so.7 -lwind.1 => /usr/lib/libwind.so.1 -lheimbase.2 => /usr/lib/libheimbase.so.2 -lsqlite3.1 => /usr/lib/libsqlite3.so.1 -lheimntlm.5 => /usr/lib/libheimntlm.so.5 -lldap.4 => /usr/lib/libldap.so.4 -llber.3 => /usr/lib/liblber.so.3 $ ldd /usr/libexec/postfix/qmgr /usr/libexec/postfix/qmgr: -lsaslc.0 => /usr/lib/libsaslc.so.0 -lcrypto.14 => /usr/lib/libcrypto.so.14 -lcrypt.1 => /lib/libcrypt.so.1 -lc.12 => /usr/lib/libc.so.12 -lgcc_s.1 => /usr/lib/libgcc_s.so.1 -lssl.14 => /usr/lib/libssl.so.14 -lgssapi.11 => /usr/lib/libgssapi.so.11 -lkrb5.27 => /usr/lib/libkrb5.so.27 -lhx509.6 => /usr/lib/libhx509.so.6 -lasn1.10 => /usr/lib/libasn1.so.10 -lcom_err.8 => /usr/lib/libcom_err.so.8 -lroken.20 => /usr/lib/libroken.so.20 -lutil.7 => /usr/lib/libutil.so.7 -lwind.1 => /usr/lib/libwind.so.1 -lheimbase.2 => /usr/lib/libheimbase.so.2 -lsqlite3.1 => /usr/lib/libsqlite3.so.1 -lheimntlm.5 => /usr/lib/libheimntlm.so.5 -lldap.4 => /usr/lib/libldap.so.4 -llber.3 => /usr/lib/liblber.so.3 That's an awful lotta libraries to rattle through! The runtime linker isn't terribly fast as it has quite a bit of computation to do, and just doing the I/O alone to page in enough of all those files on an older machine is going to take quite a long time. Now I don't know what your storage situation is on your VAXen, but if you can possibly afford to static-link your build you'll find things start so much faster you'll be VERY surprised. Static linked those two programs are just: # size /usr/libexec/postfix/master /usr/libexec/postfix/qmgr text data bss dec hex filename 674331 4428 42840 721599 b02bf /usr/libexec/postfix/master 4219996 26140 53016 4299152 419990 /usr/libexec/postfix/qmgr So, one is indeed quite large, but not huge by today's standards. A complete static-build for i386 is 1.8G (that's everything but tests, debug, and x11). Now ideally what I want to do for embedded systems is static-link every binary into one crunchgen binary. I've done this for 5.2 on i386, and the whole base system (or most of it, no compiler, tests, or x11; and no ntpd or named or postfix) is just 7.4MB compressed. Yes, just 7.4 megabytes: $ ls -l -h netbsd-NET5501.EMBED.gz -r--r--r-- 3 woods wheel 7.4M Jun 5 2016 netbsd-NET5501.EMBED.gz The whole root filesystem fits in 12.5MB, and the uncompressed bootable file is still well under 20MB: $ ls -l netbsd-NET5501.EMBED -rwxr-xr-x 1 woods wheel 18268428 Jun 5 2016 netbsd-NET5501.EMBED That's a kernel with an embedded root filesystem containing all most all the base programs (274 in total) all in one binary: $ file ramdiskbin ramdiskbin: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), statically linked, for NetBSD 5.2, stripped $ size ramdiskbin text data bss dec hex filename 9700273 270300 4222768 14193341 d892bd ramdiskbin $ ls -l ramdiskbin -rwxr-xr-x 1 woods wheel 9974568 Jun 5 2016 ramdiskbin And let me tell you, running programs on that system is so stunningly fast you might think they didn't even run even though the processor isn't a whirlwind by any stretch. Most text pages for _all_ programs will _always_ be in memory _all_ of the time, even on a rather tiny machine. To really make this useful for general NetBSD builds will take some more hacking on the build system (basically it might go something like always building a ".cro" file for every program possible, then generating a crunchgen config to link them all together and generate an mtree file to generate all the (sym)links; and possibly doing it on a per-set basis (e.g. one binary for base, one for the toolchain, one for x11, or something like that). > 133,000 lines of code is, in my book, a lot. That is not a good sign. It is a lot, but it's all very modular and very very little is used to start the default configuration. -- Greg A. Woods <gwoods%acm.org@localhost> Kelowna, BC +1 250 762-7675 RoboHack <woods%robohack.ca@localhost> Planix, Inc. <woods%planix.com@localhost> Avoncote Farms <woods%avoncote.ca@localhost>
Attachment:
pgpRl9qONsO9s.pgp
Description: OpenPGP Digital Signature