Re: Pbulk scan phase acceleration with tmpfs?

[Jonathan Perkin <jperkin%joyent.com@localhost>]

* On 2021-04-10 at 10:37 BST, Matthias Petermann wrote:

> So what came to my mind - would it make a significant difference if I 
> keep the pkgsrc structure of the sandboxes ready in the host's tmpfs 
> and mount it to the sandbox? Then this filesystem would be served 
> quasi from RAM - of which there would also be enough available. Does 
> anyone have experience with this?

The two main things that will help scan speed are:

 * Increased concurrency through additional scan processes running
   across multiple hosts and chroots.  One per CPU is a good start.

 * Enabling PBULK_CACHE_DIRECTORY (usually to something like /var/tmp in 
   each chroot, as it must be empty prior to each run).

Any other changes usually make negligible changes to scan times, whereas 
increased concurrency will, up to a point (the fastest I managed was 
around 5 minutes for a full scan) decrease times linearly.

--
Jonathan Perkin  -  Joyent, Inc.  -  www.joyent.com