Subject: Boost and threads
To: None <tv@netbsd.org>
From: Julio M. Merino Vidal <jmmv84@gmail.com>
List: tech-pkg
Date: 02/20/2005 00:31:02
Hi Tood (and list),

as you may be already aware, I'm updating boost to 1.32.0 and trying
to reorganize these packages a bit.  So far, I've created the following
packages:

- boost-build, which provides bjam.  This will always be a build-time
  dependency and is installed as a standalone package so that all other
  boost-* packages can easily use it.

- boost-headers, which provides all boost .hpp files.  This will also be
  a build-time dependency always.  Some programs may even compile with
  this package installed only, since many boost libraries are provided
  in the form of templates.

- boost-docs, which includes all the documentation in html format.

- boost, a meta-pkg which depends on all the above (and the ones
  described below).

So far, so good.  However, the problems now arise.  My initial idea (and
the one I like most) was (is) to create a standalone package for each of
the provided binary libs.  I.e., boost-regex, boost-test, boost-python
(which already exists), etc.  The point is to make these binary packages
very small dependencies for anything else that needs to use them
(contrary to the actual situation, as the 'boost' package is close to 
giant).  Furthermore, each package is kept small, so build errors are
easy to catch and affect as few dependent packages as possible.

This proposal is very easily doable, but not with the actual threading
related hacks present in pkgsrc.  ATM, the boost package installs binary
libraries for everything, all with threads disabled.  However, the
boost-thread package not only installs the threading library, but also
_all_ the other libraries compiled against threads.  (This is confusing
at the very least.)

This approach seems to be a possible focus of problems.  The user will
end up with two versions of the libraries installed on his system: ones
compiled against threads and ones which are not.  First doubt that
arises: how is he supposed to choose which version he wants to use in
his programs (specially when built w/o pkgsrc)?  Usually, on any
preconfigured system, you can assume that linking against, say
'-lboost_filesystem', will just DTRT.  But in pkgsrc, you have to
specify which library you want (-mt or not), and also indicate which
toolset it was built with (-gcc, -sunpro, etc.).

(However, there are no boost-enabled packages in pkgrsc ATM, so the
"problem" goes unnoticed.  The only one I'm aware of is monotone; this
is why I'm using it in the examples.)

And not to mention our thread_user.hpp change.  Consider, for example,
the devel/monotone package.  This one depends on boost-thread _only_
because it misteriously needs that header (but the original monotone
does not require threads at all).  Once it's built, the boost-thread
package can simply be removed and everything will be fine.  FWIW,
building boost and monotone by hand, bypassing pkgsrc, causes no
trouble WRT threads.

So... here are my main concerns:

- Is all this complexity really needed?  If so, why?  I don't see a
  pthread dependency being a big deal.  We'd even make it a build-time
  option for those systems where threading is available but they don't
  want it.  Such a (manual) option could certainly simplify things a
  lot.

- WRT thread_user.hpp... this is something very specific to pkgsrc, and
  as already said, causes some misterious problems.  Do we want to keep
  such custom thing?  (I know, if the question to the question in the
  previous point is "yes", then this one is true too.)

- I'd rather do the following: use threading by default and make all
  libraries be installed with unversioned names.  That is, the Boost
  Regex library could just install libboost_regex.{a,so}.  Doing so
  (which is what many Linux distros do) any package needing boost can
  simply link against '-lboost_regex' and get everything just working.

  FWIW, we are actually installing some unversioned libraries (without
  1_31 in them), but they still keep the toolset in their names.

  Otherwise, ugly hacks are needed like in devel/monotone (see the
  BOOST_SUFFIX variable, for example).

- boost-python is inconsistent with the existing boost and boost-thread
  packages.  What will happen when we are able build boost-python with
  threading support?  Will we have a boost-python-thread package?

- Just as one expects boost-python to install the python specific
  library, one thinks that boost-thread will install the thread specific
  library.  But that's not true.  (This is why I said that this last
  package was confusing.)

  By making each binary library a standalone package, this confusion
  could disappear, because each package could install just one lib.

- Another approach I've considered is to provide a boost-nonthread
  package and a boost-thread one.  The former could install all binary
  libs w/o threads, and the later could do the same but with threading
  enabled.

  I don't like this very much for two reasons: we have to keep 
  thread_user.hpp, and we can't install unversioned names that will
  always work.  (Well, we'd install them for boost-nonthread, and add a
  trailing -mt for the ones in boost-thread.)

- At last, on a not-very-related note, I'm curious about our invocation
  of the configure script to generate user.hpp.  If I use the provided
  header file, boost (1.32.0) builds just fine.  Unforunately, if I use
  the one generated by configure, I get multiple build-time errors
  (mostly related to wchar)...

  Not to say that user.hpp ends with a BOOST_HAS_THREADS definition in
  it (in the original boost - nonthreaded - package).  Looks strange.

Please share your comments.

Thanks!

-- 
Julio M. Merino Vidal <jmmv84@gmail.com>
http://www.livejournal.com/users/jmmv/
The NetBSD Project - http://www.NetBSD.org/