CVS commit: pkgsrc/textproc/xapian

["Amitai Schleier" <schmonz%netbsd.org@localhost>]

Module Name:    pkgsrc
Committed By:   schmonz
Date:           Wed Jul 24 10:54:36 UTC 2024

Modified Files:
        pkgsrc/textproc/xapian: Makefile.common distinfo distinfo-bindings
Removed Files:
        pkgsrc/textproc/xapian/patches: patch-common_errno__to__string.cc

Log Message:
xapian-core: update to 1.4.26. Changes:

API:

* Weight: Document that Weight statistics DOC_LENGTH_MIN, DOC_LENGTH_MAX and
  WDF_MAX are for the shard rather than the whole database.  Usually this is
  what we want as with a sharded database it gives tighter bounds and so better
  match optimisation, but it does make them unsuitable for uses such as
  calculating a suitable offset to add to every get_sumextra() to allow
  implementing a weighting formula which can give a negative term independent
  weight contribution.  This case will be addressed in the next release series
  which also provides bounds such as DB_DOC_LENGTH_MIN which are for the whole
  database.

* LMWeight: This class was meant to implement the "Language Model" Weighting
  scheme, but we've discovered the implementation was incorrect and fixing it
  requires ABI-incompatible changes.  For 1.4.x we need to leave it in place so
  as not to break existing code, but it's now deprecated and we recommend
  avoiding using it.  It will be removed in the next release series and
  replaced with new separate classes implementing Language Model weighting with
  each smoothing.  Thanks to Sourav Saha for reporting this problem.

* PL2PlusWeight: Fix bug in implementation of formula.  Our variable mean is
  1/lambda_t from the PL2+ paper, so we need to check mean>1 for lambda_t<1 but
  we were actually checking mean<1 instead.  The result of this is that PL2+
  actually returned a zero weight unless the term occurred frequently enough in
  the collection.

* TradWeight::get_maxpart() no longer forces the wdf_max value to be at least
  one.  We used to do this so that a non-existent term in the query would cause
  it not to achieve 100%, but now we calculate percentages based on the number
  of matching subqueries, and it is more natural for a non-existent term to get
  zero weight (ditto for a term which always has wdf 0).  This was already
  addressed for BM25Weight in 1.2.1 back in 2010.

* Enquire::set_expansion_scheme(): Add "prob" as new preferred name for
  probabilistic query expansion, with the previous "trad" still being accepted
  for now.

* QueryParser::set_prefix() and set_boolean_prefix(): Allow an optional
  trailing `:` on the field name.  This makes the API here more consistent with
  ranges, where you need to include the `:` if you want one.  See #720.

testsuite:

* Catch and report if a testcase causes signal SIGPIPE.

* Suppress valgrind errors about calling memmove() with overlapping source and
  destination (which is valid, valgrind is just confused when memcpy() and
  memmove() share an implementation).

* Add more testing of weighting schemes.

* Mark checkstatsweight3 with a sharded database as XFAIL (expected to fail).
  This testcase was previous not run for sharded databases, with a FIXME
  comment noting this.  Investigating shows it's due to a bug where we use the
  shard's termfreqs rather than those for the whole database for an expanded
  wildcard, but this seems complex to fix.

matcher:

* Fix minor wildcard weighting bug spotted while reading the code.  We were
  returning too high a value from the first call to get_maxpart() in some
  cases.  Mostly this just means the matcher continue working when it could
  have stopped, but it will also cause MSet::get_termweight() to return a
  higher value than the actual known upper bound.

glass backend:

* Simplify file descriptor handling for lock files on Unix-like platforms
  which don't support OFD locks.  This eliminates corner cases where we
  could end up with file descriptors without close-on-exec set in the main
  process.

remote backend:

* xapian-tcpsrv: Use _exit() instead of exit() to end child processes which
  avoids the risk of duplicated output from stdio buffers getting copied by
  fork() then flushed in both processes.

* Simplify file descriptor handling when launching prog remote.

inmemory backend:

* Fix bug adding posting entries.

build system:

* Improve probe for -Bsymbolic-functions.  MSVC doesn't support this flag, but
  it only emits a warning when it is used and that warning didn't match any of
  the patterns we already check for so we were detecting it as supported.

* Report result of probe to determine compiler support for -Werror or
  equivalent.

documentation:

* Improve MSVC build instructions.  Thanks to Baran Demir for feedback.

* Improve formatting of stat_flags API documentation.

* sorting.rst: Replace custom weighting scheme documentation with a link to the
  more complete equivalent in "Getting Started with Xapian".

* remote.rst: Update to reflect that user metadata is fully supported (since
  1.2.4).

portability:

* Fix to compile as C++20 and C++23.

* Resolve SIGPIPE issues on NetBSD, which were causing testcase keepalive1 to
  fail.  These seem to be due to SO_NOSIGPIPE not working correctly there so
  we now use MSG_NOSIGNAL instead for NetBSD.

* Include <errno.h> for sys_errlist.  We already do this for the configure
  check but were failing to when actually using sys_errlist, which probably
  affects at least NetBSD.

* configure: Fix clang detection which wasn't working when configure determined
  a -std=X option was needed to get C++11 support.  The obvious symptom was
  that --enable-werror wouldn't add -Werror.

* configure: NetBSD automatically pulls in library dependencies, so set
  link_all_deplibs_CXX=no there.

* Avoid using sprintf() if snprintf() is available, even in cases where the
  output size is bounded, to avoid deprecation warnings on macOS.  For 1.4.x
  we still fall back to sprintf() to avoid a point release breaking support
  for any platform still lacking snprintf().

* Stop linking with --enable-runtime-pseudo-reloc.  We were requiring this for
  cygwin and mingw, but from the documentation it should only be needed for a
  library which exports data symbols, which we don't do, and the build works
  without it.

* Use `override` for subclassing functors.  This is good practice as it gives a
  clear compile error if we have to change the signature of an virtual method
  on such a functor.  See #830.

* Avoid redefining MSVC-specific macros if they are already defined.  This
  avoids an MSVC warning and potential for the code to be ill-formed if the
  user defines these macros with a value other than 1.  Patch from A. Jiang
  (https://github.com/xapian/xapian/pull/334).


To generate a diff of this commit:
cvs rdiff -u -r1.23 -r1.24 pkgsrc/textproc/xapian/Makefile.common
cvs rdiff -u -r1.51 -r1.52 pkgsrc/textproc/xapian/distinfo
cvs rdiff -u -r1.29 -r1.30 pkgsrc/textproc/xapian/distinfo-bindings
cvs rdiff -u -r1.2 -r0 \
    pkgsrc/textproc/xapian/patches/patch-common_errno__to__string.cc

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.

Modified files:

Index: pkgsrc/textproc/xapian/Makefile.common
diff -u pkgsrc/textproc/xapian/Makefile.common:1.23 pkgsrc/textproc/xapian/Makefile.common:1.24
--- pkgsrc/textproc/xapian/Makefile.common:1.23 Fri Mar  8 19:00:54 2024
+++ pkgsrc/textproc/xapian/Makefile.common      Wed Jul 24 10:54:36 2024
@@ -1,4 +1,4 @@
-# $NetBSD: Makefile.common,v 1.23 2024/03/08 19:00:54 schmonz Exp $
+# $NetBSD: Makefile.common,v 1.24 2024/07/24 10:54:36 schmonz Exp $
 # used by textproc/csharp-xapian/Makefile
 # used by textproc/lua-xapian/Makefile
 # used by textproc/p5-Xapian/Makefile
@@ -9,7 +9,7 @@
 # used by textproc/xapian/Makefile
 # used by textproc/xapian-omega/Makefile
 
-VERSION=               1.4.25
+VERSION=               1.4.26
 CATEGORIES=            textproc
 MASTER_SITES=          http://oligarchy.co.uk/xapian/${VERSION}/
 EXTRACT_SUFX=          .tar.xz

Index: pkgsrc/textproc/xapian/distinfo
diff -u pkgsrc/textproc/xapian/distinfo:1.51 pkgsrc/textproc/xapian/distinfo:1.52
--- pkgsrc/textproc/xapian/distinfo:1.51        Fri Mar  8 19:00:54 2024
+++ pkgsrc/textproc/xapian/distinfo     Wed Jul 24 10:54:36 2024
@@ -1,8 +1,7 @@
-$NetBSD: distinfo,v 1.51 2024/03/08 19:00:54 schmonz Exp $
+$NetBSD: distinfo,v 1.52 2024/07/24 10:54:36 schmonz Exp $
 
-BLAKE2s (xapian-core-1.4.25.tar.xz) = 75f04fbaac1ac9d629009c8dd37f53eb69c72f6fabc8b9e89266ddf43f37c113
-SHA512 (xapian-core-1.4.25.tar.xz) = e8069f2cb75554978716a99580fb57fc73bbdd607c82394f53d07205d8c78fb59265004be0c676a6806bb2840a3933499dfc867e36a28855278569a0676d469d
-Size (xapian-core-1.4.25.tar.xz) = 3205904 bytes
-SHA1 (patch-common_errno__to__string.cc) = c5d9613e6676e1dcd3d9caabf113046d5e471454
+BLAKE2s (xapian-core-1.4.26.tar.xz) = bc264961f9274a9a11857f81beb314294de1e64c6416bfbce71cfa3f358576a1
+SHA512 (xapian-core-1.4.26.tar.xz) = ecd94d410dc94b6510c6729f276606987f9dd955cfb0520e62d46fa71645f5b140d474cda0d77078e64ee7cf9c0cd00f3aefe0e1b087739582e64bbcb9fabd4b
+Size (xapian-core-1.4.26.tar.xz) = 3246304 bytes
 SHA1 (patch-common_safesyssocket.h) = 6a619a91975283419d2c1ac70cf62e6b417fe981
 SHA1 (patch-xapian-config.in) = 470d1de7f04b8b8817abbcf56b5b71a36948de97

Index: pkgsrc/textproc/xapian/distinfo-bindings
diff -u pkgsrc/textproc/xapian/distinfo-bindings:1.29 pkgsrc/textproc/xapian/distinfo-bindings:1.30
--- pkgsrc/textproc/xapian/distinfo-bindings:1.29       Fri Mar  8 19:00:54 2024
+++ pkgsrc/textproc/xapian/distinfo-bindings    Wed Jul 24 10:54:36 2024
@@ -1,8 +1,8 @@
-$NetBSD: distinfo-bindings,v 1.29 2024/03/08 19:00:54 schmonz Exp $
+$NetBSD: distinfo-bindings,v 1.30 2024/07/24 10:54:36 schmonz Exp $
 
-BLAKE2s (xapian-bindings-1.4.25.tar.xz) = a87c0b38f9dd3530602b30017451383b7a286a4d19306a7a0a7769f08e68b397
-SHA512 (xapian-bindings-1.4.25.tar.xz) = 119f62b53d487319e33aaf0c473e479a52b3831a46b23724188dada13f341e08facc7b79641c00ba7b0507c22b55c7b700182e65b5a195cd322be2098bcca183
-Size (xapian-bindings-1.4.25.tar.xz) = 1076644 bytes
+BLAKE2s (xapian-bindings-1.4.26.tar.xz) = ba6427394041fffcbd6aa3b2c56eab8063ab8becc125e8aef6a5f5188365c7fb
+SHA512 (xapian-bindings-1.4.26.tar.xz) = 9ab6c8f7092c6d353341a2de565501ba4eadd47c78a472c54c9f41de08ac5b42382236a4b66f03257cd505464e1b6e9eaf2ab05c1d6e984cc641738dd43b3314
+Size (xapian-bindings-1.4.26.tar.xz) = 1114580 bytes
 SHA1 (patch-configure) = 6e0525fb4747759e22b64a3852df71e76cc6773e
 SHA1 (patch-lua_Makefile.in) = 7f1c5077f0d46dfdf33c2b65f144bb08d5031330
 SHA1 (patch-ruby_Makefile.in) = ddbf3ca92b11ff6955d80f6a5609e3ce36798b0b