tech-pkg archive

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]

Re: SQLite3 and JSON1

Jonathan Schleifer <> writes:

> chat/matrix-synapse requires JSON support in SQLite3. The SQLite3
> source code does not enable it by default, but the SQLite3 build
> system does by default. See

I don't follow the separation of source code vs the upstream build
system.  I went to read upstream docs and their test is also confusing:

Plus, they talk about "loadable module", but the pkgsrc sqlite package
doesn't have any modules.  Then upstream talks about an "amalgamation",
and it's not immediately clear if the library is built from that, but it
seems so.

> However, we build it without

I interpret "we" as "NetBSD base system" here.

> specifying the same flags as SQLite3's build system does by default:
> See external/public-domain/sqlite/lib/Makefile

There are N cases about JSON1 vs not:

  how does NetBSD base build sqlite3

  how does $OS build it, for all other $OS that have it in base

  how does pkgsrc build it

and I think you've said:

  NetBSD base does not enable JSON1.

  The upstream default builds it, which is therefore what hapepns in pkgsrc.

and I wonder if any other base systems have this?

> This results in pkgsrc SQLite3 and base SQLite3 to have a different
> feature set. py-sqlite3 by default depends on system SQLite3, so we
> end up with a py-sqlite3 that just fails at runtime to use JSON. Which
> is why matrix-synapse doesn't work, and currently has a MESSAGE that
> says you need to prefer pkgsrc SQLite3.

Normally, we would add some sort of package-settable variable that
packages that depend on sqlite3 and need JSON1 can set.   For example,


Then, the buildlink3/ would decline to use builtin if the
builtin lacked that feature.  This avoids skipping base implementations
that are acceptable, which could include future NetBSD versions.

> I assume a lot of Python software has this assumption, so I would
> propose that py-sqlite3 should always depend on pkgsrc SQLite3. A lot
> of C software will have the same assumption (regular SQLite3 does come
> with JSON1 - we're being odd here), but this software can just depend
> on pkgsrc SQLite3. However Python software can not do that easily, as
> py-sqlite3 also needs to link pkgsrc SQLite3 then.

I don't see any other aprpoach, as pkgsrc doesn't and can't set options
on dependent packages.

So I think py-sqlite3 has to declare SQLITE_FEATURES=JSON1, to force use
of a sqlite3 build that has JSON1, always.  I don't think the harm of
the bloat is going to be significant; on my system matrix-synapse and
gramps depend on py-sqlite3, and I already have pkgsrc sqlite3 for tcl.

> Any objections to making py-sqlite3 always use pkgsrc SQLite3?

I object to basically saying "never use builtin" because

  This doesn't provide a path to using base sqlite3 in NetBSD 10 if we
  enable JSON1.

  This stops using base sqlite3 on all platforms, even those with JSON1.

Thus we need something like SQLITE_FEATURES.

Do you or others know how many other pkgsrc platforms have sqlite in
base (probably some/most GNU/Linux)?  And how it is compiled there?

My sketch for doing this:

  Document SQLITE_FEATURES in databases/sqlite3/  Default
  it to empty.

  In builtin, check SQLITE_FEATURES for JSON1, and if so fail builtin
  detection on NetBSD.

  Set SQLITE_FEATURES=JSON1 in databases/py-sqlite3/Makefile

  Add a line to DESCR of py-sqlite3 that the package will always include

which sounds harder than it is, I hope.

Attachment: signature.asc
Description: PGP signature

Home | Main Index | Thread Index | Old Index