pkgsrc-Bugs archive
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]
pkg/57516: chat/matrix-synapse 1.86.0 unable to launch: sqlite3.IntegrityError
>Number: 57516
>Category: pkg
>Synopsis: chat/matrix-synapse 1.86.0 unable to launch: sqlite3.IntegrityError
>Confidential: no
>Severity: critical
>Priority: medium
>Responsible: pkg-manager
>State: open
>Class: sw-bug
>Submitter-Id: net
>Arrival-Date: Mon Jul 10 12:25:00 +0000 2023
>Originator: Paolo Vincenzo Olivo
>Release: pkgsrc-2023Q2
>Organization:
SDF Public Access UNIX System
>Environment:
NetBSD beastie 9.3 NetBSD 9.3 (GENERIC_KASLR) #0: Thu Aug 4 15:30:37 UTC 2022 mkrepro%mkrepro.NetBSD.org@localhost:/usr/src/sys/arch/amd64/compile/GENERIC_KASLR amd64
>Description:
After upgrading matrix-synapse to 1.86.0 - the version currently shipped with pkgsrc-2022Q3 - I encountered the following run-time error:
```
sqlite3.IntegrityError: UNIQUE constraint failed: temp_user_filters.user_id
```
This actually makes the server crash and the service from starting.
This is apparently a known bug [1] introduced in 1.86.0 and fixed [2] in 1.87.0 [3], released few days ago.
I don't know how many existing users could be affected y this, but it seems worth fixing and back-porting to 2022Q3.
[1] https://github.com/NixOS/nixpkgs/issues/239833
[2] https://github.com/matrix-org/synapse/pull/15817
[3] https://github.com/matrix-org/synapse/releases/tag/v1.87.0
>How-To-Repeat:
Upgrade chat/matrix-synapse from 1.78.0 to 1.86.0, given an existing and functional SQL synapse database containing a number of user IDs.
Attempt to restart the matrix-synapse rc.d service upon successful build, and verify that the server crashes upon start-up with the above mentioned error log.
>Fix:
Either apply upstream's patch, or update the package to release 1.87.0, which incorporates it.
I decided to patch 1.86.0 and can attest that it solves the issue.
Diff follows:
```
Index: Makefile
===================================================================
RCS file: /cvsroot/pkgsrc/chat/matrix-synapse/Makefile,v
retrieving revision 1.77
diff -u -p -r1.77 Makefile
--- Makefile 25 Jun 2023 22:54:34 -0000 1.77
+++ Makefile 10 Jul 2023 11:48:57 -0000
@@ -1,6 +1,7 @@
# $NetBSD: Makefile,v 1.77 2023/06/25 22:54:34 js Exp $
DISTNAME= matrix-synapse-1.86.0
+PKGREVISION= 1
CATEGORIES= chat
MASTER_SITES= ${MASTER_SITE_GITHUB:=matrix-org/}
GITHUB_PROJECT= synapse
@@ -104,6 +105,7 @@ BUILD_DEFS+= SYNAPSE_DEFAULT_CREATOR_P
post-patch:
# Otherwise, this file is installed and pollutes PLIST.
rm ${WRKSRC}/synapse/handlers/room.py.orig
+ rm ${WRKSRC}/synapse/storage/schema/main/delta/78/*.orig
post-install:
${INSTALL_DATA_DIR} ${DESTDIR}${DOCDIR}
Index: PLIST
===================================================================
RCS file: /cvsroot/pkgsrc/chat/matrix-synapse/PLIST,v
retrieving revision 1.42
diff -u -p -r1.42 PLIST
--- PLIST 25 Jun 2023 22:54:34 -0000 1.42
+++ PLIST 10 Jul 2023 11:48:57 -0000
@@ -1,4 +1,4 @@
-@comment $NetBSD: PLIST,v 1.42 2023/06/25 22:54:34 js Exp $
+@comment $NetBSD$
bin/export_signing_key
bin/generate_config
bin/generate_log_config
@@ -1346,6 +1346,10 @@ ${PYSITELIB}/synapse/storage/schema/main
${PYSITELIB}/synapse/storage/schema/main/delta/78/01_validate_and_update_profiles.pyc
${PYSITELIB}/synapse/storage/schema/main/delta/78/02_validate_and_update_user_filters.py
${PYSITELIB}/synapse/storage/schema/main/delta/78/02_validate_and_update_user_filters.pyc
+${PYSITELIB}/synapse/storage/schema/main/delta/78/03_remove_unused_indexes_user_filters.py
+${PYSITELIB}/synapse/storage/schema/main/delta/78/03_remove_unused_indexes_user_filters.pyc
+${PYSITELIB}/synapse/storage/schema/main/delta/78/04_add_full_user_id_index_user_filters.py
+${PYSITELIB}/synapse/storage/schema/main/delta/78/04_add_full_user_id_index_user_filters.pyc
${PYSITELIB}/synapse/storage/schema/main/full_schemas/16/application_services.sql
${PYSITELIB}/synapse/storage/schema/main/full_schemas/16/event_edges.sql
${PYSITELIB}/synapse/storage/schema/main/full_schemas/16/event_signatures.sql
Index: distinfo
===================================================================
RCS file: /cvsroot/pkgsrc/chat/matrix-synapse/distinfo,v
retrieving revision 1.57
diff -u -p -r1.57 distinfo
--- distinfo 25 Jun 2023 22:54:34 -0000 1.57
+++ distinfo 10 Jul 2023 11:48:57 -0000
@@ -170,3 +170,6 @@ SHA512 (windows_x86_64_msvc-0.36.1.crate
Size (windows_x86_64_msvc-0.36.1.crate) = 661999 bytes
SHA1 (patch-pyproject.toml) = 3144408c089de7fee47a7babccca382d1baf9bf2
SHA1 (patch-synapse_handlers_room.py) = 1bb7740b5a8385f5d0ca9c1e8c38d41a8e594473
+SHA1 (patch-synapse_storage_schema_main_delta_78_02__validate__and__update__user__filters.py) = 5545c9b3a6714f3a3607b5ab2ff2470470e625b3
+SHA1 (patch-synapse_storage_schema_main_delta_78_03__remove__unused__indexes__user__filters.py) = 516a214d2f1b1e9268b42568ab31945be3b5ecb0
+SHA1 (patch-synapse_storage_schema_main_delta_78_04__add__full__user__id__index__user__filters.py) = be8a7a174d05559bac59865224ed80cc6f86353e
Index: patches/patch-synapse_storage_schema_main_delta_78_02__validate__and__update__user__filters.py
===================================================================
RCS file: patches/patch-synapse_storage_schema_main_delta_78_02__validate__and__update__user__filters.py
diff -N patches/patch-synapse_storage_schema_main_delta_78_02__validate__and__update__user__filters.py
--- /dev/null 1 Jan 1970 00:00:00 -0000
+++ patches/patch-synapse_storage_schema_main_delta_78_02__validate__and__update__user__filters.py 10 Jul 2023 11:48:57 -0000
@@ -0,0 +1,17 @@
+$NetBSD$
+
+Fix sqlite user_filters.
+
+--- synapse/storage/schema/main/delta/78/02_validate_and_update_user_filters.py.orig 2023-06-20 15:22:50.000000000 +0000
++++ synapse/storage/schema/main/delta/78/02_validate_and_update_user_filters.py
+@@ -61,9 +61,7 @@ def run_upgrade(
+ full_user_id text NOT NULL,
+ user_id text NOT NULL,
+ filter_id bigint NOT NULL,
+- filter_json bytea NOT NULL,
+- UNIQUE (full_user_id),
+- UNIQUE (user_id)
++ filter_json bytea NOT NULL
+ )
+ """
+ cur.execute(create_sql)
Index: patches/patch-synapse_storage_schema_main_delta_78_03__remove__unused__indexes__user__filters.py
===================================================================
RCS file: patches/patch-synapse_storage_schema_main_delta_78_03__remove__unused__indexes__user__filters.py
diff -N patches/patch-synapse_storage_schema_main_delta_78_03__remove__unused__indexes__user__filters.py
--- /dev/null 1 Jan 1970 00:00:00 -0000
+++ patches/patch-synapse_storage_schema_main_delta_78_03__remove__unused__indexes__user__filters.py 10 Jul 2023 11:48:57 -0000
@@ -0,0 +1,54 @@
+$NetBSD$
+
+Fix sqlite user_filters.
+
+--- synapse/storage/schema/main/delta/78/03_remove_unused_indexes_user_filters.py.orig 2023-07-10 11:33:20.180398701 +0000
++++ synapse/storage/schema/main/delta/78/03_remove_unused_indexes_user_filters.py
+@@ -0,0 +1,47 @@
++def run_update(
++ cur: LoggingTransaction,
++ database_engine: BaseDatabaseEngine,
++ config: HomeServerConfig,
++) -> None:
++ """
++ Fix to drop unused indexes caused by incorrectly adding UNIQUE constraint to
++ columns `user_id` and `full_user_id` of table `user_filters` in previous migration.
++ """
++
++ if isinstance(database_engine, Sqlite3Engine):
++ cur.execute("DROP TABLE IF EXISTS temp_user_filters")
++ create_sql = """
++ CREATE TABLE temp_user_filters (
++ full_user_id text NOT NULL,
++ user_id text NOT NULL,
++ filter_id bigint NOT NULL,
++ filter_json bytea NOT NULL
++ )
++ """
++ cur.execute(create_sql)
++
++ copy_sql = """
++ INSERT INTO temp_user_filters (
++ user_id,
++ filter_id,
++ filter_json,
++ full_user_id)
++ SELECT user_id, filter_id, filter_json, full_user_id FROM user_filters
++ """
++ cur.execute(copy_sql)
++
++ drop_sql = """
++ DROP TABLE user_filters
++ """
++ cur.execute(drop_sql)
++
++ rename_sql = """
++ ALTER TABLE temp_user_filters RENAME to user_filters
++ """
++ cur.execute(rename_sql)
++
++ index_sql = """
++ CREATE UNIQUE INDEX IF NOT EXISTS user_filters_unique ON
++ user_filters (user_id, filter_id)
++ """
++ cur.execute(index_sql)
Index: patches/patch-synapse_storage_schema_main_delta_78_04__add__full__user__id__index__user__filters.py
===================================================================
RCS file: patches/patch-synapse_storage_schema_main_delta_78_04__add__full__user__id__index__user__filters.py
diff -N patches/patch-synapse_storage_schema_main_delta_78_04__add__full__user__id__index__user__filters.py
--- /dev/null 1 Jan 1970 00:00:00 -0000
+++ patches/patch-synapse_storage_schema_main_delta_78_04__add__full__user__id__index__user__filters.py 10 Jul 2023 11:48:57 -0000
@@ -0,0 +1,18 @@
+$NetBSD$
+
+Fix sqlite user_filters.
+
+--- synapse/storage/schema/main/delta/78/04_add_full_user_id_index_user_filters.py.orig 2023-07-10 11:33:45.237148964 +0000
++++ synapse/storage/schema/main/delta/78/04_add_full_user_id_index_user_filters.py
+@@ -0,0 +1,11 @@
++from synapse.storage.database import LoggingTransaction
++from synapse.storage.engines import BaseDatabaseEngine, Sqlite3Engine
++
++
++def run_create(cur: LoggingTransaction, database_engine: BaseDatabaseEngine) -> None:
++ if isinstance(database_engine, Sqlite3Engine):
++ idx_sql = """
++ CREATE UNIQUE INDEX IF NOT EXISTS user_filters_full_user_id_unique ON
++ user_filters (full_user_id, filter_id)
++ """
++ cur.execute(idx_sql)
```
Home |
Main Index |
Thread Index |
Old Index