[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]
Re: pkg/24255 (pkg_delete removes meta data even if deinstall fails)
The following reply was made to PR pkg/24255; it has been noted by GNATS.
From: Matthew Mondor <mm_lists%pulsar-zone.net@localhost>
Subject: Re: pkg/24255 (pkg_delete removes meta data even if deinstall
Date: Thu, 21 Apr 2011 05:49:01 -0400
On Thu, 21 Apr 2011 03:49:39 +0000 (UTC)
> Synopsis: pkg_delete removes meta data even if deinstall fails
> State-Changed-From-To: feedback->open
> State-Changed-By: dholland%NetBSD.org@localhost
> State-Changed-When: Thu, 21 Apr 2011 03:49:38 +0000
> Problem still exists. Also, submitter wrote to the gnats junkmail address
> to report that he can't test.
Just so I have a clear view of what is ideally wanted:
Currently, some files of a package might still subsist while the db
metadata is lost/deleted.
It would be possible to instead leave the meta information pending on
error, at the risk of having some files lost/missing with inconsistent
I doubt it, but could access(2) reliably be used as a first pass before
the unlink(2)/rmdir(2) phase to fail before performing any
Alternatively, should packages also support a "partial/broken" state,
where metadata would be updated to reflect actual remaining files after
an incomplete removal? Possibly keeping the original PLIST for
reference but adding a PARTIAL-PLIST for such packages?
Or should a transactional/atomic system be expected instead?
In which case I could see various imperfect solutions (FFS not
helping much here):
- Obtain a global exclusive pkg advisory lock
- Backup the package files and attempt to delete the original files; if
everything is successful, delete the backup and metadata, otherwise
restore the deleted files from the backup and preserve the metadata.
This should take into account file permissions, flags, PaX flags, etc.
As drawbacks, some space would be required to store the temporary
backup, and some effort made to sync all file permissions and flags,
code which might be duplicated among the various backup utilities
supporting some or all of these flags (not to mention the OS-specifics).
Perhaps acceptable with a dependency on pax(1)?
- Instead of deleting files, move them to a temporary "trashcan"
directory tree on the same filesystem (on write failure, fail
immediately); if all files were successfully moved, ensure that their
permissions allow deletion and recursively delete the temporary tree,
then the metadata.
On error move successfully-moved files back where they belong,
restoring their permissions, and leave the metadata as-is, since we
restored the tree state. Seems flaky; I'm not sure this would
succeed in all cases where package removal now fails. We also could
end up with a growing trash directory if files could be moved yet could
not be deleted.
- fsync(2) and release the pkg lock
Unfortunately, even the above solutions might yield inconsistent state
in the advent of a crash or user interrupt. NetBSD doesn't have
transactional FS and support syscalls, and even if it did this might
not exist on most other pkgsrc-supported OSs...
Other more far-fetched "atomic" support built on non-transactional FS:
I guess that a variant could be using a NetBSD-supported transactional
"trashcan" per file system along with its atomic-simulated
auto-restore/async-delete system. Such a trashcan might include two
sections, one for user-deleted files and another for system-deleted
files. Files would be stored with their original path/flags.
User-deleted files would require explicit "empty/restore" commands, yet
files destined for system-removal would be enclosed in transactions and
could be restored at crash/reboot in case of an incomplete transaction,
with a thread/daemon responsible for asynchroneously deleting files
deleted in complete transactions.
Transactions metadata could be implemented on top of bdb, sqlite or a
custom system supporting a transactional log (using fdatasync(2) at
reasonable intervals and automatic log file rotation/removal, etc).
pkgsrc when on NetBSD could then make use of that "atomic multi-delete
operation", despite it being non-portable. Perhaps that some files
might still be dangling over time and that a nightly script should warn
A variant of the above might be directly provided by pkgsrc, with
reliance on the required portable tools such as sqlite used for package
metadata and transactional-log, with $PKGDIR/tmp/trash used as a
temporary trash directory, an asynchroneous files cleanup cron event or
daemon, with a boot rollback/recovery script.
Unfortunately complicates pkgsrc, adds new tool dependencies and makes
it more system-invasive... Perhaps that instead of having a
boot-time recovery script and a cleanup cron event/daemon, every pkg_*
command could as a prologue perform those verification/pending
operations as needed, to minimise system invasion, with failed
procedures logged to stderr as a warning before attempting the
requested operation, with some loss of performance...
Main Index |
Thread Index |