[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]
Re: dtrace and ZFS update
as taylor requested after the previous mail about this,
I've made a copy of the freebsd files that I started with
and put them in the netbsd source tree layout so that they
can easily be diffed against the merged tree.
I've put several files in
the perl script to rearrange the cddl files from a
freebsd source tree into the netbsd source layout
(mostly directory moves, a few file renames)
the freebsd sources that I started with
those freebsd sources as rearranged by the script
diff of fbsd-r315983-remapped.tar.gz vs. the merged tree
cvs diff of netbsd -current vs. the merged tree
this version also includes some cleanup of the existing files,
mainly removing about 30 mostly useless glue header files
(which I found in the process of getting the remap script working).
I haven't conditionalized DTRACE_OPTS yet, but otherwise it should be
very close to what I'll commit.
please take a look and let me know if there are any other changes
that are desired before I commit it.
On Fri, Oct 13, 2017 at 09:56:01AM -0700, Chuck Silvers wrote:
> hi folks,
> I've been working on updating netbsd's copy of the dtrace and zfs code to rebase
> from the existing ancient opensolaris version to a recent freebsd version.
> most of the freebsd changes are pretty close to what netbsd needs,
> so that seems like a more useful upstream for us. I have things working
> well enough now that I want to share the code in preparation for committing.
> this update improves upon our existing dtrace/zfs code in several ways:
> - picks up all the upstream zfs fixes and enhancements from the last decade
> - zfs now supports mmap on netbsd, so you can run executables stored in zfs
> - dtrace fbt probes can now be used in kernel modules (such as zfs)
> here's the diff, which is based on netbsd HEAD as of
> "Tue Oct 10 20:14:48 UTC 2017":
> note that the patch needs to be applied with "patch -E" since it
> adds and removes various files.
> here's a summary of what's in the diff:
> - freebsd's dtrace/zfs code as of r315983 (2017-03-26), adapted for netbsd.
> many netbsd changes in our current version to fix warnings have been removed
> or redone to be reduce the difference from the new upstream.
> - a few updates to our copy of freebsd's libproc.
> I plan to do a wholesale resync of libproc to current freebsd
> rather than just picking up the few bits in this diff.
> - build system support for using -fno-omit-frame-pointer everywhere
> and disabling other compiler optimizations that confuse dtrace.
> I'm ok with leaving these changes disabled by default,
> or perhaps making them conditional on MKDTRACE.
> - sample kernel config changes for a couple evbarm configs (the ones I tested).
> - module/ksyms enhancements for dtrace integration.
> - genfs API enhancements to support zfs.
> - an option to have mutexes not become no-ops during a panic.
> - uvm_aobj API change to support 64-bit aobj sizes (eg. for tmpfs).
> the plan is that items listed separately above will be committed separately
> but in quick succession, to allow one kernel version increment for all of
> the ones which require that.
> here's a list of known issues with the patch:
> - unloading the zfs module fails even with no zpools imported
> if you've done much with zfs since it was loaded.
> there's some refcounting problem that I haven't tracked down yet.
> - the module refcounting for active fbt probes is bogus.
> currently module refcounting is protected by kernconfig_lock(),
> but taking that lock down in the bowels of dtrace seems likely
> to create deadlocks. I plan to do something fancier but haven't
> gotten to it yet.
> - the dtrace uregs stuff is probably still wrong.
> - the CTF typeid overflow problem is still there (more on this below).
> - a bunch of features are still not supported:
> - the ".zfs" virtual directory, eg. ".zfs/snapshot/foo@bar"
> - zvols
> - ZFS ACLs (aka. NFSv4 ACLs)
> - NFS exporting a ZFS file system
> - setting dtrace probes in application code
> - using ZFS as the root fs
> - new crypto hashes SHA512_256, skein, and edonr
> (the last one is not in freebsd yet either)
> - zio delay injection (used for testing zfs)
> - dtrace support for platforms other than x86 and arm
> I don't intend to fix any of these issues before initial commit of
> these changes, since I don't think any of them are regressions from
> what we have currently.
> there a few other bits still in the diff that are just for debugging or
> otherwise need to be cleaned up, but not very many. I'll clean them up
> before committing.
> about the CTF typeid overflow issue I mentioned above:
> CTF is the Compact Type Format data that dtrace uses to know about
> function argument types and return values. CTF encodes a type ID
> as a 15-bit value, but unfortunately a netbsd GENERIC kernel has
> more than 15 bits worth of types, so the values overflow the field.
> we don't really want to change this format unilaterally since it is
> currently the same on all dtrace-supporting operating systems, so we
> need to work around the problem for now by removing various subsystems
> from the kernel config used with dtrace, so that the number of types
> is within what CTF supports.
> to tell if your kernel will work with dtrace, run
> ctdump /path/to/kernel | grep "total number of types"
> if the number of types is less than 32768 then you're fine,
> otherwise you'll need to change your kernel config to remove
> some drivers or file systems that you won't use.
> this error is another symptom of the typeid overflow problem:
> dtrace: failed to compile script sleep.d: "/usr/lib/dtrace/psinfo.d", line 46: failed to copy type of 'pr_arglen': Type information is in parent and unavailable
> note that dtrace scripts that do not reference any argument or return types
> will work even if the kernel has too many types for CTF (but you'll need to
> remove the psinfo.d file mentioned above to remove that reference).
> I hope to integrate a solution for this CTF overflow issue before
> the first netbsd release with this new dtrace code in it (netbsd 9.0),
> but it's not clear what that solution will end up being yet.
> anyone interested in zfs or dtrace on netbsd please give this a try
> and let me know about any problems that you find.
> any comments on the diff are also welcome.
Main Index |
Thread Index |