Re: kernel libraries and dead code in MODULAR kernels

[Masao Uebayashi <uebayasi%gmail.com@localhost>]

I believe that the exact problem exists in userland's dynamically
linked libraries/programs, right?  If so, how do they deal with this
dead code problem?

On Mon, Aug 31, 2015 at 10:55 AM, Dennis Ferguson
<dennis.c.ferguson%gmail.com@localhost> wrote:
> I've been compiling the same kernel configuration with and without
> options MODULAR and noticed that the MODULAR kernels have a not
> inconsiderable amount of code that is called neither by the kernel
> nor any of the modules.  I've appended a list of the global symbols
> from libkern.[ao] that end up being included in amd64 mostly-GENERIC
> modular kernels that have no callers and are omitted from non-modular
> kernel builds.
>
> The reason for this is that while non-modular kernels build a libkern.a
> from which only code that is actually called by something is loaded into
> the kernel, modular kernels instead build a libkern.o from which everything
> is unconditionally included.  While I understand why this is done (everything
> is included in case the separately-compiled modules need it) it causes
> a difficult problem if you would like to have a modular kernel without
> the dead code.  A few of those functions are in fact no longer used and
> could just be eliminated from the build, but the rest seem to end up
> there either because their use depends on configuration options which may
> or may not be defined, or they are members of a library that is also used
> in user space that has a richer API there than the kernel makes use of, or
> the use of particular functions (e.g. in the runtime library) is dependent
> on the architecture being built.  While it would be possible to go through
> these and subset what is built for each kernel by options and architecture
> this is a whole lot of work, and a maintenance headache, compared to just
> compiling a few extra files and letting the loader figure out what the
> kernel it is linking actually wants.
>
> The particular reason I'm concerned about this is that I am now loading
> the kernel against additional libraries with APIs the kernel won't make
> full use of and don't want to make the dead code issue worse, but I've
> found that maintaining the subsets the kernel does use to be quite
> painful and annoying, particularly for one library where the subset
> depends on the architecture being built.  I wouldn't mind going through
> at the end and subsetting the files which are never going to be used out
> of the kernel build to save a bit of kernel compile time, but for things
> that are option- or architecture-dependent this is way too complex to be
> worth the modest savings in build time.  There should be a better way to
> keep the kernel free of dead code.
>
> I do notice that for modular kernels the modules are built before the kernel
> is.  Is there no way to harvest the undefined global symbols from the modules
> and include these to force the kernel link to resolve them?  Then libkern
> could go back to being libkern.a, with the bonus that using something in a
> module that the kernel doesn't have would cause a complaint when the kernel
> is loaded.
>
> Dennis Ferguson
>
> __absvdi2
> __absvsi2
> __absvti2
> __addvdi3
> __addvsi3
> __addvti3
> __ashldi3
> __ashlti3
> __ashrdi3
> __ashrti3
> __clzdi2
> __clzsi2
> __clzti2
> __cmpdi2
> __cmpti2
> __ctzdi2
> __ctzsi2
> __ctzti2
> __divdi3
> __divmoddi4
> __divmodsi4
> __divsi3
> __divti3
> __ffsdi2
> __ffsti2
> __lshrdi3
> __lshrti3
> __main
> __moddi3
> __modsi3
> __modti3
> __muldi3
> __mulodi4
> __mulosi4
> __muloti4
> __multi3
> __mulvdi3
> __mulvsi3
> __mulvti3
> __negdi2
> __negti2
> __negvdi2
> __negvsi2
> __negvti2
> __paritydi2
> __paritysi2
> __parityti2
> __popcountdi2
> __popcountsi2
> __popcountti2
> __subvdi3
> __subvsi3
> __subvti3
> __ucmpdi2
> __ucmpti2
> __udivdi3
> __udivmoddi4
> __udivmodsi4
> __udivmodti4
> __udivsi3
> __udivti3
> __umoddi3
> __umodsi3
> __umodti3
> _prop_object_externalize_append_encoded_cstring
> bswap16
> bswap32
> bswap64
> cdbr_close
> cdbr_entries
> cdbr_find
> cdbr_get
> cdbr_open_mem
> compilerrt_abort_impl
> ffs
> htonl
> htons
> imax
> imin
> lmax
> lmin
> max
> mi_vector_hash
> min
> mtprng_init32
> mtprng_initarray
> mtprng_random
> mtprng_rawrandom
> murmurhash2
> ntohl
> ntohs
> null_extant
> popcount64
> popcountl
> popcountll
> ppath_alloc
> ppath_component_at
> ppath_component_extant_dec
> ppath_component_extant_inc
> ppath_component_idx
> ppath_component_key
> ppath_component_release
> ppath_component_retain
> ppath_copy
> ppath_copydel_bool
> ppath_copydel_data
> ppath_copydel_int64
> ppath_copydel_object
> ppath_copydel_string
> ppath_copydel_uint64
> ppath_copyset_bool
> ppath_copyset_data
> ppath_copyset_int64
> ppath_copyset_object
> ppath_copyset_string
> ppath_copyset_uint64
> ppath_create
> ppath_create_bool
> ppath_create_data
> ppath_create_int64
> ppath_create_object
> ppath_create_string
> ppath_create_uint64
> ppath_delete_bool
> ppath_delete_data
> ppath_delete_int64
> ppath_delete_object
> ppath_delete_string
> ppath_delete_uint64
> ppath_dup_data
> ppath_dup_string
> ppath_extant_dec
> ppath_extant_inc
> ppath_free
> ppath_get_bool
> ppath_get_data
> ppath_get_int64
> ppath_get_object
> ppath_get_string
> ppath_get_uint64
> ppath_idx
> ppath_key
> ppath_length
> ppath_lookup
> ppath_pop
> ppath_push
> ppath_push_idx
> ppath_push_key
> ppath_release
> ppath_replace_idx
> ppath_replace_key
> ppath_retain
> ppath_set_bool
> ppath_set_data
> ppath_set_int64
> ppath_set_object
> ppath_set_string
> ppath_set_uint64
> ppath_subpath
> prop_array_add_and_rel
> prop_array_add_int16
> prop_array_add_int32
> prop_array_add_int64
> prop_array_add_int8
> prop_array_add_uint16
> prop_array_add_uint32
> prop_array_add_uint64
> prop_array_add_uint8
> prop_array_get_bool
> prop_array_get_cstring
> prop_array_get_cstring_nocopy
> prop_array_get_int16
> prop_array_get_int32
> prop_array_get_int64
> prop_array_get_int8
> prop_array_get_uint16
> prop_array_get_uint32
> prop_array_get_uint64
> prop_array_get_uint8
> prop_array_set_bool
> prop_array_set_cstring
> prop_array_set_cstring_nocopy
> prop_array_set_int16
> prop_array_set_int32
> prop_array_set_int64
> prop_array_set_int8
> prop_array_set_uint16
> prop_array_set_uint32
> prop_array_set_uint64
> prop_array_set_uint8
> prop_dictionary_ingest
> prop_ingest_context_alloc
> prop_ingest_context_error
> prop_ingest_context_free
> prop_ingest_context_key
> prop_ingest_context_private
> prop_ingest_context_type
> ptree_check
> ptree_find_filtered_node
> ptree_init
> ptree_insert_mask_node
> ptree_insert_node
> ptree_iterate
> ptree_mask_node_p
> ptree_remove_node
> random
> strcspn
> strlen
> strncat
> strpbrk
> strsep
> strspn
> strtoi
> strtoimax
> strtou
> strtoumax
> ulmax
> ulmin
>