Re: The run-time linker doesn't deal with magic symlinks (or even plain symlinks) for shared objects?

[Simon Burge <simonb%NetBSD.org@localhost>]

Johan Ihren wrote:

> I want to use the @machine magic symlink to be able to install both  
> i386 and amd64 distributions into the same root.
> 
> The primary reason is that I run NetBSD/i386 Xen3 in production, which  
> requires a release version, but I'm porting my stuff to NetBSD/amd64  
> Xen3, which is only available as -current. Being able to choose at  
> boot time which one to run would be *very* convenient, especially for  
> laptops with only one disk.
> 
> However, this just plain doesn't work. If I move /lib to a location  
> which is the target of a magic symlink /sbin/init refuses to execute  
> ("Error 2"). Having /sbin itself as /sbin --> /machine/@machine/sbin  
> seems to work fine, though. Leaving /lib alone to allow the machine to  
> reach multiuser the following is an obvious example of the problem:
> 
> - -bash-3.2# pwd
> /usr
> - -bash-3.2# uname -m
> amd64
> - -bash-3.2# type uname
> uname is hashed (/usr/bin/uname)
> - -bash-3.2# mv /usr/lib /usr/machine/amd64/lib
> - -bash-3.2# ln -s machine/@machine/lib /usr
> - -bash-3.2# uname -m
> Shared object "libc.so.12" not found
> - -bash-3.2# rm lib
> - -bash-3.2# mv machine/amd64/lib /usr
> - -bash-3.2# uname -m
> amd64
> 
> Note #1: yes, I've built custom kernels with "options MAGICLINKS", so  
> not doing "sysctl -w vfs.generic.magiclinks=1" early enough is not the  
> problem.
> 
> Note #2: I admit that I don't understand where the problem lies. I  
> thought that magic symlink support was entirely inside the kernel and  
> that the run-time linker being a userland process should deal with the  
> magic sylinks just like every other process, i.e. bascially not even  
> noticing. What am I missing?
> 
> Note #3: Umm, I just realized that this has nothing to do with magic  
> symlinks as such. It breaks just as nicely with plain ordinary symlinks.

This looks to be related to the relative symlinks that the libraries
are set up with - the problem is that "libc.so.12" points to
"../../lib/libc.so.12.159", and in your example ../../lib ends up
nowhere useful.  With this:

        opteron 12# mkdir -p machine/i386/usr
        opteron 13# mv usr/lib machine/i386/usr/lib
        opteron 14# ln -s /machine/i386/usr/lib usr/lib
        opteron 16# uname -m
        Shared object "libc.so.12" not found

it fails, but add:

        opteron 31# ln -s /lib machine/i386/
        opteron 32# uname -p
        i386

and we're all good.

You'll want to play with moving /lib to somewhere under machine/<arch>
too, as well as /libexec, /sbin and /bin and probably /libdata.  Is /dev
totally sharable between i386 and amd64?  I also wonder if any of the
databases in /var are arch-specific...

Or look at my suggestion in my other posting which describes booting off
different partitions - that seems simpler ;-)

Cheers,
Simon.