Subject: `__attribute__ ((__aligned__))' has gotta go
To: None <tech-kern@netbsd.org>
From: Eduardo Horvath <eeh@netbsd.org>
List: tech-kern
Date: 10/23/1999 11:43:20
Some time back, to make the lock_data field usable on certain
platforms that require peculiar alignments for their synchronization
primitives, `__attribute__ ((__aligned__))' was added to the MI lock.h
file.  This has been causing me no end of trouble.

(Here I describe the problems it's caused me.  Skip to the bottom if
you're not interested in a long story.)

I've been trying to get a sparcv9 cross toolchain running on the 32-bit
ELF userland for a while.  After a lot of coaxing and prodding, I manage
to get a kernel to compile and link.  So I try to boot it.  Boom, datafault.
A bit of analysis reveals that the kernel loaded, but it died trying to zero
out the BSS.  So what happened?
 
Let's verify the toolchain.  I build a new second stage bootloader to test
it out.  It works fine, but the kernel dies in the same place.  Toolchain
seems to work, maybe it's a bug in the bootloader's ELF routines.

So I add a whole bunch of debug statements to the bootloader and discover
that it's unable to find the BSS segment in the kernel.
 
OK, let's dump the ELF headers on the kernel.  There's a BSS section in
there, allright, but it's got 2**16 alignment, and there's a big hole
between it and the DATA segment.  Hmm, that doesn't look right.  Also,
the filesz and the memsz in the program headers are the same and only
go as far as the DATA segment.

I hunt down an old kernel and dump it's headers also.  There the memsz
is larger than the filesz, the BSS section is aligned 2**8, and it abuts
the DATA segment.

Linker bug?  So, let's rebuild ld.  No change.  Let's look at the linker
scripts.  They look reasonable.  Maybe the linker is reading the wrong 
script.  I generate a script that has no alignment specified for either
the DATA or BSS segment and try it out.  No change.
 
So I look in the symbol table to find what happens to be located at the
head of the BSS segment and it turns out to be some temporary from 
db_sym.o.  Look in the source file for `align' and out pops 
`ALIGNED_POINTER', but it turns out to be a red herring.  I finally
disassemble the object file and track the data object to the `static 
char tmp[256];' in db_qualify().  How could that cause aligment problems?
So I dump that header, and the BSS segment is aligned 2**8.

Then I dump all the object file headers and look for anything aligned
2**16.  Bingo!  Several files have this, so I look at the first one,
kern_malloc.o.  Look for `align' in the source, but no luck.  Then I
notice it's using locks.  So I comment out the `__attribute__ 
((__aligned__))' from lock.h and rebuild kern_malloc.o.  Alignment
is now 2**8.  So why is this happening?

(Here's the important stuff)

The gcc info doc says:

     "Whenever you leave out the alignment factor in an `aligned'
     attribute specification, the compiler automatically sets the
     alignment for the declared variable or field to the largest
     alignment which is ever used for any data type on the target
     machine you are compiling for."

The largest alignment which is ever used would be page alignment, and
the SPARC V9 ELF ABI specifies that pages have a maximum 1MB alignment.
So this %$#%%$#% directive is causing my BSS segment to be 1MB aligned,
which I don't think is the desired effect.

So, what I want to do create a new type, lock_t, and have it defined
in <machine/lock.h> so architectures that need `__attribute__ ((__aligned__))'
can have it and architectures where it causes problems don't.  Then have 
the cpu_simple_lock*() routines take a `lock_t *' instead of a 
`struct simplelock *'.  If MULTIPROCESSOR is not defined, then we can simply
add `typedef int lock_data;' to <sys/lock.h>

Comments?

Eduardo