Subject: Re: StrongARM K bug
To: Nicholas Clark <nick@flirble.org>
From: Richard Earnshaw <rearnsha@arm.com>
List: port-arm32
Date: 03/29/1999 19:49:13
> I think I wasn't clear:
> 
> I added a flag to optionally enable the cc1 workaround for the K bug

That was perfectly clear, but I still think it's the wrong approach...

> The workaround itself is to .align 3; before any LDM involving PC.
> As I understand it, the only time the compiler generates an LDM loading
> the PC is for a function return.  If this is true then the workaround
> catches all problem instances.

.align 3 (by which I assume you mean align to a 2^3 boundary -- different 
arm assemblers unfortunately put different interpretations on this 
*groan*) has no meaning unless the section that it is placed in is also 
guaranteed to be this aligned.  I can't speak for the ELF format off the 
top of my head but this is not the case for a.out which only aligns 
sections to a 4-byte boundary.  It also fills the instruction stream with 
arbitrary padding that aren't guaranteed to be nop instructions (though I 
think that they will normally have that effect with gas -- 0 == andeq r0, 
r0, r0).   Don't forget that gcc emits conditional ldm...{...,pc} as well. 
 Packing these out can really hose your performance.

> 
>  
> > Summary: I still don't think fixing the compiler is the correct way to 
> > address this bug.
> > 
> > Richard (gcc/arm maintainer).
> 
> I agree that bodging the compiler is not a permanent solution to the problem,
> and I wasn't advocating that the compiler flag be enabled by default, or that
> distributed binaries be built with the K workaround.  The discussion about
> working round the K bug arose because I'd hit specific problems compiling/
> running things (bootstrapping egcs, glibc, and perl5.005_03, ld-linux.so) that
> appeared to be due to the K bug.
> 
> I'd been sent and compiled your program, but found that it wouldn't solve my
> problems because
> 
> 1: It's for a.out, I'm using ELF, and I don't know enough about ELF formats
>    to convert your program to modify ELF

Yep, but updating it oughtn't to be that hard.  You need to find the 
executable segment(s) and scan through them.  At the end of each will 
normally be some spare bytes into which bogus instructions can be moved.

> 2: I think some of my SEGVs due to the K bug were in shared libraries.
>    Certainly the perl ones only hit with dynamic loading of the IO extension.

Extending to shared libs shouldn't be hard either; they are in effect just 
executables with slightly different headers and loading requirements.

> 3: The egcs problems were with programs compiled during the bootstrapping
>    process (gcc/genrecog), hence even if I'd created an ELF version I'd've
>    had to modify the makefile to process binaries before they were used.

Yeah, this can be a pain.  If you are trying to build a large number of 
programs (eg bootstrapping a complete netbsd/linux tree), then I really 
think there is no substitute for getting a fixed chip.

> 
> I'm not sure what the best solution is short of replacing the buggy CPU
> (which may not be possible without replacing the whole RiscPC with something
> else).  If the source is available I'm happier with modify the assembler or
> object files rather than binary patching.  Matthew Wilcox suggested to me
> that one could consider arranging to optionally compile the kernel to
> check whether a page finishes with a problem instruction, and if so lock
> that page to the next so that the second is always paged in if the first
> is paged in.  This too is messy (?This is too messy?), as you then have to
> check the subsequent page, which could make a chain, and you'd need a limit
> to stop a denial of service using a pathological binary.  Messy as it may be,
> it would allow a K processor to use unmodified binaries as is.

I don't like that particular idea either.  According to the errata 
document that Digital published, it is supposedly possible to make a 
VM-based system work by avoiding certain combinations of permission bits 
(you will have to read the document itself for the details -- I can't 
remember them).  I've yet to hear a convincing argument as to why this 
can't be done (other than "I can't be bothered").  However, I can't really 
complain, since I don't know enough about the vm code implementation to 
start fixing them.

> 
> I'm not sure if I like that idea either.  Because I'm trying to ensure that
> I've got source code for everything I'm running, personally I'm quite happy
> with a private, modified toolchain.  Hacking the kernel (Linux or BSD) appears
> to me (an outsider) like more work than obtaining a replacement CPU.
> 
> I'm being bitten by the K bug, enough to be irritated into doing something
> about it.  As it seems I'm currently pointing in the wrong direction, could
> someone suggest a more profitable avenue I should be following?
> 
> Nick
>