port-arm32/50218: ARM executables linked with native linker on cortex-a9 raises SIGILL

[mzaidibrahime%gmail.com@localhost]

>Number:         50218
>Category:       port-arm32
>Synopsis:       ARM executables linked with native linker on cortex-a9 raises SIGILL
>Confidential:   no
>Severity:       critical
>Priority:       medium
>Responsible:    port-arm32-maintainer
>State:          open
>Class:          support
>Submitter-Id:   net
>Arrival-Date:   Mon Sep 07 15:20:00 +0000 2015
>Originator:     Zaid
>Release:        5.1
>Organization:
>Environment:
NetBSD 5.1_STABLE evbarm
>Description:
I am trying to do native compilation in ARM target(armv7-a). 

This target was built with with NetBSD5.1 kernel cross compiled and userland was created with Machine as evbarm and architecture as armv7eb in NetBSD5.1 source. 

         NetBSD version:   5.1
         MACHINE:          evbarm
         MACHINE_ARCH:     armv7eb
         Build platform:   Linux 2.6.18-194.32.1.el5PAE i686
         HOST_SH:          /bin/sh

 
Kernel and userland application like (ls, chmod,etc) which were cross-compiled executes successfully.

I have also built developer tool chain and installed on the target ARM machine. 

gcc version: 4.1.3
ld version: 2.6.1
as version 2.6.1

Using this version of developer tools on the target, I am trying to compile some C programs natively. I am facing SIGILL on executing the binaries. On analyzing the objs generated by the compiler, all the instructions generated are proper ARM instructions in Big Endian. 
However on linking, the executable created does not do the endianness swapping(be8 option of the linker) *completely*. I am seeing the mix of little endian and big endian instructions in the text section. I understand that all the instructions are supposed to swapped into little endian.

Small programs(<6K size) does not result in SIGILL as all the instruction are properly swapped for endianness by the linker.

Here is an extract of
Correct instruction in the executables:

   1594:	0dc0a0e1 	stceql	0, cr10, [r0, #900]
    1598:	00dc2de9 	sbceqs	r2, ip, r9, ror #27
    159c:	04b04ce2 	ldreqt	r4, [r0], #3298
    15a0:	10a09fe5 	adcne	r9, r0, r5, ror #31


Wrong instruction which are not swapped which results in SIGILLs

    15c0:	e1a0c00d 	mov	ip, sp
    15c4:	e92dd800 	stmdb	sp!, {fp, ip, lr, pc}
    15c8:	e24cb004 	sub	fp, ip, #4	; 0x4


Want to understand how to get the correct executable on the target. Cross compilation of the C files is not an option for me.
>How-To-Repeat:
Create userland(Cross compiled) for armv7eb with NetBSD5.1 and copy the same to the ARM target. chroot to the userland. 

1. create a big static/dynamic library. now link the library to a C program and execute the program
Commands that I have used
gcc -msoft-float -dynamic -mtune=cortex-a9 -g -c example-deconstructor.o example-deconstructor.c 

gcc -I../include -msoft-float -dynamic -mtune=cortex-a9 -g -o example-deconstructor example-deconstructor.o  ./libmyx.so.2

2. Compile C file with -static option and run the executable.

SIGILL will be seen
>Fix: