Re: ASLR still problematic with -lpthread

[Piotr Meyer <aniou%smutek.pl@localhost>]

On Thu, Sep 09, 2010 at 10:10:01PM +0000, Christos Zoulas wrote:
 
> I think that the current code works purely by chance. If the stack
> base or the stack pointer happen to be in the wrong place, we'll
> simply get a core dump.  As it is, I have not seen any coredumps
> without having ASLR enabled. This code improves the situation,
> but it is unclear to me if it is ok to do:
> 
>       base = (void *)(pthread__sp() & pthread__threadmask);
> 
> on machines where the stack grows up.
> In the current scenario, we have 2 failure cases (no matter how the
> stack grows) without my hack:

[sorry for reanimating zombified thread, but now I have some spare time
for this]

I'm afraid that calculation is bad witch ASLR. With Your patch I have
less faults, but problems still exists. I made some tests and sometimes
I got sefgfaults (very, very rare) or "hangs" (app not works and waits,
probably due to stack problem). Typical case (::: marks my debug from
pthread__initmain() and +++ from pthread__id() - sp and va are values 
from va = sp & pthread__threadmask):

stack 0xbfc00000 d=0xbfe000 0xbf002000
applying to 0xbbbf8000 orig_addr=0x0 f=1002
result 0xabef9000
applying to 0xbbbff000 orig_addr=0x0 f=1
result 0xabf00000
applying to 0xbbbee000 orig_addr=0x0 f=2
result 0xabeef000
not applying to 0xabef6000 orig_addr=0xabef6000 f=12
not applying to 0xabef8000 orig_addr=0xabef8000 f=1012
applying to 0xbbbff000 orig_addr=0x0 f=1
result 0xabf00000
applying to 0xbbbf1000 orig_addr=0x0 f=2
result 0xabef2000
not applying to 0xabee4000 orig_addr=0xabee4000 f=12
not applying to 0xabee6000 orig_addr=0xabee6000 f=1012
applying to 0xbbbff000 orig_addr=0x0 f=1
result 0xabf00000
applying to 0xbbaf9000 orig_addr=0x0 f=2
result 0xabdfa000
not applying to 0xabec2000 orig_addr=0xabec2000 f=12
not applying to 0xabecb000 orig_addr=0xabecb000 f=1012

:::pthread__sp: 0xbefffdf8  base 0xbee00000  stack_size: 0x200000 

+++ sp: 0xbefffc98 va: 0xbee00000 pthread__mainbase: 0x0
+++ sp: 0xbefffc98 va: 0xbee00000 pthread__mainbase: 0x0
+++ sp: 0xbf000168 va: 0xbf000000 pthread__mainbase: 0x0
+++ sp: 0xbf000028 va: 0xbf000000 pthread__mainbase: 0x0
+++ sp: 0xbf000168 va: 0xbf000000 pthread__mainbase: 0x0
+++ sp: 0xbf000028 va: 0xbf000000 pthread__mainbase: 0x0
+++ sp: 0xbf000168 va: 0xbf000000 pthread__mainbase: 0x0
+++ sp: 0xbf000028 va: 0xbf000000 pthread__mainbase: 0x0
+++ sp: 0xbf000168 va: 0xbf000000 pthread__mainbase: 0x0
+++ sp: 0xbf000028 va: 0xbf000000 pthread__mainbase: 0x0
+++ sp: 0xbf000100 va: 0xbf000000 pthread__mainbase: 0x0
+++ sp: 0xbf000100 va: 0xbf000000 pthread__mainbase: 0x0
+++ sp: 0xbf0000f0 va: 0xbf000000 pthread__mainbase: 0x0
+++ sp: 0xbeffffb0 va: 0xbee00000 pthread__mainbase: 0x0
+++ sp: 0xbeffffb0 va: 0xbee00000 pthread__mainbase: 0x0
+++ sp: 0xbf0000f0 va: 0xbf000000 pthread__mainbase: 0x0
+++ sp: 0xbeffffb0 va: 0xbee00000 pthread__mainbase: 0x0
+++ sp: 0xbeffffb0 va: 0xbee00000 pthread__mainbase: 0x0
+++ sp: 0xbf0000f0 va: 0xbf000000 pthread__mainbase: 0x0
+++ sp: 0xbeffffb0 va: 0xbee00000 pthread__mainbase: 0x0
+++ sp: 0xbeffffb0 va: 0xbee00000 pthread__mainbase: 0x0
+++ sp: 0xbf000100 va: 0xbf000000 pthread__mainbase: 0x0
+++ sp: 0xbf0000f0 va: 0xbf000000 pthread__mainbase: 0x0
+++ sp: 0xbeffffb0 va: 0xbee00000 pthread__mainbase: 0x0
+++ sp: 0xbeffffb0 va: 0xbee00000 pthread__mainbase: 0x0
+++ sp: 0xbf000100 va: 0xbf000000 pthread__mainbase: 0x0
+++ sp: 0xbefff300 va: 0xbee00000 pthread__mainbase: 0x0
+++ sp: 0xbefff2e4 va: 0xbee00000 pthread__mainbase: 0x0
+++ sp: 0xbefff2d0 va: 0xbee00000 pthread__mainbase: 0x0
+++ sp: 0xbefff2d0 va: 0xbee00000 pthread__mainbase: 0x0
+++ sp: 0xbefff2e0 va: 0xbee00000 pthread__mainbase: 0x0
+++ sp: 0xbefff2e0 va: 0xbee00000 pthread__mainbase: 0x0
+++ sp: 0xbefff2e0 va: 0xbee00000 pthread__mainbase: 0x0
+++ sp: 0xbefff2e0 va: 0xbee00000 pthread__mainbase: 0x0
+++ sp: 0xbefff280 va: 0xbee00000 pthread__mainbase: 0x0
+++ sp: 0xbefff280 va: 0xbee00000 pthread__mainbase: 0x0
+++ sp: 0xbefff300 va: 0xbee00000 pthread__mainbase: 0x0
+++ sp: 0xbefff740 va: 0xbee00000 pthread__mainbase: 0x0
+++ sp: 0xbefff740 va: 0xbee00000 pthread__mainbase: 0x0
+++ sp: 0xbefff720 va: 0xbee00000 pthread__mainbase: 0x0
+++ sp: 0xbefff720 va: 0xbee00000 pthread__mainbase: 0x0
+++ sp: 0xbefff6a0 va: 0xbee00000 pthread__mainbase: 0x0
+++ sp: 0xbefff6a0 va: 0xbee00000 pthread__mainbase: 0x0
+++ sp: 0xbefff720 va: 0xbee00000 pthread__mainbase: 0x0
+++ sp: 0xbefff720 va: 0xbee00000 pthread__mainbase: 0x0
+++ sp: 0xbefff710 va: 0xbee00000 pthread__mainbase: 0x0
+++ sp: 0xbefff6b0 va: 0xbee00000 pthread__mainbase: 0x0
applying to 0xbbb00000 orig_addr=0x0 f=14001002
result 0xabe01000
+++ sp: 0xbefff6b0 va: 0xbee00000 pthread__mainbase: 0x0
+++ sp: 0xbefff710 va: 0xbee00000 pthread__mainbase: 0x0
+++ sp: 0xbefff750 va: 0xbee00000 pthread__mainbase: 0x0
+++ sp: 0xbefff750 va: 0xbee00000 pthread__mainbase: 0x0
+++ sp: 0xbefff740 va: 0xbee00000 pthread__mainbase: 0x0
+++ sp: 0xbefff600 va: 0xbee00000 pthread__mainbase: 0x0
+++ sp: 0xbefff600 va: 0xbee00000 pthread__mainbase: 0x0
+++ sp: 0xbefff740 va: 0xbee00000 pthread__mainbase: 0x0
+++ sp: 0xbefff5a0 va: 0xbee00000 pthread__mainbase: 0x0


pmap for this process gave me:

[...]
bbbff000-bbbfffff       4k 00000000 rw-p- (rwx) 1/0/0 00:00       0 -   [ anon ]
bbc00000-bedfffff   51200k 00000000 ---p+ (rwx) 1/0/0 00:00       0 -   [ stack 
]
bee00000-bee00fff       4k 00000000 rw-p- (rwx) 1/0/0 00:00       0 -   [ stack 
]
bee01000-bee01fff       4k 00000000 ---p- (rwx) 1/0/0 00:00       0 -   [ stack 
]
bee02000-befeffff    1976k 00000000 rw-p+ (rwx) 1/0/0 00:00       0 -   [ stack 
]
beff0000-beffffff      64k 00000000 rw-p- (rwx) 1/0/0 00:00       0 -   [ stack 
]
bf000000-bf001fff       8k 00000000 rw-p- (rwx) 1/0/0 00:00       0 -   [ stack 
]


So, I guess that problem is in default definition of pthread_id() on i386,
when stack pointer crosses boundary:

+++ sp: 0xbefffc98 va: 0xbee00000 pthread__mainbase: 0x0
+++ sp: 0xbf000168 va: 0xbf000000 pthread__mainbase: 0x0

...and after binary AND we have two different regions as stack base for
single stack.

When I define PTHREAD__HAVE_THREADREG I got, with Christos patch, working
system (I execute this simple program about 10 million times without single
failure) but impact of this change for rest of system is unclear for me.

-- 
Piotr 'aniou' Meyer