Re: bin/42158 (qemu: pthread + fork = hang)

[pooka%NetBSD.org@localhost]

Synopsis: qemu: pthread + fork = hang

State-Changed-From-To: open->analyzed
State-Changed-By: pooka%NetBSD.org@localhost
State-Changed-When: Fri, 09 Oct 2009 00:08:18 +0300
State-Changed-Why:
I analyzed this a bit:

  
(gdb) bt
#0  0xbbbdfe00 in _atomic_cas_ptr (ptr=0xbbbd4060, old=0x0, new=0xbfa00000)
    at /root/lib/libpthread/arch/i386/pthread_md.h:104
#1  0xbbbdfd7b in pthread_mutex_lock (ptm=0xbbbd4054) at pthread_mutex.c:156
#2  0xbbb7efb5 in __cxa_finalize () from /usr/lib/libc.so.12
#3  0xbbb7ef76 in exit () from /usr/lib/libc.so.12
#4  0x080484f6 in ___start ()
#5  0x08048447 in _start ()
(gdb) x/i $eip
0xbbbdfdfc <_atomic_cas_ptr+28>:        lock cmpxchg %ecx,(%esi)
(gdb) info register
eax            0x0      0
ecx            0xbfa00000       -1080033280
edx            0xbbbd4060       -1145225120
ebx            0xbbbe7144       -1145147068
esp            0xbfbfeaa8       0xbfbfeaa8
ebp            0xbfbfeb18       0xbfbfeb18
esi            0xbbbd4060       -1145225120
edi            0xbfbfeb6c       -1077941396
eip            0xbbbdfdfc       0xbbbdfdfc <_atomic_cas_ptr+28>
eflags         0x382    [ SF TF IF ]
cs             0x17     23
ss             0x1f     31
ds             0x1f     31
es             0x1f     31
fs             0x0      0
gs             0x0      0
(gdb) print *(int *)0xbbbd4060
$2 = 2
(gdb) stepi
0xbbbdfe00      104             __asm __volatile ("lock; cmpxchgl %2, %1"
(gdb) info register
eax            0x2      2
ecx            0xbfa00000       -1080033280
edx            0xbbbd4060       -1145225120
ebx            0xbbbe7144       -1145147068
esp            0xbfbfeaa8       0xbfbfeaa8
ebp            0xbfbfeb18       0xbfbfeb18
esi            0xbbbd4060       -1145225120
edi            0xbfbfeb6c       -1077941396
eip            0xbbbdfe00       0xbbbdfe00 <_atomic_cas_ptr+32>
eflags         0x346    [ PF ZF TF IF ]
cs             0x17     23
ss             0x1f     31
ds             0x1f     31
es             0x1f     31
fs             0x0      0
gs             0x0      0
(gdb)

mutex_enter(&atexit_mutex) in __cxa_finalize() works totally bonkers.
The atomic cas in the pthread mutex enter fastpath is succesful in
that it actually swaps the values.  Still, it returns the old value
of the mutex owner (which for a free recursive mutex is 0x2).  This
leads to an eventual spincycle in the slow path, since the owner
is now us and we are running.

Seems like cmpxchg fails to work properly in qemu in some circumstances.
Dunno if this is a qemu bug or NetBSD bug.  Note that ZF is set
after the instruction, so chompchomp really thinks it was doing
the right thing.

Also, it seems that usually this happens in the parent.  However, 
if the child exists before the parent, both processes fail in this
way.