Current-Users archive

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]

Re: Debugging Epiphany/Midori (webkit-gtk based) on earmv6hf (RPI 2)



Folks,

more on this topic:

This

---------8<--------------
(gdb) x/5i 0x45fd3efc
   0x45fd3efc:  add     r12, pc, #1048576       ; 0x100000
   0x45fd3f00:  add     r12, r12, #4096 ; 0x1000
   0x45fd3f04:  ldr     pc, [r12, #1652]!       ; 0x674
   0x45fd3f08:  add     r12, pc, #1048576       ; 0x100000
   0x45fd3f0c:  add     r12, r12, #4096 ; 0x1000
---------------------------

is not garbage. It's kind of a link table (plt) inside of glib2. It
makes the first 2 calls in g_dpgettext2() go to strlen() and the third
go to memcpy(). memcpy() saves the r0 and lr regisers at the very
beginning and pops them into r0 and pc at a later time in order to
return to the caller. However somewhere in the function, the stack
data seems to become corrupted and a junk value is loaded into pc,
leading to the crash. Either its value was overwritten or sp is
pointing to the wrong place.


(gdb) x/50i 0x45e80f98
   0x45e80f98 <memcpy>: push {r0, lr}        =====> SAVE REGISTERS
   0x45e80f9c <memcpy+4>: subs r2, r2, #4
   0x45e80fa0 <memcpy+8>: blt 0x45e81028 <memcpy+144>
   0x45e80fa4 <memcpy+12>: ands r12, r0, #3
   0x45e80fa8 <memcpy+16>: bne 0x45e81050 <memcpy+184>
   0x45e80fac <memcpy+20>: ands r12, r1, #3
   0x45e80fb0 <memcpy+24>: bne 0x45e81080 <memcpy+232>
   0x45e80fb4 <memcpy+28>: subs r2, r2, #8
   0x45e80fb8 <memcpy+32>: blt 0x45e81008 <memcpy+112>
   0x45e80fbc <memcpy+36>: subs r2, r2, #20
   0x45e80fc0 <memcpy+40>: blt 0x45e80ff4 <memcpy+92>
   0x45e80fc4 <memcpy+44>: push {r4} ; (str r4, [sp, #-4]!)
   0x45e80fc8 <memcpy+48>: ldm r1!, {r3, r4, r12, lr}
   0x45e80fcc <memcpy+52>: stmia r0!, {r3, r4, r12, lr}
   0x45e80fd0 <memcpy+56>: ldm r1!, {r3, r4, r12, lr}
   0x45e80fd4 <memcpy+60>: stmia r0!, {r3, r4, r12, lr}
   0x45e80fd8 <memcpy+64>: subs r2, r2, #32
   0x45e80fdc <memcpy+68>: bge 0x45e80fc8 <memcpy+48>
   0x45e80fe0 <memcpy+72>: cmn r2, #16
   0x45e80fe4 <memcpy+76>: ldmge r1!, {r3, r4, r12, lr}
   0x45e80fe8 <memcpy+80>: stmiage r0!, {r3, r4, r12, lr}
   0x45e80fec <memcpy+84>: subge r2, r2, #16
   0x45e80ff0 <memcpy+88>: pop {r4} ; (ldr r4, [sp], #4)
   0x45e80ff4 <memcpy+92>: adds r2, r2, #20
   0x45e80ff8 <memcpy+96>: ldmge r1!, {r3, r12, lr}
   0x45e80ffc <memcpy+100>: stmiage r0!, {r3, r12, lr}
   0x45e81000 <memcpy+104>: subsge r2, r2, #12
   0x45e81004 <memcpy+108>: bge 0x45e80ff8 <memcpy+96>
   0x45e81008 <memcpy+112>: adds r2, r2, #8
   0x45e8100c <memcpy+116>: blt 0x45e81028 <memcpy+144>
   0x45e81010 <memcpy+120>: subs r2, r2, #4
   0x45e81014 <memcpy+124>: ldrlt r3, [r1], #4
   0x45e81018 <memcpy+128>: strlt r3, [r0], #4
   0x45e8101c <memcpy+132>: ldmge r1!, {r3, r12}
   0x45e81020 <memcpy+136>: stmiage r0!, {r3, r12}
   0x45e81024 <memcpy+140>: subge r2, r2, #4
   0x45e81028 <memcpy+144>: adds r2, r2, #4
   0x45e8102c <memcpy+148>: popeq {r0, pc}
   0x45e81030 <memcpy+152>: cmp r2, #2
   0x45e81034 <memcpy+156>: ldrb r3, [r1], #1
   0x45e81038 <memcpy+160>: strb r3, [r0], #1
   0x45e8103c <memcpy+164>: ldrbge r3, [r1], #1
   0x45e81040 <memcpy+168>: strbge r3, [r0], #1
   0x45e81044 <memcpy+172>: ldrbgt r3, [r1], #1
   0x45e81048 <memcpy+176>: strbgt r3, [r0], #1
=> 0x45e8104c <memcpy+180>: pop {r0, pc}           ======> CRASH: PC
gets corrupted value
   0x45e81050 <memcpy+184>: rsb r12, r12, #4

Further investigation is to come ;)

2015-05-31 19:05 GMT+00:00 Jared McNeill <jmcneill%invisible.ca@localhost>:
> I saw the same issue a while back, don't remember exactly which version of
> glib2 it was. I LD_PRELOAD'd a version of g_dpgettext2 that always returned
> NULL and it made the crashes go away. g_dpgettext2 uses alloca internally,
> so I increased stack size and the crashes went away again.
>
> https://git.gnome.org/browse/glib/tree/glib/ggettext.c#n271
>
>
> On Sun, 31 May 2015, Stephan wrote:
>
>>
>> That does not help unfortunately - just tested again with ulimit -s
>> unlimited. I dont think we are hitting the stack boundary
>> because sp is still inside the stack region outlined by pmap. Have you
>> seen the very same issue as I do?
>>
>> Am 31.05.2015 19:48 schrieb "Jared McNeill" <jmcneill%invisible.ca@localhost>:
>>       Default stack size limits on evbarm are too low for webkit-gtk. Bump
>> it up with ulimit -s and the g_dpgettext2
>>       problem should go away.
>>
>>
>>
>>       On Sun, 31 May 2015, Stephan wrote:
>>
>>             Hi folks,
>>
>>             I am currently testing some applications on the RPI 2. Some
>> work
>>             pretty well, others not yet. As for webkit-gtk based browsers,
>> I am
>>             experiencing crashes from time to time.
>>
>>             One problem that occurs often seems to be related to
>> g_dpgettext2 ()
>>             from glib2. The top of the stack looks like this:
>>
>>             (gdb) bt
>>             #0  0x636f7452 in ?? ()
>>             #1  0x45ff3fa8 in g_dpgettext2 () from
>> /usr/pkg/lib/libglib-2.0.so.0
>>             #2  0x42ad6030 in gtk_stock_lookup () from
>> /usr/pkg/lib/libgtk-x11-2.0.so.0
>>             #3  0x42987b98 in gtk_action_set_stock_id () from
>>             /usr/pkg/lib/libgtk-x11-2.0.so.0
>>             #4  0x45f55cfc in g_object_set_valist () from
>> /usr/pkg/lib/libgobject-2.0.so.0
>>             #5  0x45f5642c in g_object_set () from
>> /usr/pkg/lib/libgobject-2.0.so.0
>>             #6  0x4298a27c in gtk_action_group_add_actions_full () from
>>             /usr/pkg/lib/libgtk-x11-2.0.so.0
>>             #7  0x4298a388 in gtk_action_group_add_actions () from
>>             /usr/pkg/lib/libgtk-x11-2.0.so.0
>>             #8  0x0004238c in ?? ()
>>             #9  0x45f5322c in g_object_new_internal () from
>> /usr/pkg/lib/libgobject-2.0.so.0
>>             #10 0x45f5587c in g_object_new_valist () from
>> /usr/pkg/lib/libgobject-2.0.so.0
>>             #11 0x45f55a24 in g_object_new () from
>> /usr/pkg/lib/libgobject-2.0.so.0
>>             #12 0x00043ff0 in ephy_window_new_with_chrome ()
>>             #13 0x0003ac94 in ephy_shell_new_tab_full ()
>>             #14 0x0003f81c in ?? ()
>>             #15 0x40a090e8 in webkit_marshal_OBJECT__OBJECT () from
>>             /usr/pkg/lib/libwebkitgtk-1.0.so.0
>>             #16 0x45f4e070 in g_closure_invoke () from
>> /usr/pkg/lib/libgobject-2.0.so.0
>>             #17 0x45f6154c in signal_emit_unlocked_R () from
>>             /usr/pkg/lib/libgobject-2.0.so.0
>>             #18 0x45f69278 in g_signal_emit_valist () from
>> /usr/pkg/lib/libgobject-2.0.so.0
>>             #19 0x45f69cac in g_signal_emit_by_name () from
>> /usr/pkg/lib/libgobject-2.0.so.0
>>             #20 0x409d9074 in
>>
>> WebKit::FrameLoaderClient::dispatchCreatePage(WebCore::NavigationAction
>>             const&) () from /usr/pkg/lib/libwebkitgtk-1.0.so.0
>>             ...
>>
>>             The call to the topmost frame happens from 0x45ff3fa4 in the
>> said
>>             funtion. The disassembly from within gdb loos like this:
>>
>>             (gdb) disass g_dpgettext2
>>             Dump of assembler code for function g_dpgettext2:
>>               0x45ff3f44 <+0>:     mov     r12, sp
>>               0x45ff3f48 <+4>:     push    {r4, r5, r6, r7, r8, r9, r10,
>> r11, r12, lr, pc}
>>               0x45ff3f4c <+8>:     sub     r11, r12, #4
>>               0x45ff3f50 <+12>:    sub     sp, sp, #12
>>               0x45ff3f54 <+16>:    mov     r9, r0
>>               0x45ff3f58 <+20>:    mov     r0, r1
>>               0x45ff3f5c <+24>:    mov     r6, r2
>>               0x45ff3f60 <+28>:    str     r1, [r11, #-48] ; 0x30
>>               0x45ff3f64 <+32>:    bl      0x45fd3f44
>>               0x45ff3f68 <+36>:    mov     r5, r0
>>               0x45ff3f6c <+40>:    mov     r0, r6
>>               0x45ff3f70 <+44>:    bl      0x45fd3f44
>>               0x45ff3f74 <+48>:    add     r10, r5, #1
>>               0x45ff3f78 <+52>:    add     r8, r0, #1
>>               0x45ff3f7c <+56>:    add     r3, r8, r10
>>               0x45ff3f80 <+60>:    add     r3, r3, #14
>>               0x45ff3f84 <+64>:    bic     r3, r3, #7
>>               0x45ff3f88 <+68>:    sub     sp, sp, r3
>>               0x45ff3f8c <+72>:    mov     r3, sp
>>               0x45ff3f90 <+76>:    lsr     r7, r3, #3
>>               0x45ff3f94 <+80>:    lsl     r4, r7, #3
>>               0x45ff3f98 <+84>:    ldr     r1, [r11, #-48] ; 0x30
>>               0x45ff3f9c <+88>:    mov     r2, r5
>>               0x45ff3fa0 <+92>:    mov     r0, r4
>>               0x45ff3fa4 <+96>:    bl      0x45fd3efc
>>               0x45ff3fa8 <+100>:   mov     r3, #4
>>               0x45ff3fac <+104>:   mov     r2, r8
>>               0x45ff3fb0 <+108>:   strb    r3, [r5, r7, lsl #3]
>>               ...
>>
>>             Now the jump target actually seems to be some random memory:
>>
>>             (gdb) info symbol 0x45fd3efc
>>             No symbol matches 0x45fd3efc.
>>
>>             At least it is mapped to the text section of libglib2. It
>> decodes like
>>             follows which just looks like random garbage:
>>
>>             (gdb) x/5i 0x45fd3efc
>>               0x45fd3efc:  add     r12, pc, #1048576       ; 0x100000
>>               0x45fd3f00:  add     r12, r12, #4096 ; 0x1000
>>               0x45fd3f04:  ldr     pc, [r12, #1652]!       ; 0x674
>>               0x45fd3f08:  add     r12, pc, #1048576       ; 0x100000
>>               0x45fd3f0c:  add     r12, r12, #4096 ; 0x1000
>>
>>             The disassembly of g_dpgettext2() from objdump looks weird as
>> well:
>>
>>             00033f44 <g_dpgettext2>:
>>               33f44:       e1a0c00d        mov     ip, sp
>>               33f48:       e92ddff0        push    {r4, r5, r6, r7, r8,
>> r9, sl,
>>             fp, ip, lr, pc}
>>               33f4c:       e24cb004        sub     fp, ip, #4
>>               33f50:       e24dd00c        sub     sp, sp, #12
>>               33f54:       e1a09000        mov     r9, r0
>>               33f58:       e1a00001        mov     r0, r1
>>               33f5c:       e1a06002        mov     r6, r2
>>               33f60:       e50b1030        str     r1, [fp, #-48]  ; 0x30
>>               33f64:       ebff7ff6        bl      13f44
>> <g_mem_chunk_new-0x7c>
>>               33f68:       e1a05000        mov     r5, r0
>>               33f6c:       e1a00006        mov     r0, r6
>>               33f70:       ebff7ff3        bl      13f44
>> <g_mem_chunk_new-0x7c>
>>               33f74:       e285a001        add     sl, r5, #1
>>               33f78:       e2808001        add     r8, r0, #1
>>               33f7c:       e088300a        add     r3, r8, sl
>>               33f80:       e283300e        add     r3, r3, #14
>>               33f84:       e3c33007        bic     r3, r3, #7
>>               33f88:       e04dd003        sub     sp, sp, r3
>>               33f8c:       e1a0300d        mov     r3, sp
>>               33f90:       e1a071a3        lsr     r7, r3, #3
>>               33f94:       e1a04187        lsl     r4, r7, #3
>>               33f98:       e51b1030        ldr     r1, [fp, #-48]  ; 0x30
>>               33f9c:       e1a02005        mov     r2, r5
>>               33fa0:       e1a00004        mov     r0, r4
>>               33fa4:       ebff7fd4        bl      13efc
>> <g_mem_chunk_new-0xc4>
>>               33fa8:       e3a03004        mov     r3, #4
>>               33fac:       e1a02008        mov     r2, r8
>>               33fb0:       e7c53187        strb    r3, [r5, r7, lsl #3]
>>               33fb4:       e1a01006        mov     r1, r6
>>               33fb8:       e084000a        add     r0, r4, sl
>>               33fbc:       ebff7fce        bl      13efc
>> <g_mem_chunk_new-0xc4>
>>               33fc0:       e1a00009        mov     r0, r9
>>               33fc4:       e1a01004        mov     r1, r4
>>               33fc8:       ebffff8a        bl      33df8 <g_dgettext>
>>               33fcc:       e1500004        cmp     r0, r4
>>               33fd0:       e1a08000        mov     r8, r0
>>
>>             The jumps refer to negative function offsets (for example
>>             <g_mem_chunk_new-0x7c>). Glibc (2.42.2) is from pkgsrc from
>>             ftp.netbsd.org. I compiled an own one which has exactly the
>> same
>>             issue.
>>
>>             Ideas?
>>
>>
>>
>


Home | Main Index | Thread Index | Old Index