Current-Users archive
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]
Re: Debugging Epiphany/Midori (webkit-gtk based) on earmv6hf (RPI 2)
Indeed it looks like g_alloca() is broken in this case. It is an
inline function which is supposed to allocate space on the stack (so
my previous suspicion was partly wrong).
(gdb) x/30i g_dpgettext2
0x46213fe4 <g_dpgettext2>: mov r12, sp
0x46213fe8 <g_dpgettext2+4>: push {r4, r5, r6, r7, r8, r9, r10,
r11, r12, lr, pc}
0x46213fec <g_dpgettext2+8>: sub r11, r12, #4
0x46213ff0 <g_dpgettext2+12>: sub sp, sp, #12
0x46213ff4 <g_dpgettext2+16>: mov r9, r0
0x46213ff8 <g_dpgettext2+20>: mov r0, r1
0x46213ffc <g_dpgettext2+24>: mov r6, r2
0x46214000 <g_dpgettext2+28>: str r1, [r11, #-48] ; 0x30
0x46214004 <g_dpgettext2+32>: bl 0x461f4024
0x46214008 <g_dpgettext2+36>: mov r5, r0
0x4621400c <g_dpgettext2+40>: mov r0, r6
0x46214010 <g_dpgettext2+44>: bl 0x461f4024
0x46214014 <g_dpgettext2+48>: add r10, r5, #1
0x46214018 <g_dpgettext2+52>: add r8, r0, #1
0x4621401c <g_dpgettext2+56>: add r3, r8, r10
0x46214020 <g_dpgettext2+60>: add r3, r3, #14 ==> huh (alignment
kind of thing)?
0x46214024 <g_dpgettext2+64>: bic r3, r3, #7 ==> huh?
0x46214028 <g_dpgettext2+68>: sub sp, sp, r3 ===> allocate space
IMHO it would be valid to pass the value of sp as the first argument
(mov r0, sp). However, the way how r0 is computed looks quite
obfuscated to me:
0x4621402c <g_dpgettext2+72>: mov r3, sp => ?
0x46214030 <g_dpgettext2+76>: lsr r7, r3, #3 => ?
0x46214034 <g_dpgettext2+80>: lsl r4, r7, #3 => ?
=> 0x46214038 <g_dpgettext2+84>: ldr r1, [r11, #-48] ; 0x30
0x4621403c <g_dpgettext2+88>: mov r2, r5
0x46214040 <g_dpgettext2+92>: mov r0, r4 => ?
0x46214044 <g_dpgettext2+96>: bl 0x461f3fdc
0x46214048 <g_dpgettext2+100>: mov r3, #4
0x4621404c <g_dpgettext2+104>: mov r2, r8
0x46214050 <g_dpgettext2+108>: strb r3, [r5, r7, lsl #3]
0x46214054 <g_dpgettext2+112>: mov r1, r6
0x46214058 <g_dpgettext2+116>: add r0, r4, r10
I changed the value of r0 to equal sp before memcpy() was being called
and it worked fine then.
(gdb) set $r0 = 0x7fffa3c4
...
0x45e81044 in memcpy () from /usr/lib/libc.so.12
(gdb)
0x45e81048 in memcpy () from /usr/lib/libc.so.12
(gdb)
0x45e8104c in memcpy () from /usr/lib/libc.so.12
(gdb)
0x46214048 in g_dpgettext2 () from /usr/pkg/lib/libglib-2.0.so.0
(gdb)
0x4621404c in g_dpgettext2 () from /usr/pkg/lib/libglib-2.0.so.0
2015-10-08 15:36 GMT+00:00 Stephan <stephanwib%googlemail.com@localhost>:
> One more piece of this puzzle:
>
> The destination pointer passed to memcpy() is definitely invalid.
>
> (gdb) stepi
> 0x461f3fe0 in ?? () from /usr/pkg/lib/libglib-2.0.so.0
> (gdb)
> 0x461f3fe4 in ?? () from /usr/pkg/lib/libglib-2.0.so.0
> (gdb)
> 0x45e80f98 in memcpy () from /usr/lib/libc.so.12
> (gdb) i r
> r0 0x7fffa3c0 2147460032 ==> first arg, destination ptr
> r1 0x42cb0e84 1120603780 ==> second arg, source ptr
> r2 0xb 11 ==> third arg, length
> r3 0x7fffa3c4 2147460036
> r4 0x7fffa3c0 2147460032
> r5 0xb 11
> r6 0x42cb0f74 1120604020
> r7 0xffff478 268432504
> r8 0x8 8
> r9 0x42c11b90 1119951760
> r10 0xc 12
> r11 0x7fffa418 2147460120
> r12 0x462f5b48 1177508680
> sp 0x7fffa3c4 0x7fffa3c4 ==> top element of
> g_dpgettext2()´s stack
> lr 0x46214048 1176584264
> pc 0x45e80f98 0x45e80f98 <memcpy>
> cpsr 0x20030010 537067536
>
>
> As you can see, the destination buffer starts 1 Byte below the top
> element of the callers stack. That is where the link register happens
> to be saved by the callee, leading to the pc corruption later on (even
> the callers frame would be wasted in this scenario anyway).
>
> 2015-10-06 15:07 GMT+00:00 Stephan <stephanwib%googlemail.com@localhost>:
>> Folks,
>>
>> more on this topic:
>>
>> This
>>
>> ---------8<--------------
>> (gdb) x/5i 0x45fd3efc
>> 0x45fd3efc: add r12, pc, #1048576 ; 0x100000
>> 0x45fd3f00: add r12, r12, #4096 ; 0x1000
>> 0x45fd3f04: ldr pc, [r12, #1652]! ; 0x674
>> 0x45fd3f08: add r12, pc, #1048576 ; 0x100000
>> 0x45fd3f0c: add r12, r12, #4096 ; 0x1000
>> ---------------------------
>>
>> is not garbage. It's kind of a link table (plt) inside of glib2. It
>> makes the first 2 calls in g_dpgettext2() go to strlen() and the third
>> go to memcpy(). memcpy() saves the r0 and lr regisers at the very
>> beginning and pops them into r0 and pc at a later time in order to
>> return to the caller. However somewhere in the function, the stack
>> data seems to become corrupted and a junk value is loaded into pc,
>> leading to the crash. Either its value was overwritten or sp is
>> pointing to the wrong place.
>>
>>
>> (gdb) x/50i 0x45e80f98
>> 0x45e80f98 <memcpy>: push {r0, lr} =====> SAVE REGISTERS
>> 0x45e80f9c <memcpy+4>: subs r2, r2, #4
>> 0x45e80fa0 <memcpy+8>: blt 0x45e81028 <memcpy+144>
>> 0x45e80fa4 <memcpy+12>: ands r12, r0, #3
>> 0x45e80fa8 <memcpy+16>: bne 0x45e81050 <memcpy+184>
>> 0x45e80fac <memcpy+20>: ands r12, r1, #3
>> 0x45e80fb0 <memcpy+24>: bne 0x45e81080 <memcpy+232>
>> 0x45e80fb4 <memcpy+28>: subs r2, r2, #8
>> 0x45e80fb8 <memcpy+32>: blt 0x45e81008 <memcpy+112>
>> 0x45e80fbc <memcpy+36>: subs r2, r2, #20
>> 0x45e80fc0 <memcpy+40>: blt 0x45e80ff4 <memcpy+92>
>> 0x45e80fc4 <memcpy+44>: push {r4} ; (str r4, [sp, #-4]!)
>> 0x45e80fc8 <memcpy+48>: ldm r1!, {r3, r4, r12, lr}
>> 0x45e80fcc <memcpy+52>: stmia r0!, {r3, r4, r12, lr}
>> 0x45e80fd0 <memcpy+56>: ldm r1!, {r3, r4, r12, lr}
>> 0x45e80fd4 <memcpy+60>: stmia r0!, {r3, r4, r12, lr}
>> 0x45e80fd8 <memcpy+64>: subs r2, r2, #32
>> 0x45e80fdc <memcpy+68>: bge 0x45e80fc8 <memcpy+48>
>> 0x45e80fe0 <memcpy+72>: cmn r2, #16
>> 0x45e80fe4 <memcpy+76>: ldmge r1!, {r3, r4, r12, lr}
>> 0x45e80fe8 <memcpy+80>: stmiage r0!, {r3, r4, r12, lr}
>> 0x45e80fec <memcpy+84>: subge r2, r2, #16
>> 0x45e80ff0 <memcpy+88>: pop {r4} ; (ldr r4, [sp], #4)
>> 0x45e80ff4 <memcpy+92>: adds r2, r2, #20
>> 0x45e80ff8 <memcpy+96>: ldmge r1!, {r3, r12, lr}
>> 0x45e80ffc <memcpy+100>: stmiage r0!, {r3, r12, lr}
>> 0x45e81000 <memcpy+104>: subsge r2, r2, #12
>> 0x45e81004 <memcpy+108>: bge 0x45e80ff8 <memcpy+96>
>> 0x45e81008 <memcpy+112>: adds r2, r2, #8
>> 0x45e8100c <memcpy+116>: blt 0x45e81028 <memcpy+144>
>> 0x45e81010 <memcpy+120>: subs r2, r2, #4
>> 0x45e81014 <memcpy+124>: ldrlt r3, [r1], #4
>> 0x45e81018 <memcpy+128>: strlt r3, [r0], #4
>> 0x45e8101c <memcpy+132>: ldmge r1!, {r3, r12}
>> 0x45e81020 <memcpy+136>: stmiage r0!, {r3, r12}
>> 0x45e81024 <memcpy+140>: subge r2, r2, #4
>> 0x45e81028 <memcpy+144>: adds r2, r2, #4
>> 0x45e8102c <memcpy+148>: popeq {r0, pc}
>> 0x45e81030 <memcpy+152>: cmp r2, #2
>> 0x45e81034 <memcpy+156>: ldrb r3, [r1], #1
>> 0x45e81038 <memcpy+160>: strb r3, [r0], #1
>> 0x45e8103c <memcpy+164>: ldrbge r3, [r1], #1
>> 0x45e81040 <memcpy+168>: strbge r3, [r0], #1
>> 0x45e81044 <memcpy+172>: ldrbgt r3, [r1], #1
>> 0x45e81048 <memcpy+176>: strbgt r3, [r0], #1
>> => 0x45e8104c <memcpy+180>: pop {r0, pc} ======> CRASH: PC
>> gets corrupted value
>> 0x45e81050 <memcpy+184>: rsb r12, r12, #4
>>
>> Further investigation is to come ;)
>>
>> 2015-05-31 19:05 GMT+00:00 Jared McNeill <jmcneill%invisible.ca@localhost>:
>>> I saw the same issue a while back, don't remember exactly which version of
>>> glib2 it was. I LD_PRELOAD'd a version of g_dpgettext2 that always returned
>>> NULL and it made the crashes go away. g_dpgettext2 uses alloca internally,
>>> so I increased stack size and the crashes went away again.
>>>
>>> https://git.gnome.org/browse/glib/tree/glib/ggettext.c#n271
>>>
>>>
>>> On Sun, 31 May 2015, Stephan wrote:
>>>
>>>>
>>>> That does not help unfortunately - just tested again with ulimit -s
>>>> unlimited. I dont think we are hitting the stack boundary
>>>> because sp is still inside the stack region outlined by pmap. Have you
>>>> seen the very same issue as I do?
>>>>
>>>> Am 31.05.2015 19:48 schrieb "Jared McNeill" <jmcneill%invisible.ca@localhost>:
>>>> Default stack size limits on evbarm are too low for webkit-gtk. Bump
>>>> it up with ulimit -s and the g_dpgettext2
>>>> problem should go away.
>>>>
>>>>
>>>>
>>>> On Sun, 31 May 2015, Stephan wrote:
>>>>
>>>> Hi folks,
>>>>
>>>> I am currently testing some applications on the RPI 2. Some
>>>> work
>>>> pretty well, others not yet. As for webkit-gtk based browsers,
>>>> I am
>>>> experiencing crashes from time to time.
>>>>
>>>> One problem that occurs often seems to be related to
>>>> g_dpgettext2 ()
>>>> from glib2. The top of the stack looks like this:
>>>>
>>>> (gdb) bt
>>>> #0 0x636f7452 in ?? ()
>>>> #1 0x45ff3fa8 in g_dpgettext2 () from
>>>> /usr/pkg/lib/libglib-2.0.so.0
>>>> #2 0x42ad6030 in gtk_stock_lookup () from
>>>> /usr/pkg/lib/libgtk-x11-2.0.so.0
>>>> #3 0x42987b98 in gtk_action_set_stock_id () from
>>>> /usr/pkg/lib/libgtk-x11-2.0.so.0
>>>> #4 0x45f55cfc in g_object_set_valist () from
>>>> /usr/pkg/lib/libgobject-2.0.so.0
>>>> #5 0x45f5642c in g_object_set () from
>>>> /usr/pkg/lib/libgobject-2.0.so.0
>>>> #6 0x4298a27c in gtk_action_group_add_actions_full () from
>>>> /usr/pkg/lib/libgtk-x11-2.0.so.0
>>>> #7 0x4298a388 in gtk_action_group_add_actions () from
>>>> /usr/pkg/lib/libgtk-x11-2.0.so.0
>>>> #8 0x0004238c in ?? ()
>>>> #9 0x45f5322c in g_object_new_internal () from
>>>> /usr/pkg/lib/libgobject-2.0.so.0
>>>> #10 0x45f5587c in g_object_new_valist () from
>>>> /usr/pkg/lib/libgobject-2.0.so.0
>>>> #11 0x45f55a24 in g_object_new () from
>>>> /usr/pkg/lib/libgobject-2.0.so.0
>>>> #12 0x00043ff0 in ephy_window_new_with_chrome ()
>>>> #13 0x0003ac94 in ephy_shell_new_tab_full ()
>>>> #14 0x0003f81c in ?? ()
>>>> #15 0x40a090e8 in webkit_marshal_OBJECT__OBJECT () from
>>>> /usr/pkg/lib/libwebkitgtk-1.0.so.0
>>>> #16 0x45f4e070 in g_closure_invoke () from
>>>> /usr/pkg/lib/libgobject-2.0.so.0
>>>> #17 0x45f6154c in signal_emit_unlocked_R () from
>>>> /usr/pkg/lib/libgobject-2.0.so.0
>>>> #18 0x45f69278 in g_signal_emit_valist () from
>>>> /usr/pkg/lib/libgobject-2.0.so.0
>>>> #19 0x45f69cac in g_signal_emit_by_name () from
>>>> /usr/pkg/lib/libgobject-2.0.so.0
>>>> #20 0x409d9074 in
>>>>
>>>> WebKit::FrameLoaderClient::dispatchCreatePage(WebCore::NavigationAction
>>>> const&) () from /usr/pkg/lib/libwebkitgtk-1.0.so.0
>>>> ...
>>>>
>>>> The call to the topmost frame happens from 0x45ff3fa4 in the
>>>> said
>>>> funtion. The disassembly from within gdb loos like this:
>>>>
>>>> (gdb) disass g_dpgettext2
>>>> Dump of assembler code for function g_dpgettext2:
>>>> 0x45ff3f44 <+0>: mov r12, sp
>>>> 0x45ff3f48 <+4>: push {r4, r5, r6, r7, r8, r9, r10,
>>>> r11, r12, lr, pc}
>>>> 0x45ff3f4c <+8>: sub r11, r12, #4
>>>> 0x45ff3f50 <+12>: sub sp, sp, #12
>>>> 0x45ff3f54 <+16>: mov r9, r0
>>>> 0x45ff3f58 <+20>: mov r0, r1
>>>> 0x45ff3f5c <+24>: mov r6, r2
>>>> 0x45ff3f60 <+28>: str r1, [r11, #-48] ; 0x30
>>>> 0x45ff3f64 <+32>: bl 0x45fd3f44
>>>> 0x45ff3f68 <+36>: mov r5, r0
>>>> 0x45ff3f6c <+40>: mov r0, r6
>>>> 0x45ff3f70 <+44>: bl 0x45fd3f44
>>>> 0x45ff3f74 <+48>: add r10, r5, #1
>>>> 0x45ff3f78 <+52>: add r8, r0, #1
>>>> 0x45ff3f7c <+56>: add r3, r8, r10
>>>> 0x45ff3f80 <+60>: add r3, r3, #14
>>>> 0x45ff3f84 <+64>: bic r3, r3, #7
>>>> 0x45ff3f88 <+68>: sub sp, sp, r3
>>>> 0x45ff3f8c <+72>: mov r3, sp
>>>> 0x45ff3f90 <+76>: lsr r7, r3, #3
>>>> 0x45ff3f94 <+80>: lsl r4, r7, #3
>>>> 0x45ff3f98 <+84>: ldr r1, [r11, #-48] ; 0x30
>>>> 0x45ff3f9c <+88>: mov r2, r5
>>>> 0x45ff3fa0 <+92>: mov r0, r4
>>>> 0x45ff3fa4 <+96>: bl 0x45fd3efc
>>>> 0x45ff3fa8 <+100>: mov r3, #4
>>>> 0x45ff3fac <+104>: mov r2, r8
>>>> 0x45ff3fb0 <+108>: strb r3, [r5, r7, lsl #3]
>>>> ...
>>>>
>>>> Now the jump target actually seems to be some random memory:
>>>>
>>>> (gdb) info symbol 0x45fd3efc
>>>> No symbol matches 0x45fd3efc.
>>>>
>>>> At least it is mapped to the text section of libglib2. It
>>>> decodes like
>>>> follows which just looks like random garbage:
>>>>
>>>> (gdb) x/5i 0x45fd3efc
>>>> 0x45fd3efc: add r12, pc, #1048576 ; 0x100000
>>>> 0x45fd3f00: add r12, r12, #4096 ; 0x1000
>>>> 0x45fd3f04: ldr pc, [r12, #1652]! ; 0x674
>>>> 0x45fd3f08: add r12, pc, #1048576 ; 0x100000
>>>> 0x45fd3f0c: add r12, r12, #4096 ; 0x1000
>>>>
>>>> The disassembly of g_dpgettext2() from objdump looks weird as
>>>> well:
>>>>
>>>> 00033f44 <g_dpgettext2>:
>>>> 33f44: e1a0c00d mov ip, sp
>>>> 33f48: e92ddff0 push {r4, r5, r6, r7, r8,
>>>> r9, sl,
>>>> fp, ip, lr, pc}
>>>> 33f4c: e24cb004 sub fp, ip, #4
>>>> 33f50: e24dd00c sub sp, sp, #12
>>>> 33f54: e1a09000 mov r9, r0
>>>> 33f58: e1a00001 mov r0, r1
>>>> 33f5c: e1a06002 mov r6, r2
>>>> 33f60: e50b1030 str r1, [fp, #-48] ; 0x30
>>>> 33f64: ebff7ff6 bl 13f44
>>>> <g_mem_chunk_new-0x7c>
>>>> 33f68: e1a05000 mov r5, r0
>>>> 33f6c: e1a00006 mov r0, r6
>>>> 33f70: ebff7ff3 bl 13f44
>>>> <g_mem_chunk_new-0x7c>
>>>> 33f74: e285a001 add sl, r5, #1
>>>> 33f78: e2808001 add r8, r0, #1
>>>> 33f7c: e088300a add r3, r8, sl
>>>> 33f80: e283300e add r3, r3, #14
>>>> 33f84: e3c33007 bic r3, r3, #7
>>>> 33f88: e04dd003 sub sp, sp, r3
>>>> 33f8c: e1a0300d mov r3, sp
>>>> 33f90: e1a071a3 lsr r7, r3, #3
>>>> 33f94: e1a04187 lsl r4, r7, #3
>>>> 33f98: e51b1030 ldr r1, [fp, #-48] ; 0x30
>>>> 33f9c: e1a02005 mov r2, r5
>>>> 33fa0: e1a00004 mov r0, r4
>>>> 33fa4: ebff7fd4 bl 13efc
>>>> <g_mem_chunk_new-0xc4>
>>>> 33fa8: e3a03004 mov r3, #4
>>>> 33fac: e1a02008 mov r2, r8
>>>> 33fb0: e7c53187 strb r3, [r5, r7, lsl #3]
>>>> 33fb4: e1a01006 mov r1, r6
>>>> 33fb8: e084000a add r0, r4, sl
>>>> 33fbc: ebff7fce bl 13efc
>>>> <g_mem_chunk_new-0xc4>
>>>> 33fc0: e1a00009 mov r0, r9
>>>> 33fc4: e1a01004 mov r1, r4
>>>> 33fc8: ebffff8a bl 33df8 <g_dgettext>
>>>> 33fcc: e1500004 cmp r0, r4
>>>> 33fd0: e1a08000 mov r8, r0
>>>>
>>>> The jumps refer to negative function offsets (for example
>>>> <g_mem_chunk_new-0x7c>). Glibc (2.42.2) is from pkgsrc from
>>>> ftp.netbsd.org. I compiled an own one which has exactly the
>>>> same
>>>> issue.
>>>>
>>>> Ideas?
>>>>
>>>>
>>>>
>>>
Home |
Main Index |
Thread Index |
Old Index