Re: port-macppc/54827: alloca() is broken in gcc 8 on powerpc

[Valery Ushakov <uwe%stderr.spb.ru@localhost>]

The following reply was made to PR port-macppc/54827; it has been noted by GNATS.

From: Valery Ushakov <uwe%stderr.spb.ru@localhost>
To: gnats-bugs%netbsd.org@localhost
Cc: 
Subject: Re: port-macppc/54827: alloca() is broken in gcc 8 on powerpc
Date: Fri, 3 Jan 2020 02:23:47 +0300

 When gdb starts up it registers its commands.  Among others:
 
   c = add_com_alias ("save-tracepoints", "save tracepoints", class_trace, 0);
 
 It parses "save tracepoints", finds "save", and finds its command
 table.  Then it needs to lookup "tracepoints" in that command table,
 and that ahppens in external/gpl3/gdb/dist/gdb/cli/cli-decode.c in
 lookup_cmd_1().  To perform the lookup it creates a copy of the
 command:
 
   command = (char *) alloca (len + 1);
   memcpy (command, *text, len);
   command[len] = '\0';
 
 and here we have:
 
 (gdb) p command
 $68 = 0xffffe1a0 "tracepoints"
 
 but then in the next line:
 
   /* Look it up.  */
   found = 0;
 
 the copied value in command is corrupted:
 
 (gdb) p command
 $84 = 0xffffe1a0 "tracepoi"
 
 
 That alloca call is compiled to:
 
 => <lookup_cmd_1+152>:      lwz     r9,16(r31)      # len
    <lookup_cmd_1+156>:      addi    r9,r9,1
    <lookup_cmd_1+160>:      addi    r9,r9,15
    <lookup_cmd_1+164>:      rlwinm  r9,r9,28,4,31
    <lookup_cmd_1+168>:      rlwinm  r9,r9,4,0,27    # len16 = round up to 16
    <lookup_cmd_1+172>:      lwz     r10,0(r1)	    # r10 = back chain
    <lookup_cmd_1+176>:      neg     r9,r9
    <lookup_cmd_1+180>:      stwux   r10,r1,r9	    # sp -= len16, store back chain
    <lookup_cmd_1+184>:      addi    r9,r1,8	    # skip back chain, LR slot
    <lookup_cmd_1+188>:      addi    r9,r9,15
    <lookup_cmd_1+192>:      rlwinm  r9,r9,28,4,31
    <lookup_cmd_1+196>:      rlwinm  r9,r9,4,0,27    # round up to 16
    <lookup_cmd_1+200>:      stw     r9,20(r31)	    # command = ...
 
 and then later found = 0 is compiled to:
 
 => <lookup_cmd_1+252>:      li      r9,0
    <lookup_cmd_1+256>:      stw     r9,8(r31)	    # <- overwrites!
 
 
 We start with %sp = %r31 = 0xffffe1a0.
 
 Length of "tracepoints" is 11, so alloca code pushes %sp 16 bytes,
 from 0xffffe1a0 to 0xffffe190.  The 8 bytes just below %sp are the
 back chain and the callee's LR slot, so it skips them.  Then it rounds
 up the result to 16 bytes and command gets the value 0xffffe1a0.  But
 11 bytes starting from that address are overlapping local variables
 that start at 8(%r31) (where 8 is the same offset that covers back
 chain and LR slot).
 
 
 Consider this minimized example, alloca-ppc.c:
 
 void bar(char *, int *);
 
 void
 foo()
 {
     char *p;
     int i;
 
     p = __builtin_alloca(11);
     i = 0;
     bar(p, &i);
 }
 
 
 $ powerpc--netbsd-gcc-7.4.0 -O0 -S -o alloca-ppc-7.s alloca-ppc.c 
 $ powerpc--netbsd-gcc-8.3.0 -O0 -S -o alloca-ppc-8.s alloca-ppc.c 
 $ diff -u -I ident alloca-ppc-7.s alloca-ppc-8.s 
 --- alloca-ppc-7.s 2020-01-03 00:30:37.304541529 +0300
 +++ alloca-ppc-8.s 2020-01-03 00:30:51.910782364 +0300
 @@ -17,7 +17,7 @@
    mr 31,1
    .cfi_def_cfa_register 31
    lwz 9,0(1)
 -  stwu 9,-32(1)
 +  stwu 9,-16(1)
    addi 9,1,8
    addi 9,9,15
    srwi 9,9,4
 
 
 Here gcc7 correctly accounts for 8 bytes it's going to skip later and
 moves %sp by roundup16(11 + 8) = 32 bytes.  OTOH, gcc8 doesn't take
 those 8 bytes into account and moves %sp only by roundup16(11) = 16
 bytes, creating an overlap.
 
 STARTING_FRAME_OFFSET and STACK_DYNAMIC_OFFSET in rs60000.h only got
 cosmetic changes between 7 and 8 imports in our tree, it seems.
 
 -uwe