port-amd64/44405: ddb on amd64 disassebler shows wrong register values

[Wolfgang.Stukenbrock%nagler-company.com@localhost]

>Number:         44405
>Category:       port-amd64
>Synopsis:       ddb on amd64 disassebler shows wrong register values
>Confidential:   no
>Severity:       non-critical
>Priority:       high
>Responsible:    port-amd64-maintainer
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Mon Jan 17 16:15:00 +0000 2011
>Originator:     Dr. W. Stukenbrock
>Release:        NetBSD 5.1
>Organization:
Dr. Nagler & Company GmbH
>Environment:
        
        
System: NetBSD test-s0 4.0 NetBSD 4.0 (NSW-WS) #0: Tue Aug 17 17:28:09 CEST 
2010 wgstuken@test-s0:/usr/src/sys/arch/amd64/compile/NSW-WS amd64
Architecture: x86_64
Machine: amd64
>Description:
        The disassembler of DDB for amd64 disassembles at least the cmp 
instruction in a wrong way.
        The sequence "48 3d 34 12 00 00" is disassebled to "cmpq    
$0x1234,%r8", but it should
        be "cmpq    $0x1234,%rax".
        The objdump utility does it right and produces "48 3d 34 12 00 00       
cmp    $0x1234,%rax".
        Due to the context of the function only "rax" makes sence, because "r8" 
is not used anywhere else.
>How-To-Repeat:
        Write a short function that compiles to this instruction and disasseble 
the object file
        with objdump and compare that with the ddb output. They differ in the 
register name.
>Fix:
        I'm 100% not shure, because I'm not realy confirmed with the Intel/Amd 
instruction set.

        The problem is located in "/usr/src/sys/arch/amd64/amd64/db_disasm.c".
        In the instruction table for "0x3d" is "op2(Is, A)" for the operands.
        The prefix 0x48 is loaded into the extention variable "rex" in line 
1235 and used to determine the
        size (quad-word in this case with REG_w (0x08)) in line 1280.
        In the operand switch for "A" in line 1412 the same check is used to 
determine the lower 8 register
        (rax ...) or the upper register (r8 - r15).

        I think the check should not be done with "REX_w" - "REX_x" should be 
used to select the upper
        registers.

        Someone who knows the Intel/Amd assembler much better than me should 
have a look at it.

        remark: the following instructions are affected on a change for "A" 
processing too:
          add    - 04, 05
          or     - 0c, 0d
          adc    - 14, 15
          ssb    - 1c, 1d
          and    - 24, 25
          sub    - 2c, 2d
          xor    - 34, 35
          xchg   - 91, 92, 93, 94, 95, 96, 97
          mov    - a0, a1
          test   - a8, a9
          in     - e4, e5, ed, ee
          out    - e6, e7, ee, ef

          mul, imul, div, idiv may be affected too - (Grp2 instructions)

>Unformatted: