[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]
gcc -O2 produces invalid object code (x86_64, netbsd-5 branch)
I've been debugging a strange libpam segfault on the netbsd-5 branch
(amd64) and I've concluded that gcc -O2 is incorrectly compiling the
pam_end() function in src/dist/openpam/lib/pam_end.c. gcc -O0 works fine.
It looks like lines 59 and 60 are simply skipped. Either that or they
are reordered below line 63. If pam_end() is passed a NULL pamh, it'll
segfault when it dereferences pamh at line 63.
I verified that this is a compiler problem by looking at the assembler
output. The following are the first few instructions emitted by gcc -O2
-S (see end of email for full command line):
# amd64 abi requires callee to preserve %r12
# set up call: _openpam_log(PAM_LOG_DEBUG, "pam_end", "enter")
# _openpam_log() is variadic, and the amd64 abi requires the
# caller to put the number of floating point args in %rax when
# calling a variadic function. there are no fp arguments in
# this case so set it to 0.
xorl %eax, %eax
# amd64 abi uses %rsi for the 2nd argument. save pam_end()'s
# 2nd argument (status) in %r12 since %rsi will be used to pass
# "pam_end" to _openpam_log().
movl %esi, %r12d
# amd64 abi uses %rdx for the 3rd argument. load the register
# with the string "enter"
movl $.LC0, %edx
# put "pam_end" in the 2nd argument register
movl $__PRETTY_FUNCTION__.2637, %esi
# amd64 abi requires callee to preserve %rbp
# amd64 abi uses %rdi for the 1st argument. save pam_end()'s
# 1st argument (pamh) in %rbp so that %rdi can be used for the
# call to _openpam_log().
movq %rdi, %rbp
# put PAM_LOG_DEBUG (which equals 0) in %rdi
xorl %edi, %edi
# %rbx will be used after the call to _openpam_log(), and the
# amd64 abi requires callee to preserve it
# execute _openpam_log(PAM_LOG_DEBUG, "pam_end", "enter")
# copy pamh->module_data to %rbx (the variable dp). the value
# 176 is offsetof(pam_handle_t, module_data). THIS RESULTS IN A
# SEGFAULT IF pamh is NULL!
movq 176(%rbp), %rbx
Notice that there is no check for NULL in the above assembly before pamh
is dereferenced, yet the 'if (pamh == NULL)' statement is in the source
code, clear as day.
Changing -O2 to -O0 produces correct object code.
I haven't tried any other version of gcc. netbsd-5's gcc is quite old,
so I wouldn't be surprised if the problem has already been fixed.
gcc command used to get assembler output:
/usr/obj//usr-5/tools/bin/x86_64--netbsd-gcc -Wall -Wstrict-prototypes
-Wmissing-prototypes -Wpointer-arith -Wno-sign-compare -Wno-traditional
-Wa,--fatal-warnings -Wreturn-type -Wswitch -Wshadow -Wcast-qual
-Wwrite-strings -Wextra -Wno-unused-parameter -Werror -fstack-protector
-Wstack-protector --param ssp-buffer-size=1 -ggdb -O2 -DLIB_MAJ=1
-DDEBUG=1 -DOPENPAM_DEBUG=1 -nostdinc -isystem
/usr/src/dist/openpam/lib/pam_end.c -o pam_end.s
Tools were initially built by running "BUILD-NetBSD" from the etcmanage
Main Index |
Thread Index |