Subject: dereferencing type-punned pointer will break strict-aliasing rules
To: None <tech-toolchain@NetBSD.org>
From: Bang Jun-Young <junyoung@mogua.com>
List: tech-toolchain
Date: 08/04/2004 10:32:11
Hi,

With today's netbsd-2-0 sources, usr.sbin/racoon/racoon can't be built
with -O{2,3,s}:

/u0/netbsd-2-0/src/crypto/dist/kame/racoon/crypto_openssl.c: In function `eay_str2asn1dn':
/u0/netbsd-2-0/src/crypto/dist/kame/racoon/crypto_openssl.c:186: warning: dereferencing type-punned pointer will break strict-aliasing rules
/u0/netbsd-2-0/src/crypto/dist/kame/racoon/crypto_openssl.c: In function `eay_cmp_asn1dn':
/u0/netbsd-2-0/src/crypto/dist/kame/racoon/crypto_openssl.c:215: warning: dereferencing type-punned pointer will break strict-aliasing rules
/u0/netbsd-2-0/src/crypto/dist/kame/racoon/crypto_openssl.c:218: warning: dereferencing type-punned pointer will break strict-aliasing rules

This can be fixed in three ways:

1) Add a flag to our own Makefile:

Index: Makefile
===================================================================
RCS file: /cvsroot/src/usr.sbin/racoon/racoon/Makefile,v
retrieving revision 1.19.2.2
diff -u -r1.19.2.2 Makefile
--- Makefile	17 Jun 2004 12:38:10 -0000	1.19.2.2
+++ Makefile	4 Aug 2004 01:18:41 -0000
@@ -38,6 +38,7 @@
 DPADD+=	${LIBIPSEC} ${LIBY} ${LIBL} ${LIBCRYPTO}
 
 #CFLAGS+=-g
+CFLAGS+=-fno-strict-aliasing
 OPTFLAG+=-DHAVE_PFKEYV2 -DYIPS_DEBUG
 CPPFLAGS+=-DINET6 -DHAVE_FUNCTION_MACRO=1 -DHAVE_LIBCRYPTO=1 -DHAVE_LIBL=1 -DHAVE_LIBY=1 -DENABLE_IPV6=1 -DADVAPI=1 -DHAVE_GETADDRINFO=1 -DHAVE_GETNAMEINFO=1 -DSTDC_HEADERS=1 -DHAVE_SYS_WAIT_H=1 -DHAVE_LIMITS_H=1 -DHAVE_SYS_TIME_H=1 -DHAVE_UNISTD_H=1 -DHAVE_STDARG_H=1 -DHAVE_VARARGS_H=1 -DTIME_WITH_SYS_TIME=1 -DRETSIGTYPE=void -DHAVE_VPRINTF=1 -DHAVE_GETTIMEOFDAY=1 -DHAVE_SELECT=1 -DHAVE_SOCKET=1 -DHAVE_STRERROR=1 -DHAVE_STRTOL=1 -DHAVE_STRTOUL=1 -DHAVE_STRDUP=1 -DHAVE_GETIFADDRS=1 -DHAVE_ARC4RANDOM=1 $(OPTFLAG) -DIPSEC  -I. -I${RACOONSRCDIR} -DSYSCONFDIR=\"$(ETCDIR)\"
 CPPFLAGS+=-DHAVE_OPENSSL_OPENSSLV_H=1

2) Fix crypto_openssl.c:

Index: crypto_openssl.c
===================================================================
RCS file: /cvsroot/src/crypto/dist/kame/racoon/crypto_openssl.c,v
retrieving revision 1.12.2.1
diff -u -r1.12.2.1 crypto_openssl.c
--- crypto_openssl.c	17 Jun 2004 12:38:09 -0000	1.12.2.1
+++ crypto_openssl.c	2 Aug 2004 07:39:16 -0000
@@ -133,7 +133,7 @@
 	char *field, *value;
 	int i, j;
 	vchar_t *ret;
-	caddr_t p;
+	unsigned char *p;
 
 	buf = racoon_malloc(len + 1);
 	if (!buf) {
@@ -183,7 +183,7 @@
 	if (!ret)
 		goto err;
 	p = ret->v;
-	i = i2d_X509_NAME(name, (unsigned char **)&p);
+	i = i2d_X509_NAME(name, &p);
 	if (!i)
 		goto err;
 
@@ -208,14 +208,14 @@
 	vchar_t *n1, *n2;
 {
 	X509_NAME *a = NULL, *b = NULL;
-	caddr_t p;
+	unsigned char *p;
 	int i = -1;
 
 	p = n1->v;
-	if (!d2i_X509_NAME(&a, (unsigned char **)&p, n1->l))
+	if (!d2i_X509_NAME(&a, &p, n1->l))
 		goto end;
 	p = n2->v;
-	if (!d2i_X509_NAME(&b, (unsigned char **)&p, n2->l))
+	if (!d2i_X509_NAME(&b, &p, n2->l))
 		goto end;
 
 	i = X509_NAME_cmp(a, b);

3) Modify crypto_openssl.c in other way than 2) using union.
Although this is a recommended method in other projects, IMHO it makes
code quite ugly...

Which is better? I'm leaning towards 1) since that way we can keep local
changes to kame minimal.

Jun-Young

P.S. This is the same problem as noted in lib/26516.