Subject: port-alpha/10984: RSA_generate_key() on alpha hangs
To: None <gnats-bugs@gnats.netbsd.org>
From: Hubert Feyrer <feyrer@feyrer.de>
List: netbsd-bugs
Date: 09/10/2000 17:22:16
>Number:         10984
>Category:       port-alpha
>Synopsis:       RSA_generate_key() on alpha hangs
>Confidential:   no
>Severity:       critical
>Priority:       high
>Responsible:    port-alpha-maintainer
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Sun Sep 10 17:23:00 PDT 2000
>Closed-Date:
>Last-Modified:
>Originator:     Hubert Feyrer
>Release:        1.5E
>Organization:
	
>Environment:
	
System: NetBSD jumbo 1.5E NetBSD 1.5E (JUMBO) #3: Sat Aug 26 13:59:40 EDT 2000     root@jumbo:/r0/src/sys/arch/alpha/compile/JUMBO alpha



>Description:
	I just had a build of pkgsrc/security/ssh on a Alpha running -current
	stop, with top showing:

  PID USERNAME PRI NICE   SIZE   RES STATE     TIME   WCPU    CPU COMMAND
  23171 root     104   20    71M   72M run      20.3H 99.02% 99.02% conftest

	conftest was run by configure, conftest.c is:

--- snip ---
        #include <string.h>
        #include <openssl/rand.h>
        #include <openssl/rsa.h>
        #include <openssl/bn.h>
        #include <openssl/sha.h>
        int main(void)
        {
                int num; RSA *key; static unsigned char p_in[] = "blahblah";
                unsigned char c[256], p[256];
                memset(c, 0, sizeof(c)); RAND_add(c, sizeof(c), sizeof(c));
                if ((key=RSA_generate_key(512, 3, NULL, NULL))==NULL) return(1);
                num = RSA_public_encrypt(sizeof(p_in) - 1, p_in, c, key, RSA_PKCS1_PADDING);
                return(-1 == RSA_private_decrypt(num, c, p, key, RSA_PKCS1_PADDING));
        }
--- snip ---

	i'm not a guru on either alpha nor the whole crypto code, can you
	think of anything that could hang this? By putting in some printf()s,
	it seems the RSA_generate_key() call is the problem.

	This is on 1.5E/alpha using libs from /usr/lib (cc r.c -o r -lssl
	-lcrypto).  It does *not* happen on 1.5_ALPHA/i386.

	BTW, can't we teach libssl to pull in libcrypto automatically?


>How-To-Repeat:
	Compile the above program, run it. Wait. 
	(On my P133 it runs <5 seconds).

>Fix:
	Unknown.
>Release-Note:
>Audit-Trail:
>Unformatted: