port-sparc64/57472: OpenSSL broken, affects ecdsa keys in OpenSSH

[logix%foobar.franken.de@localhost]

>Number:         57472
>Category:       port-sparc64
>Synopsis:       OpenSSL broken, affects ecdsa keys in OpenSSH
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    port-sparc64-maintainer
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Sat Jun 17 10:00:00 +0000 2023
>Originator:     Harold Gutch
>Release:        NetBSD current
>Organization:
>Environment:
NetBSD abc 10.99.4 NetBSD 10.99.4 (GENERIC) #0: Thu Jun  8 21:41:13 CEST 2023  h@bld:/home/h/netbsd/202306080500Z/usr/obj.sparc64/sys/arch/sparc64/compile/GENERIC sparc64

>Description:
Recently sshd on sparc64 started no longer accepting existing ssh ecdsa keys of any kind (including previously generated keys), claiming that they are invalid.  At the same time, generating new ecdsa keys on sparc64 created keys that were not accepted on non-sparc64.  This seems to have started with the import of OpenSSL 3.0.8 (May 7)

The issue is in src/crypto/external/bsd/openssl/dist/crypto/ec/ec_curve.c
 where NID_X9_62_prime256v1 is defined (l. ~2965), and replacing the optimized method there with 0 makes it fall back to the generic method.  For other existing keys, the same issue exists with NID_secp521r1 (l. ~2940), and in principle (not directly affecting sshd), NID_secp224r1 (l. ~2925) also has this issue.

All of this might also affect ed25519.
>How-To-Repeat:
Download a recent sparc64 iso and boot it (e.g.) on qemu.  Observe that:

A) ssh
1) on the host
$ ssh-keygen -lf /etc/ssh/ssh_host_ecdsa_key.pub 
/etc/ssh/ssh_host_ecdsa_key.pub is not a public key file.

2) over the network

$ ssh-keyscan -p 60023 -t ecdsa 127.0.0.1
# 127.0.0.1:60023 SSH-2.0-OpenSSH_9.1 NetBSD_Secure_Shell-20221004-hpn13v14-lpk

(detects the remote sshd but doesn't get a host key).

B) openssl
The following shows issues with the three curves mentioned above (NID_secp224r1, NID_secp521r1 and NID_X9_62_prime256v1).  No problems are reported on amd64.

#include <openssl/ec.h>

#include <stdlib.h>
#include <unistd.h>

int main(int argc, char *argv[])
{
	int ncurves, i, nid;
	EC_builtin_curve *r;
	EC_GROUP *group;

	ncurves = EC_get_builtin_curves(NULL, 0);

	if ((r = malloc(ncurves * sizeof(EC_builtin_curve))) == NULL) {
		perror("malloc");
		exit(-1);
	}

	if (EC_get_builtin_curves(r, ncurves) != ncurves) {
		perror("EC_get_builtin_curves");
		exit(-1);
	}

	for (i = 0; i < ncurves; i++)
	{
		group = EC_GROUP_new_by_curve_name(r[i].nid);

		nid = EC_GROUP_check_named_curve(group, 0, NULL);
		if (r[i].nid != nid) {
			printf("generating curve %d:%d yielded curve %d\n",
			    i, r[i].nid, nid);
		}
	}
}
>Fix:
Rather intrusive as it disables more assembler optimization... falling back to the default functions for only the problematic functions would be better.

--- src/crypto/external/bsd/openssl/lib/libcrypto/arch/sparc64/ec.inc.orig	2023-05-25 17:52:29.000000000 +0200
+++ src/crypto/external/bsd/openssl/lib/libcrypto/arch/sparc64/ec.inc	2023-06-16 22:50:09.278554298 +0200
@@ -1,7 +1,8 @@
 .PATH.S: ${.PARSEDIR}
-EC_SRCS += \
-ecp_nistz256-sparcv9.S
-ECCPPFLAGS+= -DECP_NISTZ256_ASM
 
 ECNI = yes
+ECNONISTZ = yes
+
+ECCPPFLAGS+= -DOPENSSL_NO_EC_NISTP_64_GCC_128
+
 .include "../../ec.inc"