lib/58656: libfetch: multiple issues with SSL+CONNECT

[rvp%SDF.ORG@localhost]

>Number:         58656
>Category:       lib
>Synopsis:       libfetch: multiple issues with SSL+CONNECT
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    lib-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Thu Aug 29 22:20:01 +0000 2024
>Originator:     RVP
>Release:        NetBSD/amd64 10.99.12
>Organization:
>Environment:
NetBSD/amd64 10.99.12
>Description:
1. Fetching files via a proxy doesn't work for HTTPS when the proxy
   server (here, `polipo`) doesn't doesn't add any HTTP headers (as
   reqd. by the RFCs):

```
$ echo $http_proxy
http://localhost:8118/

$ time -p pkgin pc libfetch
00B88063247F0000:error:0A000126:SSL routines:ssl3_read_n:unexpected eof while reading:/usr/src/crypto/external/bsd/openssl/dist/ssl/record/rec_layer_s3.c:322:
pkg_info: can't find package `https://cdn.NetBSD.org/pub/pkgsrc/packages/NetBSD/x86_64/10.0/All/libfetch-2.40.tgz', skipped
real 61.60
user 0.05
sys 0.08

$ no_proxy=\* time -p pkgin pc libfetch
Information for https://cdn.NetBSD.org/pub/pkgsrc/packages/NetBSD/x86_64/10.0/All/libfetch-2.40.tgz:
Files:
/usr/pkg/include/fetch.h
/usr/pkg/lib/libfetch.a
/usr/pkg/man/man3/fetch.3
real         0.29
user         0.07
sys          0.08

$
```

This is caused by an extra read in the NetBSD version of libfetch for
headers which never arrive. The FreeBSD library removes trailing blanks,
then checks for a 0-length HTTP header before going to read a new line.

2. Even if you fix 1), Proxy-authentication isn't done for HTTPS URLs:

```
$ cat /tmp/fetchstat.c
#include <err.h>
#include <stdio.h>
#include <stdlib.h>
#include <fetch.h>
int main(int argc, char* argv[]) {
        struct url_stat us;
        if (argc != 2)
                exit(EXIT_FAILURE);
        if (fetchStatURL(argv[1], &us, "v") != 0)
                err(1, "fetchStat(%s) failed", argv[1]);
        printf("%zu bytes\n", us.size);
        return 0;
}

$ cc -o /tmp/f /tmp/fetchstat.c -L/tmp/fetch/lib -lfetch -lssl

$ HTTP_PROXY_AUTH='basic:*:rvp:not_very_secret' /tmp/f https://cdn.NetBSD.org/pub/pkgsrc/packages/NetBSD/x86_64/10.0/All/libfetch-2.40.tgz
looking up localhost
connecting to localhost:8118
f: fetchStat(https://cdn.NetBSD.org/pub/pkgsrc/packages/NetBSD/x86_64/10.0/All/libfetch-2.40.tgz) failed: Undefined error: 0

$
```

Send Basic auth for proxy+SSL also.

3. Wrong test when reading HTTP headers in libfetch/http.c:http_connect():

```
    794                         }
    795                 } while (h < hdr_end);
    796         }
```

Surely that should be `while (h > hdr_end)` to discard headers (if any)
from the proxy server?
>How-To-Repeat:
As shown above.
>Fix:
Note this is for the libfetch in base.

---START patch---
diff -urN fetch.orig/dist/libfetch/http.c fetch/dist/libfetch/http.c
--- fetch.orig/dist/libfetch/http.c	2024-02-03 01:51:49.796872219 +0000
+++ fetch/dist/libfetch/http.c	2024-08-29 00:31:49.955206679 +0000
@@ -771,16 +771,20 @@
 				URL->host, URL->port);
 		http_cmd(conn, "Host: %s:%d\r\n",
 				URL->host, URL->port);
+		/* proxy authorization */
+		if (purl) {
+			if (*purl->user || *purl->pwd)
+				http_basic_auth(conn, "Proxy-Authorization",
+				    purl->user, purl->pwd);
+			else if ((p = getenv("HTTP_PROXY_AUTH")) != NULL && *p != '\0')
+				http_authorize(conn, "Proxy-Authorization", p);
+		}
 		http_cmd(conn, "\r\n");
 		if (http_get_reply(conn) != HTTP_OK) {
 			http_seterr(conn->err);
 			goto ouch;
 		}
-		/* Read and discard the rest of the proxy response */
-		if (fetch_getln(conn) < 0) {
-			fetch_syserr();
-			goto ouch;
-		}
+		/* Read and discard the rest of the proxy response (if any) */
 		do {
 			switch ((h = http_next_header(conn, &p))) {
 			case hdr_syserror:
@@ -792,7 +796,7 @@
 			default:
 				/* ignore */ ;
 			}
-		} while (h < hdr_end);
+		} while (h > hdr_end);
 	}
 	if (strcasecmp(URL->scheme, SCHEME_HTTPS) == 0 &&
 	    fetch_ssl(conn, URL, verbose) == -1) {
---END patch---