Re: Problems unpacking sets with ftp -o

[RVP <rvp%SDF.ORG@localhost>]

On Sat, 3 Jul 2021, Christos Zoulas wrote:

> Interesting. snprintf should not matter. Something else is going on.
> I think we should investigate why it fails instead of using the big
> signal hammer.

Well, this only happens with xz(1) *and* _only_ if the final pipe-write
gets interrupted. Neither gzip(1) nor bzip2(1) exhibit this issue
because I think, of the trio, only xz does non-blocking I/O.

This almost feels that NetBSD-10 blocker PR #56113, except that only
an interruption of the final pipe-write causes xz(1) to misbehave.

In any case, setting _IONBF on the popen()ed stream fixes the issue
(or, at least, makes it go into remission). Patch below:

---START---
diff -urN ftp.orig/fetch.c ftp/fetch.c
--- ftp.orig/fetch.c	2020-07-11 00:29:38.000000000 +0000
+++ ftp/fetch.c	2021-07-04 22:22:19.047393346 +0000
@@ -1253,6 +1253,28 @@
 }
 #endif

+/**
+ * Write buffer safe from interrupts.
+ */
+static size_t
+fullwrite(FILE *fp, const void *buf, size_t size)
+{
+	size_t n, len = size;
+	const char* p = buf;
+
+	for (; len > 0; len -= n, p += n) {
+		errno = 0;
+		n = fwrite(p, 1, len, fp);
+		if (n == 0) {
+			if (errno == EINTR)
+				clearerr(fp);
+			else
+				return 0;
+		}
+	}
+	return size;
+}
+
 /*
  * Retrieve URL, via a proxy if necessary, using HTTP.
  * If proxyenv is set, use that for the proxy, otherwise try ftp_proxy or
@@ -1526,6 +1548,7 @@
 				warn("Can't execute `%s'", savefile + 1);
 				goto cleanup_fetch_url;
 			}
+			setvbuf(fout, NULL, _IONBF, 0); /* for xz(1) */
 			closefunc = pclose;
 		}
 	}
@@ -1650,8 +1673,7 @@
 				}
 				bytes += flen;
 				bufrem -= flen;
-				if (fwrite(xferbuf, sizeof(char), flen, fout)
-				    != flen) {
+				if (fullwrite(fout, xferbuf, flen) != flen) {
 					warn("Writing `%s'", savefile);
 					goto cleanup_fetch_url;
 				}
---END---

-RVP