Re: GENERAL porting advice

[Greg Troxel <gdt%lexort.com@localhost>]

Don Y <dgy%caramail.com@localhost> writes:

> E.g., I would think it would be preferable for an author to directly
> support NetBSD in HIS released sources.  But, that means that he
> would have to take on the task of verifying his application runs on
> (some particular releases/architectures) NetBSD at each new release
> of his codebase.

Yes, the author could do that.  They could also accept patches to make
it work, submitted by others.   This is true for all upstreams and all
operating systems they could run on, and isn't special for NetBSD.

You seem to be assuming some particular affirmative testing duty, but I
don't understand where that comes from.  There are a wide range of
upstream practices about NetBSD support, ranging from hostility to
deliberate indifference, to accepting reasonable patches if submitted,
to trying to fix things, to calling for testing of betas on a
development mailinglist with people with lots of systems, and to
actually having NetBSD systems as part of CI.  There is no "would have
to".  There is simply what upstream chooses.

> This also means there can be a disconnect if his release hasn't changed
> but NetBSD has (i.e., verifying the "old, existing" release still works
> in the context of the new NetBSD release)

Yes, and this is true of every operating system.  Arguably, it is less
likely that there will be trouble on NetBSD because we have a culture of
not breaking API/ABI compat.

This is also what CI is for, adding new OS versions into the CI farm and
kicking off a run, for upstreams that are rigorous enough to be doing CI.

> Do you see the difference -- in *who* adopts responsibility for this?

If you want to take on work, just do it - I don't see why you are
asking.  If you are asking about theoretical responsiblity, I don't see
that this makes sense to talk about.  Pretty much nobody here has any
authority to allocate other people's time.

My advice is to try to get fixes upstream, meeting upstream where they
are, and pushing them gently.

>> One is that separate from pkgsrc, any published upstream program (which
>> releases tarballs or the equivalent), should be written in such a way
>> that it builds and works on any mostly-reasonable system that mostly
>> conforms to POSIX, and not doing so is a bug.  Not all upstreams see it
>> that way, but many are happy to take patches.
>
> Most of my applications were written to run on a distributed, capabilities
> based multiprocessor.  So, they would be considered "bugs" by your
> (arbitrary) requirement that they be POSIX-compliant.

I congratulate you on pulling a non-relevant example out of a hat.  I
was referring to the bulk of mainstream software, which is what it
seemed we were talking about.

> That doesn't mean that the applications themselves *can't* run under
> such a system; it just means that the assumptions made in their development
> would have to be reevaluated for an environment that can't provide those
> same guarantees.

Most programs are written for mostly POSIX, except that people that only
use Linux often use Linux-specific non-POSIX interfaces, usually (it
seems to me) because either they don't realize there is a standard, they
don't understand the point of standards, or they don't care.  I was not
discussing (and you didn't either in your first message) programs
written for environments that are not mostly like POSIX.

>> If you know that a package is
>> architecture-specific you can mark it (BROKEN, usually, because unless
>> it is support for some arch-specific cpu feature, it's a bug to not work
>> on some other system).  We are ok with this deferred evaluation.
>
> But, how do you indicate that it hasn't been BUILT on a particular
> architecture?  I.e., it COULD work but noone (i.e., me) has tried it?

We don't.  Really.  If you want to test and keep track, go for it.
There are bulk builds and you can look at them.  There is a general
(normative) expectation that software is written to standard interfaces,
uses types properly, and generally there is no reason it should fail on
any particular CPU type.  (Well, vax has non-IEEE754 floating point.)
When we figure out that a program doesn't work where it should, we mark
it BROKEN.  We do not have a mechanism for recording where it has been
tested.

>> Well, upstream packages should have tests,
>
> "Should" and "do" are spelled very differently, in practice.  :<

I know that.  But if upstream doesn't have tests, and it builds but
doesn't work, nobody should be surprised, and that's just how it is.  If
you want to test, fix, file bugs upstream --- great.

pkgsrc does not require that upstream packages be tested on all of our
vast number of OS/versions/cpu combinations.  We do expect that updates
will not break things, and that's a bit fuzzy, met with a combination of
being able to trust upstream (many upstreams' micro-labeled releases
really are micro releases) and some testing.

> I would hate to waste time building a *package* if the preferred solution
> is to get the author to make the required changes.  So, this changes
> the primary interactions when making those patches:  do you interact
> with pkgsrc (and staff) or with the original author.

I think you are missing the point that even if the upstream builds and
installs perfectly under NetBSD, we still want a package.  It
encapsulates the build instructions and produces a binary installable
package.  The only difference for programs with portability bugs is
adding some patches.  Many of the things I package have zero patches, or
only things to adapt to pkgsrc build norms (where upstream isn't wrong).

But I tried to be clear: the first approach IMHO should be to make the
upstream program work when you follow its build instructions, with fixes
applied upstream.

> An author concerned about having his codebase run on as many platforms
> as possible would likely welcome said patches -- he just hasn't had the
> time/resources to make them, himself.  Burying them in pkgsrc is a
> disservice to him.

Yes, that's why I said upstream first is the best approach, and that in
pkgsrc it is policy to file bugs/patches upstream.

> OTOH, as many authors (esp code that isn't actively maintained) don't
> have those resources, interactions with them are likely a waste of effort.

Yes, but pkgsrc doctrine is to file the bug anyway, so it's available in
the tracker if anyone else looks (I have often found bugs/fixes filed by
FreeBSD people very helpful), and we don't have to talk about why it
isn't filed (which takes longer than filing it).