Source-Changes-HG archive
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]
[pkgsrc/trunk]: pkgsrc/doc doc/pkgsrc.*: regen
details: https://anonhg.NetBSD.org/pkgsrc/rev/7383a87c5ef8
branches: trunk
changeset: 432480:7383a87c5ef8
user: rillig <rillig%pkgsrc.org@localhost>
date: Fri May 22 18:51:44 2020 +0000
description:
doc/pkgsrc.*: regen
diffstat:
doc/pkgsrc.html | 1603 ++++++++++++++++++++++++++++++---------------------
doc/pkgsrc.txt | 1696 +++++++++++++++++++++++++++++++-----------------------
2 files changed, 1909 insertions(+), 1390 deletions(-)
diffs (truncated from 4062 to 300 lines):
diff -r 05f9aa85f131 -r 7383a87c5ef8 doc/pkgsrc.html
--- a/doc/pkgsrc.html Fri May 22 18:50:08 2020 +0000
+++ b/doc/pkgsrc.html Fri May 22 18:51:44 2020 +0000
@@ -124,8 +124,23 @@
<dt><span class="sect1"><a href="#bulk.pbulk">8.2. Running a pbulk-style bulk build</a></span></dt>
<dd><dl><dt><span class="sect2"><a href="#bulk.pbulk.conf">8.2.1. Configuration</a></span></dt></dl></dd>
<dt><span class="sect1"><a href="#bulk.req">8.3. Requirements of a full bulk build</a></span></dt>
-<dt><span class="sect1"><a href="#creating-cdroms">8.4. Creating a multiple CD-ROM packages collection</a></span></dt>
-<dd><dl><dt><span class="sect2"><a href="#cdpack-example">8.4.1. Example of cdpack</a></span></dt></dl></dd>
+<dt><span class="sect1"><a href="#bulk.var">8.4. Bulk build variants</a></span></dt>
+<dd><dl>
+<dt><span class="sect2"><a href="#bulk.var.subst_noop">8.4.1. Strict SUBST blocks</a></span></dt>
+<dt><span class="sect2"><a href="#bulk.var.confopt">8.4.2. Detect unknown configure options</a></span></dt>
+<dt><span class="sect2"><a href="#bulk.var.comperr">8.4.3. Detect classes of bugs by forcing compiler warnings</a></span></dt>
+<dt><span class="sect2"><a href="#bulk.var.dirs">8.4.4. Use custom directories</a></span></dt>
+<dt><span class="sect2"><a href="#bulk.var.warn">8.4.5. Turn warnings into errors</a></span></dt>
+<dt><span class="sect2"><a href="#bulk.var.pkglint">8.4.6. Reject packages for which pkglint reports errors</a></span></dt>
+<dt><span class="sect2"><a href="#bulk.var.strings">8.4.7. Reject packages that contain forbidden strings</a></span></dt>
+<dt><span class="sect2"><a href="#bulk.var.test">8.4.8. Reject packages whose self-test fails</a></span></dt>
+<dt><span class="sect2"><a href="#bulk.var.shvar">8.4.9. Reject packages that use undefined shell variables</a></span></dt>
+<dt><span class="sect2"><a href="#bulk.var.quiet">8.4.10. Turn off verbose logging</a></span></dt>
+<dt><span class="sect2"><a href="#bulk.var.options">8.4.11. Select random sets of options</a></span></dt>
+<dt><span class="sect2"><a href="#bulk.var.build_defs">8.4.12. Select random configurations of BUILD_DEFS</a></span></dt>
+</dl></dd>
+<dt><span class="sect1"><a href="#creating-cdroms">8.5. Creating a multiple CD-ROM packages collection</a></span></dt>
+<dd><dl><dt><span class="sect2"><a href="#cdpack-example">8.5.1. Example of cdpack</a></span></dt></dl></dd>
</dl></dd>
<dt><span class="chapter"><a href="#files">9. Directory layout of the installed files</a></span></dt>
<dd><dl>
@@ -869,8 +884,23 @@
<dt><span class="sect1"><a href="#bulk.pbulk">8.2. Running a pbulk-style bulk build</a></span></dt>
<dd><dl><dt><span class="sect2"><a href="#bulk.pbulk.conf">8.2.1. Configuration</a></span></dt></dl></dd>
<dt><span class="sect1"><a href="#bulk.req">8.3. Requirements of a full bulk build</a></span></dt>
-<dt><span class="sect1"><a href="#creating-cdroms">8.4. Creating a multiple CD-ROM packages collection</a></span></dt>
-<dd><dl><dt><span class="sect2"><a href="#cdpack-example">8.4.1. Example of cdpack</a></span></dt></dl></dd>
+<dt><span class="sect1"><a href="#bulk.var">8.4. Bulk build variants</a></span></dt>
+<dd><dl>
+<dt><span class="sect2"><a href="#bulk.var.subst_noop">8.4.1. Strict SUBST blocks</a></span></dt>
+<dt><span class="sect2"><a href="#bulk.var.confopt">8.4.2. Detect unknown configure options</a></span></dt>
+<dt><span class="sect2"><a href="#bulk.var.comperr">8.4.3. Detect classes of bugs by forcing compiler warnings</a></span></dt>
+<dt><span class="sect2"><a href="#bulk.var.dirs">8.4.4. Use custom directories</a></span></dt>
+<dt><span class="sect2"><a href="#bulk.var.warn">8.4.5. Turn warnings into errors</a></span></dt>
+<dt><span class="sect2"><a href="#bulk.var.pkglint">8.4.6. Reject packages for which pkglint reports errors</a></span></dt>
+<dt><span class="sect2"><a href="#bulk.var.strings">8.4.7. Reject packages that contain forbidden strings</a></span></dt>
+<dt><span class="sect2"><a href="#bulk.var.test">8.4.8. Reject packages whose self-test fails</a></span></dt>
+<dt><span class="sect2"><a href="#bulk.var.shvar">8.4.9. Reject packages that use undefined shell variables</a></span></dt>
+<dt><span class="sect2"><a href="#bulk.var.quiet">8.4.10. Turn off verbose logging</a></span></dt>
+<dt><span class="sect2"><a href="#bulk.var.options">8.4.11. Select random sets of options</a></span></dt>
+<dt><span class="sect2"><a href="#bulk.var.build_defs">8.4.12. Select random configurations of BUILD_DEFS</a></span></dt>
+</dl></dd>
+<dt><span class="sect1"><a href="#creating-cdroms">8.5. Creating a multiple CD-ROM packages collection</a></span></dt>
+<dd><dl><dt><span class="sect2"><a href="#cdpack-example">8.5.1. Example of cdpack</a></span></dt></dl></dd>
</dl></dd>
<dt><span class="chapter"><a href="#files">9. Directory layout of the installed files</a></span></dt>
<dd><dl>
@@ -2061,8 +2091,23 @@
<dt><span class="sect1"><a href="#bulk.pbulk">8.2. Running a pbulk-style bulk build</a></span></dt>
<dd><dl><dt><span class="sect2"><a href="#bulk.pbulk.conf">8.2.1. Configuration</a></span></dt></dl></dd>
<dt><span class="sect1"><a href="#bulk.req">8.3. Requirements of a full bulk build</a></span></dt>
-<dt><span class="sect1"><a href="#creating-cdroms">8.4. Creating a multiple CD-ROM packages collection</a></span></dt>
-<dd><dl><dt><span class="sect2"><a href="#cdpack-example">8.4.1. Example of cdpack</a></span></dt></dl></dd>
+<dt><span class="sect1"><a href="#bulk.var">8.4. Bulk build variants</a></span></dt>
+<dd><dl>
+<dt><span class="sect2"><a href="#bulk.var.subst_noop">8.4.1. Strict SUBST blocks</a></span></dt>
+<dt><span class="sect2"><a href="#bulk.var.confopt">8.4.2. Detect unknown configure options</a></span></dt>
+<dt><span class="sect2"><a href="#bulk.var.comperr">8.4.3. Detect classes of bugs by forcing compiler warnings</a></span></dt>
+<dt><span class="sect2"><a href="#bulk.var.dirs">8.4.4. Use custom directories</a></span></dt>
+<dt><span class="sect2"><a href="#bulk.var.warn">8.4.5. Turn warnings into errors</a></span></dt>
+<dt><span class="sect2"><a href="#bulk.var.pkglint">8.4.6. Reject packages for which pkglint reports errors</a></span></dt>
+<dt><span class="sect2"><a href="#bulk.var.strings">8.4.7. Reject packages that contain forbidden strings</a></span></dt>
+<dt><span class="sect2"><a href="#bulk.var.test">8.4.8. Reject packages whose self-test fails</a></span></dt>
+<dt><span class="sect2"><a href="#bulk.var.shvar">8.4.9. Reject packages that use undefined shell variables</a></span></dt>
+<dt><span class="sect2"><a href="#bulk.var.quiet">8.4.10. Turn off verbose logging</a></span></dt>
+<dt><span class="sect2"><a href="#bulk.var.options">8.4.11. Select random sets of options</a></span></dt>
+<dt><span class="sect2"><a href="#bulk.var.build_defs">8.4.12. Select random configurations of BUILD_DEFS</a></span></dt>
+</dl></dd>
+<dt><span class="sect1"><a href="#creating-cdroms">8.5. Creating a multiple CD-ROM packages collection</a></span></dt>
+<dd><dl><dt><span class="sect2"><a href="#cdpack-example">8.5.1. Example of cdpack</a></span></dt></dl></dd>
</dl>
</div>
<p>For a number of reasons, you may want to build binary packages
@@ -2186,7 +2231,227 @@
</div>
<div class="sect1">
<div class="titlepage"><div><div><h2 class="title" style="clear: both">
-<a name="creating-cdroms"></a>8.4. Creating a multiple CD-ROM packages collection</h2></div></div></div>
+<a name="bulk.var"></a>8.4. Bulk build variants</h2></div></div></div>
+<p>To ensure that pkgsrc packages work in different configurations, it
+makes sense to run non-default bulk builds from time to time. This
+section lists some ideas for bulk builds that intentionally let packages
+fail if they don't follow the pkgsrc style.</p>
+<div class="sect2">
+<div class="titlepage"><div><div><h3 class="title">
+<a name="bulk.var.subst_noop"></a>8.4.1. Strict SUBST blocks</h3></div></div></div>
+<p>Up to May 2020, the SUBST blocks ignored files that didn't exist,
+as well as substitutions that didn't have any effect. There were quite a
+few SUBST blocks that were redundant, and these have been removed
+already.</p>
+<p>The next step would be to not only check that each filename pattern
+has an effect but also that each substitution in SUBST_SED or SUBST_VARS
+applies to at least one file.</p>
+<p>To do this, <code class="filename">mk/subst.mk</code> would have to be
+adjusted, in a similar way as the check for no-op SUBST_FILES. There are
+several regression tests in <code class="filename">regress/infra-unittests</code>
+that help to get all edge cases correct.</p>
+<p>When a package fails this additional check, there are various
+possible causes why the <code class="varname">SUBST_SED</code> became a
+no-op.</p>
+<div class="orderedlist"><ol class="orderedlist" type="1">
+<li class="listitem"><p>The pattern used to be found in a former version of the
+package, but is not needed anymore. In that case, just remove
+it.</p></li>
+<li class="listitem"><p>The pattern contains a typo. In that case, fix the typo
+and bump <code class="varname">PKGREVISION</code>, since the fixed typo will
+probably modify the resulting binary package.</p></li>
+<li class="listitem"><p>There is a patch that is applied before the SUBST block,
+and the patch accidentally contains the change that was intended for the
+SUBST block. In that case, remove the respective hunk from the
+patch.</p></li>
+</ol></div>
+</div>
+<div class="sect2">
+<div class="titlepage"><div><div><h3 class="title">
+<a name="bulk.var.confopt"></a>8.4.2. Detect unknown configure options</h3></div></div></div>
+<p>Add the following line to <a class="link" href="#mk.conf"><code class="filename">mk.conf</code></a>.</p>
+<pre class="programlisting">
+GNU_CONFIGURE_STRICT= yes
+</pre>
+<p>When a package fails this additional check, the most common cause
+is that the configure option was valid for an older version of the
+package but does not apply anymore. In that case, just remove it.</p>
+</div>
+<div class="sect2">
+<div class="titlepage"><div><div><h3 class="title">
+<a name="bulk.var.comperr"></a>8.4.3. Detect classes of bugs by forcing compiler warnings</h3></div></div></div>
+<p>The job of a compiler is not restricted to producing executable
+code, most compilers also detects typical mistakes.</p>
+<p>Add the following line to <a class="link" href="#mk.conf"><code class="filename">mk.conf</code></a>.</p>
+<pre class="programlisting">
+CFLAGS+= -Werror=char-subscripts
+</pre>
+<p>When a package fails this additional check, first document the
+circumstances in which the compiler produced the error message. This
+includes:</p>
+<div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; ">
+<li class="listitem"><p>The platform
+(<code class="varname">MACHINE_PLATFORM</code>)</p></li>
+<li class="listitem"><p>The source file</p></li>
+<li class="listitem"><p>An excerpt of the code. GCC and Clang already do this as
+part of the diagnostic.</p></li>
+<li class="listitem"><p>The error message from the compiler.</p></li>
+</ul></div>
+<p>If a package produces these error messages, but the package is
+fine, document this in the package Makefile, like this:</p>
+<pre class="programlisting">
+# Version ${VERSION} failed on ${MACHINE_PLATFORM}:
+# error message
+# code
+# reason why the code does not need to be fixed
+BUILDLINK_TRANSFORM+= rm:-Werror=char-subscripts
+</pre>
+<p>If the error messages from the compiler are valid and the code
+needs to be fixed, prepare a patch for a single source file, or if it's a
+one-liner fix, add a SUBST block to the package Makefile. In any case,
+report it to the upstream authors of the package.</p>
+</div>
+<div class="sect2">
+<div class="titlepage"><div><div><h3 class="title">
+<a name="bulk.var.dirs"></a>8.4.4. Use custom directories</h3></div></div></div>
+<p>Some directories like <code class="varname">PREFIX</code>,
+<code class="varname">VARBASE</code>, <code class="varname">PKG_SYSCONFDIR</code>,
+<code class="varname">PKGMANDIR</code>, <code class="varname">PKG_INFODIR</code> can be
+configured in pkgsrc. Set these to arbitrary paths during bootstrap or
+afterwards in <a class="link" href="#mk.conf"><code class="filename">mk.conf</code></a>.</p>
+<pre class="programlisting">
+PREFIX= /a-random-uuid
+PKG_SYSCONFDIR= /a-random-uuid
+VARBASE= /a-random-uuid
+PKGMANDIR= a-random-uuid
+PKG_INFODIR= a-random-uuid
+</pre>
+</div>
+<div class="sect2">
+<div class="titlepage"><div><div><h3 class="title">
+<a name="bulk.var.warn"></a>8.4.5. Turn warnings into errors</h3></div></div></div>
+<p>When building a package, warnings are typically ignored since they
+just flow by and do not cause the build to fail immediately. To find
+these warnings, redefine them to errors in <a class="link" href="#mk.conf"><code class="filename">mk.conf</code></a>.</p>
+<pre class="programlisting">
+DELAYED_WARNING_MSG= ${DELAYED_ERROR_MSG} "(was warning)"
+WARNING_MSG= ${FAIL_MSG} "(was warning)"
+</pre>
+<p>(There are many more classes of warnings in pkgsrc, and most of
+them can be redefined with a simple definition like above.</p>
+<p>If a package suggest to add <code class="varname">USE_TOOLS+=perl</code> to
+the package Makefile, research whether the package actually needs Perl.
+If it does, add <code class="varname">USE_TOOLS+=perl</code> to the package
+Makefile, and if it doesn't, add
+<code class="varname">TOOLS_BROKEN+=perl</code>.</p>
+</div>
+<div class="sect2">
+<div class="titlepage"><div><div><h3 class="title">
+<a name="bulk.var.pkglint"></a>8.4.6. Reject packages for which pkglint reports errors</h3></div></div></div>
+<p>Using pkglint as part of the regular build process is mostly a
+waste of time. If you want to fix some of the warnings, just run pkglint
+recursively on the whole pkgsrc tree. This will take a few minutes (up to
+10), which is much faster than a complete bulk build.</p>
+</div>
+<div class="sect2">
+<div class="titlepage"><div><div><h3 class="title">
+<a name="bulk.var.strings"></a>8.4.7. Reject packages that contain forbidden strings</h3></div></div></div>
+<p>To ensure that the binary packages don't contain references to the
+build directory, there is already <code class="varname">CHECK_WRKREF</code>. If
+that variable includes the item <code class="literal">extra</code>, it is
+possible to define additional patterns that must not appear in any
+installed file. This is specified in <a class="link" href="#mk.conf"><code class="filename">mk.conf</code></a>.</p>
+<pre class="programlisting">
+CHECK_WRKREF= extra
+CHECK_WRKREF_EXTRA_DIRS+= /usr/local
+CHECK_WRKREF_EXTRA_DIRS+= /usr/pkg
+CHECK_WRKREF_EXTRA_DIRS+= @[A-Z][A-Z]*@
+</pre>
+<p>The above patterns will probably generate many false positives,
+therefore the results need to be taken with a grain of salt.</p>
+</div>
+<div class="sect2">
+<div class="titlepage"><div><div><h3 class="title">
+<a name="bulk.var.test"></a>8.4.8. Reject packages whose self-test fails</h3></div></div></div>
+<p>To run the test suites that come with each package, add this line
+to <a class="link" href="#mk.conf"><code class="filename">mk.conf</code></a>.</p>
+<pre class="programlisting">
+PKGSRC_RUN_TEST= yes
+</pre>
+<p>Be prepared that even the most basic packages fail this test. When
+doing a bulk build with this, it will often abort in the early phase
+where the packages are scanned for their dependencies since there are
+cyclic dependencies. There is still a lot to do in this area.</p>
+</div>
+<div class="sect2">
+<div class="titlepage"><div><div><h3 class="title">
+<a name="bulk.var.shvar"></a>8.4.9. Reject packages that use undefined shell variables</h3></div></div></div>
+<p>To catch typos in the shell snippets from the Makefile fragments,
+add the <code class="literal">-u</code> flag to most of the commands by adding this
+line to <a class="link" href="#mk.conf"><code class="filename">mk.conf</code></a>.</p>
+<pre class="programlisting">
+RUN= @set -eu;
+</pre>
+<p>See <code class="filename">mk/misc/common.mk</code> for the existing
+definition.</p>
+</div>
+<div class="sect2">
+<div class="titlepage"><div><div><h3 class="title">
+<a name="bulk.var.quiet"></a>8.4.10. Turn off verbose logging</h3></div></div></div>
+<p>The build logs of a package are often quite long. This allows error
+messages or other interesting details to hide between the noise. To make
+the actual error message stand out more, add these lines to
+<a class="link" href="#mk.conf"><code class="filename">mk.conf</code></a>.</p>
+<pre class="programlisting">
+GNU_CONFIGURE_QUIET= yes
+MAKE_FLAGS+= -s
+</pre>
+<p>The <code class="literal">-s</code> option works for both GNU Make and BSD
+Make. On exotic platforms with their own make, it may be a little
+different.</p>
+</div>
+<div class="sect2">
+<div class="titlepage"><div><div><h3 class="title">
+<a name="bulk.var.options"></a>8.4.11. Select random sets of options</h3></div></div></div>
+<p>Most bulk builds run with the default package options. This means
+that other combinations of options are not regularly tested. To do this,
+run a bulk build with these configurations.</p>
+<div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; ">
+<li class="listitem"><p>no options enabled</p></li>
+<li class="listitem"><p>all options enabled</p></li>
+<li class="listitem"><p>2n + 0</p></li>
+<li class="listitem"><p>2n + 1</p></li>
+<li class="listitem"><p>4n + 0..1</p></li>
+<li class="listitem"><p>4n + 2..3</p></li>
+<li class="listitem"><p>8n + 0..3</p></li>
+<li class="listitem"><p>8n + 4..7</p></li>
+<li class="listitem"><p>2048n + 0..1023</p></li>
+<li class="listitem"><p>2048n + 1024..2047</p></li>
+</ul></div>
+<p>Open questions are:</p>
+<div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; ">
+<li class="listitem"><p>how to collect all options from the entire
+pkgsrc</p></li>
+<li class="listitem"><p>how to handle mutually exclusive
+options</p></li>
+<li class="listitem"><p>the sets of mutually exclusive options are defined
+per-package</p></li>
+<li class="listitem"><p>the sets of nonempty sets are defined
+per-package</p></li>
+</ul></div>
+</div>
+<div class="sect2">
+<div class="titlepage"><div><div><h3 class="title">
+<a name="bulk.var.build_defs"></a>8.4.12. Select random configurations of BUILD_DEFS</h3></div></div></div>
+<p>Just like the <code class="varname">PKG_OPTIONS</code>, the
+<code class="varname">BUILD_DEFS</code> also allow different variants of pkgsrc to
Home |
Main Index |
Thread Index |
Old Index