CVS commit: pkgsrc

To: pkgsrc-changes%NetBSD.org@localhost
Subject: CVS commit: pkgsrc
From: "Roland Illig" <rillig%netbsd.org@localhost>
Date: Sun, 5 May 2019 18:36:05 +0000

Module Name:    pkgsrc
Committed By:   rillig
Date:           Sun May  5 18:36:05 UTC 2019

Modified Files:
        pkgsrc/doc/guide: Makefile
        pkgsrc/doc/guide/files: build.xml
        pkgsrc/mk/fetch: sites.mk
Added Files:
        pkgsrc/doc/guide/files: fill-placeholders.py help-topics.xml
Removed Files:
        pkgsrc/doc/guide/files: help-topics.gen.py help-topics.tmpl.xml

Log Message:
doc/guide: fill in the MASTER_SITE variables automatically

Keeping these two lists in sync is not something that humans should do.


To generate a diff of this commit:
cvs rdiff -u -r1.54 -r1.55 pkgsrc/doc/guide/Makefile
cvs rdiff -u -r1.78 -r1.79 pkgsrc/doc/guide/files/build.xml
cvs rdiff -u -r0 -r1.1 pkgsrc/doc/guide/files/fill-placeholders.py \
    pkgsrc/doc/guide/files/help-topics.xml
cvs rdiff -u -r1.1 -r0 pkgsrc/doc/guide/files/help-topics.gen.py \
    pkgsrc/doc/guide/files/help-topics.tmpl.xml
cvs rdiff -u -r1.162 -r1.163 pkgsrc/mk/fetch/sites.mk

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.

Modified files:

Index: pkgsrc/doc/guide/Makefile
diff -u pkgsrc/doc/guide/Makefile:1.54 pkgsrc/doc/guide/Makefile:1.55
--- pkgsrc/doc/guide/Makefile:1.54      Mon Apr 29 16:18:41 2019
+++ pkgsrc/doc/guide/Makefile   Sun May  5 18:36:05 2019
@@ -1,4 +1,4 @@
-# $NetBSD: Makefile,v 1.54 2019/04/29 16:18:41 rillig Exp $
+# $NetBSD: Makefile,v 1.55 2019/05/05 18:36:05 rillig Exp $
 
 DISTNAME=              pkgsrc-guide-${PKGVERSION}
 CATEGORIES=            # empty
@@ -66,7 +66,7 @@ pre-extract:
 pre-configure: generate-help-topics
 generate-help-topics: .PHONY
        ${RUN} ${MAKE} help topic=:index > ${WRKSRC}/help-topics.data
-       ${RUN} cd ${WRKSRC} && python help-topics.gen.py
+       ${RUN} cd ${WRKSRC} && env PKGSRCDIR=${PKGSRCDIR} python fill-placeholders.py *.xml
 
 do-build:
 .for _output_ in ${OUTPUTS}

Index: pkgsrc/doc/guide/files/build.xml
diff -u pkgsrc/doc/guide/files/build.xml:1.78 pkgsrc/doc/guide/files/build.xml:1.79
--- pkgsrc/doc/guide/files/build.xml:1.78       Fri May 26 17:59:37 2017
+++ pkgsrc/doc/guide/files/build.xml    Sun May  5 18:36:05 2019
@@ -1,4 +1,4 @@
-<!-- $NetBSD: build.xml,v 1.78 2017/05/26 17:59:37 leot Exp $ -->
+<!-- $NetBSD: build.xml,v 1.79 2019/05/05 18:36:05 rillig Exp $ -->
 
 <chapter id="build">
   <title>The build process</title>
@@ -249,41 +249,7 @@ MASTER_SITES=   -http://www.example.com/
       packages.  The names of the variables should speak for
       themselves.</para>
 
-      <!-- sort mk/fetch/sites.mk | sed -n 's/\(^MA[A-Z_]*\).*/    ${\1}/p' -->
-
-<programlisting>
-${MASTER_SITE_APACHE}
-${MASTER_SITE_BACKUP}
-${MASTER_SITE_CYGWIN}
-${MASTER_SITE_DEBIAN}
-${MASTER_SITE_FREEBSD}
-${MASTER_SITE_FREEBSD_LOCAL}
-${MASTER_SITE_GENTOO}
-${MASTER_SITE_GNOME}
-${MASTER_SITE_GNU}
-${MASTER_SITE_GNUSTEP}
-${MASTER_SITE_HASKELL_HACKAGE}
-${MASTER_SITE_IFARCHIVE}
-${MASTER_SITE_KDE}
-${MASTER_SITE_MOZILLA}
-${MASTER_SITE_MOZILLA_ALL}
-${MASTER_SITE_MOZILLA_ESR}
-${MASTER_SITE_MYSQL}
-${MASTER_SITE_NETLIB}
-${MASTER_SITE_OPENOFFICE}
-${MASTER_SITE_OSDN}
-${MASTER_SITE_PERL_CPAN}
-${MASTER_SITE_PGSQL}
-${MASTER_SITE_RUBYGEMS}
-${MASTER_SITE_R_CRAN}
-${MASTER_SITE_SOURCEFORGE}
-${MASTER_SITE_SUNSITE}
-${MASTER_SITE_SUSE}
-${MASTER_SITE_TEX_CTAN}
-${MASTER_SITE_XCONTRIB}
-${MASTER_SITE_XEMACS}
-${MASTER_SITE_XORG}
-</programlisting>
+@master_sites@
 
       <para>Some explanations for the less self-explaining ones:
       <varname>MASTER_SITE_BACKUP</varname> contains backup sites

Index: pkgsrc/mk/fetch/sites.mk
diff -u pkgsrc/mk/fetch/sites.mk:1.162 pkgsrc/mk/fetch/sites.mk:1.163
--- pkgsrc/mk/fetch/sites.mk:1.162      Mon Feb  4 09:36:41 2019
+++ pkgsrc/mk/fetch/sites.mk    Sun May  5 18:36:05 2019
@@ -1,10 +1,9 @@
-# $NetBSD: sites.mk,v 1.162 2019/02/04 09:36:41 wiz Exp $
+# $NetBSD: sites.mk,v 1.163 2019/05/05 18:36:05 rillig Exp $
 #
 # This Makefile fragment defines read-only MASTER_SITE_* variables
 # representing some well-known master distribution sites for software.
 #
-# When variables are added or removed, the pkgsrc guide (build.xml)
-# should also be updated.
+# These variables are also listed in the pkgsrc guide.
 
 MASTER_SITE_XCONTRIB+=  \
        ftp://ftp.gwdg.de/pub/x11/x.org/contrib/ \

Added files:

Index: pkgsrc/doc/guide/files/fill-placeholders.py
diff -u /dev/null pkgsrc/doc/guide/files/fill-placeholders.py:1.1
--- /dev/null   Sun May  5 18:36:05 2019
+++ pkgsrc/doc/guide/files/fill-placeholders.py Sun May  5 18:36:05 2019
@@ -0,0 +1,88 @@
+#! python
+# $NetBSD: fill-placeholders.py,v 1.1 2019/05/05 18:36:05 rillig Exp $
+
+"""
+Fills in some sections of data that are determined directly from the
+pkgsrc code, such as variable names or help topics.
+"""
+
+import filecmp
+import os
+import re
+import sys
+from typing import List, Match
+from xml.sax.saxutils import escape as to_xml
+
+pkgsrcdir = os.environ['PKGSRCDIR']
+
+
+def read_lines(filename: str) -> List[str]:
+    with open(filename, 'r') as f:
+        return f.readlines()
+
+
+def help_topics() -> str:
+    # type="vert" would spread the columns over several pages,
+    # starting with aaaa | pppp on the first page, which is
+    # unintuitive in both the PDF and the HTML version.
+    out = ['<simplelist type="horiz" columns="2">']
+    for line in read_lines('help-topics.data')[2:]:
+        topic = line.strip()
+        out.append(f'<member>{to_xml(topic)}</member>')
+    out.append('</simplelist>')
+
+    return '\n'.join(out)
+
+
+def master_sites() -> str:
+    sites = []
+
+    for line in read_lines(f'{pkgsrcdir}/mk/fetch/sites.mk'):
+        m = re.match(r'^(MAS\w+)', line)
+        if m:
+            sites.append(m[1])
+
+    out = ['<simplelist type="horiz" columns="2">']
+    for site in sorted(sites):
+        out.append(f'<member>{to_xml(site)}</member>')
+    out.append('</simplelist>')
+
+    return '\n'.join(out)
+
+
+def process(filename: str, placeholders: set):
+    tmpl_filename = f'{filename}.tmpl'
+    tmp_filename = f'{filename}.tmp'
+    in_filename = tmpl_filename if os.path.isfile(tmpl_filename) else filename
+    out_filename = filename
+
+    phs = dict()
+    for p in placeholders:
+        phs[p.__name__] = p
+
+    def repl(m: Match):
+        return phs[m[1]]() if m[1] in phs else m[0]
+
+    out = []
+    for line in read_lines(in_filename):
+        out.append(re.sub(r'@(\w+)@', repl, line))
+
+    with open(tmp_filename, 'w') as f:
+        f.writelines(out)
+
+    if filecmp.cmp(tmp_filename, out_filename):
+        os.remove(tmp_filename)
+    elif in_filename == tmpl_filename:
+        os.replace(tmp_filename, out_filename)
+    else:
+        os.rename(out_filename, tmpl_filename)
+        os.rename(tmp_filename, out_filename)
+
+
+def main():
+    for filename in sys.argv[1:]:
+        process(filename, {help_topics, master_sites})
+
+
+if __name__ == '__main__':
+    main()
Index: pkgsrc/doc/guide/files/help-topics.xml
diff -u /dev/null pkgsrc/doc/guide/files/help-topics.xml:1.1
--- /dev/null   Sun May  5 18:36:05 2019
+++ pkgsrc/doc/guide/files/help-topics.xml      Sun May  5 18:36:05 2019
@@ -0,0 +1,13 @@
+<!-- $NetBSD: help-topics.xml,v 1.1 2019/05/05 18:36:05 rillig Exp $ -->
+
+<appendix id="help-topics">
+<title>Help topics</title>
+
+<para>
+       The following list contains all help topics that are available
+       when running <command>bmake help topic=:index</command>.
+</para>
+
+@help_topics@
+
+</appendix>

Prev by Date: CVS commit: [pkgsrc-2019Q1] pkgsrc/doc
Next by Date: CVS commit: pkgsrc/graphics/pngcrush
Previous by Thread: CVS commit: [pkgsrc-2019Q1] pkgsrc/doc
Next by Thread: CVS commit: pkgsrc/graphics/pngcrush
Indexes:

Home | Main Index | Thread Index | Old Index