CVS commit: othersrc/external/historical/eawk

["Alistair G. Crooks" <agc%netbsd.org@localhost>]

Module Name:    othersrc
Committed By:   agc
Date:           Wed Aug 31 04:19:41 UTC 2011

Update of /cvsroot/othersrc/external/historical/eawk
In directory ivanova.netbsd.org:/tmp/cvs-serv23294

Log Message:
Embedded Awk
============

This is a heresy I have done, of my own free will and volition, and
which I now know as being a sin.

Firstly, I have butchered the one true awk source code, made
it re-entrant and embeddable in C programs, and now present it
as a library (libeawk) and a small driver program (eawk).  The
driver program now uses getopt_long, and gives a good idea of
how to use eawk in embedded code.

Furthermore, I have "added" to the one true language.  The
additions are 4 functions:

        dlopen(handle, shared object name)
        dlproto(handle, C function prototype as a string)
        dlcall(handle, function, function args...)
        dlclose(handle)

which allows you to do such abominations as:

        dlopen(libc, "libc");
        dlproto(libc, "long write(int, awkptr, long)")
        dlcall(libc, "write", 1, "hi\n", 3)
        dlclose(libc)

(i.e. allows interfacing to shared libraries and shared objects without
any C glue or other shim in between the scripting language and the
compiled library).

Please note that you can specify the prototype at the same time as the
foreign function call, with dlcall:

        dlopen(libc, "libc");
        dlcall(libc, "long write(int, awkptr, long)", 1, "hi\n", 3)

and then:

% eawk 'BEGIN { dlopen(libc, "libc"); dlcall(libc, "int printf(awkptr)", "Hello 
world\n") }' /dev/null
Hello world
%

In fact, the following scripts are all equivalent:

% eawk 'BEGIN { dlopen(libc, "libc"); dlcall(libc, "long write(int, awkptr, 
long)", 1, "Hello world\n", 12) }' /dev/null
Hello world
% eawk 'BEGIN { dlopen(libc, "libc"); dlcall(libc, "int printf(awkptr)", "Hello 
world\n") }' /dev/null
Hello world
% eawk 'BEGIN { dlopen(libc, "libc"); dlcall(libc, "int fprintf(cvar, awkptr)", 
"stdout", "Hello world\n") }' /dev/null
Hello world

The type of arguments, and the return type, given in the dlproto()
calls is important:

        awkptr - a string as passed from the eawk script
        cptr - a pointer to an object in the compiled shared object
        cref - the address of a pointer to an object in the compiled
                shared object. this is used to map the construct:
                        &cp
                into an awk string
        cvar - the awk string which maps to a compiled well-known
                variable in the compiled shared object, typically
                stdin, stdout and stderr
        void - no return type
        bool - the boolean type
        int - standard integer type on this machine
        long - native long on this machine
        int64 - 64-bit data type

In order to support foreign functions which typically use a structure
passed into every function as a handle (very much like the eawk
implementation here), I'd also added two other functions which can be
called from scripts:

        buf = dlalloc(size)
        dlfree(buf)

and also a new prototype keyword called "cptr" - this is functionally
equivalent to long, but more intuitively obvious that the argument should
be pre-allocated storage (at the native layer).

% eawk 'BEGIN {
        dlopen(libc, "libc")
        size = 1024
        buf = dlalloc(size)
        dlcall(libc, "int snprintf(cptr, int, awkptr, int)", buf, size, 
"allocated size is %d\n", size)
        dlcall(libc, "int printf(cptr)", buf)
        dlfree(buf)
}' /dev/null
allocated size is 1024
%

Finally, we need a way to get information back from C structures and
storage into an awk script, and we do that with the

        var = dlfield(storage, offset, type)

function. This can be used as follows:

% eawk 'BEGIN {
        dlopen(libc, "libc")
        st = dlalloc(1024)
        dlcall(libc, "int stat(awkptr, cptr)", "/etc/group", st)
        mode = dlfield(st, 8, "int16")
        printf("%s mode is %o\n", "/etc/group", mode)
        dlfree(st)
}' /dev/null
mode is 100644
%

To illustrate some of the dlcall features a bit further, this script
will print out the keys in the user's keyring, by direct calling of
exported frunctionality from libnetpgp:

% eawk '
BEGIN {
        dlopen(libc, "libc")
        dlopen(libnetpgp, "libnetpgp")
        netpgp = dlalloc(2048)
        ret = dlcall(libnetpgp, "int netpgp_set_homedir(cptr, awkptr, awkptr, 
int)", netpgp, ENVIRON["HOME"], "/.gnupg", quiet = 1)
        ret = dlcall(libnetpgp, "int netpgp_init(cptr)", netpgp)
}
END {
        ret = dlcall(libnetpgp, "int netpgp_list_keys_json(cptr, cref, int)", 
netpgp, json, psigs = 0)
        ret = dlcall(libnetpgp, "int netpgp_format_json(cvar, cptr, int)", 
"stdout", json, psigs = 0)
}' /dev/null
126 keys found
signature  2048/RSA (Encrypt or Sign) 1b68dcfcc0596823 2004-01-12
Key fingerprint: d415 9deb 336d e4cc cdfa 00cd 1b68 dcfc c059 6823 
uid              Alistair Crooks <alistair%hockley-crooks.com@localhost> 
uid              Alistair Crooks <agc%pkgsrc.org@localhost> 
uid              Alistair Crooks <agc%netbsd.org@localhost> 
uid              Alistair Crooks <agc%alistaircrooks.com@localhost> 
uid              Alistair Crooks (Yahoo!) <agcrooks%yahoo-inc.com@localhost> 
encryption 2048/RSA (Encrypt or Sign) 79deb61e488eee74 2004-01-12
...
%

Note that the prototype for netpgp_list_keys_json() above is:

        int netpgp_list_keys_json(netpgp_t *, char **, const int);

and the prototype for netpgp_format_json is:

        int netpgp_format_json(void *, const char *, const int);

and so the signature of the compiled function in the shared object
defines the type of arguments that are passed via dlcall.

Another example is that of calculating a digest using the SHA256_File
function in libc.

% eawk 'BEGIN {
        f = "../Makefile"
        dlopen(libc, "libc")
        buf = dlalloc(65)
        dlcall(libc, "cptr SHA256_File(awkptr, cptr)", f, buf)
        dlcall(libc, "int printf(awkptr, awkptr, cptr)", "SHA256 (%s) = %s\n", 
f, buf) }')
}'
SHA256 (../Makefile) = 
a6ccb2e57801867720b434d8dfc248d62389c518457ea1a022861819151f2b1f
% digest sha256 ../Makefile
SHA256 (../Makefile) = 
a6ccb2e57801867720b434d8dfc248d62389c518457ea1a022861819151f2b1f
%

I've had these changes around in my own tree for over 2 and a half years.
It's finally time to commit them.

Alistair Crooks
Wed Aug 30 19:45:50 PDT 2011

Status:

Vendor Tag:     BWK
Release Tags:   eawk-base
                
N othersrc/external/historical/eawk/Makefile
N othersrc/external/historical/eawk/dist/awkgram.y
N othersrc/external/historical/eawk/dist/FIXES
N othersrc/external/historical/eawk/dist/Makefile
N othersrc/external/historical/eawk/dist/README
N othersrc/external/historical/eawk/dist/dl9.sh
N othersrc/external/historical/eawk/dist/EAWK
N othersrc/external/historical/eawk/dist/eawk.1
N othersrc/external/historical/eawk/dist/awkre.h
N othersrc/external/historical/eawk/dist/dlA.sh
N othersrc/external/historical/eawk/dist/dlB.sh
N othersrc/external/historical/eawk/dist/dlC.sh
N othersrc/external/historical/eawk/dist/libeawk.3
N othersrc/external/historical/eawk/dist/dl.sh
N othersrc/external/historical/eawk/dist/dl2.sh
N othersrc/external/historical/eawk/dist/b.c
N othersrc/external/historical/eawk/dist/dl3.sh
N othersrc/external/historical/eawk/dist/dl4.sh
N othersrc/external/historical/eawk/dist/dl5.sh
N othersrc/external/historical/eawk/dist/dl6.sh
N othersrc/external/historical/eawk/dist/dl7.sh
N othersrc/external/historical/eawk/dist/lex.c
N othersrc/external/historical/eawk/dist/dl8.sh
N othersrc/external/historical/eawk/dist/proto.h
N othersrc/external/historical/eawk/dist/lib.c
N othersrc/external/historical/eawk/dist/main.c
N othersrc/external/historical/eawk/dist/parse.c
N othersrc/external/historical/eawk/dist/proctab.c
N othersrc/external/historical/eawk/dist/run.c
N othersrc/external/historical/eawk/dist/eawk.h
N othersrc/external/historical/eawk/dist/tran.c
N othersrc/external/historical/eawk/eawk/Makefile
N othersrc/external/historical/eawk/eawk/expected
N othersrc/external/historical/eawk/libeawk/Makefile
N othersrc/external/historical/eawk/libeawk/shlib_version

No conflicts created by this import