Re: traditional cpp

[David Holland <dholland-tech%netbsd.org@localhost>]

On Mon, Dec 20, 2010 at 09:51:21AM -0500, der Mouse wrote:
 > > I also apparently don't have the right ideas about comment removal in
 > > traditional cpp, because it currently exhibits the behavior that
 > 
 > >    #define STOP */
 > >    #define START /*
 > >    /* hello there STOP fnord START ho hum */
 > 
 > > outputs "fnord", which cannot be what anyone intended.
 > 
 > That's an interesting corner case.
 > 
 > I'm not sure what I think the right thing is (but, I agree, that ain't
 > it), and I no longer have a real old-school cpp on hand to try, at
 > least not at ready hand.  (I may have a 4.4Lite CD or some such with
 > one.  I'm not sure.)

Neither do I.

 > > How's it supposed to be done?  Right now it's removing comments after
 > > macro expansion, but if it removes comments before macro expansion
 > > then /**/ doesn't work for pasting...
 > 
 > Remove comments after macro expansion, yes.  If the /* on the START
 > definition begins a comment, it continues up through the end of the
 > next line; if not, then the next line is a comment, and macro expansion
 > never happens inside comments.

That's what it does that creates the crazy behavior: it removes
comments entirely after expansion. This means that START is defined to
the string /*, and STOP to */, which as far as processing so far is
concerned are just strings, so the text before comment removal is

   /* hello there */ fnord /* ho hum */

I think it has to recognize comment delimiters no later than at the
same time it's doing expansion, so it can avoid doing expansion inside
comments. That will stop this particular insanity, but it's not good
enough to handle comments on directive lines, so

   #define FOO foo /*
   #undef FOO
   #define FOO bar */
   FOO

would yield

   bar */

which definitely isn't right either.

But if you drop comments on directive lines *before* expansion to make
this problem go away, the /**/ comments used for token pasting
disappear and nothing works.

I'm now really curious what a genuine legacy cpp really does. Maybe it
does two comment-stripping passes, one handling multiline comments
before interpreting directives and one handling same-line comments
afterwards? I wonder what

   /* blah
   */#define FOO foo
   FOO

vs.

   /**/#define FOO foo

vs.

   /* blah */#define FOO foo

generates.

 > I wrote a preprocessor back in the '90s, mostly to relieve a few
 > annoying limitations such as the inability of a macro to expand to a
 > macro definition.  It treates @ kind of like # but able to appear
 > anywhere, and has (integer-valued) variables, looping constructs, and
 > the like. I didn't mention it because I thought we already had plenty
 > of preprocessors, but if anyone wants a copy, I'll be happy to send it
 > out or put it up for ftp.

There aren't any *good* macro preprocessors, but I don't think either
cpp or m4 is a good starting point for a new good one. (The best
candidate starting point I know of is make's input preprocessor; this
is the main reason I'm working on rationalizing it.)

-- 
David A. Holland
dholland%netbsd.org@localhost