CVS commit: pkgsrc/devel/pcre2

["Benny Siegert" <bsiegert%netbsd.org@localhost>]

Module Name:    pkgsrc
Committed By:   bsiegert
Date:           Tue Dec 29 14:40:20 UTC 2015

Modified Files:
        pkgsrc/devel/pcre2: Makefile PLIST buildlink3.mk distinfo

Log Message:
Update pcre2 to 10.20. Fix CVE-2015-8381.

Version 10.20 30-June-2015
--------------------------

1. Callouts with string arguments have been added.

2. Assertion code generator in JIT has been optimized.

3. The invalid pattern (?(?C) has a missing assertion condition at the end. The
pcre2_compile() function read past the end of the input before diagnosing an
error. This bug was discovered by the LLVM fuzzer.

4. Implemented pcre2_callout_enumerate().

5. Fix JIT compilation of conditional blocks whose assertion is converted to
(*FAIL). E.g: /(?(?!))/.

6. The pattern /(?(?!)^)/ caused references to random memory. This bug was
discovered by the LLVM fuzzer.

7. The assertion (?!) is optimized to (*FAIL). This was not handled correctly
when this assertion was used as a condition, for example (?(?!)a|b). In
pcre2_match() it worked by luck; in pcre2_dfa_match() it gave an incorrect
error about an unsupported item.

8. For some types of pattern, for example /Z*(|d*){216}/, the auto-
possessification code could take exponential time to complete. A recursion
depth limit of 1000 has been imposed to limit the resources used by this
optimization. This infelicity was discovered by the LLVM fuzzer.

9. A pattern such as /(*UTF)[\S\V\H]/, which contains a negated special class
such as \S in non-UCP mode, explicit wide characters (> 255) can be ignored
because \S ensures they are all in the class. The code for doing this was
interacting badly with the code for computing the amount of space needed to
compile the pattern, leading to a buffer overflow. This bug was discovered by
the LLVM fuzzer.

10. A pattern such as /((?2)+)((?1))/ which has mutual recursion nested inside
other kinds of group caused stack overflow at compile time. This bug was
discovered by the LLVM fuzzer.

11. A pattern such as /(?1)(?#?'){8}(a)/ which had a parenthesized comment
between a subroutine call and its quantifier was incorrectly compiled, leading
to buffer overflow or other errors. This bug was discovered by the LLVM fuzzer.

12. The illegal pattern /(?(?<E>.*!.*)?)/ was not being diagnosed as missing an
assertion after (?(. The code was failing to check the character after (?(?<
for the ! or = that would indicate a lookbehind assertion. This bug was
discovered by the LLVM fuzzer.

13. A pattern such as /X((?2)()*+){2}+/ which has a possessive quantifier with
a fixed maximum following a group that contains a subroutine reference was
incorrectly compiled and could trigger buffer overflow. This bug was discovered
by the LLVM fuzzer.

14. Negative relative recursive references such as (?-7) to non-existent
subpatterns were not being diagnosed and could lead to unpredictable behaviour.
This bug was discovered by the LLVM fuzzer.

15. The bug fixed in 14 was due to an integer variable that was unsigned when
it should have been signed. Some other "int" variables, having been checked,
have either been changed to uint32_t or commented as "must be signed".

16. A mutual recursion within a lookbehind assertion such as (?<=((?2))((?1)))
caused a stack overflow instead of the diagnosis of a non-fixed length
lookbehind assertion. This bug was discovered by the LLVM fuzzer.

17. The use of \K in a positive lookbehind assertion in a non-anchored pattern
(e.g. /(?<=\Ka)/) could make pcre2grep loop.

18. There was a similar problem to 17 in pcre2test for global matches, though
the code there did catch the loop.

19. If a greedy quantified \X was preceded by \C in UTF mode (e.g. \C\X*),
and a subsequent item in the pattern caused a non-match, backtracking over the
repeated \X did not stop, but carried on past the start of the subject, causing
reference to random memory and/or a segfault. There were also some other cases
where backtracking after \C could crash. This set of bugs was discovered by the
LLVM fuzzer.

20. The function for finding the minimum length of a matching string could take
a very long time if mutual recursion was present many times in a pattern, for
example, /((?2){73}(?2))((?1))/. A better mutual recursion detection method has
been implemented. This infelicity was discovered by the LLVM fuzzer.

21. Implemented PCRE2_NEVER_BACKSLASH_C.

22. The feature for string replication in pcre2test could read from freed
memory if the replication required a buffer to be extended, and it was not
working properly in 16-bit and 32-bit modes. This issue was discovered by a
fuzzer: see http://lcamtuf.coredump.cx/afl/.

23. Added the PCRE2_ALT_CIRCUMFLEX option.

24. Adjust the treatment of \8 and \9 to be the same as the current Perl
behaviour.

25. Static linking against the PCRE2 library using the pkg-config module was
failing on missing pthread symbols.

26. If a group that contained a recursive back reference also contained a
forward reference subroutine call followed by a non-forward-reference
subroutine call, for example /.((?2)(?R)\1)()/, pcre2_compile() failed to
compile correct code, leading to undefined behaviour or an internally detected
error. This bug was discovered by the LLVM fuzzer.

27. Quantification of certain items (e.g. atomic back references) could cause
incorrect code to be compiled when recursive forward references were involved.
For example, in this pattern: /(?1)()((((((\1++))\x85)+)|))/. This bug was
discovered by the LLVM fuzzer.

28. A repeated conditional group whose condition was a reference by name caused
a buffer overflow if there was more than one group with the given name. This
bug was discovered by the LLVM fuzzer.

29. A recursive back reference by name within a group that had the same name as
another group caused a buffer overflow. For example: /(?J)(?'d'(?'d'\g{d}))/.
This bug was discovered by the LLVM fuzzer.

30. A forward reference by name to a group whose number is the same as the
current group, for example in this pattern: /(?|(\k'Pm')|(?'Pm'))/, caused a
buffer overflow at compile time. This bug was discovered by the LLVM fuzzer.

31. Fix -fsanitize=undefined warnings for left shifts of 1 by 31 (it treats 1
as an int; fixed by writing it as 1u).

32. Fix pcre2grep compile when -std=c99 is used with gcc, though it still gives
a warning for "fileno" unless -std=gnu99 us used.

33. A lookbehind assertion within a set of mutually recursive subpatterns could
provoke a buffer overflow. This bug was discovered by the LLVM fuzzer.

34. Give an error for an empty subpattern name such as (?'').

35. Make pcre2test give an error if a pattern that follows #forbud_utf contains
\P, \p, or \X.

36. The way named subpatterns are handled has been refactored. There is now a
pre-pass over the regex which does nothing other than identify named
subpatterns and count the total captures. This means that information about
named patterns is known before the rest of the compile. In particular, it means
that forward references can be checked as they are encountered. Previously, the
code for handling forward references was contorted and led to several errors in
computing the memory requirements for some patterns, leading to buffer
overflows.

37. There was no check for integer overflow in subroutine calls such as (?123).

38. The table entry for \l in EBCDIC environments was incorrect, leading to its
being treated as a literal 'l' instead of causing an error.

39. If a non-capturing group containing a conditional group that could match
an empty string was repeated, it was not identified as matching an empty string
itself. For example: /^(?:(?(1)x|)+)+$()/.

40. In an EBCDIC environment, pcretest was mishandling the escape sequences
\a and \e in test subject lines.

41. In an EBCDIC environment, \a in a pattern was converted to the ASCII
instead of the EBCDIC value.

42. The handling of \c in an EBCDIC environment has been revised so that it is
now compatible with the specification in Perl's perlebcdic page.

43. Single character repetition in JIT has been improved. 20-30% speedup
was achieved on certain patterns.

44. The EBCDIC character 0x41 is a non-breaking space, equivalent to 0xa0 in
ASCII/Unicode. This has now been added to the list of characters that are
recognized as white space in EBCDIC.

45. When PCRE2 was compiled without Unicode support, the use of \p and \P gave
an error (correctly) when used outside a class, but did not give an error
within a class.

46. \h within a class was incorrectly compiled in EBCDIC environments.

47. JIT should return with error when the compiled pattern requires
more stack space than the maximum.

48. Fixed a memory leak in pcre2grep when a locale is set.


To generate a diff of this commit:
cvs rdiff -u -r1.1 -r1.2 pkgsrc/devel/pcre2/Makefile pkgsrc/devel/pcre2/PLIST \
    pkgsrc/devel/pcre2/buildlink3.mk
cvs rdiff -u -r1.2 -r1.3 pkgsrc/devel/pcre2/distinfo

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.