pkgsrc-WIP-changes archive
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]
compiler-rt-netbsd: Sync with a local copy
Module Name: pkgsrc-wip
Committed By: Kamil Rytarowski <n54%gmx.com@localhost>
Pushed By: kamil
Date: Thu May 10 04:51:56 2018 +0200
Changeset: dd71fa7899cb0a7ea578b9c589378c6657a1e1e6
Modified Files:
compiler-rt-netbsd/TODO.check
compiler-rt-netbsd/TODO.fuzzer
compiler-rt-netbsd/distinfo
Added Files:
compiler-rt-netbsd/patches/patch-lib_fuzzer_FuzzerLoop.cpp
Log Message:
compiler-rt-netbsd: Sync with a local copy
1. Add a local kludge for FuzzerLoop under SIGALARM (timeout).
2. Inline a kernel WIP patch to unbreak few remaining tests.
3. Remove old notes about profiling features vs gnu gprof.
4. Note the idea of using EXTERNAL_TOOLCHAIN.
$ make check-fuzzer
********************
Testing: 0 .. 10.. 20.. 30.. 40.. 50.. 60.. 70.. 80.. 90..
Testing Time: 56.25s
********************
Failing Tests (2):
LLVMFuzzer :: merge-posix.test
LLVMFuzzer :: trace-malloc-threaded.test
Expected Passes : 105
Unsupported Tests : 8
Unexpected Failures: 2
Sponsored by <The NetBSD Foundation>
To see a diff of this commit:
https://wip.pkgsrc.org/cgi-bin/gitweb.cgi?p=pkgsrc-wip.git;a=commitdiff;h=dd71fa7899cb0a7ea578b9c589378c6657a1e1e6
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
diffstat:
compiler-rt-netbsd/TODO.check | 31 ++----
compiler-rt-netbsd/TODO.fuzzer | 114 ++++++---------------
compiler-rt-netbsd/distinfo | 1 +
.../patches/patch-lib_fuzzer_FuzzerLoop.cpp | 13 +++
4 files changed, 58 insertions(+), 101 deletions(-)
diffs:
diff --git a/compiler-rt-netbsd/TODO.check b/compiler-rt-netbsd/TODO.check
index 97a29fc072..b0f1c74b1e 100644
--- a/compiler-rt-netbsd/TODO.check
+++ b/compiler-rt-netbsd/TODO.check
@@ -239,40 +239,30 @@ Failing Tests (31):
$ check-fuzzer-default
********************
-Testing Time: 94.86s
+Testing: 0 .. 10.. 20.. 30.. 40.. 50.. 60.. 70.. 80.. 90..
+Testing Time: 54.91s
********************
-Failing Tests (8):
+Failing Tests (2):
LLVMFuzzer :: merge-posix.test
- LLVMFuzzer :: strncmp.test
LLVMFuzzer :: trace-malloc-threaded.test
- LLVMFuzzer :: trace-malloc-unbalanced.test
- LLVMFuzzer :: value-profile-cmp.test
- LLVMFuzzer :: value-profile-load.test
- LLVMFuzzer :: value-profile-set.test
- LLVMFuzzer :: value-profile-switch.test
- Expected Passes : 99
+ Expected Passes : 105
Unsupported Tests : 8
- Unexpected Failures: 8
+ Unexpected Failures: 2
$ check-fuzzer
********************
-Testing Time: 141.30s
+Testing: 0 .. 10.. 20.. 30.. 40.. 50.. 60.. 70.. 80.. 90..
+Testing Time: 54.82s
********************
-Failing Tests (8):
+Failing Tests (2):
LLVMFuzzer :: merge-posix.test
- LLVMFuzzer :: strncmp.test
LLVMFuzzer :: trace-malloc-threaded.test
- LLVMFuzzer :: trace-malloc-unbalanced.test
- LLVMFuzzer :: value-profile-cmp.test
- LLVMFuzzer :: value-profile-load.test
- LLVMFuzzer :: value-profile-set.test
- LLVMFuzzer :: value-profile-switch.test
- Expected Passes : 99
+ Expected Passes : 105
Unsupported Tests : 8
- Unexpected Failures: 8
+ Unexpected Failures: 2
$ check-fuzzer-unit
@@ -597,7 +587,6 @@ Testing Time: 1.55s
Expected Passes : 7
Unsupported Tests : 1
-
$ check-scudo
********************
diff --git a/compiler-rt-netbsd/TODO.fuzzer b/compiler-rt-netbsd/TODO.fuzzer
index 50f6b3f875..d3e17f0f0f 100644
--- a/compiler-rt-netbsd/TODO.fuzzer
+++ b/compiler-rt-netbsd/TODO.fuzzer
@@ -1,99 +1,53 @@
-compiler-rt profile v.s. -pg
-I. compiler-rt profile
- Similar to GCC gcov tool, LLVM can provide llvm-cov to interpret profiling data for each line of code.
- One can add "--coverage" or "-fcoverage-mapping" flags to generate necessary information for this tool.
- This is pure software method, which means the information is generated from AST or something else.
-II. -pg
- This flag is usually used to collect information for gprof. The unit is function instead of source line.
-Summary:
- The basic units of profiling are different. llvm-cov/gcov is line-based, while gprof is function-based.
-Do we need llvm-cov for libFuzzer?
- I don't think so. It seems there is no relationship between these two.
-III. SanitizerCoverage (https://clang.llvm.org/docs/SanitizerCoverage.html)
- There are command line arguments like:
- -fsanitize-coverage=trace-pc-guard,indirect-calls,trace-gep,trace-div,trace-cmp
- We would like to cover them in llvm / clang tests (wherever applicable). At least covering
- the netbsd--x86_64 target.
- Examples of tests: clang/test/Driver/fsanitize-coverage.c clang/test/Driver/fuzzer.c
- SanitizerCoverage is required for at least a part of the libfuzzer functionality.
+Algorithm of libfuzzer & sanitizers porting for userland (by joerg)
-sanitizers:
- - ASan (amd64, i386), UBSan (all ports), MSan (amd64), TSan (amd64) - ported and mostly functional
- - LSan - in progresss (scheduled after ptrace(2) kernel fixes)
- - src/ integration - TODO:
- * import compiler-rt into src/
- * add Makefile glue to build and install compiler-rt features (check cmake/config-ix.cmake)
- along the MKLLVM target
- * add MKSANITIZER build option, to prebuild libraries and programs with a desired sanitizer
+(1) Run build.sh -T somewhere -V MKLLVM=yes ... tools
+(2) Replace somewhere/bin/TARGET-clang with a symlink to a normally
+(via cmake) built clang binary.
+(3) Run build.sh -T somewhereelse -V MKLLVM=yes -V EXTERNAL_TOOLCHAIN=somewhere ... release
+
+Expectations:
+ - prepare a bootstrap.sh script to build a distribution with compiler-rt
+ features, that is using EXTERNAL_TOOLCHAIN
+ - ship compiler-rt in basesystem
+ - add MKSANITIZER build option to prebuild libraries and programs with a
+ desired sanitizer
$ make check-fuzzer
********************
-Testing Time: 94.86s
+Testing: 0 .. 10.. 20.. 30.. 40.. 50.. 60.. 70.. 80.. 90..
+Testing Time: 56.25s
********************
-Failing Tests (8):
+Failing Tests (2):
LLVMFuzzer :: merge-posix.test
- LLVMFuzzer :: strncmp.test
LLVMFuzzer :: trace-malloc-threaded.test
- LLVMFuzzer :: trace-malloc-unbalanced.test
- LLVMFuzzer :: value-profile-cmp.test
- LLVMFuzzer :: value-profile-load.test
- LLVMFuzzer :: value-profile-set.test
- LLVMFuzzer :: value-profile-switch.test
- Expected Passes : 99
+ Expected Passes : 105
Unsupported Tests : 8
- Unexpected Failures: 8
-
-
-libFuzzer Failed cases (make check-fuzzer):
-I. hang of fuzzer-timeout.test
-
- kamil: It seems to be an issue with Thread-Local-Storage in the
- SIGALARM signal handler... but it's not reproducible for me
- out of the libfuzzer context.
-
- With the following workaround the tests passes:
-
-```
-diff --git a/lib/fuzzer/FuzzerLoop.cpp b/lib/fuzzer/FuzzerLoop.cpp
-index 6cc220d97..a67431c71 100644
---- a/lib/fuzzer/FuzzerLoop.cpp
-+++ b/lib/fuzzer/FuzzerLoop.cpp
-@@ -274,7 +274,7 @@ NO_SANITIZE_MEMORY
- void Fuzzer::AlarmCallback() {
- assert(Options.UnitTimeoutSec > 0);
- // In Windows Alarm callback is executed by a different thread.
--#if !LIBFUZZER_WINDOWS
-+#if !LIBFUZZER_WINDOWS && !LIBFUZZER_NETBSD
- if (!InFuzzingThread())
- return;
- #endif
-```
+ Unexpected Failures: 2
- Under investigation.
+Reason unknown of failures (environment mismatch?) and ignored.
-II. unknown hangs
+A local WIP kernel patch for tests:
- LLVMFuzzer :: value-profile-cmp.test
- LLVMFuzzer :: value-profile-load.test
- LLVMFuzzer :: value-profile-set.test
- LLVMFuzzer :: value-profile-switch.test
- LLVMFuzzer :: strncmp.test
+Index: sys/uvm/uvm_map.c
+===================================================================
+RCS file: /cvsroot/src/sys/uvm/uvm_map.c,v
+retrieving revision 1.354
+diff -u -r1.354 uvm_map.c
+--- sys/uvm/uvm_map.c 6 Feb 2018 09:20:29 -0000 1.354
++++ sys/uvm/uvm_map.c 10 May 2018 01:21:10 -0000
+@@ -4972,7 +4972,7 @@
+ return EINVAL;
- workaround: kill with pkill(1) or kill(1)
-
- reason unknown
-
-III. SIGXFSZ handler bug?
- LLVMFuzzer :: merge-posix.test
-
-IV. Unknown
- LLVMFuzzer :: trace-malloc-threaded.test
- LLVMFuzzer :: trace-malloc-unbalanced.test
+ if (oldp) {
+- if (*oldlenp > 1024 * 1024)
++ if (*oldlenp > 10 * 1024 * 1024)
+ return E2BIG;
+ count = *oldlenp / elem_size;
+ if (count == 0)
TODO:
- - research the remaining failures (assignee: kamil)
- oom-killer, research a crasher and share with kernel developers (assignee: plusun)
- oom-killer, add a new ATF test to check oom-killer (assignee: plusun)
- upstream semaphore fixes to reviews.llvm.org (assignee: plusun)
diff --git a/compiler-rt-netbsd/distinfo b/compiler-rt-netbsd/distinfo
index 4dd3fc8748..8a48b8ce35 100644
--- a/compiler-rt-netbsd/distinfo
+++ b/compiler-rt-netbsd/distinfo
@@ -5,6 +5,7 @@ SHA1 (patch-cmake_config-ix.cmake) = 91ffb8c1e65394c7c6e9956379373b28348e3f1f
SHA1 (patch-lib_asan_asan__mapping.h) = 1975314eb61f14b9fabbcfee20de63dc724d809c
SHA1 (patch-lib_asan_asan__posix.cc) = 0f596061c078eaab0b5b4b183550aa7820677f46
SHA1 (patch-lib_asan_asan__thread.cc) = 6408c904c1f3adbebff56aaf63f953f5b8b02bef
+SHA1 (patch-lib_fuzzer_FuzzerLoop.cpp) = fe6804c47064d10f311082a919ce84ebcd19bb4b
SHA1 (patch-lib_fuzzer_FuzzerShmemPosix.cpp) = e2540a9385d641c77a2b5da49a2ac66313db8a94
SHA1 (patch-lib_fuzzer_tests_CMakeLists.txt) = 0e559ebb791942cba9c83f3098d7752f91e1f187
SHA1 (patch-lib_msan_msan__interceptors.cc) = 79bfe137923597d79bb467d0f282d57221079a85
diff --git a/compiler-rt-netbsd/patches/patch-lib_fuzzer_FuzzerLoop.cpp b/compiler-rt-netbsd/patches/patch-lib_fuzzer_FuzzerLoop.cpp
new file mode 100644
index 0000000000..807cba276a
--- /dev/null
+++ b/compiler-rt-netbsd/patches/patch-lib_fuzzer_FuzzerLoop.cpp
@@ -0,0 +1,13 @@
+$NetBSD$
+
+--- lib/fuzzer/FuzzerLoop.cpp.orig 2018-02-25 10:48:27.000000000 +0000
++++ lib/fuzzer/FuzzerLoop.cpp
+@@ -269,7 +269,7 @@ NO_SANITIZE_MEMORY
+ void Fuzzer::AlarmCallback() {
+ assert(Options.UnitTimeoutSec > 0);
+ // In Windows Alarm callback is executed by a different thread.
+-#if !LIBFUZZER_WINDOWS
++#if !LIBFUZZER_WINDOWS && !LIBFUZZER_NETBSD # XXX: why?
+ if (!InFuzzingThread())
+ return;
+ #endif
Home |
Main Index |
Thread Index |
Old Index