Add minimal LLVM sanity coverage (sancov) reporting for unittests.

Add minimal LLVM sanity coverage (sancov) reporting for unittests.

This CL enables generating *.sancov data. Blacklist for sancov tool is
provided, too. Sancov tool for report generation needs to be build from
llvm compiler-rt sources (llvm 3.9.0 or newer).

See http://clang.llvm.org/docs/SanitizerCoverage.html .

BUG=webrtc:6136
R=phoglund@webrtc.org
TBR=kjellander@webrtc.org

Committed: https://crrev.com/9ddac18d1c04f3cd2afcebf1c5c295895f9c0d53
Cr-Commit-Position: refs/heads/master@{#13506}

[johan, 2016-07-14 13:43:18]

Description was changed from

==========
Add minimal LLVM sanity coverage (sancov) reporting for unittests.

This CL enables generating *.sancov data. Blacklist for sancov tool is
provided, too. Sancov tool for report generation needs to be build from
llvm compiler-rt sources (llvm 3.9.0 or newer).

See http://clang.llvm.org/docs/SanitizerCoverage.html .

BUG=
==========

to

==========
Add minimal LLVM sanity coverage (sancov) reporting for unittests.

This CL enables generating *.sancov data. Blacklist for sancov tool is
provided, too. Sancov tool for report generation needs to be build from
llvm compiler-rt sources (llvm 3.9.0 or newer).

See http://clang.llvm.org/docs/SanitizerCoverage.html .

BUG=
==========

[johan, 2016-07-14 13:43:18]

johan@webrtc.org changed reviewers:
+ kjellander@webrtc.org, phoglund@webrtc.org

[phoglund, 2016-07-19 19:07:00]

I'm terribly sorry, I missed this review. Feel free to ping if I don't respond
within one working day :)

[phoglund, 2016-07-19 19:12:40]

https://codereview.webrtc.org/2144273002/diff/1/webrtc/build/common.gypi
File webrtc/build/common.gypi (right):

https://codereview.webrtc.org/2144273002/diff/1/webrtc/build/common.gypi#newc...
webrtc/build/common.gypi:106: 'webrtc_sanitize_coverage%': "",
We need to have parity between gn and gyp; please add this variable to
build/webrtc.gni, and the flag fiddling to webrtc/BUILD.gn (in
common_inherited_config). Ensure it works with for instance

gn gen out/Default --args="webrtc_sanitize_coverage=func"
ninja -C out/Default some_target

[johan, 2016-07-21 12:33:22]

https://codereview.webrtc.org/2144273002/diff/1/webrtc/build/common.gypi
File webrtc/build/common.gypi (right):

https://codereview.webrtc.org/2144273002/diff/1/webrtc/build/common.gypi#newc...
webrtc/build/common.gypi:106: 'webrtc_sanitize_coverage%': "",
Done, gn files use name pattern "rtc_.*" while gyp files use 'webrtc_.*' for
arguments.
gn gen out/Default --args='rtc_sanitize_coverage="func"'

[phoglund, 2016-07-21 12:49:33]

lgtm. Please file a bug (type feature, title "Make it possible to report
coverage in WebRTC" or something like that, and set the BUG= field in the CL
desc before submitting.

[johan, 2016-07-21 13:40:13]

Description was changed from

==========
Add minimal LLVM sanity coverage (sancov) reporting for unittests.

This CL enables generating *.sancov data. Blacklist for sancov tool is
provided, too. Sancov tool for report generation needs to be build from
llvm compiler-rt sources (llvm 3.9.0 or newer).

See http://clang.llvm.org/docs/SanitizerCoverage.html .

BUG=
==========

to

==========
Add minimal LLVM sanity coverage (sancov) reporting for unittests.

This CL enables generating *.sancov data. Blacklist for sancov tool is
provided, too. Sancov tool for report generation needs to be build from
llvm compiler-rt sources (llvm 3.9.0 or newer).

See http://clang.llvm.org/docs/SanitizerCoverage.html .

BUG=6136
==========

[johan, 2016-07-21 13:40:40]

Description was changed from

==========
Add minimal LLVM sanity coverage (sancov) reporting for unittests.

This CL enables generating *.sancov data. Blacklist for sancov tool is
provided, too. Sancov tool for report generation needs to be build from
llvm compiler-rt sources (llvm 3.9.0 or newer).

See http://clang.llvm.org/docs/SanitizerCoverage.html .

BUG=6136
==========

to

==========
Add minimal LLVM sanity coverage (sancov) reporting for unittests.

This CL enables generating *.sancov data. Blacklist for sancov tool is
provided, too. Sancov tool for report generation needs to be build from
llvm compiler-rt sources (llvm 3.9.0 or newer).

See http://clang.llvm.org/docs/SanitizerCoverage.html .

BUG=webrtc:6136
==========

[johan, 2016-07-21 13:41:09]

The CQ bit was checked by johan@webrtc.org

[commit-bot: I haz the power, 2016-07-21 13:41:12]

CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.webrtc.org/...

[commit-bot: I haz the power, 2016-07-21 13:45:52]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-07-21 13:45:53]

Try jobs failed on following builders:
  linux_libfuzzer_rel on master.tryserver.webrtc (JOB_FAILED,
http://build.chromium.org/p/tryserver.webrtc/builders/linux_libfuzzer_rel/bui...)

[johan, 2016-07-21 14:51:13]

Bug created and linked.
Presubmit hook complains on missing lgtm from owner of webrtc/BUILD.gn and
webrtc/build/*  . That would be kjellander, who is ooo. Wait until he is back?

[phoglund, 2016-07-21 15:11:17]

No, add a TBR=kjellander@webrtc.org to CL desc and land; he's out this and next
week. I think that's reasonable to do in this situation, and I think I'm as much
an expert as anyone on gyp and GN in this project.

[johan, 2016-07-22 06:32:06]

Description was changed from

==========
Add minimal LLVM sanity coverage (sancov) reporting for unittests.

This CL enables generating *.sancov data. Blacklist for sancov tool is
provided, too. Sancov tool for report generation needs to be build from
llvm compiler-rt sources (llvm 3.9.0 or newer).

See http://clang.llvm.org/docs/SanitizerCoverage.html .

BUG=webrtc:6136
==========

to

==========
Add minimal LLVM sanity coverage (sancov) reporting for unittests.

This CL enables generating *.sancov data. Blacklist for sancov tool is
provided, too. Sancov tool for report generation needs to be build from
llvm compiler-rt sources (llvm 3.9.0 or newer).

See http://clang.llvm.org/docs/SanitizerCoverage.html .

BUG=webrtc:6136
TBR=kjellander@webrtc.org
==========

[johan, 2016-07-22 06:42:47]

The CQ bit was checked by johan@webrtc.org

[commit-bot: I haz the power, 2016-07-22 06:42:55]

CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.webrtc.org/...

[commit-bot: I haz the power, 2016-07-22 06:45:04]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-07-22 06:45:06]

Try jobs failed on following builders:
  linux_libfuzzer_rel on master.tryserver.webrtc (JOB_FAILED,
http://build.chromium.org/p/tryserver.webrtc/builders/linux_libfuzzer_rel/bui...)

[commit-bot: I haz the power, 2016-07-22 06:57:43]

Description was changed from

==========
Add minimal LLVM sanity coverage (sancov) reporting for unittests.

This CL enables generating *.sancov data. Blacklist for sancov tool is
provided, too. Sancov tool for report generation needs to be build from
llvm compiler-rt sources (llvm 3.9.0 or newer).

See http://clang.llvm.org/docs/SanitizerCoverage.html .

BUG=webrtc:6136
TBR=kjellander@webrtc.org
==========

to

==========
Add minimal LLVM sanity coverage (sancov) reporting for unittests.

This CL enables generating *.sancov data. Blacklist for sancov tool is
provided, too. Sancov tool for report generation needs to be build from
llvm compiler-rt sources (llvm 3.9.0 or newer).

See http://clang.llvm.org/docs/SanitizerCoverage.html .

BUG=webrtc:6136
R=phoglund@webrtc.org
TBR=kjellander@webrtc.org

Committed: https://crrev.com/9ddac18d1c04f3cd2afcebf1c5c295895f9c0d53
Cr-Commit-Position: refs/heads/master@{#13506}
==========

[johan, 2016-07-22 06:57:43]

Description was changed from

==========
Add minimal LLVM sanity coverage (sancov) reporting for unittests.

This CL enables generating *.sancov data. Blacklist for sancov tool is
provided, too. Sancov tool for report generation needs to be build from
llvm compiler-rt sources (llvm 3.9.0 or newer).

See http://clang.llvm.org/docs/SanitizerCoverage.html .

BUG=webrtc:6136
TBR=kjellander@webrtc.org
==========

to

==========
Add minimal LLVM sanity coverage (sancov) reporting for unittests.

This CL enables generating *.sancov data. Blacklist for sancov tool is
provided, too. Sancov tool for report generation needs to be build from
llvm compiler-rt sources (llvm 3.9.0 or newer).

See http://clang.llvm.org/docs/SanitizerCoverage.html .

BUG=webrtc:6136
R=phoglund@webrtc.org
TBR=kjellander@webrtc.org

Committed:
https://chromium.googlesource.com/external/webrtc/+/9ddac18d1c04f3cd2afcebf1c...
==========

[commit-bot: I haz the power, 2016-07-22 06:57:44]

Patchset 2 (id:??) landed as
https://crrev.com/9ddac18d1c04f3cd2afcebf1c5c295895f9c0d53
Cr-Commit-Position: refs/heads/master@{#13506}

[johan, 2016-07-22 06:57:45]

Committed patchset #2 (id:20001) manually as
9ddac18d1c04f3cd2afcebf1c5c295895f9c0d53 (presubmit successful).

[kjellander_webrtc, 2016-07-23 15:11:31]

kjellander@webrtc.org changed reviewers:
+ machenbach@chromium.org

[kjellander_webrtc, 2016-07-23 15:11:32]

lgtm but I'd appreciate if machenbach@ could give input as well.

[Michael Achenbach, 2016-07-23 20:29:02]

The CL looks consistent. I know too little about the webrtc test runner (I
assume gtest?). Some thoughts:
- It might be prudent to allow specifying the directory where the sancov files
will end up. Otherwise I think it's in the CWD (might be harder to clean up in
the end). This dir is passed as a runtime option in e.g. the UBSAN_OPTIONS.
- In v8 we ran into the max_pid boundary. E.g. in one test runner execution we
might get more processes in total than linux' max_pid limit. Since sancov spits
out files called exe_name.pid.sancov, you risk a name clash as pid will rotate
after reaching the max.
- You can follow the work on our prototype here
https://bugs.chromium.org/p/chromium/issues/detail?id=568949 - first gcov, later
sancov
- Note that we don't use the sancov exe, just the python script