Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(108)

Issue 1308403003: Refactoring full stack and loopback tests (Closed)

Created:
5 years, 4 months ago by ivica
Modified:
5 years, 3 months ago
CC:
webrtc-reviews_webrtc.org, yujie_mao (webrtc), Andrew MacDonald, stefan-webrtc, tterriberry_mozilla.com, qiang.lu, niklas.enbom, mflodman, perkj_webrtc, andresp, sprang_webrtc
Base URL:
https://chromium.googlesource.com/external/webrtc.git@master
Target Ref:
refs/pending/heads/master
Project:
webrtc
Visibility:
Public.

Description

Refactoring full stack and loopback tests Refactoring full stack, video and screenshare tests to use the same code basis for parametrization and initialization. This patch is done on top of recently commited full stack graphs CL https://codereview.webrtc.org/1289933003/, but virtually no changes have been made to full_stack_plot.py nor to the VideoAnalyzer in full stack, except moving it to video_quality_test.cc. Also, full_stack_samples.cc (build target) was removed and replaced with -output_filename and -duration cmdline arguments in video_loopback and screenshare_loopback. The important things to review: - video_quality_test.h Is the structure of Params good? (examples of usage can be found in full_stack.cc, video_loopback.cc and screenshare_loopback.cc) - video_quality_test.cc Is the initialization correct? The case for using Analyzer and using local renderer are different, can they be further merged? - webrtc_tests.gypi Reproducing the different bitrate settings the full stack and loopback tests had was a little bit tricky. To support both simultaneously, I added BitrateConfig to the Params struct, as well as separate start_bitrate and target_bitrate flags for loopback tests. Note: Side-by-side diff for video_quality_test.cc compares that file directly with the old full_stack.cc, so changes to VideoAnalyzer are clearly visible. Note: Recent CL I've committed added -num_temporal_layers and -sl_discard_threshold args to loopback tests. This was removed here. Support for streams and SVC will be added in a CL following this one. Committed: https://crrev.com/5d6a06c1d29a2061bcf4b321ffceab477a404d51 Cr-Commit-Position: refs/heads/master@{#9969}

Patch Set 1 #

Patch Set 2 : rebase master (old) and rm full_stack_samples.cc #

Patch Set 3 : rebase master (including first graphs commit) #

Patch Set 4 : Renaming full_stack_base to video_quality_test #

Patch Set 5 : Removing full_stack.h from gyp file #

Patch Set 6 : Removing unnecessary includes #

Patch Set 7 : cpp standard apparently doesn't like unnamed structs #

Patch Set 8 : Fixing casts #

Patch Set 9 : Another cast #

Patch Set 10 : Adding parentheses #

Patch Set 11 : Rebase fixes. Temporarily disabling SLs in loopback tests. #

Patch Set 12 : Hopefully fixed the gyp file and the casts. #

Patch Set 13 : Disabling a dependency for android only. Rebase master. #

Total comments: 14

Patch Set 14 : Addressing comments #

Patch Set 15 : Removing a comment. #

Total comments: 12

Patch Set 16 : Adding some comments #

Patch Set 17 : Addressing comments #

Patch Set 18 : Fixing perf tests #

Patch Set 19 : rebase updates #

Unified diffs Side-by-side diffs Delta from patch set Stats (+569 lines, -1777 lines) Patch
D webrtc/video/full_stack.h View 1 2 1 chunk +0 lines, -52 lines 0 comments Download
M webrtc/video/full_stack.cc View 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 2 chunks +83 lines, -821 lines 0 comments Download
M webrtc/video/full_stack_plot.py View 1 2 3 4 chunks +2 lines, -4 lines 0 comments Download
D webrtc/video/full_stack_quality_sampler.cc View 1 2 1 chunk +0 lines, -142 lines 0 comments Download
D webrtc/video/loopback.h View 1 2 3 4 5 6 7 8 9 10 1 chunk +0 lines, -63 lines 0 comments Download
M webrtc/video/loopback.cc View 1 2 3 4 5 6 7 8 9 10 11 12 13 1 chunk +0 lines, -183 lines 0 comments Download
M webrtc/video/screenshare_loopback.cc View 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 5 chunks +70 lines, -144 lines 0 comments Download
M webrtc/video/video_loopback.cc View 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 4 chunks +83 lines, -33 lines 0 comments Download
A webrtc/video/video_quality_test.h View 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 1 chunk +86 lines, -0 lines 0 comments Download
A + webrtc/video/video_quality_test.cc View 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 13 chunks +230 lines, -303 lines 0 comments Download
M webrtc/webrtc_tests.gypi View 1 2 3 4 5 6 7 8 9 10 11 12 4 chunks +15 lines, -32 lines 0 comments Download

Messages

Total messages: 70 (30 generated)
ivica
5 years, 4 months ago (2015-08-25 07:44:43 UTC) #2
commit-bot: I haz the power
Dry run: CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/patch-status/1308403003/60001 View timeline at https://chromium-cq-status.appspot.com/patch-timeline/1308403003/60001
5 years, 3 months ago (2015-09-07 14:38:32 UTC) #4
commit-bot: I haz the power
Dry run: Try jobs failed on following builders: win_rel on tryserver.webrtc (JOB_FAILED, http://build.chromium.org/p/tryserver.webrtc/builders/win_rel/builds/9410)
5 years, 3 months ago (2015-09-07 14:41:56 UTC) #6
commit-bot: I haz the power
Dry run: CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/patch-status/1308403003/80001 View timeline at https://chromium-cq-status.appspot.com/patch-timeline/1308403003/80001
5 years, 3 months ago (2015-09-07 14:51:03 UTC) #8
commit-bot: I haz the power
Dry run: Try jobs failed on following builders: win_rel on tryserver.webrtc (JOB_FAILED, http://build.chromium.org/p/tryserver.webrtc/builders/win_rel/builds/9412)
5 years, 3 months ago (2015-09-07 14:53:36 UTC) #10
commit-bot: I haz the power
Dry run: CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/patch-status/1308403003/120001 View timeline at https://chromium-cq-status.appspot.com/patch-timeline/1308403003/120001
5 years, 3 months ago (2015-09-07 17:03:16 UTC) #12
commit-bot: I haz the power
Dry run: Try jobs failed on following builders: win_x64_rel on tryserver.webrtc (JOB_FAILED, http://build.chromium.org/p/tryserver.webrtc/builders/win_x64_rel/builds/9229)
5 years, 3 months ago (2015-09-07 17:08:47 UTC) #14
commit-bot: I haz the power
Dry run: CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/patch-status/1308403003/140001 View timeline at https://chromium-cq-status.appspot.com/patch-timeline/1308403003/140001
5 years, 3 months ago (2015-09-07 17:38:52 UTC) #16
commit-bot: I haz the power
Dry run: Try jobs failed on following builders: win_x64_rel on tryserver.webrtc (JOB_FAILED, http://build.chromium.org/p/tryserver.webrtc/builders/win_x64_rel/builds/9231)
5 years, 3 months ago (2015-09-07 17:42:26 UTC) #18
commit-bot: I haz the power
Dry run: CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/patch-status/1308403003/160001 View timeline at https://chromium-cq-status.appspot.com/patch-timeline/1308403003/160001
5 years, 3 months ago (2015-09-07 18:13:20 UTC) #20
commit-bot: I haz the power
Dry run: Try jobs failed on following builders: linux on tryserver.webrtc (JOB_FAILED, http://build.chromium.org/p/tryserver.webrtc/builders/linux/builds/9475) linux_tsan2 on ...
5 years, 3 months ago (2015-09-07 18:14:55 UTC) #22
commit-bot: I haz the power
Dry run: CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/patch-status/1308403003/180001 View timeline at https://chromium-cq-status.appspot.com/patch-timeline/1308403003/180001
5 years, 3 months ago (2015-09-07 18:16:41 UTC) #24
commit-bot: I haz the power
Dry run: Try jobs failed on following builders: android_clang on tryserver.webrtc (JOB_FAILED, http://build.chromium.org/p/tryserver.webrtc/builders/android_clang/builds/7443)
5 years, 3 months ago (2015-09-07 18:22:35 UTC) #26
commit-bot: I haz the power
Dry run: CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/patch-status/1308403003/220001 View timeline at https://chromium-cq-status.appspot.com/patch-timeline/1308403003/220001
5 years, 3 months ago (2015-09-08 11:56:48 UTC) #28
commit-bot: I haz the power
Dry run: Try jobs failed on following builders: win on tryserver.webrtc (JOB_FAILED, http://build.chromium.org/p/tryserver.webrtc/builders/win/builds/9562) win_rel on ...
5 years, 3 months ago (2015-09-08 11:59:03 UTC) #30
commit-bot: I haz the power
Dry run: CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/patch-status/1308403003/240001 View timeline at https://chromium-cq-status.appspot.com/patch-timeline/1308403003/240001
5 years, 3 months ago (2015-09-08 13:10:36 UTC) #32
commit-bot: I haz the power
Dry run: Try jobs failed on following builders: android on tryserver.webrtc (JOB_TIMED_OUT, no build URL) ...
5 years, 3 months ago (2015-09-08 15:10:53 UTC) #34
ivica
5 years, 3 months ago (2015-09-10 07:56:57 UTC) #36
sprang_webrtc
https://codereview.webrtc.org/1308403003/diff/240001/webrtc/video/loopback.cc File webrtc/video/loopback.cc (right): https://codereview.webrtc.org/1308403003/diff/240001/webrtc/video/loopback.cc#newcode48 webrtc/video/loopback.cc:48: // Empty. Use Run() instead. Should this ever be ...
5 years, 3 months ago (2015-09-11 08:01:34 UTC) #37
ivica
Full stack and loopback tests had different bitrates configuration, so I tried to reproduce that ...
5 years, 3 months ago (2015-09-14 17:12:55 UTC) #38
commit-bot: I haz the power
Dry run: CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/patch-status/1308403003/260001 View timeline at https://chromium-cq-status.appspot.com/patch-timeline/1308403003/260001
5 years, 3 months ago (2015-09-14 17:14:38 UTC) #40
commit-bot: I haz the power
Dry run: Try jobs failed on following builders: android on tryserver.webrtc (JOB_TIMED_OUT, no build URL)
5 years, 3 months ago (2015-09-14 19:14:51 UTC) #42
commit-bot: I haz the power
Dry run: CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/patch-status/1308403003/280001 View timeline at https://chromium-cq-status.appspot.com/patch-timeline/1308403003/280001
5 years, 3 months ago (2015-09-15 07:38:24 UTC) #44
commit-bot: I haz the power
Dry run: Try jobs failed on following builders: win_x64_rel on tryserver.webrtc (JOB_FAILED, http://build.chromium.org/p/tryserver.webrtc/builders/win_x64_rel/builds/9336)
5 years, 3 months ago (2015-09-15 07:53:06 UTC) #46
sprang_webrtc
lgtm. pbos@ ptal!
5 years, 3 months ago (2015-09-15 15:49:12 UTC) #47
pbos-webrtc
https://codereview.webrtc.org/1308403003/diff/280001/webrtc/video/loopback.cc File webrtc/video/loopback.cc (left): https://codereview.webrtc.org/1308403003/diff/280001/webrtc/video/loopback.cc#oldcode1 webrtc/video/loopback.cc:1: /* Diff says empty file, not deleted. Is this ...
5 years, 3 months ago (2015-09-16 12:02:33 UTC) #48
ivica
https://codereview.webrtc.org/1308403003/diff/280001/webrtc/video/loopback.cc File webrtc/video/loopback.cc (left): https://codereview.webrtc.org/1308403003/diff/280001/webrtc/video/loopback.cc#oldcode1 webrtc/video/loopback.cc:1: /* On 2015/09/16 12:02:33, pbos-webrtc wrote: > Diff says ...
5 years, 3 months ago (2015-09-16 12:51:21 UTC) #49
pbos-webrtc
lgtm
5 years, 3 months ago (2015-09-16 12:53:37 UTC) #50
commit-bot: I haz the power
Dry run: CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/patch-status/1308403003/320001 View timeline at https://chromium-cq-status.appspot.com/patch-timeline/1308403003/320001
5 years, 3 months ago (2015-09-16 12:54:08 UTC) #52
commit-bot: I haz the power
Dry run: Try jobs failed on following builders: android on tryserver.webrtc (JOB_TIMED_OUT, no build URL) ...
5 years, 3 months ago (2015-09-16 14:54:24 UTC) #54
commit-bot: I haz the power
Dry run: CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/patch-status/1308403003/340001 View timeline at https://chromium-cq-status.appspot.com/patch-timeline/1308403003/340001
5 years, 3 months ago (2015-09-16 16:08:12 UTC) #56
ivica
Hopefully, now all tests give the same results as before. I was missing min_transmit_bitrate and ...
5 years, 3 months ago (2015-09-16 16:09:22 UTC) #57
pbos-webrtc
lgtm, I was meaning to ask what that 2 was but forgot :( sorry about ...
5 years, 3 months ago (2015-09-16 16:14:19 UTC) #58
commit-bot: I haz the power
Dry run: Try jobs failed on following builders: android on tryserver.webrtc (JOB_TIMED_OUT, no build URL) ...
5 years, 3 months ago (2015-09-16 18:08:35 UTC) #60
commit-bot: I haz the power
Dry run: CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/patch-status/1308403003/360001 View timeline at https://chromium-cq-status.appspot.com/patch-timeline/1308403003/360001
5 years, 3 months ago (2015-09-17 07:53:18 UTC) #62
pbos-webrtc
lgtm
5 years, 3 months ago (2015-09-17 08:30:57 UTC) #63
commit-bot: I haz the power
Dry run: This issue passed the CQ dry run.
5 years, 3 months ago (2015-09-17 11:56:09 UTC) #65
commit-bot: I haz the power
CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/patch-status/1308403003/360001 View timeline at https://chromium-cq-status.appspot.com/patch-timeline/1308403003/360001
5 years, 3 months ago (2015-09-17 12:27:45 UTC) #68
commit-bot: I haz the power
Committed patchset #19 (id:360001)
5 years, 3 months ago (2015-09-17 12:30:28 UTC) #69
commit-bot: I haz the power
5 years, 3 months ago (2015-09-17 12:30:39 UTC) #70
Message was sent while issue was closed.
Patchset 19 (id:??) landed as
https://crrev.com/5d6a06c1d29a2061bcf4b321ffceab477a404d51
Cr-Commit-Position: refs/heads/master@{#9969}

Powered by Google App Engine
This is Rietveld 408576698