Chromium Code Reviews
|
|
Chromium Code Reviews
Issue
2806373002:
Fix swarming tests not running in parallel (Closed)
|
Created:
3 years, 8 months ago by kjellander_webrtc Modified:
3 years, 8 months ago CC:
webrtc-reviews_webrtc.org, mithro Target Ref:
refs/heads/master Project:
webrtc Visibility:
Public. |
DescriptionFix swarming tests not running in parallel
Due to recent Chrome infra changes in
https://chromium-review.googlesource.com/c/472290/
tests running on swarming are now assumed to emit JSON results
or will be marked as failing. This requires us to use our
gtest-parallel wrapper for all our Swarming tests
(or implement the --isolated-script-test-output flag, which
normally only is implemented by the Chromium test launcher).
The low_bandwidth_audio_test can actually run in parallel,
so just change that.
The webrtc_nonparallel_tests cannot, so this CL changes MB
to pass --workers=1 flag to gtest-parallel, which makes the
tests run in sequence. This adds a little confusion but the root
problem is really that our gtest-parallel script [1] does a lot more
than just running the tests in parallel these days, so it should
probably be renamed.
Also make sure gtest-parallel-wrapper.py [2] consumes the
--isolated-script-test-chartjson-output flag (unused) so we don't
pass it on to the test executable.
[1]: https://chromium.googlesource.com/external/github.com/google/gtest-parallel/+/master/gtest-parallel
[2]: https://chromium.googlesource.com/external/webrtc/+/master/tools-webrtc/gtest-parallel-wrapper.py
BUG=709988
TBR=ehmaldonado@webrtc.org
NOTRY=True
Review-Url: https://codereview.webrtc.org/2806373002
Cr-Commit-Position: refs/heads/master@{#17646}
Committed: https://chromium.googlesource.com/external/webrtc/+/382f2b2c456e87a06df678b9a52986529e637300
Patch Set 1 #Patch Set 2 : Fixing webrtc_nonparallel_tests #Patch Set 3 : Eat --isolated-script-test-chartjson-output flag #Patch Set 4 : Move workers flag #Patch Set 5 : Move flag after executable #Patch Set 6 : Don't add flag for memcheck #
Messages
Total messages: 32 (24 generated)
Description was changed from ========== Run low_bandwidth_audio_test in parallel Due to recent Chrome infra changes in https://chromium-review.googlesource.com/c/472290/ tests running on swarming are now assumed to emit JSON results or will be marked as failing. This reqiures us to use our gtest-parallel wrapper for all our Swarming tests (or implement the --isolated-script-test-output and --isolated-script-test-chartjson-output flags, which normally only is implemented by the Chromium test launcher). BUG=None TBR=oprypin@webrtc.org ========== to ========== Run low_bandwidth_audio_test in parallel Due to recent Chrome infra changes in https://chromium-review.googlesource.com/c/472290/ tests running on swarming are now assumed to emit JSON results or will be marked as failing. This reqiures us to use our gtest-parallel wrapper for all our Swarming tests (or implement the --isolated-script-test-output and --isolated-script-test-chartjson-output flags, which normally only is implemented by the Chromium test launcher). BUG=None TBR=oprypin@webrtc.org ==========
kjellander@webrtc.org changed reviewers: - oprypin@webrtc.org
On 2017/04/10 18:03:45, kjellander_webrtc wrote: > mailto:kjellander@webrtc.org changed reviewers: > - mailto:oprypin@webrtc.org This doesn't help since webrtc_nonparallel_tests also runs without the script. I'll try to update the recipes instead.
Description was changed from ========== Run low_bandwidth_audio_test in parallel Due to recent Chrome infra changes in https://chromium-review.googlesource.com/c/472290/ tests running on swarming are now assumed to emit JSON results or will be marked as failing. This reqiures us to use our gtest-parallel wrapper for all our Swarming tests (or implement the --isolated-script-test-output and --isolated-script-test-chartjson-output flags, which normally only is implemented by the Chromium test launcher). BUG=None TBR=oprypin@webrtc.org ========== to ========== Fix swarming tests not running in parallel Due to recent Chrome infra changes in https://chromium-review.googlesource.com/c/472290/ tests running on swarming are now assumed to emit JSON results or will be marked as failing. This requires us to use our gtest-parallel wrapper for all our Swarming tests (or implement the --isolated-script-test-output and --isolated-script-test-chartjson-output flags, which normally only is implemented by the Chromium test launcher). The low_bandwidth_audio_test can actually run in parallel, so just change that. The webrtc_nonparallel_tests cannot, so this CL changes the MB to pass --workers=1 flag to gtest-parallel, which makes the test run in sequence. BUG=709988 TBR=oprypin@webrtc.org ==========
kjellander@webrtc.org changed reviewers: + oprypin@webrtc.org
Description was changed from ========== Fix swarming tests not running in parallel Due to recent Chrome infra changes in https://chromium-review.googlesource.com/c/472290/ tests running on swarming are now assumed to emit JSON results or will be marked as failing. This requires us to use our gtest-parallel wrapper for all our Swarming tests (or implement the --isolated-script-test-output and --isolated-script-test-chartjson-output flags, which normally only is implemented by the Chromium test launcher). The low_bandwidth_audio_test can actually run in parallel, so just change that. The webrtc_nonparallel_tests cannot, so this CL changes the MB to pass --workers=1 flag to gtest-parallel, which makes the test run in sequence. BUG=709988 TBR=oprypin@webrtc.org ========== to ========== Fix swarming tests not running in parallel Due to recent Chrome infra changes in https://chromium-review.googlesource.com/c/472290/ tests running on swarming are now assumed to emit JSON results or will be marked as failing. This requires us to use our gtest-parallel wrapper for all our Swarming tests (or implement the --isolated-script-test-output flag, which normally only is implemented by the Chromium test launcher). The low_bandwidth_audio_test can actually run in parallel, so just change that. The webrtc_nonparallel_tests cannot, so this CL changes MB to pass --workers=1 flag to gtest-parallel, which makes the test run in sequence. This adds a little confusion but the root problem is really that our gtest-parallel script does a lot more than just running the tests in parallel these days, so it should probably be renamed. BUG=709988 TBR=oprypin@webrtc.org ==========
Description was changed from ========== Fix swarming tests not running in parallel Due to recent Chrome infra changes in https://chromium-review.googlesource.com/c/472290/ tests running on swarming are now assumed to emit JSON results or will be marked as failing. This requires us to use our gtest-parallel wrapper for all our Swarming tests (or implement the --isolated-script-test-output flag, which normally only is implemented by the Chromium test launcher). The low_bandwidth_audio_test can actually run in parallel, so just change that. The webrtc_nonparallel_tests cannot, so this CL changes MB to pass --workers=1 flag to gtest-parallel, which makes the test run in sequence. This adds a little confusion but the root problem is really that our gtest-parallel script does a lot more than just running the tests in parallel these days, so it should probably be renamed. BUG=709988 TBR=oprypin@webrtc.org ========== to ========== Fix swarming tests not running in parallel Due to recent Chrome infra changes in https://chromium-review.googlesource.com/c/472290/ tests running on swarming are now assumed to emit JSON results or will be marked as failing. This requires us to use our gtest-parallel wrapper for all our Swarming tests (or implement the --isolated-script-test-output flag, which normally only is implemented by the Chromium test launcher). The low_bandwidth_audio_test can actually run in parallel, so just change that. The webrtc_nonparallel_tests cannot, so this CL changes MB to pass --workers=1 flag to gtest-parallel, which makes the test run in sequence. This adds a little confusion but the root problem is really that our gtest-parallel script does a lot more than just running the tests in parallel these days, so it should probably be renamed. BUG=709988 TBR=oprypin@webrtc.org ==========
Description was changed from ========== Fix swarming tests not running in parallel Due to recent Chrome infra changes in https://chromium-review.googlesource.com/c/472290/ tests running on swarming are now assumed to emit JSON results or will be marked as failing. This requires us to use our gtest-parallel wrapper for all our Swarming tests (or implement the --isolated-script-test-output flag, which normally only is implemented by the Chromium test launcher). The low_bandwidth_audio_test can actually run in parallel, so just change that. The webrtc_nonparallel_tests cannot, so this CL changes MB to pass --workers=1 flag to gtest-parallel, which makes the test run in sequence. This adds a little confusion but the root problem is really that our gtest-parallel script does a lot more than just running the tests in parallel these days, so it should probably be renamed. BUG=709988 TBR=oprypin@webrtc.org ========== to ========== Fix swarming tests not running in parallel Due to recent Chrome infra changes in https://chromium-review.googlesource.com/c/472290/ tests running on swarming are now assumed to emit JSON results or will be marked as failing. This requires us to use our gtest-parallel wrapper for all our Swarming tests (or implement the --isolated-script-test-output flag, which normally only is implemented by the Chromium test launcher). The low_bandwidth_audio_test can actually run in parallel, so just change that. The webrtc_nonparallel_tests cannot, so this CL changes MB to pass --workers=1 flag to gtest-parallel, which makes the test run in sequence. This adds a little confusion but the root problem is really that our gtest-parallel script does a lot more than just running the tests in parallel these days, so it should probably be renamed. BUG=709988 TBR=ehmaldonado@webrtc.org ==========
kjellander@webrtc.org changed reviewers: + ehmaldonado@webrtc.org
Patchset #3 (id:40001) has been deleted
Patchset #4 (id:80001) has been deleted
kjellander@webrtc.org changed reviewers: - ehmaldonado@webrtc.org, oprypin@webrtc.org
On 2017/04/10 18:04:10, kjellander_webrtc wrote: > On 2017/04/10 18:03:45, kjellander_webrtc wrote: > > mailto:kjellander@webrtc.org changed reviewers: > > - mailto:oprypin@webrtc.org > > This doesn't help since webrtc_nonparallel_tests also runs without the script. > I'll try to update the recipes instead. Actually, it's better to do it here instead. I'll make webrtc_nonparallel_tests use the wrapper script as well, but run the tests in sequence.
Description was changed from ========== Fix swarming tests not running in parallel Due to recent Chrome infra changes in https://chromium-review.googlesource.com/c/472290/ tests running on swarming are now assumed to emit JSON results or will be marked as failing. This requires us to use our gtest-parallel wrapper for all our Swarming tests (or implement the --isolated-script-test-output flag, which normally only is implemented by the Chromium test launcher). The low_bandwidth_audio_test can actually run in parallel, so just change that. The webrtc_nonparallel_tests cannot, so this CL changes MB to pass --workers=1 flag to gtest-parallel, which makes the test run in sequence. This adds a little confusion but the root problem is really that our gtest-parallel script does a lot more than just running the tests in parallel these days, so it should probably be renamed. BUG=709988 TBR=ehmaldonado@webrtc.org ========== to ========== Fix swarming tests not running in parallel Due to recent Chrome infra changes in https://chromium-review.googlesource.com/c/472290/ tests running on swarming are now assumed to emit JSON results or will be marked as failing. This requires us to use our gtest-parallel wrapper for all our Swarming tests (or implement the --isolated-script-test-output flag, which normally only is implemented by the Chromium test launcher). The low_bandwidth_audio_test can actually run in parallel, so just change that. The webrtc_nonparallel_tests cannot, so this CL changes MB to pass --workers=1 flag to gtest-parallel, which makes the test run in sequence. This adds a little confusion but the root problem is really that our gtest-parallel script [1] does a lot more than just running the tests in parallel these days, so it should probably be renamed. [1] https://chromium.googlesource.com/external/github.com/google/gtest-parallel/+... BUG=709988 TBR=ehmaldonado@webrtc.org ==========
Description was changed from ========== Fix swarming tests not running in parallel Due to recent Chrome infra changes in https://chromium-review.googlesource.com/c/472290/ tests running on swarming are now assumed to emit JSON results or will be marked as failing. This requires us to use our gtest-parallel wrapper for all our Swarming tests (or implement the --isolated-script-test-output flag, which normally only is implemented by the Chromium test launcher). The low_bandwidth_audio_test can actually run in parallel, so just change that. The webrtc_nonparallel_tests cannot, so this CL changes MB to pass --workers=1 flag to gtest-parallel, which makes the test run in sequence. This adds a little confusion but the root problem is really that our gtest-parallel script [1] does a lot more than just running the tests in parallel these days, so it should probably be renamed. [1] https://chromium.googlesource.com/external/github.com/google/gtest-parallel/+... BUG=709988 TBR=ehmaldonado@webrtc.org ========== to ========== Fix swarming tests not running in parallel Due to recent Chrome infra changes in https://chromium-review.googlesource.com/c/472290/ tests running on swarming are now assumed to emit JSON results or will be marked as failing. This requires us to use our gtest-parallel wrapper for all our Swarming tests (or implement the --isolated-script-test-output flag, which normally only is implemented by the Chromium test launcher). The low_bandwidth_audio_test can actually run in parallel, so just change that. The webrtc_nonparallel_tests cannot, so this CL changes MB to pass --workers=1 flag to gtest-parallel, which makes the tests run in sequence. This adds a little confusion but the root problem is really that our gtest-parallel script [1] does a lot more than just running the tests in parallel these days, so it should probably be renamed. [1] https://chromium.googlesource.com/external/github.com/google/gtest-parallel/+... BUG=709988 TBR=ehmaldonado@webrtc.org ==========
Description was changed from ========== Fix swarming tests not running in parallel Due to recent Chrome infra changes in https://chromium-review.googlesource.com/c/472290/ tests running on swarming are now assumed to emit JSON results or will be marked as failing. This requires us to use our gtest-parallel wrapper for all our Swarming tests (or implement the --isolated-script-test-output flag, which normally only is implemented by the Chromium test launcher). The low_bandwidth_audio_test can actually run in parallel, so just change that. The webrtc_nonparallel_tests cannot, so this CL changes MB to pass --workers=1 flag to gtest-parallel, which makes the tests run in sequence. This adds a little confusion but the root problem is really that our gtest-parallel script [1] does a lot more than just running the tests in parallel these days, so it should probably be renamed. [1] https://chromium.googlesource.com/external/github.com/google/gtest-parallel/+... BUG=709988 TBR=ehmaldonado@webrtc.org ========== to ========== Fix swarming tests not running in parallel Due to recent Chrome infra changes in https://chromium-review.googlesource.com/c/472290/ tests running on swarming are now assumed to emit JSON results or will be marked as failing. This requires us to use our gtest-parallel wrapper for all our Swarming tests (or implement the --isolated-script-test-output flag, which normally only is implemented by the Chromium test launcher). The low_bandwidth_audio_test can actually run in parallel, so just change that. The webrtc_nonparallel_tests cannot, so this CL changes MB to pass --workers=1 flag to gtest-parallel, which makes the tests run in sequence. This adds a little confusion but the root problem is really that our gtest-parallel script [1] does a lot more than just running the tests in parallel these days, so it should probably be renamed. Also make sure gtest-parallel-wrapper.py [2] consumes the --isolated-script-test-chartjson-output flag (unused) so we don't pass it on to the test executable. [1]: https://chromium.googlesource.com/external/github.com/google/gtest-parallel/+... [2]: https://chromium.googlesource.com/external/webrtc/+/master/tools-webrtc/gtest... BUG=709988 TBR=ehmaldonado@webrtc.org ==========
kjellander@webrtc.org changed reviewers: + ehmaldonado@webrtc.org, mbonadei@webrtc.org
Now I think I nailed this.
The CQ bit was checked by kjellander@webrtc.org to run a CQ dry run
Dry run: CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.webrtc.org/...
On 2017/04/11 09:20:51, kjellander_webrtc wrote: > Now I think I nailed this. lgtm
The CQ bit was unchecked by commit-bot@chromium.org
Dry run: Try jobs failed on following builders: linux_memcheck on master.tryserver.webrtc (JOB_FAILED, http://build.chromium.org/p/tryserver.webrtc/builders/linux_memcheck/builds/7132)
On 2017/04/11 09:29:48, mbonadei wrote: > On 2017/04/11 09:20:51, kjellander_webrtc wrote: > > Now I think I nailed this. > > lgtm *phew*, thanks to us having memcheck enabled by default I caught the fact that --workers=1 cannot be passed for that bot. Made a small fix for that in PS#6 and will submit that now.
Description was changed from ========== Fix swarming tests not running in parallel Due to recent Chrome infra changes in https://chromium-review.googlesource.com/c/472290/ tests running on swarming are now assumed to emit JSON results or will be marked as failing. This requires us to use our gtest-parallel wrapper for all our Swarming tests (or implement the --isolated-script-test-output flag, which normally only is implemented by the Chromium test launcher). The low_bandwidth_audio_test can actually run in parallel, so just change that. The webrtc_nonparallel_tests cannot, so this CL changes MB to pass --workers=1 flag to gtest-parallel, which makes the tests run in sequence. This adds a little confusion but the root problem is really that our gtest-parallel script [1] does a lot more than just running the tests in parallel these days, so it should probably be renamed. Also make sure gtest-parallel-wrapper.py [2] consumes the --isolated-script-test-chartjson-output flag (unused) so we don't pass it on to the test executable. [1]: https://chromium.googlesource.com/external/github.com/google/gtest-parallel/+... [2]: https://chromium.googlesource.com/external/webrtc/+/master/tools-webrtc/gtest... BUG=709988 TBR=ehmaldonado@webrtc.org ========== to ========== Fix swarming tests not running in parallel Due to recent Chrome infra changes in https://chromium-review.googlesource.com/c/472290/ tests running on swarming are now assumed to emit JSON results or will be marked as failing. This requires us to use our gtest-parallel wrapper for all our Swarming tests (or implement the --isolated-script-test-output flag, which normally only is implemented by the Chromium test launcher). The low_bandwidth_audio_test can actually run in parallel, so just change that. The webrtc_nonparallel_tests cannot, so this CL changes MB to pass --workers=1 flag to gtest-parallel, which makes the tests run in sequence. This adds a little confusion but the root problem is really that our gtest-parallel script [1] does a lot more than just running the tests in parallel these days, so it should probably be renamed. Also make sure gtest-parallel-wrapper.py [2] consumes the --isolated-script-test-chartjson-output flag (unused) so we don't pass it on to the test executable. [1]: https://chromium.googlesource.com/external/github.com/google/gtest-parallel/+... [2]: https://chromium.googlesource.com/external/webrtc/+/master/tools-webrtc/gtest... BUG=709988 TBR=ehmaldonado@webrtc.org NOTRY=True ==========
The CQ bit was checked by kjellander@webrtc.org
The patchset sent to the CQ was uploaded after l-g-t-m from mbonadei@webrtc.org Link to the patchset: https://codereview.webrtc.org/2806373002/#ps140001 (title: "Don't add flag for memcheck")
CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.webrtc.org/...
CQ is committing da patch.
Bot data: {"patchset_id": 140001, "attempt_start_ts": 1491908640789720,
"parent_rev": "4fa0c4f97fec5fd7d52ed505ae1d086ded6ceb9a", "commit_rev":
"382f2b2c456e87a06df678b9a52986529e637300"}
Message was sent while issue was closed.
Description was changed from ========== Fix swarming tests not running in parallel Due to recent Chrome infra changes in https://chromium-review.googlesource.com/c/472290/ tests running on swarming are now assumed to emit JSON results or will be marked as failing. This requires us to use our gtest-parallel wrapper for all our Swarming tests (or implement the --isolated-script-test-output flag, which normally only is implemented by the Chromium test launcher). The low_bandwidth_audio_test can actually run in parallel, so just change that. The webrtc_nonparallel_tests cannot, so this CL changes MB to pass --workers=1 flag to gtest-parallel, which makes the tests run in sequence. This adds a little confusion but the root problem is really that our gtest-parallel script [1] does a lot more than just running the tests in parallel these days, so it should probably be renamed. Also make sure gtest-parallel-wrapper.py [2] consumes the --isolated-script-test-chartjson-output flag (unused) so we don't pass it on to the test executable. [1]: https://chromium.googlesource.com/external/github.com/google/gtest-parallel/+... [2]: https://chromium.googlesource.com/external/webrtc/+/master/tools-webrtc/gtest... BUG=709988 TBR=ehmaldonado@webrtc.org NOTRY=True ========== to ========== Fix swarming tests not running in parallel Due to recent Chrome infra changes in https://chromium-review.googlesource.com/c/472290/ tests running on swarming are now assumed to emit JSON results or will be marked as failing. This requires us to use our gtest-parallel wrapper for all our Swarming tests (or implement the --isolated-script-test-output flag, which normally only is implemented by the Chromium test launcher). The low_bandwidth_audio_test can actually run in parallel, so just change that. The webrtc_nonparallel_tests cannot, so this CL changes MB to pass --workers=1 flag to gtest-parallel, which makes the tests run in sequence. This adds a little confusion but the root problem is really that our gtest-parallel script [1] does a lot more than just running the tests in parallel these days, so it should probably be renamed. Also make sure gtest-parallel-wrapper.py [2] consumes the --isolated-script-test-chartjson-output flag (unused) so we don't pass it on to the test executable. [1]: https://chromium.googlesource.com/external/github.com/google/gtest-parallel/+... [2]: https://chromium.googlesource.com/external/webrtc/+/master/tools-webrtc/gtest... BUG=709988 TBR=ehmaldonado@webrtc.org NOTRY=True Review-Url: https://codereview.webrtc.org/2806373002 Cr-Commit-Position: refs/heads/master@{#17646} Committed: https://chromium.googlesource.com/external/webrtc/+/382f2b2c456e87a06df678b9a... ==========
Message was sent while issue was closed.
Committed patchset #6 (id:140001) as https://chromium.googlesource.com/external/webrtc/+/382f2b2c456e87a06df678b9a...
Message was sent while issue was closed.
On 2017/04/11 11:03:43, kjellander_webrtc wrote: > *phew*, thanks to us having memcheck enabled by default I caught the fact that > --workers=1 cannot be passed for that bot. > Made a small fix for that in PS#6 and will submit that now. Yeah, that was almost impossible to catch. |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
