|
|
Created:
5 years, 1 month ago by pbos-webrtc Modified:
5 years ago CC:
webrtc-reviews_webrtc.org, yujie_mao (webrtc), Andrew MacDonald, stefan-webrtc, tterriberry_mozilla.com, qiang.lu, niklas.enbom, mflodman, peah-webrtc, the sun, perkj_webrtc, andresp Base URL:
https://chromium.googlesource.com/external/webrtc.git@master Target Ref:
refs/pending/heads/master Project:
webrtc Visibility:
Public. |
DescriptionRemove 'video_engine_core_unittests' binary.
Merges tests into 'video_engine_tests' to reduce the number of test
targets.
BUG=webrtc:1695
R=kjellander@webrtc.org, phoglund@webrtc.org
Committed: https://crrev.com/bc32ab458baca116ec571ad5e0b1199502101334
Cr-Commit-Position: refs/heads/master@{#10891}
Patch Set 1 #Patch Set 2 : rebase #Patch Set 3 : rebase #
Messages
Total messages: 34 (12 generated)
PTAL, today is your lucky day.
What did we decide on this one?
kjellander@google.com changed reviewers: + kjellander@google.com, phoglund@webrtc.org
Giving this some thought, I think I prefer moving tests into four large binaries: webrtc_unittests (fast) webrtc_tests (larger) webrtc_perf_tests (for perf measurements, running on baremetal) webrtc_device_tests (tests that require hardware) This CL merges tests from the first category into the second, which is not the direction I'd prefer.
On 2015/11/18 19:58:53, kjellander (google.com) wrote: > Giving this some thought, I think I prefer moving tests into four large > binaries: > webrtc_unittests (fast) > webrtc_tests (larger) > webrtc_perf_tests (for perf measurements, running on baremetal) > webrtc_device_tests (tests that require hardware) > > This CL merges tests from the first category into the second, which is not the > direction I'd prefer. I'd like to merge the first two of those in this suggestion, which this is in line with. Filtering tests is more important when developing (only run my tests), and running tests on the bot will be faster if we have fewer targets. Also we're historically not very good at making that distinction, modules_unittests is currently significantly slower than modules_tests. :)
On 2015/11/19 10:44:12, pbos-webrtc wrote: > On 2015/11/18 19:58:53, kjellander (http://google.com) wrote: > > Giving this some thought, I think I prefer moving tests into four large > > binaries: > > webrtc_unittests (fast) > > webrtc_tests (larger) > > webrtc_perf_tests (for perf measurements, running on baremetal) > > webrtc_device_tests (tests that require hardware) > > > > This CL merges tests from the first category into the second, which is not the > > direction I'd prefer. > > I'd like to merge the first two of those in this suggestion, which this is in > line with. Filtering tests is more important when developing (only run my > tests), and running tests on the bot will be faster if we have fewer targets. I guess I can live with that, if Patrik agrees. > Also we're historically not very good at making that distinction, > modules_unittests is currently significantly slower than modules_tests. :) Right, but old wrongdoing is not a good reason for continue doing something wrong ;-)
On 2015/11/19 12:17:06, kjellander (google.com) wrote: > On 2015/11/19 10:44:12, pbos-webrtc wrote: > > On 2015/11/18 19:58:53, kjellander (http://google.com) wrote: > > > Giving this some thought, I think I prefer moving tests into four large > > > binaries: > > > webrtc_unittests (fast) > > > webrtc_tests (larger) > > > webrtc_perf_tests (for perf measurements, running on baremetal) > > > webrtc_device_tests (tests that require hardware) > > > > > > This CL merges tests from the first category into the second, which is not > the > > > direction I'd prefer. > > > > I'd like to merge the first two of those in this suggestion, which this is in > > line with. Filtering tests is more important when developing (only run my > > tests), and running tests on the bot will be faster if we have fewer targets. > > I guess I can live with that, if Patrik agrees. phoglund@: WDYT? Less is more? :) > > Also we're historically not very good at making that distinction, > > modules_unittests is currently significantly slower than modules_tests. :) > > Right, but old wrongdoing is not a good reason for continue doing something > wrong ;-) Yeah, but my point is that it's both hard to find a strict line for when an unittest becomes a test (if measured in milliseconds), and that we're not very likely to actually enforce it. :)
On 2015/11/19 12:18:31, pbos-webrtc wrote: > On 2015/11/19 12:17:06, kjellander (http://google.com) wrote: > > On 2015/11/19 10:44:12, pbos-webrtc wrote: > > > On 2015/11/18 19:58:53, kjellander (http://google.com) wrote: > > > > Giving this some thought, I think I prefer moving tests into four large > > > > binaries: > > > > webrtc_unittests (fast) > > > > webrtc_tests (larger) > > > > webrtc_perf_tests (for perf measurements, running on baremetal) > > > > webrtc_device_tests (tests that require hardware) > > > > > > > > This CL merges tests from the first category into the second, which is not > > the > > > > direction I'd prefer. > > > > > > I'd like to merge the first two of those in this suggestion, which this is > in > > > line with. Filtering tests is more important when developing (only run my > > > tests), and running tests on the bot will be faster if we have fewer > targets. > > > > I guess I can live with that, if Patrik agrees. > > phoglund@: WDYT? Less is more? :) > > > > > Also we're historically not very good at making that distinction, > > > modules_unittests is currently significantly slower than modules_tests. :) > > > > Right, but old wrongdoing is not a good reason for continue doing something > > wrong ;-) > > Yeah, but my point is that it's both hard to find a strict line for when an > unittest becomes a test (if measured in milliseconds), and that we're not very > likely to actually enforce it. :) The only reason I can see for splitting up binaries is if devs build and run the tests often: they will filter so runtime isn't a big deal, but small binaries are faster to build. Otherwise it's all upsides with fewer binaries; probably less linker overhead, less config, less looking for the proper binary to put new tests in, faster execution on the bots if our parallelization is efficient. You are right that few unit tests in WebRTC are proper, clean unit tests, so I don't see a reason to try and make that distinction. I think the only important distinction nowadays is parallel and nonparallel. Ergo: ship it. lgtm
Description was changed from ========== Remove 'video_engine_core_unittests' binary. Merges tests into 'video_engine_tests' to reduce the number of test targets. BUG=webrtc:1695 R=kjellander@webrtc.org ========== to ========== Remove 'video_engine_core_unittests' binary. Merges tests into 'video_engine_tests' to reduce the number of test targets. BUG=webrtc:1695 R=kjellander@webrtc.org, phoglund@webrtc.org ==========
On 2015/11/19 13:06:54, phoglund wrote: > On 2015/11/19 12:18:31, pbos-webrtc wrote: > > On 2015/11/19 12:17:06, kjellander (http://google.com) wrote: > > > On 2015/11/19 10:44:12, pbos-webrtc wrote: > > > > On 2015/11/18 19:58:53, kjellander (http://google.com) wrote: > > > > > Giving this some thought, I think I prefer moving tests into four large > > > > > binaries: > > > > > webrtc_unittests (fast) > > > > > webrtc_tests (larger) > > > > > webrtc_perf_tests (for perf measurements, running on baremetal) > > > > > webrtc_device_tests (tests that require hardware) > > > > > > > > > > This CL merges tests from the first category into the second, which is > not > > > the > > > > > direction I'd prefer. > > > > > > > > I'd like to merge the first two of those in this suggestion, which this is > > in > > > > line with. Filtering tests is more important when developing (only run my > > > > tests), and running tests on the bot will be faster if we have fewer > > targets. > > > > > > I guess I can live with that, if Patrik agrees. > > > > phoglund@: WDYT? Less is more? :) > > > > > > > > Also we're historically not very good at making that distinction, > > > > modules_unittests is currently significantly slower than modules_tests. :) > > > > > > Right, but old wrongdoing is not a good reason for continue doing something > > > wrong ;-) > > > > Yeah, but my point is that it's both hard to find a strict line for when an > > unittest becomes a test (if measured in milliseconds), and that we're not very > > likely to actually enforce it. :) > > The only reason I can see for splitting up binaries is if devs build and run > the tests often: they will filter so runtime isn't a big deal, but small > binaries > are faster to build. > > Otherwise it's all upsides with fewer binaries; probably less linker overhead, > less config, less looking for the proper binary to put new tests in, faster > execution on the bots if our parallelization is efficient. > > You are right that few unit tests in WebRTC are proper, clean unit tests, > so I don't see a reason to try and make that distinction. I think the only > important distinction nowadays is parallel and nonparallel. > > Ergo: ship it. lgtm Alright then we agree on reducing the numbers. Just let me remove the test from the bots before committing this. That is, wait for https://codereview.chromium.org/1454423003/ to land first.
On 2015/11/19 14:24:59, kjellander (webrtc) wrote: > On 2015/11/19 13:06:54, phoglund wrote: > > On 2015/11/19 12:18:31, pbos-webrtc wrote: > > > On 2015/11/19 12:17:06, kjellander (http://google.com) wrote: > > > > On 2015/11/19 10:44:12, pbos-webrtc wrote: > > > > > On 2015/11/18 19:58:53, kjellander (http://google.com) wrote: > > > > > > Giving this some thought, I think I prefer moving tests into four > large > > > > > > binaries: > > > > > > webrtc_unittests (fast) > > > > > > webrtc_tests (larger) > > > > > > webrtc_perf_tests (for perf measurements, running on baremetal) > > > > > > webrtc_device_tests (tests that require hardware) > > > > > > > > > > > > This CL merges tests from the first category into the second, which is > > not > > > > the > > > > > > direction I'd prefer. > > > > > > > > > > I'd like to merge the first two of those in this suggestion, which this > is > > > in > > > > > line with. Filtering tests is more important when developing (only run > my > > > > > tests), and running tests on the bot will be faster if we have fewer > > > targets. > > > > > > > > I guess I can live with that, if Patrik agrees. > > > > > > phoglund@: WDYT? Less is more? :) > > > > > > > > > > > Also we're historically not very good at making that distinction, > > > > > modules_unittests is currently significantly slower than modules_tests. > :) > > > > > > > > Right, but old wrongdoing is not a good reason for continue doing > something > > > > wrong ;-) > > > > > > Yeah, but my point is that it's both hard to find a strict line for when an > > > unittest becomes a test (if measured in milliseconds), and that we're not > very > > > likely to actually enforce it. :) > > > > The only reason I can see for splitting up binaries is if devs build and run > > the tests often: they will filter so runtime isn't a big deal, but small > > binaries > > are faster to build. > > > > Otherwise it's all upsides with fewer binaries; probably less linker overhead, > > less config, less looking for the proper binary to put new tests in, faster > > execution on the bots if our parallelization is efficient. > > > > You are right that few unit tests in WebRTC are proper, clean unit tests, > > so I don't see a reason to try and make that distinction. I think the only > > important distinction nowadays is parallel and nonparallel. > > > > Ergo: ship it. lgtm > > Alright then we agree on reducing the numbers. Just let me remove the test from > the bots before committing this. > That is, wait for https://codereview.chromium.org/1454423003/?_ga=1.86706012.532910034.1361177937 to land first. This is landed, oui? :)
On 2015/11/27 16:54:19, pbos-webrtc wrote: > On 2015/11/19 14:24:59, kjellander (webrtc) wrote: > > On 2015/11/19 13:06:54, phoglund wrote: > > > On 2015/11/19 12:18:31, pbos-webrtc wrote: > > > > On 2015/11/19 12:17:06, kjellander (http://google.com) wrote: > > > > > On 2015/11/19 10:44:12, pbos-webrtc wrote: > > > > > > On 2015/11/18 19:58:53, kjellander (http://google.com) wrote: > > > > > > > Giving this some thought, I think I prefer moving tests into four > > large > > > > > > > binaries: > > > > > > > webrtc_unittests (fast) > > > > > > > webrtc_tests (larger) > > > > > > > webrtc_perf_tests (for perf measurements, running on baremetal) > > > > > > > webrtc_device_tests (tests that require hardware) > > > > > > > > > > > > > > This CL merges tests from the first category into the second, which > is > > > not > > > > > the > > > > > > > direction I'd prefer. > > > > > > > > > > > > I'd like to merge the first two of those in this suggestion, which > this > > is > > > > in > > > > > > line with. Filtering tests is more important when developing (only run > > my > > > > > > tests), and running tests on the bot will be faster if we have fewer > > > > targets. > > > > > > > > > > I guess I can live with that, if Patrik agrees. > > > > > > > > phoglund@: WDYT? Less is more? :) > > > > > > > > > > > > > > Also we're historically not very good at making that distinction, > > > > > > modules_unittests is currently significantly slower than > modules_tests. > > :) > > > > > > > > > > Right, but old wrongdoing is not a good reason for continue doing > > something > > > > > wrong ;-) > > > > > > > > Yeah, but my point is that it's both hard to find a strict line for when > an > > > > unittest becomes a test (if measured in milliseconds), and that we're not > > very > > > > likely to actually enforce it. :) > > > > > > The only reason I can see for splitting up binaries is if devs build and run > > > the tests often: they will filter so runtime isn't a big deal, but small > > > binaries > > > are faster to build. > > > > > > Otherwise it's all upsides with fewer binaries; probably less linker > overhead, > > > less config, less looking for the proper binary to put new tests in, faster > > > execution on the bots if our parallelization is efficient. > > > > > > You are right that few unit tests in WebRTC are proper, clean unit tests, > > > so I don't see a reason to try and make that distinction. I think the only > > > important distinction nowadays is parallel and nonparallel. > > > > > > Ergo: ship it. lgtm > > > > Alright then we agree on reducing the numbers. Just let me remove the test > from > > the bots before committing this. > > That is, wait for > https://codereview.chromium.org/1454423003/?_ga=1.86706012.532910034.1361177937 > to land first. > > This is landed, oui? :) Oui, monsieur.
On 2015/11/27 18:24:15, kjellander (webrtc) wrote: > On 2015/11/27 16:54:19, pbos-webrtc wrote: > > On 2015/11/19 14:24:59, kjellander (webrtc) wrote: > > > On 2015/11/19 13:06:54, phoglund wrote: > > > > On 2015/11/19 12:18:31, pbos-webrtc wrote: > > > > > On 2015/11/19 12:17:06, kjellander (http://google.com) wrote: > > > > > > On 2015/11/19 10:44:12, pbos-webrtc wrote: > > > > > > > On 2015/11/18 19:58:53, kjellander (http://google.com) wrote: > > > > > > > > Giving this some thought, I think I prefer moving tests into four > > > large > > > > > > > > binaries: > > > > > > > > webrtc_unittests (fast) > > > > > > > > webrtc_tests (larger) > > > > > > > > webrtc_perf_tests (for perf measurements, running on baremetal) > > > > > > > > webrtc_device_tests (tests that require hardware) > > > > > > > > > > > > > > > > This CL merges tests from the first category into the second, > which > > is > > > > not > > > > > > the > > > > > > > > direction I'd prefer. > > > > > > > > > > > > > > I'd like to merge the first two of those in this suggestion, which > > this > > > is > > > > > in > > > > > > > line with. Filtering tests is more important when developing (only > run > > > my > > > > > > > tests), and running tests on the bot will be faster if we have fewer > > > > > targets. > > > > > > > > > > > > I guess I can live with that, if Patrik agrees. > > > > > > > > > > phoglund@: WDYT? Less is more? :) > > > > > > > > > > > > > > > > > Also we're historically not very good at making that distinction, > > > > > > > modules_unittests is currently significantly slower than > > modules_tests. > > > :) > > > > > > > > > > > > Right, but old wrongdoing is not a good reason for continue doing > > > something > > > > > > wrong ;-) > > > > > > > > > > Yeah, but my point is that it's both hard to find a strict line for when > > an > > > > > unittest becomes a test (if measured in milliseconds), and that we're > not > > > very > > > > > likely to actually enforce it. :) > > > > > > > > The only reason I can see for splitting up binaries is if devs build and > run > > > > the tests often: they will filter so runtime isn't a big deal, but small > > > > binaries > > > > are faster to build. > > > > > > > > Otherwise it's all upsides with fewer binaries; probably less linker > > overhead, > > > > less config, less looking for the proper binary to put new tests in, > faster > > > > execution on the bots if our parallelization is efficient. > > > > > > > > You are right that few unit tests in WebRTC are proper, clean unit tests, > > > > so I don't see a reason to try and make that distinction. I think the only > > > > important distinction nowadays is parallel and nonparallel. > > > > > > > > Ergo: ship it. lgtm > > > > > > Alright then we agree on reducing the numbers. Just let me remove the test > > from > > > the bots before committing this. > > > That is, wait for > > > https://codereview.chromium.org/1454423003/?_ga=1.86706012.532910034.1361177937 > > to land first. > > > > This is landed, oui? :) > > Oui, monsieur. wouldst thou revieweth my changelist?
lgtm è voila!
rebase
The CQ bit was checked by pbos@webrtc.org
The patchset sent to the CQ was uploaded after l-g-t-m from phoglund@webrtc.org, kjellander@webrtc.org Link to the patchset: https://codereview.webrtc.org/1409803007/#ps20001 (title: "rebase")
CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/patch-status/1409803007/20001 View timeline at https://chromium-cq-status.appspot.com/patch-timeline/1409803007/20001
The CQ bit was unchecked by commit-bot@chromium.org
Try jobs failed on following builders: linux_tsan2 on tryserver.webrtc (JOB_FAILED, http://build.chromium.org/p/tryserver.webrtc/builders/linux_tsan2/builds/8322)
The CQ bit was checked by pbos@webrtc.org
CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/patch-status/1409803007/20001 View timeline at https://chromium-cq-status.appspot.com/patch-timeline/1409803007/20001
The CQ bit was unchecked by commit-bot@chromium.org
Try jobs failed on following builders: win_drmemory_light on tryserver.webrtc (JOB_FAILED, http://build.chromium.org/p/tryserver.webrtc/builders/win_drmemory_light/buil...)
rebase
The CQ bit was checked by pbos@webrtc.org
The patchset sent to the CQ was uploaded after l-g-t-m from phoglund@webrtc.org, kjellander@webrtc.org Link to the patchset: https://codereview.webrtc.org/1409803007/#ps40001 (title: "rebase")
CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/patch-status/1409803007/40001 View timeline at https://chromium-cq-status.appspot.com/patch-timeline/1409803007/40001
The CQ bit was unchecked by commit-bot@chromium.org
Try jobs failed on following builders: android_arm64_rel on tryserver.webrtc (JOB_TIMED_OUT, no build URL) linux_baremetal on tryserver.webrtc (JOB_TIMED_OUT, no build URL) linux_gn_dbg on tryserver.webrtc (JOB_TIMED_OUT, no build URL) linux_msan on tryserver.webrtc (JOB_TIMED_OUT, no build URL) mac_baremetal on tryserver.webrtc (JOB_TIMED_OUT, no build URL) mac_rel on tryserver.webrtc (JOB_TIMED_OUT, no build URL) mac_x64_gn_rel on tryserver.webrtc (JOB_TIMED_OUT, no build URL) win_baremetal on tryserver.webrtc (JOB_TIMED_OUT, no build URL)
Message was sent while issue was closed.
Description was changed from ========== Remove 'video_engine_core_unittests' binary. Merges tests into 'video_engine_tests' to reduce the number of test targets. BUG=webrtc:1695 R=kjellander@webrtc.org, phoglund@webrtc.org ========== to ========== Remove 'video_engine_core_unittests' binary. Merges tests into 'video_engine_tests' to reduce the number of test targets. BUG=webrtc:1695 R=kjellander@webrtc.org, phoglund@webrtc.org Committed: https://chromium.googlesource.com/external/webrtc/+/bc32ab458baca116ec571ad5e... ==========
Message was sent while issue was closed.
Description was changed from ========== Remove 'video_engine_core_unittests' binary. Merges tests into 'video_engine_tests' to reduce the number of test targets. BUG=webrtc:1695 R=kjellander@webrtc.org, phoglund@webrtc.org Committed: https://chromium.googlesource.com/external/webrtc/+/bc32ab458baca116ec571ad5e... ========== to ========== Remove 'video_engine_core_unittests' binary. Merges tests into 'video_engine_tests' to reduce the number of test targets. BUG=webrtc:1695 R=kjellander@webrtc.org, phoglund@webrtc.org Committed: https://crrev.com/bc32ab458baca116ec571ad5e0b1199502101334 Cr-Commit-Position: refs/heads/master@{#10891} ==========
Message was sent while issue was closed.
Committed patchset #3 (id:40001) manually as bc32ab458baca116ec571ad5e0b1199502101334 (presubmit successful).
Message was sent while issue was closed.
Patchset 3 (id:??) landed as https://crrev.com/bc32ab458baca116ec571ad5e0b1199502101334 Cr-Commit-Position: refs/heads/master@{#10891} |