Added GetCpuTime to base/ to get total CPU time consumed by process for perf tests.

webrtc/base/cpu_time_unittest.cc

Index: webrtc/base/cpu_time_unittest.cc
diff --git a/webrtc/base/cpu_time_unittest.cc b/webrtc/base/cpu_time_unittest.cc
new file mode 100644
index 0000000000000000000000000000000000000000..0bde1dfe31428184e4d8d59be94f166a048e3114
--- /dev/null
+++ b/webrtc/base/cpu_time_unittest.cc
@@ -0,0 +1,72 @@
+/*
+ *  Copyright (c) 2017 The WebRTC project authors. All Rights Reserved.
+ *
+ *  Use of this source code is governed by a BSD-style license
+ *  that can be found in the LICENSE file in the root of the source
+ *  tree. An additional intellectual property rights grant can be found
+ *  in the file PATENTS.  All contributing project authors may
+ *  be found in the AUTHORS file in the root of the source tree.
+ */
+
+#include "webrtc/base/cpu_time.h"
+#include "webrtc/base/platform_thread.h"
+#include "webrtc/base/timeutils.h"
+#include "webrtc/test/gtest.h"
+
+namespace internal {

[nisse-webrtc, 2017/02/15 14:27:35]

Just use the unnamed namespace. My understanding is that internal (or rather,
webrtc::internal) is mainly internal things which for some reason has to be put
in otherwise public header file.

[ilnik, 2017/02/15 14:40:45]

Done.

+const double kAllowedErrorSec = 0.1;
+const int kProcessingTimeSec = 1;
+
+bool WaitingFunction(void* counter_pointer) {
+  int64_t& counter = *reinterpret_cast<int64_t*>(counter_pointer);
+  counter = 0;
+  int64_t stop_time =
+      rtc::SystemTimeNanos() + kProcessingTimeSec * rtc::kNumNanosecsPerSec;
+  while (rtc::SystemTimeNanos() < stop_time) {
+    counter++;
+  }
+  return true;
+}
+}  // namespace internal
+
+namespace rtc {
+
+TEST(GetCpuTimeTest, SingleThread) {
+  int64_t start = GetCpuTime();
+  int64_t* counter = new int64_t;
+  ::internal::WaitingFunction(reinterpret_cast<void*>(counter));
+  EXPECT_GT(*counter, 0);
+  delete counter;
+  int64_t duration = GetCpuTime() - start;
+  EXPECT_GE(static_cast<double>(duration) / kNumNanosecsPerSec,
+            ::internal::kProcessingTimeSec - ::internal::kAllowedErrorSec);

[nisse-webrtc, 2017/02/15 14:27:35]

Is there anyway you can rearrange this test to not have a tolerance threshold?

If you spin based on wallclock time (as you do above), then you should get the
strict bound cputime <= wallclock time, with a possibly large difference if
competing for cpu with other threads. (Maybe add a little margin, in case the
two clocks for some reason aren't based on exactly the same frequency).

A strict bound in the other direction is harder, but maybe it's sufficient for
this test to check that the measured cputime is > 0?

[ilnik, 2017/02/15 14:40:45]

Unfortunately, these CPU counters are not very precise. On my machine I get
consistently 0.001-0.002s error. I could decrease the tolerance a bit.

[nisse-webrtc, 2017/02/15 14:48:33]

I'm more concerned that tests may be flaky, i.e., randomly fail if the machine
running them happen to have high load. So if you need a threshold, please make
include a pretty large margin.

[ilnik, 2017/02/15 15:07:10]

That's the great thing about CPU time - it should not depend on load of the
system as CPU time of only current process is measured. There are some issues
with precision, but it doesn't change over time (on my linux workstation it's
always 0.001-0.002s less than expected).

[ilnik, 2017/02/15 15:19:16]

Also, I just realized, that's the problem with the tests: they are affected by
system load. These 0.001 lost seconds are other processes stealing work from my
waiting function. Unfortunately, it's impossible to test CPU time without
writing function which takes exactly fixed time to compute using wall timers.

I think unittests here are even more complicated than the function itself and
are not needed.

[nisse-webrtc, 2017/02/15 15:34:35]

I think you could do a meaningful test like this:

const int64_t kTestTimeNs = 300 * rtc::kNumMillisecondsPerSecond;

int64_t wall_clock_start = rtc::TimeNanos();
int64_t cpu_clock_start = rtc::GetProcessCpuTime();

while (rtc::TimeNanos() < wall_clock_start + kTestTimeNs)
  ;

int64_t cpu_clock_duration = rtc::GetPRocesscpuTime() - cpu_clock_start;
int64_t wall_clock_duration = rtc::TimeNanos() - wall_clock_start;

EXPECT_GT(cpu_clock_duration, 0);
EXPECT_LE(cpu_clock_duration, wall_clock_duration);

+  EXPECT_LE(static_cast<double>(duration) / kNumNanosecsPerSec,
+            ::internal::kProcessingTimeSec + ::internal::kAllowedErrorSec);
+}
+
+TEST(GetCpuTimeTest, TwoThreads) {

[nisse-webrtc, 2017/02/15 14:27:35]

This makes me wonder what the cputime is going to be used for. For the usecases
I imagine, I'd expect a thread-specific cpu timer is more relevant. I.e.,
CLOCK_THREAD_CPUTIME_ID on linux (no idea how these things work on windows, or
if RUSAGE_THREAD is supported on Mac OS).  

[ilnik, 2017/02/15 14:40:45]

I am going to measure total CPU time consumed by test to infer total cpu usage. 

[nisse-webrtc, 2017/02/15 14:48:33]

I see. Then I have two suggestions:

1. Rename it to include "Process" somewhere in the name. Otherwise, I think
including other threads will be unexpected to anyone looking to using this for
benchmarking.

2. If it's intended for tests only, move it to a test target in the BUILD.gn
file.

[ilnik, 2017/02/15 15:07:10]

1. is GetProcessCpuTime() ok?

2. I could just include it to the source set named "rtc_base_tests_utils".
But none of the stuff there has any unittests. Should I purge my unittest. It
seams to pass on every trybot and there is no changes for that function intended
in the future.

[nisse-webrtc, 2017/02/15 15:34:35]

I think so.
I don't think unit tests are an obstacle to putting things in that target.

+  int64_t start = GetCpuTime();
+  int64_t* counter1 = new int64_t;
+  int64_t* counter2 = new int64_t;
+  PlatformThread thread1(::internal::WaitingFunction,
+                         reinterpret_cast<void*>(counter1), "Thread1");
+  PlatformThread thread2(::internal::WaitingFunction,
+                         reinterpret_cast<void*>(counter2), "Thread2");
+  thread1.Start();
+  thread2.Start();
+  thread1.Stop();
+  thread2.Stop();
+
+  EXPECT_GE(*counter1, 0);
+  EXPECT_GE(*counter2, 0);
+  delete counter1;
+  delete counter2;
+  int64_t duration = GetCpuTime() - start;
+  EXPECT_GE(static_cast<double>(duration) / kNumNanosecsPerSec,
+            2 * ::internal::kProcessingTimeSec - ::internal::kAllowedErrorSec);
+  //  If .Stop() also consumes CPU time, there will be 3 active threads.
+  EXPECT_LE(static_cast<double>(duration) / kNumNanosecsPerSec,
+            3 * ::internal::kProcessingTimeSec + ::internal::kAllowedErrorSec);
+}
+
+}  // namespace rtc