New rtc dump analyzing tool in Python

tools/py_event_log_analyzer/misc.py

Index: tools/py_event_log_analyzer/misc.py
diff --git a/tools/py_event_log_analyzer/misc.py b/tools/py_event_log_analyzer/misc.py
new file mode 100644
index 0000000000000000000000000000000000000000..fe188cbd5517e9c46ba3cbb7589c97280df61a98
--- /dev/null
+++ b/tools/py_event_log_analyzer/misc.py
@@ -0,0 +1,75 @@
+#  Copyright (c) 2016 The WebRTC project authors. All Rights Reserved.
+#
+#  Use of this source code is governed by a BSD-style license
+#  that can be found in the LICENSE file in the root of the source
+#  tree. An additional intellectual property rights grant can be found
+#  in the file PATENTS.  All contributing project authors may
+#  be found in the AUTHORS file in the root of the source tree.
+
+"""Utility functions for calculating statistics.
+"""
+
+from __future__ import division
+import collections
+
+
+def count_reordered(sequence_numbers):
+  """Returns number of indices `i` for which
+  sequence_numbres[i] >=  sequence_numbres[i+1]

[kwiberg-webrtc, 2016/05/25 12:42:41]

There's an extra space here. Also, Spanish-sounding spelling!

[hlundin-webrtc, 2016/05/25 12:54:23]

numbres -> numbers, twice.

+  """
+  largest = sequence_numbers[0]

[peah-webrtc, 2016/05/26 06:44:18]

I also think the numbres<->numbers confusion will cause a runtime error on this
line.

+  result = 0
+  for sequence_number in sequence_numbers:

[hlundin-webrtc, 2016/05/25 12:54:23]

You could start the loop on the second item, but it doesn't matter in practice.

[peah-webrtc, 2016/05/26 06:44:18]

I don't think this tests really what the method states it tests.
For instance (assuming increasing sequence numbers)
0 5 3 2
would only be counted as 1 reordered number but according to the method
specification
a reordered number is when sequence_numbres[i] >=  sequence_numbres[i+1]
which would mean that both 5 and 3 are reordered which gives a count of 2.

Please either change the reordering counting, or change the method description
to match what is done.

[aleloi2, 2016/05/30 14:57:55]

I rewrote it to match the specification.

+    if sequence_number < largest:
+      result += 1
+    else:
+      largest = sequence_number
+  return result

[kwiberg-webrtc, 2016/05/25 12:42:41]

Unit test this function!

[kwiberg-webrtc, 2016/05/25 12:42:41]

Hmm.

  return sum(1 for (s1, s2) in zip(sequence_numbers,
                                   sequence_numbers[1:])
             if s1 >= s2)

Or more efficiently,

  return sum(1 for (s1, s2) in itertools.izip(
                 sequence_numbers,
                 itertools.islice(sequence_numbers, 1, None))
             if s1 >= s2)

[aleloi2, 2016/05/30 14:57:55]

Your suggestion is simpler to read. I went with the first one, because it is
easier to read, and because I thought it was the simplest way to handle py2/3
differences. py3's zip behaves as py2's itertools.izip (which doesn't exist in
py3).

+
+
+def ssrc_size_table(data_points):

[hlundin-webrtc, 2016/05/25 12:54:23]

The naming of this and the next functions are inconsistent. This is called
*size* while the next is *percent* -- both return a fraction.

[hlundin-webrtc, 2016/05/25 12:54:23]

Can't this be implemented in terms of percent_table below?

[aleloi2, 2016/05/30 14:57:55]

Both changed in next version.

+  """Returns mapping from a SSRC to its relative occurance proportion in
+  the data.
+  """
+  d = collections.Counter(dt.ssrc for dt in data_points)
+  total = sum(d.values())
+  for key in d:
+    d[key] /= total
+  return d

[kwiberg-webrtc, 2016/05/25 12:42:41]

This method is practically identical to percent_table; remove it.

[aleloi2, 2016/05/30 14:57:55]

See next version!

+
+
+def percent_table(data):

[kwiberg-webrtc, 2016/05/25 12:42:41]

The name is misleading, since you give relative frequencies in the [0, 1] range.

+  """Returns mapping from data element to its relative occurance
+  proportion in the data.
+  """
+  d = collections.Counter(data)
+  total = sum(d.values())

[peah-webrtc, 2016/05/26 06:44:18]

This looks very similar to lines 35-38. It would be nice to unify these methods
to avoid code duplication.

+  for key in d:
+    d[key] /= total
+  return d
+
+
+def unwrap(data, mod):
+  """Unwraps `data` modulo `mod`.
+
+  If data contains consecutive elements with difference above `mod/2`,
+  an integer multiple of `mod` is added to one of the elements in a
+  way that makes the sum of absolute consecutive differences
+  minimized. E.g. unwrap([0, 1, 2, 0, 1, 2, 7, 8], 3) -> [0, 1, 2, 3,
+  4, 5, 4, 5]

[kwiberg-webrtc, 2016/05/25 12:42:41]

This description isn't quite correct. You also add multiples of mod to elements
not in a pair with too-large difference (such as the 5th element in the list in
your example). Perhaps something like this:

"Add integer multiples of mod to all elements of data except the first, such
that all pairs of consecutive elements (a, b) satisfy -mod / 2 <= b - a < mod /
2."

[aleloi2, 2016/05/30 14:57:55]

Thank you, updated!

+
+  """
+  prev = data[0]
+  for idxm1, curr in enumerate(data[1:]):
+    idx = idxm1 + 1
+
+    # calculate positive & negative modular difference
+    delta_pos = (curr-prev)%mod
+    delta_neg = (curr-prev)%mod - mod
+
+    # update next value with minimal absolute difference from previous
+    delta = delta_pos if abs(delta_pos) < abs(delta_neg) else delta_neg
+    curr = prev + delta
+    data[idx] = curr
+    prev = curr
+  return data

[kwiberg-webrtc, 2016/05/25 12:42:41]

Unit test this function!

[kwiberg-webrtc, 2016/05/25 12:42:41]

I think this does the same thing with less overhead:

for i in range(1, len(data)):
  data[i] = (data[i - 1] +
             (data[i] - data[i - 1] + mod // 2) % mod - mod // 2)

[aleloi2, 2016/05/30 14:57:55]

Yes, it does the same thing simpler. Changed!