Script to start stubbed loopback video test with Espresso

webrtc/tools/test_stubbed_loopback_video.py

Index: webrtc/tools/test_stubbed_loopback_video.py
diff --git a/webrtc/tools/test_stubbed_loopback_video.py b/webrtc/tools/test_stubbed_loopback_video.py
new file mode 100755
index 0000000000000000000000000000000000000000..aada1269e78029bc1e74a76613454674ce2306c1
--- /dev/null
+++ b/webrtc/tools/test_stubbed_loopback_video.py
@@ -0,0 +1,111 @@
+#!/usr/bin/env python
+# Copyright (c) 2017 The WebRTC project authors. All Rights Reserved.
+#
+# Use of this source code is governed by a BSD-style license
+# that can be found in the LICENSE file in the root of the source
+# tree. An additional intellectual property rights grant can be found
+# in the file PATENTS.  All contributing project authors may
+# be found in the AUTHORS file in the root of the source tree.
+
+import argparse
+import os
+import shutil
+import subprocess
+import sys
+import tempfile
+
+SCRIPT_DIR = os.path.dirname(os.path.abspath(__file__))
+
+WEBRTC_DIR = os.path.normpath(SCRIPT_DIR + '/../../')
+
+def _ParseArgs():
+  parser = argparse.ArgumentParser(description='Start loopback video analysis')
+  parser.add_argument('--source_dir',
+      default=WEBRTC_DIR, help='The path to the WebRTC source directory')
+  parser.add_argument('build_dir_android',
+      help='The path to the build directory for Android')
+  parser.add_argument('--build_dir_x86',
+      help='The path to the build directory for building locally')
+  parser.add_argument('--temp_dir',
+      help='A temporary directory to put the output')
+
+  args = parser.parse_args()
+  return args
+
+def main():
+  args = _ParseArgs()
+
+  source_dir = args.source_dir
+  build_dir_android = args.build_dir_android
+  build_dir_x86 = args.build_dir_x86
+  temp_dir = args.temp_dir
+  if not temp_dir:
+    temp_dir = tempfile.mkdtemp()
+  else:
+    if not os.path.exists(temp_dir):
+      os.makedirs(temp_dir)
+
+  if not build_dir_x86:

[ehmaldonado_webrtc, 2017/01/17 16:02:36]

Why not do the same with build_dir_android?

[mandermo, 2017/01/18 10:21:24]

I got the impression that in the CI the android version is already built, while
the x86 version is not.

+    build_dir_x86 = temp_dir + '/LocalBuild'

[ehmaldonado_webrtc, 2017/01/17 16:02:36]

Maybe use rather os.path.join() here and below to join paths?

[mandermo, 2017/01/18 10:21:24]

Done.

+    subprocess.check_call(['gn', 'gen', build_dir_x86])
+    subprocess.check_call(['ninja', '-C', build_dir_x86, 'frame_analyzer'])
+
+  # Download ffmpeg and zxing.
+  download_script = source_dir + \
+      '/tools-webrtc/video_quality_toolchain/download.py'
+  subprocess.check_call([download_script])
+
+  # Run the Espresso code.
+  espresso_target = build_dir_android + \
+      '/bin/run_AppRTCMobileTestStubbedVideoIO'
+  subprocess.check_call([espresso_target])
+
+  # Pull the output video.
+  test_video = temp_dir + '/test_video.y4m'
+  subprocess.check_call(['adb', 'pull', '/sdcard/output.y4m', test_video])
+
+  test_video_yuv = temp_dir + '/test_video.yuv'
+
+  ffmpeg_path = source_dir + \
+      '/tools-webrtc/video_quality_toolchain/linux/ffmpeg'
+
+  def convert_video(input_video, output_video):
+    subprocess.check_call([ffmpeg_path, '-y', '-i', input_video, output_video])
+
+  # Convert the test video.
+  convert_video(test_video, test_video_yuv)
+
+  reference_video = source_dir + '/resources/reference_video_640x360_30fps.y4m'
+
+  reference_video_yuv = temp_dir + '/reference_video_640x360_30fps.yuv'
+
+  # Convert the reference video.
+  convert_video(reference_video, reference_video_yuv)
+
+  # Run compare script.
+  compare_script = source_dir + '/webrtc/tools/compare_videos.py'
+  zxing_path = source_dir + '/tools-webrtc/video_quality_toolchain/linux/zxing'
+
+  # The frame_analyzer binary should be built for local computer and not for
+  # Android
+  frame_analyzer = build_dir_x86 + '/frame_analyzer'
+
+  frame_width = 640
+  frame_height = 360
+
+  stats_file_ref = temp_dir + '/stats_ref.txt'
+  stats_file_test = temp_dir + '/stats_test.txt'
+
+  subprocess.check_call([
+      compare_script, '--ref_video', reference_video_yuv,
+      '--test_video', test_video_yuv, '--yuv_frame_width', str(frame_width),
+      '--yuv_frame_height', str(frame_height),
+      '--stats_file_ref', stats_file_ref,
+      '--stats_file_test', stats_file_test, '--frame_analyzer', frame_analyzer,
+      '--ffmpeg_path', ffmpeg_path, '--zxing_path', zxing_path])
+
+  shutil.rmtree(temp_dir)

[ehmaldonado_webrtc, 2017/01/17 16:02:36]

Just curious: Why request a temp_dir as a flag if the results won't be saved?

[mandermo, 2017/01/18 10:21:24]

Maybe there is no need actually, I thought maybe that some CI environment needed
exact control of where to save temporaries. We can check with kjellander@ what
he thinks.

[kjellander_webrtc, 2017/01/19 13:55:34]

I thought it could be useful to pass ${ISOLATED_OUTDIR} as in
https://cs.chromium.org/chromium/src/tools/mb/mb.py?rcl=0&l=1100 for things like
this, to get the stats files archived (and also in this case the built frame
analyzer binary).

We might want to avoid archiving the reference_video_640x360_30fps.yuv for each
build though, since it would be a lot of waste archiving an identical file each
time. The test_video.y4m is interesting though.

+
+if __name__ == '__main__':
+  sys.exit(main())
+