Script to start stubbed loopback video test with Espresso

webrtc/tools/test_stubbed_loopback_video.py

Index: webrtc/tools/test_stubbed_loopback_video.py
diff --git a/webrtc/tools/test_stubbed_loopback_video.py b/webrtc/tools/test_stubbed_loopback_video.py
new file mode 100755
index 0000000000000000000000000000000000000000..39ec8d39bac16cc235d5fdf530ec05f98cb9dbb8
--- /dev/null
+++ b/webrtc/tools/test_stubbed_loopback_video.py
@@ -0,0 +1,128 @@
+#!/usr/bin/env python
+# Copyright (c) 2017 The WebRTC project authors. All Rights Reserved.
+#
+# Use of this source code is governed by a BSD-style license
+# that can be found in the LICENSE file in the root of the source
+# tree. An additional intellectual property rights grant can be found
+# in the file PATENTS.  All contributing project authors may
+# be found in the AUTHORS file in the root of the source tree.
+
+"""
+This script is the wrapper that starts a loopback call with stubbed video in
+and out. It then analyses the video quality of the output video against the
+reference input video.
+
+It expect to be given the webrtc output build directory as the first argument
+all other arguments are optional.
+
+It assumes you have a Android device plugged in and that you have installed
+AppRTCMobile.apk and AppRTCMobileTestStubbedVideoIO.apk on the device.

[kjellander_webrtc, 2017/01/20 08:18:56]

These don't need to be installed, it will be taken care of by the
run_AppRTCMobileTestStubbedVideoIO script.

[mandermo, 2017/01/23 09:09:30]

Ok, have removed that from the help text.

+"""
+
+import argparse
+import os
+import shutil
+import subprocess
+import sys
+import tempfile
+
+SCRIPT_DIR = os.path.dirname(os.path.abspath(__file__))
+
+SRC_DIR = os.path.normpath(os.path.join(SCRIPT_DIR, os.pardir, os.pardir))
+
+
+def _ParseArgs():
+  parser = argparse.ArgumentParser(description='Start loopback video analysis')
+  parser.add_argument('--source_dir',
+      default=SRC_DIR, help='The path to the WebRTC source directory')
+  parser.add_argument('build_dir_android',
+      help='The path to the build directory for Android')
+  parser.add_argument('--build_dir_x86',
+      help='The path to the build directory for building locally')
+  parser.add_argument('--temp_dir',
+      help='A temporary directory to put the output')
+
+  args = parser.parse_args()
+  return args
+
+
+def main():
+  args = _ParseArgs()
+
+  source_dir = args.source_dir
+  build_dir_android = args.build_dir_android
+  build_dir_x86 = args.build_dir_x86
+  temp_dir = args.temp_dir
+  if not temp_dir:
+    temp_dir = tempfile.mkdtemp()
+  else:
+    if not os.path.exists(temp_dir):
+      os.makedirs(temp_dir)
+
+  if not build_dir_x86:
+    build_dir_x86 = os.path.join(temp_dir, 'LocalBuild')
+    subprocess.check_call(['gn', 'gen', build_dir_x86])

[kjellander_webrtc, 2017/01/20 08:18:56]

It'll be hard to debug what's going on if a command fails since
subprocess.check_call doesn't print what command it invoked.

Can you wrap the check_call invocations in a helper function and utilize the
logging module like:
def _RunCommand(argv, **kwargs):
  logging.info('Running %r', argv)
  subprocess.check_call(argv, **kwargs)

Then initialize logging like:
logging.basicConfig(level=logging.INFO)

[mandermo, 2017/01/23 09:09:30]

Done. Where is best to call logging.basicConfig(level=logging.INFO)? Now I put
it outside a function in before the function declaration have started. Is it
better in main()?

[kjellander_webrtc, 2017/01/23 10:31:47]

You can do it anywhere as long as it executes before logging.info actually
executes. I suggest in the beginning of main() rather than the global scope.

[mandermo, 2017/01/23 12:54:55]

Moved to top of main().

+    subprocess.check_call(['ninja', '-C', build_dir_x86, 'frame_analyzer'])
+
+  toolchain_dir = os.path.join(source_dir, 'tools-webrtc',
+      'video_quality_toolchain')
+
+  # Download ffmpeg and zxing.
+  download_script = os.path.join(toolchain_dir, 'download.py')
+  subprocess.check_call([download_script])
+
+  # Run the Espresso code.
+  espresso_target = os.path.join(build_dir_android,
+      'bin', 'run_AppRTCMobileTestStubbedVideoIO')
+  subprocess.check_call([espresso_target])
+
+  # Pull the output video.
+  test_video = os.path.join(temp_dir, 'test_video.y4m')
+  subprocess.check_call(['adb', 'pull', '/sdcard/output.y4m', test_video])
+
+  test_video_yuv = os.path.join(temp_dir, 'test_video.yuv')
+
+  ffmpeg_path = os.path.join(toolchain_dir, 'linux', 'ffmpeg')
+
+  def convert_video(input_video, output_video):
+    subprocess.check_call([ffmpeg_path, '-y', '-i', input_video, output_video])
+
+  convert_video(test_video, test_video_yuv)
+
+  reference_video = os.path.join(source_dir,
+      'resources', 'reference_video_640x360_30fps.y4m')
+
+  reference_video_yuv = os.path.join(temp_dir,
+      'reference_video_640x360_30fps.yuv')
+
+  convert_video(reference_video, reference_video_yuv)
+
+  # Run compare script.
+  compare_script = os.path.join(source_dir, 'webrtc', 'tools',
+      'compare_videos.py')
+  zxing_path = os.path.join(toolchain_dir, 'linux', 'zxing')
+
+  # The frame_analyzer binary should be built for local computer and not for
+  # Android
+  frame_analyzer = os.path.join(build_dir_x86, 'frame_analyzer')
+
+  frame_width = 640
+  frame_height = 360
+
+  stats_file_ref = os.path.join(temp_dir, 'stats_ref.txt')
+  stats_file_test = os.path.join(temp_dir, 'stats_test.txt')
+
+  subprocess.check_call([
+      compare_script, '--ref_video', reference_video_yuv,
+      '--test_video', test_video_yuv, '--yuv_frame_width', str(frame_width),
+      '--yuv_frame_height', str(frame_height),
+      '--stats_file_ref', stats_file_ref,
+      '--stats_file_test', stats_file_test, '--frame_analyzer', frame_analyzer,
+      '--ffmpeg_path', ffmpeg_path, '--zxing_path', zxing_path])
+
+  shutil.rmtree(temp_dir)
+
+
+if __name__ == '__main__':
+  sys.exit(main())
+