Support for video file instead of camera and output video out to file

webrtc/api/android/java/src/org/webrtc/GlVideoFileRenderer.java

Index: webrtc/api/android/java/src/org/webrtc/GlVideoFileRenderer.java
diff --git a/webrtc/api/android/java/src/org/webrtc/GlVideoFileRenderer.java b/webrtc/api/android/java/src/org/webrtc/GlVideoFileRenderer.java
new file mode 100644
index 0000000000000000000000000000000000000000..6040231df8e9c20a1ba1cfde72ccbde05649b5fe
--- /dev/null
+++ b/webrtc/api/android/java/src/org/webrtc/GlVideoFileRenderer.java
@@ -0,0 +1,152 @@
+/*
+ *  Copyright 2016 The WebRTC Project Authors. All rights reserved.
+ *
+ *  Use of this source code is governed by a BSD-style license
+ *  that can be found in the LICENSE file in the root of the source
+ *  tree. An additional intellectual property rights grant can be found
+ *  in the file PATENTS.  All contributing project authors may
+ *  be found in the AUTHORS file in the root of the source tree.
+ */
+package org.webrtc;
+
+import android.os.Handler;
+import android.os.HandlerThread;
+
+import java.nio.ByteBuffer;
+import java.io.FileOutputStream;
+import java.io.IOException;
+
+/**
+ * Can be used to saves the video frames to file.
+ */
+public class GlVideoFileRenderer implements VideoRenderer.Callbacks {
+  private static final String TAG = "GlVideoFileRenderer";
+
+  private SurfaceTextureHelper.YuvConverter yuvConverter;

[sakal, 2016/09/27 07:54:28]

nit: Please remove excessive empty lines.

[mandermo, 2016/10/04 14:56:56]

Done.

+
+  private HandlerThread renderThread;
+
+  private Handler renderThreadHandler;
+
+  private FileOutputStream videoOutFile;
+  private int outputFileWidth;
+  private int outputFileHeight;
+
+  public GlVideoFileRenderer(
+      String outputFile, int outputFileWidth, int outputFileHeight,
+      EglBase.Context sharedContext) throws IOException {
+    yuvConverter = new SurfaceTextureHelper.YuvConverter(
+        sharedContext);

[sakal, 2016/09/27 07:54:27]

nit: Does this fit on the same line?

[mandermo, 2016/10/04 14:56:56]

Done.

+
+    this.outputFileWidth = outputFileWidth;
+    this.outputFileHeight = outputFileHeight;
+
+    videoOutFile = new FileOutputStream(outputFile);
+    videoOutFile.write((
+          "YUV4MPEG2 C420 W" + outputFileWidth +
+          " H" + outputFileHeight + " Ip F30:1 A1:1\n").getBytes());
+
+    renderThread = new HandlerThread(TAG);
+    renderThread.start();
+    renderThreadHandler = new Handler(renderThread.getLooper());
+  }
+
+  @Override
+  public void renderFrame(final VideoRenderer.I420Frame frame) {
+    renderThreadHandler.post(new Runnable() {
+      @Override
+      public void run() {
+        renderFrameOnRenderThread(frame);
+      }
+    });
+  }
+
+  private void renderFrameOnRenderThread(VideoRenderer.I420Frame frame) {
+    float frameAspectRatio = (frame.rotationDegree % 180 == 0) ?
+      (float) frame.width / frame.height
+      : (float) frame.height / frame.width;
+
+    final float[] rotatedSamplingMatrix =
+      RendererCommon.rotateTextureMatrix(frame.samplingMatrix, frame.rotationDegree);

[sakal, 2016/09/27 07:54:28]

nit: indentation

+    final float[] layoutMatrix = RendererCommon.getLayoutMatrix(
+        false, frameAspectRatio, (float) outputFileWidth / outputFileHeight);
+    final float[] texMatrix = RendererCommon.multiplyMatrices(rotatedSamplingMatrix, layoutMatrix);
+
+    int frameSize = outputFileWidth*outputFileHeight*3/2;

[sakal, 2016/09/27 07:54:27]

nit: spaces

How does this work with uneven resolutions?

[mandermo, 2016/10/04 14:56:56]

Have added a check in the constructor and throws an exception if it is odd.

+    ByteBuffer frameBytes = ByteBuffer.allocateDirect(frameSize);

[sakal, 2016/09/27 07:54:28]

Can we allocate this in the constructor?

Java documentation:
The buffers returned by this method typically have somewhat higher allocation
and deallocation costs than non-direct buffers. The contents of direct buffers
may reside outside of the normal garbage-collected heap, and so their impact
upon the memory footprint of an application might not be obvious. It is
therefore recommended that direct buffers be allocated primarily for large,
long-lived buffers that are subject to the underlying system's native I/O
operations. In general it is best to allocate direct buffers only when they
yield a measureable gain in program performance.

[mandermo, 2016/10/04 14:56:56]

Moved allocation to constructor and renamed to outputFrameBuffer.

+
+    int oesTextureId = frame.textureId;
+
+    if (!frame.yuvFrame) {
+      yuvConverter.convert(
+          frameBytes, outputFileWidth, outputFileHeight, outputFileWidth,
+          oesTextureId, texMatrix);
+
+      VideoRenderer.renderFrameDone(frame);
+
+      try {
+        videoOutFile.write("FRAME\n".getBytes());
+
+        int width = outputFileWidth;
+        int height = outputFileHeight;
+        int stride = width;
+        byte[] data = frameBytes.array();
+        int offset = frameBytes.arrayOffset();
+
+        Logging.d(TAG,
+            "arrayOffset(): " + frameBytes.arrayOffset() +
+            " hasArray: " + frameBytes.hasArray());
+        // Write Y
+        videoOutFile.write(data, offset, width*height);

[sakal, 2016/09/27 07:54:27]

nit: spaces

[mandermo, 2016/10/04 14:56:56]

Done.

+
+        // Write U
+        for (int r = height; r<height*3/2; ++r) {

[sakal, 2016/09/27 07:54:27]

nit: spaces

[mandermo, 2016/10/04 14:56:56]

Done.

+          videoOutFile.write(data, offset + r * stride, stride / 2);
+        }
+
+        // Write V
+        for (int r = height; r<height*3/2; ++r) {

[sakal, 2016/09/27 07:54:28]

nit: spaces

[mandermo, 2016/10/04 14:56:56]

Done.

+          videoOutFile.write(data, offset + r * stride + stride / 2, stride / 2);
+        }
+      }
+      catch (IOException e) {
+        Logging.d(TAG, "Failed to write to file for video out");
+      }
+    }
+    else {

[sakal, 2016/09/27 07:54:27]

nit: move on the same line with the closing bracket

+      try {
+        videoOutFile.write("FRAME\n".getBytes());
+        
+        int outputFrameSize = outputFileWidth * outputFileHeight * 3 / 2;

[magjed_webrtc, 2016/09/26 11:40:02]

You already have this variable in |frameSize|. Also, you should handle odd
widths properly.

[mandermo, 2016/10/04 14:56:56]

Using frameSize now. See my response for odd widths in FileVideoCapturer.

+        ByteBuffer outputBuffer = ByteBuffer.allocateDirect(outputFrameSize);
+        nativeI420Scale(
+            frame.yuvPlanes[0], frame.yuvStrides[0],
+            frame.yuvPlanes[1], frame.yuvStrides[1],
+            frame.yuvPlanes[2], frame.yuvStrides[2],
+            frame.width, frame.height,
+            outputBuffer, outputFileWidth, outputFileHeight);
+        videoOutFile.write(outputBuffer.array(), outputBuffer.arrayOffset(), outputFrameSize);

[magjed_webrtc, 2016/09/26 11:40:02]

You are not calling VideoRenderer.renderFrameDone in this path, so you will be
leaking all frames. Have you tried exercising this path in a local test?

[kjellander_webrtc, 2016/09/27 11:30:21]

That sounds bad. Can we please get a basic unit test for each of these two code
paths?

[mandermo, 2016/10/04 14:56:56]

Missed that call. Have that path with an instance of this class as a
localRenderer for CallActivity. Did not play enough to get a crash from the
leak.

+      }
+      catch (IOException e) {
+        Logging.d(TAG, "Failed to write to file for video out");
+        throw new RuntimeException(e);
+      }
+    }
+  }
+
+  public void release() {
+    try {
+      videoOutFile.close();
+    }
+    catch (IOException e) {
+      Logging.d(TAG, "Error closing output video file");
+    }
+  }
+
+  public static native void nativeI420Scale(
+      ByteBuffer srcY, int strideY,
+      ByteBuffer srcU, int strideU,
+      ByteBuffer srcV, int strideV,
+      int width, int height, ByteBuffer dst, int dstWidth, int dstHeight);
+}
+