Add texture support to HardwareVideoEncoder.

webrtc/sdk/android/src/java/org/webrtc/HardwareVideoEncoder.java

Index: webrtc/sdk/android/src/java/org/webrtc/HardwareVideoEncoder.java
diff --git a/webrtc/sdk/android/src/java/org/webrtc/HardwareVideoEncoder.java b/webrtc/sdk/android/src/java/org/webrtc/HardwareVideoEncoder.java
index 5e8725777c5145ccdfce786a263b884278fe8653..d045276cc45225737529fe17090c1e31a3c60cb0 100644
--- a/webrtc/sdk/android/src/java/org/webrtc/HardwareVideoEncoder.java
+++ b/webrtc/sdk/android/src/java/org/webrtc/HardwareVideoEncoder.java
@@ -11,10 +11,13 @@
 package org.webrtc;
 
 import android.annotation.TargetApi;
+import android.graphics.Matrix;
 import android.media.MediaCodec;
 import android.media.MediaCodecInfo;
 import android.media.MediaFormat;
+import android.opengl.GLES20;
 import android.os.Bundle;
+import android.view.Surface;
 import java.io.IOException;
 import java.nio.ByteBuffer;
 import java.util.Arrays;
@@ -73,6 +76,12 @@ class HardwareVideoEncoder implements VideoEncoder {
   // value to send exceptions thrown during release back to the encoder thread.
   private volatile Exception shutdownException = null;
 
+  // Surface objects for texture-mode encoding.

[pthatcher1, 2017/07/17 22:50:19]

Would it make sense to write a comment explaining what they are and how they
interact?

[mellem, 2017/07/17 23:22:00]

Done.

+  private EglBase14.Context sharedContext;
+  private EglBase14 eglBase;
+  private Surface inputSurface;
+  private GlRectDrawer drawer;

[pthatcher1, 2017/07/17 22:50:19]

It would be nice if these had a common name that made it clear it was for
texture encoding.  Perhaps something like textureContext, textureBase,
textureSurface, textureDrawer?

[mellem, 2017/07/17 23:22:01]

Done.

+
   private MediaCodec codec;
   private Callback callback;
 
@@ -97,15 +106,21 @@ class HardwareVideoEncoder implements VideoEncoder {
    * @throws IllegalArgumentException if colorFormat is unsupported
    */
   public HardwareVideoEncoder(String codecName, VideoCodecType codecType, int colorFormat,
-      int keyFrameIntervalSec, int forceKeyFrameIntervalMs, BitrateAdjuster bitrateAdjuster) {
+      int keyFrameIntervalSec, int forceKeyFrameIntervalMs, BitrateAdjuster bitrateAdjuster,
+      EglBase14.Context sharedContext) {
     this.codecName = codecName;
     this.codecType = codecType;
     this.colorFormat = colorFormat;
-    this.inputColorFormat = ColorFormat.valueOf(colorFormat);
+    if (sharedContext == null) {
+      this.inputColorFormat = ColorFormat.valueOf(colorFormat);
+    } else {
+      this.inputColorFormat = null;

[pthatcher1, 2017/07/17 22:50:19]

Can you write a comment explaining why the inputColorFormat is null when the
texture shared context is non-null?

[mellem, 2017/07/17 23:22:00]

Done.

+    }
     this.keyFrameIntervalSec = keyFrameIntervalSec;
     this.forcedKeyFrameMs = forceKeyFrameIntervalMs;
     this.bitrateAdjuster = bitrateAdjuster;
     this.outputBuilders = new LinkedBlockingDeque<>();
+    this.sharedContext = sharedContext;
   }
 
   @Override
@@ -144,6 +159,15 @@ class HardwareVideoEncoder implements VideoEncoder {
       format.setInteger(MediaFormat.KEY_I_FRAME_INTERVAL, keyFrameIntervalSec);
       Logging.d(TAG, "Format: " + format);
       codec.configure(format, null, null, MediaCodec.CONFIGURE_FLAG_ENCODE);
+
+      if (sharedContext != null) {
+        // Surface mode.

[pthatcher1, 2017/07/17 22:50:19]

Above it was called "texture mode", not it says "surface mode".  That's the same
thing, I'm assuming.  If so, can we pick one?

[mellem, 2017/07/17 23:22:00]

Done.

+        eglBase = new EglBase14(sharedContext, EglBase.CONFIG_RECORDABLE);
+        inputSurface = codec.createInputSurface();
+        eglBase.createSurface(inputSurface);
+        drawer = new GlRectDrawer();
+      }
+
       codec.start();
     } catch (IllegalStateException e) {
       Logging.e(TAG, "initEncode failed", e);
@@ -161,6 +185,9 @@ class HardwareVideoEncoder implements VideoEncoder {
   @Override
   public VideoCodecStatus release() {
     try {
+      if (outputThread == null) {
+        return VideoCodecStatus.OK;
+      }

[pthatcher1, 2017/07/17 22:50:19]

Is this only true when we're in texture mode?  If we're not an there's so
outputThread, wouldn't that be an error?

[mellem, 2017/07/17 23:22:00]

No, this is just a fix for a bug that I whacked.  I noticed it because I had
test errors while writing texture support.  If init fails, the output thread may
still be null when release() is called.  In that case, we can't join the output
thread.

       // The outputThread actually stops and releases the codec once running is false.
       running = false;
       if (!ThreadUtils.joinUninterruptibly(outputThread, MEDIA_CODEC_RELEASE_TIMEOUT_MS)) {
@@ -176,6 +203,19 @@ class HardwareVideoEncoder implements VideoEncoder {
       codec = null;
       outputThread = null;
       outputBuilders.clear();
+
+      if (drawer != null) {
+        drawer.release();
+        drawer = null;
+      }
+      if (eglBase != null) {
+        eglBase.release();
+        eglBase = null;
+      }
+      if (inputSurface != null) {
+        inputSurface.release();
+        inputSurface = null;
+      }

[pthatcher1, 2017/07/17 22:50:19]

Would it make sense to put the 4 texture things in one "TextureModeState" class?
 Then it could have a release() and you'd only need one of these checks.

[mellem, 2017/07/17 23:22:01]

It's possible to fail init partway through allocating these (eg. if
createInputSurface throws).  We need to release whatever we created.  So I think
each needs a separate null check.

     }
     return VideoCodecStatus.OK;
   }
@@ -196,37 +236,12 @@ class HardwareVideoEncoder implements VideoEncoder {
       }
     }
 
-    // No timeout.  Don't block for an input buffer, drop frames if the encoder falls behind.
-    int index;
-    try {
-      index = codec.dequeueInputBuffer(0 /* timeout */);
-    } catch (IllegalStateException e) {
-      Logging.e(TAG, "dequeueInputBuffer failed", e);
-      return VideoCodecStatus.FALLBACK_SOFTWARE;
-    }
-
-    if (index == -1) {
-      // Encoder is falling behind.  No input buffers available.  Drop the frame.
-      Logging.e(TAG, "Dropped frame, no input buffers available");
-      return VideoCodecStatus.OK; // See webrtc bug 2887.
-    }
     if (outputBuilders.size() > MAX_ENCODER_Q_SIZE) {
       // Too many frames in the encoder.  Drop this frame.
       Logging.e(TAG, "Dropped frame, encoder queue full");
       return VideoCodecStatus.OK; // See webrtc bug 2887.
     }
 
-    // TODO(mellem):  Add support for input surfaces and textures.
-    ByteBuffer buffer;
-    try {
-      buffer = codec.getInputBuffers()[index];
-    } catch (IllegalStateException e) {
-      Logging.e(TAG, "getInputBuffers failed", e);
-      return VideoCodecStatus.FALLBACK_SOFTWARE;
-    }
-    VideoFrame.I420Buffer i420 = videoFrame.getBuffer().toI420();
-    inputColorFormat.fillBufferFromI420(buffer, i420);
-
     boolean requestedKeyFrame = false;
     for (EncodedImage.FrameType frameType : encodeInfo.frameTypes) {
       if (frameType == EncodedImage.FrameType.VideoFrameKey) {
@@ -241,9 +256,10 @@ class HardwareVideoEncoder implements VideoEncoder {
       requestKeyFrame(presentationTimestampMs);
     }
 
+    VideoFrame.Buffer videoFrameBuffer = videoFrame.getBuffer();
     // Number of bytes in the video buffer. Y channel is sampled at one byte per pixel; U and V are
     // subsampled at one byte per four pixels.
-    int bufferSize = videoFrame.getBuffer().getHeight() * videoFrame.getBuffer().getWidth() * 3 / 2;
+    int bufferSize = videoFrameBuffer.getHeight() * videoFrameBuffer.getWidth() * 3 / 2;
     EncodedImage.Builder builder = EncodedImage.builder()
                                        .setCaptureTimeMs(presentationTimestampMs)
                                        .setCompleteFrame(true)
@@ -251,17 +267,89 @@ class HardwareVideoEncoder implements VideoEncoder {
                                        .setEncodedHeight(videoFrame.getHeight())
                                        .setRotation(videoFrame.getRotation());
     outputBuilders.offer(builder);
-    try {
-      codec.queueInputBuffer(
-          index, 0 /* offset */, bufferSize, presentationTimestampUs, 0 /* flags */);
-    } catch (IllegalStateException e) {
-      Logging.e(TAG, "queueInputBuffer failed", e);
-      // Keep the output builders in sync with buffers in the codec.
-      outputBuilders.pollLast();
-      // IllegalStateException thrown when the codec is in the wrong state.
-      return VideoCodecStatus.FALLBACK_SOFTWARE;
+
+    if (videoFrameBuffer instanceof VideoFrame.TextureBuffer) {

[sakal, 2017/07/17 12:25:41]

nit: I would prefer these cases in separate methods.

[mellem, 2017/07/17 17:49:29]

Done.

+      VideoFrame.TextureBuffer textureBuffer = (VideoFrame.TextureBuffer) videoFrameBuffer;
+
+      // TODO(mellem):  Put this matrix manipulation in a helper.

[sakal, 2017/07/17 12:25:41]

I have implemented such helper here
https://chromium-review.googlesource.com/c/555516/4/webrtc/sdk/android/api/or...

feel free to copy

[mellem, 2017/07/17 17:49:30]

Done.  Thanks for writing that.

+      // The android.graphics.Matrix looks like this:
+      // [x1 y1 w1]
+      // [x2 y2 w2]
+      // [x3 y3 w3]
+      // We want to contruct a matrix that looks like this:
+      // [x1 y1  0 w1]
+      // [x2 y2  0 w2]
+      // [ 0  0  1  0]
+      // [x3 y3  0 w3]
+      Matrix matrix = videoFrame.getTransformMatrix();
+      float[] matrix3x3 = new float[9];
+      matrix.getValues(matrix3x3);
+
+      float[] transformationMatrix = new float[16];
+      transformationMatrix[0 * 4 + 0] = matrix3x3[0 * 3 + 0];
+      transformationMatrix[0 * 4 + 1] = matrix3x3[0 * 3 + 1];
+      transformationMatrix[0 * 4 + 3] = matrix3x3[0 * 3 + 2];
+      transformationMatrix[1 * 4 + 0] = matrix3x3[1 * 3 + 0];
+      transformationMatrix[1 * 4 + 1] = matrix3x3[1 * 3 + 1];
+      transformationMatrix[1 * 4 + 3] = matrix3x3[1 * 3 + 2];
+      transformationMatrix[2 * 4 + 2] = 1; // Z-scale should be 1.
+      transformationMatrix[3 * 4 + 0] = matrix3x3[2 * 3 + 0];
+      transformationMatrix[3 * 4 + 1] = matrix3x3[2 * 3 + 1];
+      transformationMatrix[3 * 4 + 3] = matrix3x3[2 * 3 + 2];
+
+      try {
+        eglBase.makeCurrent();
+        // TODO(perkj): glClear() shouldn't be necessary since every pixel is covered anyway,
+        // but it's a workaround for bug webrtc:5147.
+        GLES20.glClear(GLES20.GL_COLOR_BUFFER_BIT);
+        drawer.drawOes(

[sakal, 2017/07/17 12:25:42]

nit: TextureFrame might be RGB frame in the future. Can we call drawRgb if
getType is RGB?

[mellem, 2017/07/17 17:49:29]

Done.

+            textureBuffer.getTextureId(), transformationMatrix, width, height, 0, 0, width, height);
+        eglBase.swapBuffers(videoFrame.getTimestampNs());
+      } catch (RuntimeException e) {
+        Logging.e(TAG, "encodeTexture failed", e);
+        // Keep the output builders in sync with buffers in the codec.
+        outputBuilders.pollLast();
+        return VideoCodecStatus.ERROR;
+      }
+      return VideoCodecStatus.OK;
+    } else {
+      // No timeout.  Don't block for an input buffer, drop frames if the encoder falls behind.

[sakal, 2017/07/17 12:25:42]

Can we at least log a clear error if the mode doesn't match what kind of frames
we receive? How hard would it be to reinitialize the encoder?

[mellem, 2017/07/17 17:49:30]

Logging added.  We can actually process a texture buffer in non-texture mode
(texture buffers still have toI420), just not the other way around.  Or maybe we
can draw the byte-buffer onto a surface right here?

As it stands, it would be quite hard to re-initialize into a different mode.  I
would have to refactor code so that we can find an appropriate color format
during init.  Byte to texture would not work anyway if we're missing the EGL
context.  Texture to byte might not work if the codec supports texture-mode but
none of our byte-buffer color formats.

I'd rather not attempt it, since existing code CHECK-fails if it gets handed a
frame in the wrong mode:

https://cs.chromium.org/chromium/src/third_party/webrtc/sdk/android/src/jni/a...

https://cs.chromium.org/chromium/src/third_party/webrtc/sdk/android/src/jni/a...

[sakal, 2017/07/18 08:46:13]

We reconfigure the encoder before we hit that check:
https://cs.chromium.org/chromium/src/third_party/webrtc/sdk/android/src/jni/a...

But we can implement this in a follow-up if needed.

[mellem, 2017/07/18 17:11:57]

Acknowledged.

+      int index;
+      try {
+        index = codec.dequeueInputBuffer(0 /* timeout */);
+      } catch (IllegalStateException e) {
+        Logging.e(TAG, "dequeueInputBuffer failed", e);
+        return VideoCodecStatus.FALLBACK_SOFTWARE;

[sakal, 2017/07/17 12:25:42]

Can you just replace all FALLBACK_SOTWAREs with ERRORs and we will deal with the
fallback in C++.

[mellem, 2017/07/17 17:49:29]

Done.

+      }
+
+      if (index == -1) {
+        // Encoder is falling behind.  No input buffers available.  Drop the frame.
+        Logging.e(TAG, "Dropped frame, no input buffers available");
+        return VideoCodecStatus.OK; // See webrtc bug 2887.
+      }
+
+      ByteBuffer buffer;
+      try {
+        buffer = codec.getInputBuffers()[index];
+      } catch (IllegalStateException e) {
+        Logging.e(TAG, "getInputBuffers failed", e);
+        return VideoCodecStatus.FALLBACK_SOFTWARE;
+      }
+      VideoFrame.I420Buffer i420 = videoFrameBuffer.toI420();

[sakal, 2017/07/17 12:25:41]

toI420 will return "a new instance". Therefore, we should release the i420
buffer.

[mellem, 2017/07/17 17:49:29]

Done.

+      inputColorFormat.fillBufferFromI420(buffer, i420);
+
+      try {
+        codec.queueInputBuffer(
+            index, 0 /* offset */, bufferSize, presentationTimestampUs, 0 /* flags */);
+      } catch (IllegalStateException e) {
+        Logging.e(TAG, "queueInputBuffer failed", e);
+        // Keep the output builders in sync with buffers in the codec.
+        outputBuilders.pollLast();
+        // IllegalStateException thrown when the codec is in the wrong state.
+        return VideoCodecStatus.FALLBACK_SOFTWARE;
+      }
+      return VideoCodecStatus.OK;
     }
-    return VideoCodecStatus.OK;
   }
 
   @Override