MediaCodecVideoEncoder add support to encode from textures

talk/app/webrtc/androidtests/src/org/webrtc/MediaCodecVideoEncoderTest.java

 /*
  * libjingle
  * Copyright 2015 Google Inc.
  *
  * Redistribution and use in source and binary forms, with or without
  * modification, are permitted provided that the following conditions are met:
  *
  *  1. Redistributions of source code must retain the above copyright notice,
  *     this list of conditions and the following disclaimer.
  *  2. Redistributions in binary form must reproduce the above copyright notice,
  *     this list of conditions and the following disclaimer in the documentation
  *     and/or other materials provided with the distribution.
  *  3. The name of the author may not be used to endorse or promote products
  *     derived from this software without specific prior written permission.
  *
  * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR IMPLIED
  * WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
  * MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO
  * EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
  * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
  * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS;
  * OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
  * WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
  * OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
  * ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
  */
 package org.webrtc;
 
 import java.nio.ByteBuffer;
-
+import android.annotation.TargetApi;
+import android.opengl.GLES11Ext;
+import android.opengl.GLES20;
+import android.os.Build;
 import android.test.ActivityTestCase;
 import android.test.suitebuilder.annotation.SmallTest;
 import android.util.Log;

[magjed_webrtc, 2015/11/18 13:12:52]

The import order is wrong.
"Import statements are divided into the following groups, in this order, with
each group separated by a single blank line:

All static imports in a single group
com.google imports (only if this source file is in the com.google package space)
Third-party imports, one group per top-level package, in ASCII sort order
for example: android, com, junit, org, sun
java imports
javax imports"
https://google.github.io/styleguide/javaguide.html#s3.3.3-import-ordering-and...

[perkj_webrtc, 2015/11/18 14:51:20]

We strictly don't do 

+import org.webrtc.MediaCodecVideoEncoder.OutputBufferInfo;
+import javax.microedition.khronos.egl.EGL10;
 
-import org.webrtc.MediaCodecVideoEncoder.OutputBufferInfo;
-
+@TargetApi(Build.VERSION_CODES.ICE_CREAM_SANDWICH_MR1)
 public final class MediaCodecVideoEncoderTest extends ActivityTestCase {
   final static String TAG = "MediaCodecVideoEncoderTest";
 
   @SmallTest
-  public static void testInitReleaseUsingByteBuffer() {
+  public static void testReInitializeUsingByteBuffer() {

[magjed_webrtc, 2015/11/18 13:12:52]

Why did you modify this test in this CL? And why did you remove the previous
simpler test with just init() and release()?

[perkj_webrtc, 2015/11/18 14:51:20]

this is how we use it from c++ sometimes. It felt unnecessary to test both init
release and init release init release...

[magjed_webrtc, 2015/11/18 15:38:12]

Yeah maybe. I don't know much about our testing 'style guide', but it's nice to
have small tests that only test one behaviour. You can probably ask phoglund@
what the preferred way is - overlapping small tests or a few big tests.

     if (!MediaCodecVideoEncoder.isVp8HwSupported()) {
       Log.i(TAG,
             "Hardware does not support VP8 encoding, skipping testInitReleaseUsingByteBuffer");
       return;
     }
     MediaCodecVideoEncoder encoder = new MediaCodecVideoEncoder();
     assertTrue(encoder.initEncode(
-        MediaCodecVideoEncoder.VideoCodecType.VIDEO_CODEC_VP8, 640, 480, 300, 30));
+        MediaCodecVideoEncoder.VideoCodecType.VIDEO_CODEC_VP8, 640, 480, 300, 30, null));
+    encoder.release();
+    assertTrue(encoder.initEncode(
+        MediaCodecVideoEncoder.VideoCodecType.VIDEO_CODEC_VP8, 640, 480, 300, 30, null));
     encoder.release();
   }
 
+  @SmallTest
+  public static void testReInitilizeUsingTextures() {
+    if (!MediaCodecVideoEncoder.isVp8HwSupportedUsingTextures()) {
+      Log.i(TAG, "hardware does not support VP8 encoding, skipping testEncoderUsingTextures");
+      return;
+    }
+    MediaCodecVideoEncoder encoder = new MediaCodecVideoEncoder();
+    assertTrue(encoder.initEncode(
+        MediaCodecVideoEncoder.VideoCodecType.VIDEO_CODEC_VP8, 640, 480, 300, 30,
+        EGL10.EGL_NO_CONTEXT));
+    encoder.release();
+    assertTrue(encoder.initEncode(
+        MediaCodecVideoEncoder.VideoCodecType.VIDEO_CODEC_VP8, 640, 480, 300, 30,
+        EGL10.EGL_NO_CONTEXT));
+    encoder.release();
+  }
+
   @SmallTest
   public static void testEncoderUsingByteBuffer() throws InterruptedException {
     if (!MediaCodecVideoEncoder.isVp8HwSupported()) {
       Log.i(TAG, "Hardware does not support VP8 encoding, skipping testEncoderUsingByteBuffer");
       return;
     }
 
     final int width = 640;
     final int height = 480;
     final int min_size = width * height * 3 / 2;
     final long presentationTimestampUs = 2;
 
     MediaCodecVideoEncoder encoder = new MediaCodecVideoEncoder();
 
     assertTrue(encoder.initEncode(
-        MediaCodecVideoEncoder.VideoCodecType.VIDEO_CODEC_VP8, width, height, 300, 30));
+        MediaCodecVideoEncoder.VideoCodecType.VIDEO_CODEC_VP8, width, height, 300, 30, null));
     ByteBuffer[] inputBuffers = encoder.getInputBuffers();
     assertNotNull(inputBuffers);
     assertTrue(min_size <= inputBuffers[0].capacity());
 
     int bufferIndex;
     do {
       Thread.sleep(10);
       bufferIndex = encoder.dequeueInputBuffer();
     } while (bufferIndex == -1); // |-1| is returned when there is no buffer available yet.
 
     assertTrue(bufferIndex >= 0);
     assertTrue(bufferIndex < inputBuffers.length);
     assertTrue(encoder.encodeBuffer(true, bufferIndex, min_size, presentationTimestampUs));
 
     OutputBufferInfo info;
     do {
       info = encoder.dequeueOutputBuffer();
       Thread.sleep(10);
     } while (info == null);
     assertTrue(info.index >= 0);
     assertEquals(presentationTimestampUs, info.presentationTimestampUs);
     assertTrue(info.buffer.capacity() > 0);
     encoder.releaseOutputBuffer(info.index);
 
     encoder.release();
   }
+
+  @SmallTest
+  public static void testEncoderUsingTextures() throws InterruptedException {
+    if (!MediaCodecVideoEncoder.isVp8HwSupportedUsingTextures()) {
+      Log.i(TAG, "Hardware does not support VP8 encoding, skipping testEncoderUsingTextures");
+      return;
+    }
+
+    final int width = 640;
+    final int height = 480;
+    final long presentationTs = 2;
+
+    final EglBase eglOesBase = new EglBase(EGL10.EGL_NO_CONTEXT, EglBase.ConfigType.PLAIN);

[magjed_webrtc, 2015/11/18 13:12:52]

You must make |eglOesBase| current, otherwise it has no purpose. Replace with
this code:
Final EglBase eglOesBase =
    new EglBase(sharedContext, EglBase.ConfigType.PIXEL_BUFFER);
eglOesBase.createDummyPbufferSurface();
eglOesBase.makeCurrent();

[perkj_webrtc, 2015/11/18 14:51:20]

not sure I understand.

[magjed_webrtc, 2015/11/18 15:38:12]

First, you need a current EGLContext for all GLES calls. I'm not sure if another
EGLContext is current here, or if you are relying on undefined behaviour and
generating a texture outside an EGLContext. Also, if you don't make |eglOesBase|
current, you don't create |oesTextureId| in the |eglOesBase| namespace. Then the
encoder can't access it in encoder.encodeTexture(), because it is not a valid
texture id in the encode EGLContext namespace. I think this test is currently
sampling from an invalid texture.

+    int oesTextureId = GlUtil.generateTexture(GLES11Ext.GL_TEXTURE_EXTERNAL_OES);
+
+    MediaCodecVideoEncoder encoder = new MediaCodecVideoEncoder();
+
+    assertTrue(encoder.initEncode(
+        MediaCodecVideoEncoder.VideoCodecType.VIDEO_CODEC_VP8, width, height, 300, 30,
+        eglOesBase.getContext()));
+    assertTrue(encoder.encodeTexture(true, oesTextureId, RendererCommon.identityMatrix(),
+        presentationTs));
+
+    // It should be Ok to delete the texture after calling encodeTexture.
+    GLES20.glDeleteTextures(1, new int[] {oesTextureId}, 0);
+
+    OutputBufferInfo info = encoder.dequeueOutputBuffer();
+    while (info == null) {
+      info = encoder.dequeueOutputBuffer();
+      Thread.sleep(20);
+    }
+    assertTrue(info.index != -1);
+    assertTrue(info.buffer.capacity() > 0);
+    encoder.releaseOutputBuffer(info.index);
+
+    encoder.release();
+    eglOesBase.release();
+  }
 }