Add a field trial to produce VideoFrames in camera capturers.

Add a field trial to produce VideoFrames in camera capturers.

The field trials enables producing new VideoFrames in camera classes.
This field trial should be enabled if VideoSinks are used.

BUG=webrtc:7749, webrtc:7760
Review-Url: https://codereview.webrtc.org/2984633002
Cr-Commit-Position: refs/heads/master@{#19467}
Committed: https://chromium.googlesource.com/external/webrtc/+/91073ae78fb9d49f302ffb8a3142cf67a55e8118

[sakal, 2017-07-21 12:45:36]

Patchset #2 (id:20001) has been deleted

[sakal, 2017-07-24 08:56:56]

sakal@webrtc.org changed reviewers:
+ magjed@webrtc.org

[sakal, 2017-07-24 08:56:57]

PTAL

[magjed_webrtc, 2017-07-24 09:51:19]

https://codereview.webrtc.org/2984633002/diff/140001/webrtc/sdk/android/src/j...
File webrtc/sdk/android/src/java/org/webrtc/I420BufferImpl.java (right):

https://codereview.webrtc.org/2984633002/diff/140001/webrtc/sdk/android/src/j...
webrtc/sdk/android/src/java/org/webrtc/I420BufferImpl.java:72: public ByteBuffer
getBuffer() {
name it: getByteBuffer

https://codereview.webrtc.org/2984633002/diff/140001/webrtc/sdk/android/src/j...
File webrtc/sdk/android/src/jni/camera1session_jni.cc (right):

https://codereview.webrtc.org/2984633002/diff/140001/webrtc/sdk/android/src/j...
webrtc/sdk/android/src/jni/camera1session_jni.cc:42: int ret =
libyuv::NV21ToI420(src_y, width, src_uv, width, dst_y, dst_stride_y,
This is a performance regression compared to the existing
AndroidVideoTrackSource::OnByteBufferFrameCaptured conversion which uses
webrtc::NV12ToI420Scaler. Is it possible to fix?

[sakal, 2017-07-27 09:06:34]

Patchset #8 (id:160001) has been deleted

[sakal, 2017-08-15 12:40:09]

On 2017/07/24 09:51:19, magjed_webrtc wrote:
>
https://codereview.webrtc.org/2984633002/diff/140001/webrtc/sdk/android/src/j...
> File webrtc/sdk/android/src/java/org/webrtc/I420BufferImpl.java (right):
> 
>
https://codereview.webrtc.org/2984633002/diff/140001/webrtc/sdk/android/src/j...
> webrtc/sdk/android/src/java/org/webrtc/I420BufferImpl.java:72: public
ByteBuffer
> getBuffer() {
> name it: getByteBuffer
> 
>
https://codereview.webrtc.org/2984633002/diff/140001/webrtc/sdk/android/src/j...
> File webrtc/sdk/android/src/jni/camera1session_jni.cc (right):
> 
>
https://codereview.webrtc.org/2984633002/diff/140001/webrtc/sdk/android/src/j...
> webrtc/sdk/android/src/jni/camera1session_jni.cc:42: int ret =
> libyuv::NV21ToI420(src_y, width, src_uv, width, dst_y, dst_stride_y,
> This is a performance regression compared to the existing
> AndroidVideoTrackSource::OnByteBufferFrameCaptured conversion which uses
> webrtc::NV12ToI420Scaler. Is it possible to fix?

[sakal, 2017-08-15 12:41:04]

sakal@webrtc.org changed reviewers:
+ kthelgason@webrtc.org

[sakal, 2017-08-15 12:41:05]

PTAL

https://codereview.webrtc.org/2984633002/diff/140001/webrtc/sdk/android/src/j...
File webrtc/sdk/android/src/java/org/webrtc/I420BufferImpl.java (right):

https://codereview.webrtc.org/2984633002/diff/140001/webrtc/sdk/android/src/j...
webrtc/sdk/android/src/java/org/webrtc/I420BufferImpl.java:72: public ByteBuffer
getBuffer() {
On 2017/07/24 09:51:18, magjed_webrtc wrote:
> name it: getByteBuffer

The method no longer exists.

https://codereview.webrtc.org/2984633002/diff/140001/webrtc/sdk/android/src/j...
File webrtc/sdk/android/src/jni/camera1session_jni.cc (right):

https://codereview.webrtc.org/2984633002/diff/140001/webrtc/sdk/android/src/j...
webrtc/sdk/android/src/jni/camera1session_jni.cc:42: int ret =
libyuv::NV21ToI420(src_y, width, src_uv, width, dst_y, dst_stride_y,
On 2017/07/24 09:51:18, magjed_webrtc wrote:
> This is a performance regression compared to the existing
> AndroidVideoTrackSource::OnByteBufferFrameCaptured conversion which uses
> webrtc::NV12ToI420Scaler. Is it possible to fix?

NV21Buffer class should fix this.

[sakal, 2017-08-15 12:42:41]

Description was changed from

==========
Change camera implementations to produce VideoFrames.

BUG=
==========

to

==========
Add a field trial to produce VideoFrames in camera capturers.

The field trials enables producing new VideoFrames in camera classes. You
should enable this field trial if you use VideoSinks.

BUG=webrtc:7749, webrtc:7760
==========

[sakal, 2017-08-15 12:43:16]

Description was changed from

==========
Add a field trial to produce VideoFrames in camera capturers.

The field trials enables producing new VideoFrames in camera classes. You
should enable this field trial if you use VideoSinks.

BUG=webrtc:7749, webrtc:7760
==========

to

==========
Add a field trial to produce VideoFrames in camera capturers.

The field trials enables producing new VideoFrames in camera classes.
This field trial should be enabled if VideoSinks are used.

BUG=webrtc:7749, webrtc:7760
==========

[kthelgason, 2017-08-17 11:19:40]

lgtm

[sakal, 2017-08-17 11:21:15]

The CQ bit was checked by sakal@webrtc.org

[commit-bot: I haz the power, 2017-08-17 11:21:29]

CQ is trying da patch.

Follow status at:
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.webrtc.org/...

[commit-bot: I haz the power, 2017-08-17 13:02:28]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2017-08-17 13:02:29]

Try jobs failed on following builders:
  android_arm64_rel on master.tryserver.webrtc (JOB_FAILED,
http://build.chromium.org/p/tryserver.webrtc/builders/android_arm64_rel/build...)

[sakal, 2017-08-22 07:22:50]

The CQ bit was checked by sakal@webrtc.org to run a CQ dry run

[commit-bot: I haz the power, 2017-08-22 07:22:57]

Dry run: CQ is trying da patch.

Follow status at:
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.webrtc.org/...

[commit-bot: I haz the power, 2017-08-22 07:25:26]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2017-08-22 07:25:28]

Dry run: Try jobs failed on following builders:
  ios_arm64_rel on master.tryserver.webrtc (JOB_FAILED,
http://build.chromium.org/p/tryserver.webrtc/builders/ios_arm64_rel/builds/22823)
  presubmit on master.tryserver.webrtc (JOB_FAILED,
http://build.chromium.org/p/tryserver.webrtc/builders/presubmit/builds/20509)
  win_clang_rel on master.tryserver.webrtc (JOB_FAILED,
http://build.chromium.org/p/tryserver.webrtc/builders/win_clang_rel/builds/16284)
  win_dbg on master.tryserver.webrtc (JOB_FAILED,
http://build.chromium.org/p/tryserver.webrtc/builders/win_dbg/builds/21573)

[sakal, 2017-08-22 08:24:18]

The CQ bit was checked by sakal@webrtc.org to run a CQ dry run

[commit-bot: I haz the power, 2017-08-22 08:24:27]

Dry run: CQ is trying da patch.

Follow status at:
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.webrtc.org/...

[magjed_webrtc, 2017-08-22 08:53:09]

https://codereview.webrtc.org/2984633002/diff/240001/webrtc/sdk/android/src/j...
File webrtc/sdk/android/src/java/org/webrtc/Camera1Session.java (right):

https://codereview.webrtc.org/2984633002/diff/240001/webrtc/sdk/android/src/j...
webrtc/sdk/android/src/java/org/webrtc/Camera1Session.java:21: import
java.util.HashMap;
This is unused.

https://codereview.webrtc.org/2984633002/diff/240001/webrtc/sdk/android/src/j...
webrtc/sdk/android/src/java/org/webrtc/Camera1Session.java:23: import
java.util.Map;
This is unused.

https://codereview.webrtc.org/2984633002/diff/240001/webrtc/sdk/android/src/j...
webrtc/sdk/android/src/java/org/webrtc/Camera1Session.java:372: private static
native void nativeNV21ToI420(int width, int height, byte[] src, ByteBuffer dst);
This is not used?

https://codereview.webrtc.org/2984633002/diff/240001/webrtc/sdk/android/src/j...
File webrtc/sdk/android/src/java/org/webrtc/Camera2Session.java (right):

https://codereview.webrtc.org/2984633002/diff/240001/webrtc/sdk/android/src/j...
webrtc/sdk/android/src/java/org/webrtc/Camera2Session.java:232:
frameMatrix.preTranslate(0.5f, 0.5f);
Revert these changes just to be safe and perform the
convertMatrixToAndroidGraphicsMatrix inside the field trial path instead.

https://codereview.webrtc.org/2984633002/diff/240001/webrtc/sdk/android/src/j...
File webrtc/sdk/android/src/java/org/webrtc/NV21Buffer.java (right):

https://codereview.webrtc.org/2984633002/diff/240001/webrtc/sdk/android/src/j...
webrtc/sdk/android/src/java/org/webrtc/NV21Buffer.java:15: public class
NV21Buffer implements VideoFrame.Buffer {
Maybe you want to land this class in a separate CL?

https://codereview.webrtc.org/2984633002/diff/240001/webrtc/sdk/android/src/j...
webrtc/sdk/android/src/java/org/webrtc/NV21Buffer.java:43: return
(VideoFrame.I420Buffer) cropAndScale(0, 0, width, height, width, height);
Comment the literals.

https://codereview.webrtc.org/2984633002/diff/240001/webrtc/sdk/android/src/j...
File webrtc/sdk/android/src/jni/nv21buffer_jni.cc (right):

https://codereview.webrtc.org/2984633002/diff/240001/webrtc/sdk/android/src/j...
webrtc/sdk/android/src/jni/nv21buffer_jni.cc:63: std::vector<uint8_t>
tmp_buffer(tmp_size);
Use NV12ToI420Scaler instead. Still, I'm a bit worried about the temporary
memory allocations we do per frame. This is a regression from the current
pipeline.

[commit-bot: I haz the power, 2017-08-22 08:58:19]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2017-08-22 08:58:19]

Dry run: Try jobs failed on following builders:
  android_arm64_rel on master.tryserver.webrtc (JOB_FAILED,
http://build.chromium.org/p/tryserver.webrtc/builders/android_arm64_rel/build...)

[sakal, 2017-08-22 11:45:31]

https://codereview.webrtc.org/2984633002/diff/240001/webrtc/sdk/android/src/j...
File webrtc/sdk/android/src/java/org/webrtc/Camera1Session.java (right):

https://codereview.webrtc.org/2984633002/diff/240001/webrtc/sdk/android/src/j...
webrtc/sdk/android/src/java/org/webrtc/Camera1Session.java:21: import
java.util.HashMap;
On 2017/08/22 08:53:08, magjed_webrtc wrote:
> This is unused.

Done.

https://codereview.webrtc.org/2984633002/diff/240001/webrtc/sdk/android/src/j...
webrtc/sdk/android/src/java/org/webrtc/Camera1Session.java:23: import
java.util.Map;
On 2017/08/22 08:53:08, magjed_webrtc wrote:
> This is unused.

Done.

https://codereview.webrtc.org/2984633002/diff/240001/webrtc/sdk/android/src/j...
webrtc/sdk/android/src/java/org/webrtc/Camera1Session.java:372: private static
native void nativeNV21ToI420(int width, int height, byte[] src, ByteBuffer dst);
On 2017/08/22 08:53:08, magjed_webrtc wrote:
> This is not used?

Done.

https://codereview.webrtc.org/2984633002/diff/240001/webrtc/sdk/android/src/j...
File webrtc/sdk/android/src/java/org/webrtc/Camera2Session.java (right):

https://codereview.webrtc.org/2984633002/diff/240001/webrtc/sdk/android/src/j...
webrtc/sdk/android/src/java/org/webrtc/Camera2Session.java:232:
frameMatrix.preTranslate(0.5f, 0.5f);
On 2017/08/22 08:53:08, magjed_webrtc wrote:
> Revert these changes just to be safe and perform the
> convertMatrixToAndroidGraphicsMatrix inside the field trial path instead.

Done.

https://codereview.webrtc.org/2984633002/diff/240001/webrtc/sdk/android/src/j...
File webrtc/sdk/android/src/java/org/webrtc/NV21Buffer.java (right):

https://codereview.webrtc.org/2984633002/diff/240001/webrtc/sdk/android/src/j...
webrtc/sdk/android/src/java/org/webrtc/NV21Buffer.java:15: public class
NV21Buffer implements VideoFrame.Buffer {
On 2017/08/22 08:53:08, magjed_webrtc wrote:
> Maybe you want to land this class in a separate CL?

Acknowledged but let's land this with the other code since this will only be
used from here.

https://codereview.webrtc.org/2984633002/diff/240001/webrtc/sdk/android/src/j...
webrtc/sdk/android/src/java/org/webrtc/NV21Buffer.java:43: return
(VideoFrame.I420Buffer) cropAndScale(0, 0, width, height, width, height);
On 2017/08/22 08:53:08, magjed_webrtc wrote:
> Comment the literals.

Done.

https://codereview.webrtc.org/2984633002/diff/240001/webrtc/sdk/android/src/j...
File webrtc/sdk/android/src/jni/nv21buffer_jni.cc (right):

https://codereview.webrtc.org/2984633002/diff/240001/webrtc/sdk/android/src/j...
webrtc/sdk/android/src/jni/nv21buffer_jni.cc:63: std::vector<uint8_t>
tmp_buffer(tmp_size);
On 2017/08/22 08:53:08, magjed_webrtc wrote:
> Use NV12ToI420Scaler instead. Still, I'm a bit worried about the temporary
> memory allocations we do per frame. This is a regression from the current
> pipeline.

Done.

[sakal, 2017-08-22 12:06:49]

The CQ bit was checked by sakal@webrtc.org to run a CQ dry run

[commit-bot: I haz the power, 2017-08-22 12:06:59]

Dry run: CQ is trying da patch.

Follow status at:
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.webrtc.org/...

[commit-bot: I haz the power, 2017-08-22 14:09:15]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2017-08-22 14:09:16]

Dry run: Try jobs failed on following builders:
  android_arm64_rel on master.tryserver.webrtc (JOB_FAILED,
http://build.chromium.org/p/tryserver.webrtc/builders/android_arm64_rel/build...)

[magjed_webrtc, 2017-08-23 13:28:09]

lgtm

[sakal, 2017-08-23 13:28:22]

The CQ bit was checked by sakal@webrtc.org

[sakal, 2017-08-23 13:28:23]

The patchset sent to the CQ was uploaded after l-g-t-m from
kthelgason@webrtc.org
Link to the patchset: https://codereview.webrtc.org/2984633002/#ps260001 (title:
"Address comments.")

[commit-bot: I haz the power, 2017-08-23 13:28:26]

CQ is trying da patch.

Follow status at:
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.webrtc.org/...

[sakal, 2017-08-23 13:29:59]

The CQ bit was unchecked by sakal@webrtc.org

[sakal, 2017-08-23 14:00:38]

The CQ bit was checked by sakal@webrtc.org

[sakal, 2017-08-23 14:00:38]

The patchset sent to the CQ was uploaded after l-g-t-m from magjed@webrtc.org,
kthelgason@webrtc.org
Link to the patchset: https://codereview.webrtc.org/2984633002/#ps280001 (title:
"Fix error.")

[commit-bot: I haz the power, 2017-08-23 14:00:47]

CQ is trying da patch.

Follow status at:
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.webrtc.org/...

[commit-bot: I haz the power, 2017-08-23 15:50:24]

CQ is committing da patch.

Bot data: {"patchset_id": 280001, "attempt_start_ts": 1503496838478870,
"parent_rev": "fd7cffa43366d5fb8a95da71fffa7f872037fefe", "commit_rev":
"91073ae78fb9d49f302ffb8a3142cf67a55e8118"}

[commit-bot: I haz the power, 2017-08-23 15:50:30]

Description was changed from

==========
Add a field trial to produce VideoFrames in camera capturers.

The field trials enables producing new VideoFrames in camera classes.
This field trial should be enabled if VideoSinks are used.

BUG=webrtc:7749, webrtc:7760
==========

to

==========
Add a field trial to produce VideoFrames in camera capturers.

The field trials enables producing new VideoFrames in camera classes.
This field trial should be enabled if VideoSinks are used.

BUG=webrtc:7749, webrtc:7760

Review-Url: https://codereview.webrtc.org/2984633002
Cr-Commit-Position: refs/heads/master@{#19467}
Committed:
https://chromium.googlesource.com/external/webrtc/+/91073ae78fb9d49f302ffb8a3...
==========

[commit-bot: I haz the power, 2017-08-23 15:50:31]

Committed patchset #13 (id:280001) as
https://chromium.googlesource.com/external/webrtc/+/91073ae78fb9d49f302ffb8a3...