change videotoolbox format from NV12 to I420

change videotoolbox format from NV12 to I420

change videotoolbox encoder input pixelformat from NV12 to I420,
also change decoder output from NV12 to I420.
I use instruments to test the CPU usage change, there is about 2%~3% CPU drop after remove the conversion from NV12 to I420.

BUG=webrtc:5230

[wangrui, 2015-12-10 08:15:55]

wangruihit@gmail.com changed reviewers:
+ marpan@webrtc.org, pbos@webrtc.org, stefan@webrtc.org, tkchin@webrtc.org

[pbos-webrtc, 2015-12-10 12:05:18]

I think tkchin is better for reviewing this than me

https://codereview.webrtc.org/1515873002/diff/1/webrtc/modules/video_coding/c...
File webrtc/modules/video_coding/codecs/h264/h264_video_toolbox_decoder.cc
(right):

https://codereview.webrtc.org/1515873002/diff/1/webrtc/modules/video_coding/c...
webrtc/modules/video_coding/codecs/h264/h264_video_toolbox_decoder.cc:57: OSType
format = CVPixelBufferGetPixelFormatType(pixel_buffer);
This is not style compliant (indentation etc), please run git cl format.

[pbos-webrtc, 2015-12-10 12:06:30]

Please also change the CL description (Edit Issue) to 

<short description, one line>

<long description>

BUG=...

(Don't put BUG= at top, that becomes the subject for the CL)

[tkchin_webrtc, 2015-12-10 22:10:23]

On 2015/12/10 12:06:30, pbos-webrtc wrote:
> Please also change the CL description (Edit Issue) to 
> 
> <short description, one line>
> 
> <long description>
> 
> BUG=...
> 
> (Don't put BUG= at top, that becomes the subject for the CL)

What is the performance impact? If I420 is not a hardware-supported format, then
iOS is likely performing a conversion behind the scenes anyway from the hardware
output into the requested format. We should explicitly measure to get an idea of
what's going on.

[wangrui, 2015-12-11 03:18:46]

On 2015/12/10 22:10:23, tkchin_webrtc wrote:
> On 2015/12/10 12:06:30, pbos-webrtc wrote:
> > Please also change the CL description (Edit Issue) to 
> > 
> > <short description, one line>
> > 
> > <long description>
> > 
> > BUG=...
> > 
> > (Don't put BUG= at top, that becomes the subject for the CL)
> 
> What is the performance impact? If I420 is not a hardware-supported format,
then
> iOS is likely performing a conversion behind the scenes anyway from the
hardware
> output into the requested format. We should explicitly measure to get an idea
of
> what's going on.

I tested with my demo with iPhone 5C, iOS 9.1, 
I see about 2%~3% CPU drop when changed from NV12 to I420, so that should prove
I420 is supported.

[wangrui, 2015-12-11 03:22:33]

Description was changed from

==========
BUG=webrtc:5230
change videotoolbox encoder input pixelformat from NV12 to I420,
also change decoder output from NV12 to I420.
==========

to

==========
change videotoolbox format from NV12 to I420

change videotoolbox encoder input pixelformat from NV12 to I420,
also change decoder output from NV12 to I420.
I use instruments to test the CPU usage change, there is about 2%~3% CPU drop
after remove the conversion from NV12 to I420.

BUG=webrtc:5230
==========

[wangrui, 2015-12-11 03:43:10]

modify code styles on patch set 2.
and test performances after change from NV12 to I420.

[wangrui, 2016-01-05 01:18:27]

On 2015/12/10 22:10:23, tkchin_webrtc wrote:
> On 2015/12/10 12:06:30, pbos-webrtc wrote:
> > Please also change the CL description (Edit Issue) to 
> > 
> > <short description, one line>
> > 
> > <long description>
> > 
> > BUG=...
> > 
> > (Don't put BUG= at top, that becomes the subject for the CL)
> 
> What is the performance impact? If I420 is not a hardware-supported format,
then
> iOS is likely performing a conversion behind the scenes anyway from the
hardware
> output into the requested format. We should explicitly measure to get an idea
of
> what's going on.

according to comments in VideoToolbox framework:

The compression session creates this pixel buffer pool based on
the compressor's pixel buffer attributes and any pixel buffer
attributes passed in to VTCompressionSessionCreate.  If the
source pixel buffer attributes and the compressor pixel buffer
attributes cannot be reconciled, the pool is based on the source
pixel buffer attributes and the Video Toolbox converts each CVImageBuffer
internally.

So VideoToolbox will do internal conversion between different formats, that
could be a hardware conversion,
so why are we doing an external conversion(definitely soft) when there is an
internal one(could be hardware)?
please consider this. thanks.

[pbos-webrtc, 2016-05-01 23:50:18]

pbos@webrtc.org changed reviewers:
- pbos@webrtc.org