Integrate Intelligibility with APM

webrtc/modules/audio_processing/audio_buffer.cc

Index: webrtc/modules/audio_processing/audio_buffer.cc
diff --git a/webrtc/modules/audio_processing/audio_buffer.cc b/webrtc/modules/audio_processing/audio_buffer.cc
index 04dcaea799d60af6bbc48d899e9ded8134d6ce03..cc8375e449b1e300b96d616c7d6cd782e4e618e8 100644
--- a/webrtc/modules/audio_processing/audio_buffer.cc
+++ b/webrtc/modules/audio_processing/audio_buffer.cc
@@ -436,20 +436,28 @@ void AudioBuffer::DeinterleaveFrom(AudioFrame* frame) {
 }
 
 void AudioBuffer::InterleaveTo(AudioFrame* frame, bool data_changed) const {
-  assert(proc_num_frames_ == output_num_frames_);
-  assert(num_channels_ == num_input_channels_);
-  assert(frame->num_channels_ == num_channels_);
-  assert(frame->samples_per_channel_ == proc_num_frames_);
   frame->vad_activity_ = activity_;
-
   if (!data_changed) {
     return;
   }
 
-  Interleave(data_->ibuf()->channels(),
-             proc_num_frames_,
-             num_channels_,
-             frame->data_);
+  assert(proc_num_frames_ == output_num_frames_);
+  assert(frame->num_channels_ == num_channels_ || num_channels_ == 1);
+  assert(frame->samples_per_channel_ == proc_num_frames_);
+
+  if (frame->num_channels_ == num_channels_) {
+    Interleave(data_->ibuf()->channels(), proc_num_frames_, num_channels_,
+               frame->data_);
+  } else {
+    // Copy single AudioBuffer channel into all AudioFrame channels

[Andrew MacDonald, 2015/07/24 23:50:39]

Period at the end, and can you note that this is specifically for
intelligibility enhancement?

This is a problem (amongst other things) with introducing this render-side
processing. APM wasn't designed to handle it and so you're kind of shoe-horning
it in. I don't have a better suggestion without large refactoring though.

Ideally you would do your processing on however many channels you received
before downmixing to mono. But that would add a bunch of complexity to a system
that's already straining under the load. I think this upmixing is OK, but you
should note in the interface that if intelligibility enhancement is enabled the
reverse stream will become an upmixed mono signal.

[ekm, 2015/07/29 00:37:19]

Done.

+    rtc::scoped_ptr<int16_t*> channel_ptr_copies(
+        new int16_t*[frame->num_channels_]);

[Andrew MacDonald, 2015/07/24 23:50:39]

Arg, why did you switch to using dynamic allocation?

But anyway, perhaps better to add a UpmixMonoToInterleave? See the new
DownmixInterleavedToMono Michael added here:
https://chromium.googlesource.com/external/webrtc/+/master/webrtc/common_audi...

[ekm, 2015/07/29 00:37:19]

Done.

+    for (int i = 0; i < frame->num_channels_; ++i) {
+      channel_ptr_copies.get()[i] = data_->ibuf()->channels()[0];
+    }
+    Interleave(channel_ptr_copies.get(), proc_num_frames_, num_channels_,
+               frame->data_);
+  }
 }
 
 void AudioBuffer::CopyLowPassToReference() {