Added first layer of the echo canceller 3 functionality

webrtc/modules/audio_processing/aec3/block_framer.cc

+/*
+ *  Copyright (c) 2016 The WebRTC project authors. All Rights Reserved.
+ *
+ *  Use of this source code is governed by a BSD-style license
+ *  that can be found in the LICENSE file in the root of the source
+ *  tree. An additional intellectual property rights grant can be found
+ *  in the file PATENTS.  All contributing project authors may
+ *  be found in the AUTHORS file in the root of the source tree.
+ */
+
+#include "webrtc/modules/audio_processing/aec3/block_framer.h"
+
+#include <algorithm>
+
+#include "webrtc/base/checks.h"
+
+namespace webrtc {
+
+BlockFramer::BlockFramer(size_t num_bands)
+    : num_bands_(num_bands),
+      buffer_(num_bands_, std::vector<float>(kBlockSize, 0.f)) {}

[aleloi, 2016/12/23 14:28:38]

Suggestion: initialize the buffer with 16 zeroes and possibly reserve space for
64. That would introduce less delay, and the calling scheme will still work.

[peah-webrtc, 2017/01/02 08:45:10]

That does not seem to work.
I think the reason is the way that BlockFramer is used together with
FrameBlocker.

For each frame that FrameBlocker receives, BlockFramer should output exactly one
frame (otherwise there is a gap in the stream). This means for insteance that
for a framelength of 80 samples (the same thing happens for 160 sample frame
lengths)
Frame #1
FrameBlocker input size: 80
FrameBlocker output num blocks: 1
FrameBlocker buffer size (after frame being received): 16
BlockFramer input num blocks: 1
FrameBlocker output size: 80
FrameBlocker buffer size (before frame is produced): x
FrameBlocker buffer size (after frame is produced): x-16

Frame #2
FrameBlocker input size: 80
FrameBlocker output num blocks: 1
FrameBlocker buffer size (after frame being received): 32
BlockFramer input num blocks: 1
FrameBlocker output size: 80
FrameBlocker buffer size (before frame is produced): x-16
FrameBlocker buffer size (after frame is produced): x-32

Frame #3
FrameBlocker input size: 80
FrameBlocker output num blocks: 1
FrameBlocker buffer size (after frame being received): 48
BlockFramer input num blocks: 1
FrameBlocker output size: 80
FrameBlocker buffer size (before frame is produced): x-32
FrameBlocker buffer size (after frame is produced): x-48

Frame #4
FrameBlocker input size: 80
FrameBlocker output num blocks: 2
FrameBlocker buffer size (after frame being received): 0
BlockFramer input num blocks: 2
FrameBlocker output size: 80
FrameBlocker buffer size (before frame is produced): x-48
FrameBlocker buffer size (after frame is produced but before block 2 is
received): x-64
FrameBlocker buffer size (after frame is produced and after block 2 is
received): x


In order to ensure that the BlockFramer buffer size does not go below 0, the
initial buffer size x must be chosen sufficiently large. The example above I
think shows that x at least needs to be 64 for that to be the case.

Currently, the case when more 2 blocks are received by BlockFramer when one
single frame is produced is handled in such a way that the extra block is
received via a separate method call (InsertBlock). If we would bundle that call
with the extraction of the frame (in InsertBlockAndExtractSubFrame) that would
allow for the buffer size to be reduced to 48. This would, however, have the
consequence that an extra block would (for performance reasons) need to be
cached on the EchoCanceller3 state so in total that would result in an increase
of 48 samples in state memory footprint. Furthermore, that reduction to a buffer
size of 48 would not correspond to a lower signal delay. Therefore, I think it
is better to use an initial (and maximum) buffer size of 64 inside BlockFramer.

Sorry about the long explanation, but I needed to convince myself as well that
my reasoning was correct. 

To summarize, in order to maintain a continuous output stream I think an initial
buffer size of 64 is needed. It is possible to jump-start it by filling it with
true output samples, but at some time during the first frames, depending on how
this is done, there will anyway be a lack of samples when the output frame is to
be produced. Those samples will need to be filled by zeros as it is not possible
to wait with producing the frame. And that insertion of zeros will again revert
the delay  reduction achieved by the jump-start.

WDYT? Does this make sense?

[aleloi, 2017/01/09 13:49:28]

Ok, I see. Sorry for the delay.

+
+BlockFramer::~BlockFramer() = default;
+
+// All the constants are chosen so that the buffer is either empty or has enough
+// samples for InsertBlockAndExtractSubFrame to produce a frame. In order to
+// achieve this, the InsertBlockAndExtractSubFrame and InsertBlock methods need
+// to be called in the correct order.
+void BlockFramer::InsertBlock(const std::vector<std::vector<float>>& block) {
+  RTC_DCHECK_EQ(num_bands_, block.size());
+  for (size_t i = 0; i < num_bands_; ++i) {
+    RTC_DCHECK_EQ(kBlockSize, block[i].size());
+    RTC_DCHECK_EQ(0, buffer_[i].size());
+    buffer_[i].insert(buffer_[i].begin(), block[i].begin(), block[i].end());
+  }
+}
+
+void BlockFramer::InsertBlockAndExtractSubFrame(
+    const std::vector<std::vector<float>>& block,
+    std::vector<rtc::ArrayView<float>>* sub_frame) {
+  RTC_DCHECK_EQ(num_bands_, block.size());
+  RTC_DCHECK_EQ(num_bands_, sub_frame->size());
+  for (size_t i = 0; i < num_bands_; ++i) {
+    RTC_DCHECK_LE(kSubFrameLength, buffer_[i].size() + kBlockSize);
+    RTC_DCHECK_EQ(kBlockSize, block[i].size());
+    RTC_DCHECK_GE(kBlockSize, buffer_[i].size());
+    RTC_DCHECK_EQ(kSubFrameLength, (*sub_frame)[i].size());
+    const int samples_to_frame = kSubFrameLength - buffer_[i].size();
+    std::copy(buffer_[i].begin(), buffer_[i].end(), (*sub_frame)[i].begin());
+    std::copy(block[i].begin(), block[i].begin() + samples_to_frame,
+              (*sub_frame)[i].begin() + buffer_[i].size());
+    buffer_[i].resize(0);
+    buffer_[i].insert(buffer_[i].begin(), block[i].begin() + samples_to_frame,
+                      block[i].end());
+  }
+}
+
+}  // namespace webrtc