audioproc_f with simulated mic analog gain

webrtc/modules/audio_processing/test/fake_recording_device.cc

Index: webrtc/modules/audio_processing/test/fake_recording_device.cc
diff --git a/webrtc/modules/audio_processing/test/fake_recording_device.cc b/webrtc/modules/audio_processing/test/fake_recording_device.cc
new file mode 100644
index 0000000000000000000000000000000000000000..8bf37d0a10a37baf7f218ab92b096419ba69ad00
--- /dev/null
+++ b/webrtc/modules/audio_processing/test/fake_recording_device.cc
@@ -0,0 +1,85 @@
+/*
+ *  Copyright (c) 2017 The WebRTC project authors. All Rights Reserved.
+ *
+ *  Use of this source code is governed by a BSD-style license
+ *  that can be found in the LICENSE file in the root of the source
+ *  tree. An additional intellectual property rights grant can be found
+ *  in the file PATENTS.  All contributing project authors may
+ *  be found in the AUTHORS file in the root of the source tree.
+ */
+
+#include "webrtc/modules/audio_processing/test/fake_recording_device.h"
+
+#include "webrtc/base/logging.h"
+
+namespace webrtc {
+
+FakeRecordingDevice::FakeRecordingDevice(LevelToScalingMappingKind mapping_kind)
+    : mapping_kind_(mapping_kind) {}
+
+FakeRecordingDevice::~FakeRecordingDevice() = default;
+
+void FakeRecordingDevice::SimulateAnalogGain(
+    std::vector<rtc::ArrayView<const float>> src,
+    std::vector<rtc::ArrayView<float>> dest,
+    int level,
+    int real_device_level) {
+  RTC_DCHECK_EQ(src.size(), dest.size());
+  const float scaling_factor = ComputeCompoundScalingFactor(
+      level, real_device_level);
+
+  for (size_t i = 0; i < src.size(); ++i) {
+    RTC_DCHECK_EQ(src[i].size(), dest[i].size());
+    std::transform(src[i].begin(), src[i].end(), dest[i].begin(),
+                   [scaling_factor](float x) { return scaling_factor * x; });

[peah-webrtc, 2017/05/05 20:25:21]

This scaling could create absolute sample values above 1.0f. Saturation is
needed for this to be realistic.

[AleBzk, 2017/05/16 08:53:03]

Good point. And we should apply saturation twice:
1. apply the undo level (if any) and use hard-clipping
2. apply the scaling factor and use hard-clipping

If everything is done at once and there is an undo level causing saturation with
a low scaling factor, we then miss the clipping in the signal.

Do you agree?

[peah-webrtc, 2017/05/16 12:19:35]

I don't see why we need the hard-clipping in step 1. 

+  }
+}
+
+void FakeRecordingDevice::SimulateAnalogGain(const AudioFrame* src,
+                                             AudioFrame* dest,
+                                             int level,
+                                             int real_device_level) {
+  RTC_DCHECK_EQ(src->num_channels_, dest->num_channels_);
+  RTC_DCHECK_EQ(src->samples_per_channel_, dest->samples_per_channel_);
+  const size_t number_of_samples =
+      src->samples_per_channel_ * src->num_channels_;
+  RTC_DCHECK_LE(number_of_samples, AudioFrame::kMaxDataSizeSamples);
+  const float scaling_factor = ComputeCompoundScalingFactor(
+      level, real_device_level);
+
+  std::transform(src->data_, src->data_ + number_of_samples, dest->data_,
+                 [scaling_factor](int16_t x) {
+                   return rtc::saturated_cast<int16_t>(scaling_factor * x);

[peah-webrtc, 2017/05/05 20:25:21]

This simulation of the microphone scaling is too simple, but could serve as a
starting point. However, as the rest of the code indicates that the microphone
type (kind) only affects the gain curve mapping I want to point this out already
now.

For instance, to properly simulate a microphone (for the purpose of the AGC
development) you will almost immediately run into the need for simulating
soft-saturation, which is not handled in the current design.

I would rather have prefferred a layout where there is a call to a separate
microphone simulation method for each of the microphone types supported.

The same applies to the method on line 22.

[AleBzk, 2017/05/16 08:53:03]

Great point! Thanks.

I redesigned what goes on behind the scenes.
Now, we have callbacks responsible for changing a sample given the mic level to
simulate and the optional real device level. These callbacks can also depend on
the sample itself, which is the case for soft-clipping (e.g.,
https://ccrma.stanford.edu/~jos/pasp/Soft_Clipping.html).

This design also allows to have memory. It is only sufficient to add private
members in FakeRecordingDevice. Then the returned callbacks will have read-write
access to such members (which can be used to implement memory). I checked this
by adding an int member which is incremented every time
AnalogGainSimulatorIdentityFloat is called. It worked (and obviously I removed
that code).

+                 });
+}
+
+float FakeRecordingDevice::ComputeCompoundScalingFactor(
+    int level, int real_device_level) const {
+  const float scaling_factor = GetScalingFactor(level);
+  const float undo_factor = (real_device_level == kRealDeviceLevelUnknown) ?
+      1.0f : GetScalingFactor(real_device_level);
+
+  if (undo_factor == 0) {
+    LOG(LS_VERBOSE) << "[fake rec device] Cannot undo analog level of 0.";

[peah-webrtc, 2017/05/05 20:25:21]

As this is only simulation/testing code, you should not include error handling.
Instead just DCHECK as that will simplify the code a lot. That is also how it is
handled elsewhere in audioproc_f.

Silently (at least if you don't write out the verbose log-lines) will easily
cause this error to go unnoticed.

[aleloi, 2017/05/08 10:15:23]

I think a level of 0 can appear in real recordings. Should that be treated as an
error (and crash in debug-mode with DCHECK)? WDYT?

[peah-webrtc, 2017/05/08 11:41:33]

No, nothing that occurs in real recordings should be considered an error.

[AleBzk, 2017/05/16 08:53:03]

I'm a bit confused here.
We shouldn't include error handling, but we should neither treat a 0 level as an
error.
Then, the correct way of dealing with 0 real device levels should be the
original approach from Alex, that is logging and skipping the level undo part.
WDYT?

For now I went for the DCHECK way.

[peah-webrtc, 2017/05/16 12:19:35]

The action to take there is totally dependent on the simulation of the
microphone. My point was mainly that it does not make sense to consider
something that is in a real device an error, as that in my mind rather means
that the microphone simulation is wrong (what is more correct, the reality or
the simulation ;-) ).

If you insist, I'm fine with a DCHECK for now, as this is not yet really used,
but once you do this properly the DCHECK does not make sense at all.

+    return scaling_factor;
+  } else {
+    return scaling_factor / undo_factor;
+  }
+}
+
+float FakeRecordingDevice::GetScalingFactor(int level) const {
+  RTC_DCHECK_LE(0, level);
+  RTC_DCHECK_LE(level, 255);
+
+  switch (mapping_kind_) {
+    case LevelToScalingMappingKind::kLinear: {
+      return static_cast<float>(level) / 255.0f;
+    }
+    case LevelToScalingMappingKind::kIdentity: {
+      return 1.0f;
+    }
+    default: { RTC_NOTREACHED(); }
+  }
+}
+
+}  // namespace webrtc