Make the nonlinear beamformer steerable

webrtc/modules/audio_processing/beamformer/nonlinear_beamformer.h

 /*
  *  Copyright (c) 2014 The WebRTC project authors. All Rights Reserved.
  *
  *  Use of this source code is governed by a BSD-style license
  *  that can be found in the LICENSE file in the root of the source
  *  tree. An additional intellectual property rights grant can be found
  *  in the file PATENTS.  All contributing project authors may
  *  be found in the AUTHORS file in the root of the source tree.
  */
 
@@ skipping 13 matching lines @@
 // Enhances sound sources coming directly in front of a uniform linear array
 // and suppresses sound sources coming from all other directions. Operates on
 // multichannel signals and produces single-channel output.
 //
 // The implemented nonlinear postfilter algorithm taken from "A Robust Nonlinear
 // Beamforming Postprocessor" by Bastiaan Kleijn.
 class NonlinearBeamformer
   : public Beamformer<float>,
     public LappedTransform::Callback {
  public:
-  explicit NonlinearBeamformer(const std::vector<Point>& array_geometry);
+  explicit NonlinearBeamformer(const std::vector<Point>& array_geometry,
+                               float target_angle_radians = M_PI / 2.f);

[Andrew MacDonald, 2015/10/14 22:12:31]

Since we're exposing a setter, I think it's redundant to also take a target
angle at construction time. WDYT?

[aluebs-webrtc, 2015/10/20 00:04:20]

Since it has a default value, it is not necessary to specify it. I was just
feeling it would be inefficient to initialize all matrices twice, once with the
default value and once with the real one, just because the user needs to set the
angle with the setter. If you still think it is redundant, I am happy to remove,
I don't have such a strong opinion.

 
   // Sample rate corresponds to the lower band.
   // Needs to be called before the NonlinearBeamformer can be used.
   void Initialize(int chunk_size_ms, int sample_rate_hz) override;
 
   // Process one time-domain chunk of audio. The audio is expected to be split
   // into frequency bands inside the ChannelBuffer. The number of frames and
   // channels must correspond to the constructor parameters. The same
   // ChannelBuffer can be passed in as |input| and |output|.
   void ProcessChunk(const ChannelBuffer<float>& input,
                     ChannelBuffer<float>* output) override;
 
+  void SteerBeam(float target_angle_radians);

[peah-webrtc, 2015/10/13 12:53:41]

My gut feeling is that it would be simpler for an outside user to use degrees
when steering the beam. At least I need to think a second time to get a feeling
for what angle a certain value in radians corresponds to.

[Andrew MacDonald, 2015/10/14 22:12:31]

This seems like a fairly generic beamformer task. We should probably add it to
Beamformer. And since we have an existing method taking a SphericalPointf, this
should as well for consistency.

(For comparison, consider AimAt in MR)

[aluebs-webrtc, 2015/10/20 00:04:20]

Agreed. Added.

[aluebs-webrtc, 2015/10/20 00:04:20]

That is true for human input, but this is probably be set automatically by some
BemaformingManager which uses a Localizer (returns radians) to decide where to
steer towards. Plus all trigonometric functions take radians by default. I would
use as a standard to always use radians except for human user input, as I did
for the test.

[peah-webrtc, 2015/10/20 21:22:33]

Acknowledged.

+
   bool IsInBeam(const SphericalPointf& spherical_point) override;
 
   // After processing each block |is_target_present_| is set to true if the
   // target signal es present and to false otherwise. This methods can be called
   // to know if the data is target signal or interference and process it
   // accordingly.
   bool is_target_present() override { return is_target_present_; }
 
  protected:
   // Process one frequency-domain block of audio. This is where the fun
   // happens. Implements LappedTransform::Callback.
   void ProcessAudioBlock(const complex<float>* const* input,
                          int num_input_channels,
                          size_t num_freq_bins,
                          int num_output_channels,
                          complex<float>* const* output) override;
 
  private:
   typedef Matrix<float> MatrixF;
   typedef ComplexMatrix<float> ComplexMatrixF;
   typedef complex<float> complex_f;
 
-  void InitFrequencyCorrectionRanges();
+  void InitLowFrequencyCorrectionRanges();
+  void InitHighFrequencyCorrectionRanges();
   void InitInterfAngles();
   void InitDelaySumMasks();
   void InitTargetCovMats();
+  void InitDifuseCovMats();
   void InitInterfCovMats();
+  void NormalizeCovMats();
 
   // Calculates postfilter masks that minimize the mean squared error of our
   // estimation of the desired signal.
   float CalculatePostfilterMask(const ComplexMatrixF& interf_cov_mat,
                                 float rpsiw,
                                 float ratio_rxiw_rxim,
                                 float rmxi_r);
 
   // Prevents the postfilter masks from degenerating too quickly (a cause of
   // musical noise).
@@ skipping 42 matching lines @@
   size_t high_mean_start_bin_;
   size_t high_mean_end_bin_;
 
   // Quickly varying mask updated every block.
   float new_mask_[kNumFreqBins];
   // Time smoothed mask.
   float time_smooth_mask_[kNumFreqBins];
   // Time and frequency smoothed mask.
   float final_mask_[kNumFreqBins];
 
+  // For both target and interference angles, PI / 2 is perpendicular to the
+  // microphone array, facing forwards. The positive direction goes
+  // counterclockwise.

[Andrew MacDonald, 2015/10/14 22:12:31]

This is really about our coordinate system, and specifically how we convert from
Cartesian to spherical (since the microphone positions are Cartesian).

I think this documentation belongs with the declaration of SphericalPoint.

Just as a reference, see this for the definition of Carteisan mic position
coordinates in Chromium:
https://code.google.com/p/chromium/codesearch#chromium/src/media/audio/audio_...

[aluebs-webrtc, 2015/10/20 00:04:20]

I am not sure if we want to impose a specific convention to the general Point
class, I don't have a strong opinion. Removed this comment and added
documentation for CartesianPoint and SphericalPoint following the Chromium
example you suggested.

+  float target_angle_radians_;
   // Angles of the interferer scenarios.
   std::vector<float> interf_angles_radians_;
 
   // Array of length |kNumFreqBins|, Matrix of size |1| x |num_channels_|.
   ComplexMatrixF delay_sum_masks_[kNumFreqBins];
   ComplexMatrixF normalized_delay_sum_masks_[kNumFreqBins];
 
-  // Array of length |kNumFreqBins|, Matrix of size |num_input_channels_| x
+  // Arrays of length |kNumFreqBins|, Matrix of size |num_input_channels_| x
   // |num_input_channels_|.
   ComplexMatrixF target_cov_mats_[kNumFreqBins];
-
+  ComplexMatrixF uniform_cov_mat_[kNumFreqBins];
   // Array of length |kNumFreqBins|, Matrix of size |num_input_channels_| x
   // |num_input_channels_|. ScopedVector has a size equal to the number of
   // interferer scenarios.
   ScopedVector<ComplexMatrixF> interf_cov_mats_[kNumFreqBins];
 
   // Of length |kNumFreqBins|.
   float wave_numbers_[kNumFreqBins];
 
   // Preallocated for ProcessAudioBlock()
   // Of length |kNumFreqBins|.
@@ skipping 12 matching lines @@
   // Number of blocks after which the data is considered interference if the
   // mask does not pass |kMaskSignalThreshold|.
   size_t hold_target_blocks_;
   // Number of blocks since the last mask that passed |kMaskSignalThreshold|.
   size_t interference_blocks_count_;
 };
 
 }  // namespace webrtc
 
 #endif  // WEBRTC_MODULES_AUDIO_PROCESSING_BEAMFORMER_NONLINEAR_BEAMFORMER_H_