Support for remote audio into tracks

talk/app/webrtc/webrtcsession.h

 /*
  * libjingle
  * Copyright 2012 Google Inc.
  *
  * Redistribution and use in source and binary forms, with or without
  * modification, are permitted provided that the following conditions are met:
  *
  *  1. Redistributions of source code must retain the above copyright notice,
  *     this list of conditions and the following disclaimer.
  *  2. Redistributions in binary form must reproduce the above copyright notice,
@@ skipping 20 matching lines @@
 #include <string>
 #include <vector>
 
 #include "talk/app/webrtc/datachannel.h"
 #include "talk/app/webrtc/dtmfsender.h"
 #include "talk/app/webrtc/mediacontroller.h"
 #include "talk/app/webrtc/mediastreamprovider.h"
 #include "talk/app/webrtc/peerconnectioninterface.h"
 #include "talk/app/webrtc/statstypes.h"
 #include "talk/media/base/mediachannel.h"
-#include "webrtc/p2p/base/transportcontroller.h"
 #include "talk/session/media/mediasession.h"
 #include "webrtc/base/sigslot.h"
 #include "webrtc/base/sslidentity.h"
 #include "webrtc/base/thread.h"
+#include "webrtc/p2p/base/transportcontroller.h"
 
 namespace cricket {
 
 class ChannelManager;
 class DataChannel;
 class StatsReport;
 class VideoCapturer;
 class VideoChannel;
 class VoiceChannel;
 
@@ skipping 187 matching lines @@
   virtual bool GetLocalTrackIdBySsrc(uint32_t ssrc, std::string* track_id);
   virtual bool GetRemoteTrackIdBySsrc(uint32_t ssrc, std::string* track_id);
 
   // AudioMediaProviderInterface implementation.
   void SetAudioPlayout(uint32_t ssrc, bool enable) override;
   void SetAudioSend(uint32_t ssrc,
                     bool enable,
                     const cricket::AudioOptions& options,
                     cricket::AudioRenderer* renderer) override;
   void SetAudioPlayoutVolume(uint32_t ssrc, double volume) override;
+  void SetRawAudioSink(uint32_t ssrc,

[juberti, 2015/12/12 00:43:09]

Is there a reason why this doesn't use cricket::AudioRenderer (similar to
SetAudioSend above and SetVideoPlayout below)? It would be nice to maintain
symmetry between audio and video.

[tommi, 2015/12/12 01:09:11]

I started out with that but Fredrik and I ended up with this approach.  Here are
some of the reasons:

* There are then three layers of sinks.  AudioTrackSinkInterface,
AudioRenderer::Sink and the sink inside of WebRTC, I was using VoEMediaProcess. 
Now there are two layers (internal/external)

* Fredrik is refactoring the voice engine and the last interface may well be
going away.  So there's a desire to have one that replaces it and ideally has
some of the things we haven't had, such as timestamp.  So, we decided to add
this new sink and leave the other alone.  Since it's a part of the new
sanctioned voe code, it makes sense to use it from the new libjingle code
(mediastreamprovider.h) which is already using related types such as
AudioReceiveStream.

* Lifetime management is slightly less confusing with the current approach. 
Since the provider and consumer of the audio data will have different lifetimes,
they need to both be aware of each other and either can terminate the sink. 
AudioRenderer::Sink has an OnClose method for the provider to notify the
consumer and the consumer would typically disconnect a sink by calling
AudioRenderer->SetSink(nullptr) or comparable method, to clear it. This kind of
worked.  That is until I realized I had layers of implementations of SetSink and
all of the implementations of SetSink, had different lifetimes.  So the source
might have a pointer to a WebRtcSession (which implements
AudioProviderInterface) but the the actual sink pointer could be sent down
further layers to a different object that actually provides the data and has a
longer lifetime than the session object.  This would cause OnClose() to be
called at the right time, but when the session died, there was no way of knowing
it for the source object, so if it tried to terminate the sink, there would be a
UAF.
  So I decided that it would be better to adapt a model where holding a pointer
back to the provider wasn't necessary and that the way a sink would be notified
that data will no longer be provided, would be by deletion of the sink.  This
forces the consumer to have a separate sink implementation which takes care of
the notifications for both sides and its implementation is ultimately up to the
consumer.
  OnClose() isn't necessary in that model so I didn't see much point in using
AudioRenderer::Sink differently than it is intended to be used and also outside
of the context of AudioRenderer.

* AudioRenderer is in cricket which adds one more namespace into the mix (there
are one or two other types from cricket in the PeerConnection API but the intent
is to migrate types into webrtc/rtc).

* There may be more properties that will accompany the audio data in the future,
so we opted for a struct to hold the data for slightly improved future
compatibility instead of changing function prototypes.

+                       rtc::scoped_ptr<AudioSinkInterface> sink) override;
 
   // Implements VideoMediaProviderInterface.
   bool SetCaptureDevice(uint32_t ssrc, cricket::VideoCapturer* camera) override;
   void SetVideoPlayout(uint32_t ssrc,
                        bool enable,
                        cricket::VideoRenderer* renderer) override;
   void SetVideoSend(uint32_t ssrc,
                     bool enable,
                     const cricket::VideoOptions* options) override;
 
@@ skipping 245 matching lines @@
   PeerConnectionInterface::BundlePolicy bundle_policy_;
 
   // Declares the RTCP mux policy for the WebRTCSession.
   PeerConnectionInterface::RtcpMuxPolicy rtcp_mux_policy_;
 
   RTC_DISALLOW_COPY_AND_ASSIGN(WebRtcSession);
 };
 }  // namespace webrtc
 
 #endif  // TALK_APP_WEBRTC_WEBRTCSESSION_H_