Bug 1014393 - Move AudioGenerator to its own files so it can be used in more media gtests. r=pehrsons Differential Revision: https://phabricator.services.mozilla.com/D35389

Bug 1014393 - MediaEncoder better orders frames passed to the muxer. r=pehrsons Update MediaEncoder to pass frames to the muxer in order of their time stamps. This should prevent the currently possible scenario where audio and video frames are written with non-monotonically increasing timestamps (in violation of the webm spec). Differential Revision: https://phabricator.services.mozilla.com/D35388

Bug 1014393 - Use MediaQueue to store frames pending write to muxer in MediaEncoder. r=pehrsons MediaQueue provides a better interface for interleaving frames when writing to the muxer (this change will follow in another changeset). The queue interface provides a nicer abstraction than manually managing a nsTArray. MozReview-Commit-ID: 5V5XmYODFdA Differential Revision: https://phabricator.services.mozilla.com/D35387

Bug 1014393 - Update EncodedFrame class to more closely resemble MediaData class. r=pehrsons This changes EncodedFrame to behave more like MediaData, so that EncodedFrame can be used with the MediaQueue data structure. It also provides a somewhat more consistent interface across media data types. MozReview-Commit-ID: I2o6n30ErxB Differential Revision: https://phabricator.services.mozilla.com/D35386

Bug 1014393 - Shift responsibility of adjusting packets with opus codec delay to MediaEncoder. r=pehrsons Move the responsibility of adjusting opus frame timestamps to the MediaEncoder. This was previously done by the EbmlComposer, but doing so in the MediaEncoder means we can have greater control over handling of time codes and interleaving of frames. MozReview-Commit-ID: 2g9cy1IqOph Differential Revision: https://phabricator.services.mozilla.com/D35385

Bug 1014393 - Remove EncodedFrameContainer. r=pehrsons Remove EncodedFrameContainer and clean up areas where it was used. EncodedFrameContainer provided a wrapper around an nsTArray<RefPtr<EncodedFrame>>, but it simplifies the code to simply expose this array. Also clean up unused enums in EncodedFrame, and clean up some of the outdated comments for our encoded frame handling. MozReview-Commit-ID: Bh3VKesVoJE Differential Revision: https://phabricator.services.mozilla.com/D35384

Bug 1014393 - Separate MediaEncoders encode and mux steps. r=pehrsons Separating the encode and mux steps allows for better control over interleaving audio and video data. If encode and mux are done in a single step it's possible to mux large amounts of audio or video data which should have been interleaved with the other data type to give correctly ordered time stamps in the target container. MozReview-Commit-ID: CBYemrkpyF5 Differential Revision: https://phabricator.services.mozilla.com/D35383

Bug 1324552 - Implement AudioContext baseLatency and outputLatency attributes. r=pehrsons,baku Differential Revision: https://phabricator.services.mozilla.com/D37280

Bug 1324552 - Implement AudioContext baseLatency and outputLatency attributes. r=pehrsons,baku Differential Revision: https://phabricator.services.mozilla.com/D37280

Bug 1528031 - Update tests to work with [SecureContext] navigator.mediaDevices. r=pehrsons,smaug Differential Revision: https://phabricator.services.mozilla.com/D35970

Bug 1528031 - Require [SecureContext] for navigator.mediaDevices & navigator.mozGetUserMedia(). r=pehrsons,smaug Differential Revision: https://phabricator.services.mozilla.com/D33836

Bug 1528031 - Update tests to work with [SecureContext] navigator.mediaDevices. r=pehrsons,smaug Differential Revision: https://phabricator.services.mozilla.com/D35970

Bug 1528031 - Require [SecureContext] for navigator.mediaDevices & navigator.mozGetUserMedia(). r=pehrsons,smaug Differential Revision: https://phabricator.services.mozilla.com/D33836

Bug 1429847 - Promote the GraphRunner thread. r=pehrsons Differential Revision: https://phabricator.services.mozilla.com/D32499

Bug 1552571: Optimize GetUserMediaWindowListener's source listener iteration and ownership. r=pehrsons Differential Revision: https://phabricator.services.mozilla.com/D31687

Bug 1553215 - Implement the correct behaviour for MediaStreamAudioSource. r=karlt,pehrsons It should pick and lock to the right track, regardless of if it's still in the MediaStream. The test fix is because we don't expose tracks until the HTMLMediaElement has loaded loading I think. This alignes with a couple spec changes: - https://github.com/WebAudio/web-audio-api/issues/264 - https://github.com/WebAudio/web-audio-api/pull/1829/files and also throws when no valid track are found in the MediaStream, either because it's all video tracks or because it has no tracks. Differential Revision: https://phabricator.services.mozilla.com/D32176

Bug 1553215 - Start testing MediaStreamAudioSourceNode in WPT. r=karlt,pehrsons It's not complete I think, but it covers the constructor and some bits of behaviour. Differential Revision: https://phabricator.services.mozilla.com/D32175

Bug 1553215 - Implement the correct behaviour for MediaStreamAudioSource. r=karlt,pehrsons It should pick and lock to the right track, regardless of if it's still in the MediaStream. The test fix is because we don't expose tracks until the HTMLMediaElement has loaded loading I think. This alignes with a couple spec changes: - https://github.com/WebAudio/web-audio-api/issues/264 - https://github.com/WebAudio/web-audio-api/pull/1829/files and also throws when no valid track are found in the MediaStream, either because it's all video tracks or because it has no tracks. Differential Revision: https://phabricator.services.mozilla.com/D32176

Bug 1553215 - Start testing MediaStreamAudioSourceNode in WPT. r=karlt,pehrsons It's not complete I think, but it covers the constructor and some bits of behaviour. Differential Revision: https://phabricator.services.mozilla.com/D32175

Bug 1553215 - Implement the correct behaviour for MediaStreamAudioSource. r=karlt,pehrsons It should pick and lock to the right track, regardless of if it's still in the MediaStream. The test fix is because we don't expose tracks until the HTMLMediaElement has loaded loading I think. This alignes with a couple spec changes: - https://github.com/WebAudio/web-audio-api/issues/264 - https://github.com/WebAudio/web-audio-api/pull/1829/files and also throws when no valid track are found in the MediaStream, either because it's all video tracks or because it has no tracks. Differential Revision: https://phabricator.services.mozilla.com/D32176

Bug 1553215 - Start testing MediaStreamAudioSourceNode in WPT. r=karlt,pehrsons It's not complete I think, but it covers the constructor and some bits of behaviour. Differential Revision: https://phabricator.services.mozilla.com/D32175

Bug 1552571: Optimize GetUserMediaWindowListener's source listener iteration and ownership. r=pehrsons Differential Revision: https://phabricator.services.mozilla.com/D31687

Bug 1550955 - Make StopTrack() API require a hard reference. r=pehrsons Differential Revision: https://phabricator.services.mozilla.com/D31555

Bug 1335740 - Update wpt & mochitests to work w/[SecureContext] navigator.mediaDevices. r=pehrsons Differential Revision: https://phabricator.services.mozilla.com/D20425

Bug 1335740 - getUserMedia() Add 2 prefs to control A) NotAllowedError in http (pref'd on), and B) [SecureContext] navigator.mediaDevices (pref'd off) r=bzbarsky,pehrsons Differential Revision: https://phabricator.services.mozilla.com/D19549

Bug 1541290 - Close audio input from main thread to ensure that the message is not executed on a closed graph. r=pehrsons CloseAudioInut method posts a message, to the graph thread, in order to close the input asynchonously. When CloseAudioInput method was being executed from a thread other than the main thread, a runnable would be posted to main thread in order to post the async message from there. That was a risky path because when the graph was shutting down there were no guarantee that the close-input message would reach the graph thread before destroy takes place. By limiting CloseAudioInput to main thread it is ensured that the close-input message will be executed before destroy. Differential Revision: https://phabricator.services.mozilla.com/D27751

Bug 1531833 - When the input is voice, activate the global communication mode. r=pehrsons Differential Revision: https://phabricator.services.mozilla.com/D21738

Bug 1531833 - Add a way to tell the graph driver that the audio input is voice. r=pehrsons Differential Revision: https://phabricator.services.mozilla.com/D21737

Bug 1531833 - Expose a way to set a global "communication mode" in CubebUtils, to be use by real-time duplex audio users. r=pehrsons Differential Revision: https://phabricator.services.mozilla.com/D21736

Bug 1531833 - When the input is voice, activate the global communication mode. r=pehrsons Differential Revision: https://phabricator.services.mozilla.com/D21738

Bug 1531833 - Add a way to tell the graph driver that the audio input is voice. r=pehrsons Differential Revision: https://phabricator.services.mozilla.com/D21737

Bug 1531833 - Expose a way to set a global "communication mode" in CubebUtils, to be use by real-time duplex audio users. r=pehrsons Differential Revision: https://phabricator.services.mozilla.com/D21736

Bug 1324548 - Add MediaStreamTrackAudioSourceNode. r=pehrsons,baku MozReview-Commit-ID: IdVqfNigMyu Differential Revision: https://phabricator.services.mozilla.com/D15958

Bug 1324548 - Tests for MediaStreamTrackAudioSourceNode. r=pehrsons Differential Revision: https://phabricator.services.mozilla.com/D16066

Bug 1324548 - Add MediaStreamTrackAudioSourceNode. r=pehrsons,baku MozReview-Commit-ID: IdVqfNigMyu Differential Revision: https://phabricator.services.mozilla.com/D15958

Bug 1324548 - Tests for MediaStreamTrackAudioSourceNode. r=pehrsons Differential Revision: https://phabricator.services.mozilla.com/D16066

Bug 1324548 - Add MediaStreamTrackAudioSourceNode. r=pehrsons,baku MozReview-Commit-ID: IdVqfNigMyu Differential Revision: https://phabricator.services.mozilla.com/D15958

Bug 1324548 - Tests for MediaStreamTrackAudioSourceNode. r=pehrsons Differential Revision: https://phabricator.services.mozilla.com/D16066

Bug 1540251 - Workaround unset NAL size in WebrtcGmpVideoEncoder::Encoded; r=pehrsons OpenH264 1.8.1 occasionally generates a size of 0x01000000. This is a magic value in the NAL which should be replaced with a valid size, but for some reason this is not always happening. If we return early here, encoding will continue to work as expected. This workaround can be removed once this issue is addressed upstream, although that may require a new release of OpenH264. Differential Revision: https://phabricator.services.mozilla.com/D25468

Bug 1539220 - Prefer non-RGB24 capabilities when available; r=pehrsons We've hit a number of problems with handling of RGB24 video capture on Windows. This adds a check that will ignore any RGB24 capture capabilities when determining a best match if there are other capabilities available to workaround the problems. Differential Revision: https://phabricator.services.mozilla.com/D25449

Bug 1532465 - Ensure we AddRef prior to early return in MediaType::Assign; r=pehrsons Differential Revision: https://phabricator.services.mozilla.com/D23143

Bug 1521169 - Ensure that streamCount is at least one in CreateEncoderStreams; r=pehrsons We currently have a check that we create at least one stream in CreateEncoderStreams. This is failing occasionally. This adds a new check to the top of the method to check that at least one stream has been requested. This will help to narrow down where the problem is occurring. Differential Revision: https://phabricator.services.mozilla.com/D24550

Bug 1521169 - Check that at least one stream is requested in ConfigureSendMediaCodec; r=pehrsons This moves the existing release assert in CreateSendStream that ensures we have requested at least one stream to ConfigureSendMediaCodec. This covers the case where we don't recreate the send stream when only stream attributes have changed. It does not appear to be possible for no streams to be configured by the Transceivers code. Once we've verified that is the case with the release assert, we can remove it and use error handling code instead. Differential Revision: https://phabricator.services.mozilla.com/D20548

Bug 1521169 - Remove VideoEncoderConfigBuilder; r=pehrsons The SimulcastStreamConfig code is unused. Once that is removed, all that remains is a thin wrapper around webrtc::VideoEncoderConfig. We might as well use it directly. Differential Revision: https://phabricator.services.mozilla.com/D24549

Bug 1521169 - Rename SimulcastEncoding to Encoding in VideoCodecConfig; r=pehrsons The Transceivers code just calls these encodings, and we use them even if we're not doing simulcast, so we should just call them encodings as well. Differential Revision: https://phabricator.services.mozilla.com/D24548

Bug 1538640 wait for GraphRunner thread shutdown r=pehrsons Differential Revision: https://phabricator.services.mozilla.com/D24673

bug 1538630 Check a predicate when waiting on condition variables in GraphRunner r=pehrsons so as to respond appropriately to spurious wakeups. Checking the predicate before the first wait in Run() makes the mStarted notification unnecessary. Differential Revision: https://phabricator.services.mozilla.com/D24672

Bug 1533252 - P4. Serialize all the video MediaCapabilities queries. r=pehrsons Differential Revision: https://phabricator.services.mozilla.com/D23653

Bug 1533252 - P3. Rename LocalAllocPolicy into SingleAllocPolicy. r=pehrsons It's more descriptive with what it actually does. Differential Revision: https://phabricator.services.mozilla.com/D23652

Bug 1533252 - P2. Make LocalAllocPolicy inherit from AllocPolicy. r=pehrsons Despite its name and apparent use LocalAllocPolicy could only handle a single call to Alloc() at a time which was okay due to how it was used by the MediaFormatReader. However, we do want to handle multiple calls to Alloc(). These changes allow to perform simultaneous requests and have them be processed one at a time serially. Differential Revision: https://phabricator.services.mozilla.com/D23651