Bug 1546154 p6: Remove MSCOMInitThreadPoolListener. r=padenot Threads are implicitly members of the multi-threaded apartment and calls to CoInitializeEx (and CoUninitializeEx) cause user32 to load. Differential Revision: https://phabricator.services.mozilla.com/D124934

Bug 1678060: Disable this test on linux gli jobs. r=jmaher,padenot Depends on D125896 Differential Revision: https://phabricator.services.mozilla.com/D125898

Bug 1652294 - Relax the test of the number of enumerated devices. r=padenot Differential Revision: https://phabricator.services.mozilla.com/D85559

Bug 1546154 p6: Remove MSCOMInitThreadPoolListener. r=padenot Threads are implicitly members of the multi-threaded apartment and calls to CoInitializeEx (and CoUninitializeEx) cause user32 to load. Differential Revision: https://phabricator.services.mozilla.com/D124934

Bug 1721731 - disable 'test_streams_element_capture_mediatrack.html' on Android debug. r=padenot Differential Revision: https://phabricator.services.mozilla.com/D125250

Bug 1725810 - Fix audio distortion r=padenot When two successive `AudioInputProcessing::NotifyInputData` calls are followed by only one `AudioInputProcessing::ProcessInput` call, the audio input data in the first `NotifyInputData` call will be dropped silently, and these incomplete audio data will lead to unrecognizable sounds. To avoid dropping any audio input, we need to queue all the unprocessed audio input data from `NotifyInputData` instead of keeping tracking the newest one only. Depends on D123996 Differential Revision: https://phabricator.services.mozilla.com/D123840

Bug 1725810 - Remove `AudioDataBuffers` r=padenot `AudioDataBuffers` is used to hold the information of the audio input and output. However, nothing special needs to be done there. Depends on D123997 Differential Revision: https://phabricator.services.mozilla.com/D123996

Bug 1725810 - Add a test case simulate the issue r=padenot Add a test for `AudioInputProcessing` to demonstrate the audio distortion problem Differential Revision: https://phabricator.services.mozilla.com/D123997

Bug 1725137 - Fix gtest suite from dom to be buildable outside unified. r=padenot Differential Revision: https://phabricator.services.mozilla.com/D122338

Bug 1718630 - Fix browsertime YTP widevine on windows r=perftest-reviewers,davehunt,padenot Differential Revision: https://phabricator.services.mozilla.com/D122853

Bug 1718630 - Fix browsertime YTP widevine on windows r=perftest-reviewers,davehunt,padenot Differential Revision: https://phabricator.services.mozilla.com/D122853

Bug 1726061 - Make mLiveBufferingAppended optional r=padenot,pehrsons `mLiveBufferingAppended` [1] can be wrapped by a `Maybe` naturally since it should only be used when `mLiveFramesAppended` [2] is `true`. It's easier to make sure `mLiveBufferingAppended` be used in a reasonable timing by doing so, instead of checking `mLiveFramesAppended` every time before using `mLiveBufferingAppended`. [1] https://searchfox.org/mozilla-central/rev/d3683dbb252506400c71256ef3994cdbdfb71ada/dom/media/webrtc/MediaEngineWebRTCAudio.h#236 [2] https://searchfox.org/mozilla-central/rev/d3683dbb252506400c71256ef3994cdbdfb71ada/dom/media/webrtc/MediaEngineWebRTCAudio.h#230 Differential Revision: https://phabricator.services.mozilla.com/D122797

Bug 1725523 - Use BinaryPath::GetFile to find mozav* libraries. r=padenot Instead of deriving their location from the location of liblgplglibs. Differential Revision: https://phabricator.services.mozilla.com/D122545

Bug 1725523 - Use BinaryPath::GetFile to find mozav* libraries. r=padenot Instead of deriving their location from the location of liblgplglibs. Differential Revision: https://phabricator.services.mozilla.com/D122545

Bug 1721569 - Make TracePayload easier to modify, and use maximum space for name - r=padenot An upcoming patch will change the size of ProfilerThreadId, which causes troubles with TracePayload. Trying to move members around didn't help, I'm guessing some padding remains, which cannot be accounted for by just enumerating member sizes. The proposed solution is to wrap all members in a macro (so they don't need to be repeated), and create a private struct with these members and a character, in order to find the exact offset at which the name will actually start. This is uglier, but more flexible, and allows future changes (internal or external) more easily, without having to modify the name-size formula. Differential Revision: https://phabricator.services.mozilla.com/D121050

Bug 1723799 - part2 : add test. r=padenot,webidl,smaug Differential Revision: https://phabricator.services.mozilla.com/D121681

Bug 1723799 - part1 : collect height for visible video playtime. r=padenot Differential Revision: https://phabricator.services.mozilla.com/D121680

Bug 1719183 - part3 : let running audio context to keep tab awake. r=padenot,nika Let running AudioContext also prevents tab from being suspended by asking/revoking the page awake via the browsing context. Differential Revision: https://phabricator.services.mozilla.com/D119839

Bug 1722440 - Update cubeb to e1456788. r=cubeb-reviewers,padenot Differential Revision: https://phabricator.services.mozilla.com/D120925

no bug - fx doc: adjust the link to 'contribute to moz projects' r=padenot DONTBUILD Differential Revision: https://phabricator.services.mozilla.com/D120974

Bug 1701383: Move the deregistration of the device changed callback to mInitShutdownThread. r=padenot Differential Revision: https://phabricator.services.mozilla.com/D118860

Bug 1716248 - Change AppendAndConsumeChunk's input parameter type r=padenot It's better to change the input parameter's type of `AppendAndConsumeChunk` from `AudioChunk*` to `AudioChunk&&` since the `AudioChunk` will be consumed once it's fed to this function. One benefit for doing so is to prevent the consumed `AudioChunk` from being used again after it's moved/consumed. Gecko has a clang-tidy check, bugprone-use-after-move [1], to avoid this kind of error. We should utilize this check instead of catching used-after-move error by human eyes. [1]: https://searchfox.org/mozilla-central/rev/f351e19360729b351bfc7c1386d6e4ca4ea676e2/tools/clang-tidy/config.yaml#70 Differential Revision: https://phabricator.services.mozilla.com/D117714

Bug 1716248 - Change AppendAndConsumeChunk's input parameter type r=padenot It's better to change the input parameter's type of `AppendAndConsumeChunk` from `AudioChunk*` to `AudioChunk&&` since the `AudioChunk` will be consumed once it's fed to this function. One benefit for doing so is to prevent the consumed `AudioChunk` from being used again after it's moved/consumed. Gecko has a clang-tidy check, bugprone-use-after-move [1], to avoid this kind of error. We should utilize this check instead of catching used-after-move error by human eyes. Differential Revision: https://phabricator.services.mozilla.com/D117714

Bug 1702646 - Add an util-function to append interleaved buffer in AudioSegment r=padenot Add an utility function named AppendFromInterleavedBuffer in AudioSegment to append data from the given interleaved buffer. This function does the same job as what AudioInputProcessing::InsertInGraph and NativeInputTrack::ProcessInput were used to do. As a result, these two functions can be eliminated or simplified. Depends on D116673 Differential Revision: https://phabricator.services.mozilla.com/D116674

Bug 1702646 - Fix TestAudioInputProcessing tests r=padenot The AudioInputProcessing now needs to call ProcessInput to process the given input data instead of processing them in NotifyInputData Differential Revision: https://phabricator.services.mozilla.com/D116673

Bug 1702646 - Share AudioSegments among AudioInputTrack r=padenot Each AudioInputTrack has its own AudioSegments storing the input audio data. When the AudioInputTrack is in pass-through mode, the AudioSegment is just the data de-interleaved from its raw data, without any audio processing magic applied on it. If there are multiple AudioInputTracks in pass-through mode exist in the same graph, then all of their AudioSegments are same. Before this patch, each of these AudioInputTracks allocates its own space to store its own AudioSegments even those data are same. This patch makes it possible for these AudioInputTracks to share the same AudioSegment data they need. By creating the AudioSegment in the NativeInputTrack, which is mapped to one specific device and is connected to the AudioInputTrack, the AudioInputTrack can fetch the AudioSegment data when they need and then append shared-references of the AudioChunk, inside the fetched AudioSegment, into their own AudioSegment. Therefore, we can have some AudioChunks created by the NativeInputTrack and shared among the AudioInputTracks in pass-through mode. Differential Revision: https://phabricator.services.mozilla.com/D114801

Bug 1702646 - Process audio input in ProcessInput r=padenot Instead of receiving and processing audio input data in AudioInputProcessing::NotifyInputData, we can get the audio input data from the connected source via the MediaInputPort and process it in AudioInputTrack::ProcessInput Differential Revision: https://phabricator.services.mozilla.com/D114807

Bug 1702646 - Use NativeInputTrack to deliver audio data r=padenot - Create a NativeInputTrack that is managed by main-thread in MTG - Replace mInputDeviceUsers by a <device-id, NativeInputTrack> map Differential Revision: https://phabricator.services.mozilla.com/D113680

Bug 1517199 - part6 : add A/V sync test case for changing playback rate dynamically. r=padenot Differential Revision: https://phabricator.services.mozilla.com/D116285

Bug 1517199 - part5 : handle the playback rate change on video track and add crash test. r=padenot When we adjust the playback rate on the audio track, the audio clock time would be no longer align with the graph time. Eg. playback rate=2, when the graph time passes 10s, the audio clock time actually already goes forward 20s. After audio track ended, the video track would start to drive the clock time but the video clock time is align with the graph time, which means it would be smaller than the audio clock in that situation. Therefore, we have to ignore the video clock time in that case. In addition, this patch also address the duration change on video frames based on the playback rate. Differential Revision: https://phabricator.services.mozilla.com/D115035

Bug 1517199 - part4 : add gtest. r=padenot Differential Revision: https://phabricator.services.mozilla.com/D114922

Bug 1517199 - part3 : add more logs. r=padenot Differential Revision: https://phabricator.services.mozilla.com/D114921

Bug 1517199 - part2 : enable wpts. r=padenot Differential Revision: https://phabricator.services.mozilla.com/D114916

Bug 1517199 - part1 : time stretching samples in AudioDecoderInputTrack. r=padenot By using the soundtouch library, this patch implements the time stretching on the samples in AudioDecoderInputTrack when its playback rate is not 1.0f, in order to support changing playback rate on the captured media stream. As the time stretcher has to be initialized by a fixed channel count, we would perform a realtime up-mix/down-mix for those audio chunks which have different channel count thant AudioDecoderInputTrack's initial channel count. Differential Revision: https://phabricator.services.mozilla.com/D114560

Bug 1702646 - Share AudioSegments among AudioInputTrack r=padenot Each AudioInputTrack has its own AudioSegments storing the input audio data. When the AudioInputTrack is in pass-through mode, the AudioSegment is just the data de-interleaved from its raw data, without any audio processing magic applied on it. If there are multiple AudioInputTracks in pass-through mode exist in the same graph, then all of their AudioSegments are same. Before this patch, each of these AudioInputTracks allocates its own space to store its own AudioSegments even those data are same. This patch makes it possible for these AudioInputTracks to share the same AudioSegment data they need. By creating the AudioSegment in the NativeInputTrack, which is mapped to one specific device and is connected to the AudioInputTrack, the AudioInputTrack can fetch the AudioSegment data when they need and then append shared-references of the AudioChunk, inside the fetched AudioSegment, into their own AudioSegment. Therefore, we can have some AudioChunks created by the NativeInputTrack and shared among the AudioInputTracks in pass-through mode. Differential Revision: https://phabricator.services.mozilla.com/D114801

Bug 1702646 - Process audio input in ProcessInput r=padenot Instead of receiving and processing audio input data in AudioInputProcessing::NotifyInputData, we can get the audio input data from the connected source via the MediaInputPort and process it in AudioInputTrack::ProcessInput Differential Revision: https://phabricator.services.mozilla.com/D114807

Bug 1702646 - Use NativeInputTrack to deliver audio data r=padenot - Create a NativeInputTrack that is managed by main-thread in MTG - Replace mInputDeviceUsers by a <device-id, NativeInputTrack> map Differential Revision: https://phabricator.services.mozilla.com/D113680

Bug 1712598 - part2 : add a test for A/V sync. r=padenot Differential Revision: https://phabricator.services.mozilla.com/D116055

Bug 1712598 - part1 : fill silence if detecting a gap in audio. r=padenot If there is a gap bewteen the audio that is going to be appended next and the amount of frames that we've appended, then we need to fill silence data in order to keep A/V sync correct. We did that for the DecodedStream [1] before, but that got removed during the refactoring. [1] https://searchfox.org/mozilla-central/rev/36181d2c169bafd5e13c534851c8b25d1567cfc3/dom/media/mediasink/DecodedStream.cpp#660-667 Differential Revision: https://phabricator.services.mozilla.com/D115857

Bug 1712595 - ensure AudioSink won't discard frames that are not played yet. r=padenot When we switch to the DecodedStream from AudioSink. AudioSink would store some frames in its own media queue before those audio frames get played. After AudioSink got shutdown, it doesn't push those frames back to the original audio queue. Therefore, when DecodedStream starts, it try to retrieve audio data from the audio queue, but it doesn't know that there is already a gap between "the audio DecodedStream is going to play" and "the audio that AudioSink had played". Differential Revision: https://phabricator.services.mozilla.com/D115749

Bug 1712621 - Update cubeb to 5ebe69c. r=cubeb-reviewers,padenot Differential Revision: https://phabricator.services.mozilla.com/D115837

Bug 1712031 - Adjust the type of fallbackIterations to match that of IterationCount(). r=padenot Differential Revision: https://phabricator.services.mozilla.com/D115564

Bug 1709899 - batch all data together when appending data to 'AudioDecoderInputTrack' to further ensure the free spaces in SPSC queue. r=padenot. Differential Revision: https://phabricator.services.mozilla.com/D114559

Bug 1710095 - Update audioipc to 7537bfad for memmap -> memmap2 transition. r=chunmin,cubeb-reviewers,padenot Differential Revision: https://phabricator.services.mozilla.com/D114665

Bug 1709684 - disable test on OSX. r=padenot I acutally discovered that during testing the bug1709130, but I forgot to update that in D114150. We would get an decoding error during testing, which is a different issue from what bug1709130 encountered. That error only happens on testing, directly playing that video on Firefox works. So disabling the test on OSX for now, we can dig into that later if that really causes a problem. Differential Revision: https://phabricator.services.mozilla.com/D114400

Bug 1709130 - part1 : extend the SPSC queue's capacity for AudioDecoderInputTrack. r=padenot In the initial implementation, we only used 20 capacity for SPSC queue and added some mechanisms (batched data to save the free space of SPSC queue/clean SPSC queue in each graph iteration) to prevent it from being full. But that seems not working in some extreme cases where decoding is too fast and the graph thread can't catch up the speef of processing the input. So extend the queue size to 40 that should allow SPSC to work without being full again. Differential Revision: https://phabricator.services.mozilla.com/D114150

Bug 1708179 Test PannerNode.setPosition() throws with parameter out of range of float r=padenot Depends on D113943 Differential Revision: https://phabricator.services.mozilla.com/D113944

Bug 1708179 throw when PannerNode param value a not in the range of float r=padenot This is similar to what WebIDL bindings would do for us when the AudioParam setters are called. setPosition() "is equivalent to setting positionX.value, positionY.value, and positionZ.value attribute directly with the x, y and z parameters, respectively." https://webaudio.github.io/web-audio-api/#dom-pannernode-setposition Differential Revision: https://phabricator.services.mozilla.com/D113943

Bug 1708179 switch from PannerNode.setPosition() parameters close to DBL_MAX to FLT_MAX r=padenot The large numbers were to test https://hg.mozilla.org/integration/mozilla-inbound/rev/084b01d1b6b3 but were ignored after https://hg.mozilla.org/integration/mozilla-inbound/rev/8799895ca858 Depends on D113941 Differential Revision: https://phabricator.services.mozilla.com/D113942

Bug 1708179 Test AudioParam value setter throws during setValueCurveAtTime() interval r=padenot Depends on D113940 Differential Revision: https://phabricator.services.mozilla.com/D113941