Since navigating away from a page with an active AudioContext will close
it internally, this patch fixes a similar issue to bug 1180535 for Web
Audio too.
Because these nodes are only registered on PannerNodes if they have a
stream, they need only be unregistered once after the stream is destroyed.
Previously unregistration happened on finish, unlink, and destructor.
--HG--
extra : rebase_source : 7dab08fe4b5648e2a04495d6f570727684247f51
This conversion was done with the script:
find . -name '*.cpp' -o -name '*.h' -o -name '*.mm' -o -name '*.idl' | \
egrep -v 'cairo-win32-refptr.h|RefPtr.h|TestRefPtr.cpp' | \
xargs sed -i -e 's/mozilla::TemporaryRef</already_AddRefed</g' \
-e 's/TemporaryRef</already_AddRefed</g'
Manual fixups were performed in the following instances:
- We handled mfbt/RefPtr.h manually so as to not convert TemporaryRef itself
into already_AddRefed.
- The following files had explicit Move() calls added to make up for the lack
of a copy constructor on already_AddRefed:
dom/base/ImageEncoder.cpp
dom/media/MediaTaskQueue.{h,cpp}
dom/media/webaudio/PannerNode.cpp
- A redundant overload for MediaTaskQueue::Dispatch was deleted.
- A few manual fixups were required in mfbt/tests/TestRefPtr.cpp.
- Comments, using declarations, and forward declarations relating to
TemporaryRef in dom/canvas/ and gfx/layers/ were changed to refer to
already_AddRefed.
It would be nice to remove the argument in a separate patch, but we can't
perform MediaResource reads on the main thread, so the SilentReadAt stuff
needs to happen at the same time as the off-main-thread stuff.
It would be nice to remove the argument in a separate patch, but we can't
perform MediaResource reads on the main thread, so the SilentReadAt stuff
needs to happen at the same time as the off-main-thread stuff.
Backed out changeset 1dcfd15bb387 (bug 974089)
Backed out changeset 8c6a7d5efe3a (bug 974089)
Backed out changeset e511c79f2211 (bug 974089)
CLOSED TREE
When the underlying audio stream platform is slow to start, it can take a little
while for the AudioContext to switch from its initial "suspended" state to
"running".
Authors can call AudioContext.close() right after (new AudioContext()),
that was resulting in a bogus "closed" -> "running" transition, that is now
handled properly by bailing out: the context will simply not report a switch to
"running".
Having this implicit conversion means that we can silently do extra
refcounting when it's completely unnecessary. It's also an obstacle to
making RefPtr more nsRefPtr-like, so let's get rid of it.
- Relevant spec text:
- http://webaudio.github.io/web-audio-api/#widl-AudioContext-suspend-Promise
- http://webaudio.github.io/web-audio-api/#widl-AudioContext-resume-Promise
- http://webaudio.github.io/web-audio-api/#widl-AudioContext-close-Promise
- http://webaudio.github.io/web-audio-api/#widl-AudioContext-state
- http://webaudio.github.io/web-audio-api/#widl-AudioContext-onstatechange
- In a couple words, the behavior we want:
- Closed context cannot have new nodes created, but can do decodeAudioData,
and create buffers, and such.
- OfflineAudioContexts don't support those methods, transitions happen at
startRendering and at the end of processing. onstatechange is used to make
this observable.
- (regular) AudioContexts support those methods. The promises and
onstatechange should be resolved/called when the operation has actually
completed on the rendering thread. Once a context has been closed, it
cannot transition back to "running". An AudioContext switches to "running"
when the audio callback start running, this allow authors to know how long
the audio stack takes to start running.
- MediaStreams that feed in/go out of a suspended graph should respectively
not buffer at the graph input, and output silence
- suspended context should not be doing much on the CPU, and we should try
to pause audio streams if we can (this behaviour is the main reason we need
this in the first place, for saving battery on mobile, and CPU on all
platforms)
- Now, the implementation:
- AudioNodeStreams are now tagged with a context id, to be able to operate
on all the streams of a given AudioContext on the Graph thread without
having to go and lock everytime to touch the AudioContext. This happens in
the AudioNodeStream ctor. IDs are of course constant for the lifetime of the
node.
- When an AudioContext goes into suspended mode, streams for this
AudioContext are moved out of the mStreams array to a second array,
mSuspendedStreams. Streams in mSuspendedStream are not ordered, and are not
processed.
- The MSG will automatically switch to a SystemClockDriver when it finds
that there are no more AudioNodeStream/Stream with an audio track. This is
how pausing the audio subsystem and saving battery works. Subsequently, when
the MSG finds that there are only streams in mSuspendedStreams, it will go
to sleep (block on a monitor), so we save CPU, but it does not shut itself
down. This is mostly not a new behaviour (this is what the MSG does since
the refactoring), but is important to note.
- Promises are gripped (addref-ed) on the main thread, and then shepherd
down other threads and to the GraphDriver, if needed (sometimes we can
resolve them right away). They move between threads as void* to prevent
calling methods on them, as they are not thread safe. Then, the driver
executes the operation, and when it's done (initializing and closing audio
streams can take some time), we send the promise back to the main thread,
and resolve it, casting back to Promise* after asserting we're back on the
main thread. This way, we can send them back on the main thread once an
operation has complete (suspending an audio stream, starting it again on
resume(), etc.), without having to do bookkeeping between suspend calls and
their result. Promises are not thread safe, so we can't move them around
AddRef-ed.
- The stream destruction logic now takes into account that a stream can be
destroyed while not being in mStreams.
- A graph can now switch GraphDriver twice or more per iteration, for
example if an author goes suspend()/resume()/suspend() in the same script.
- Some operation have to be done on suspended stream, so we now use double
for-loop around mSuspendedStreams and mStreams in some places in
MediaStreamGraph.cpp.
- A tricky part was making sure everything worked at AudioContext
boundaries. TrackUnionStream that have one of their input stream suspended
append null ticks instead.
- The graph ordering algorithm had to be altered to not include suspended
streams.
- There are some edge cases (adding a stream on a suspended graph, calling
suspend/resume when a graph has just been close()d).
- Relevant spec text:
- http://webaudio.github.io/web-audio-api/#widl-AudioContext-suspend-Promise
- http://webaudio.github.io/web-audio-api/#widl-AudioContext-resume-Promise
- http://webaudio.github.io/web-audio-api/#widl-AudioContext-close-Promise
- http://webaudio.github.io/web-audio-api/#widl-AudioContext-state
- http://webaudio.github.io/web-audio-api/#widl-AudioContext-onstatechange
- In a couple words, the behavior we want:
- Closed context cannot have new nodes created, but can do decodeAudioData,
and create buffers, and such.
- OfflineAudioContexts don't support those methods, transitions happen at
startRendering and at the end of processing. onstatechange is used to make
this observable.
- (regular) AudioContexts support those methods. The promises and
onstatechange should be resolved/called when the operation has actually
completed on the rendering thread. Once a context has been closed, it
cannot transition back to "running". An AudioContext switches to "running"
when the audio callback start running, this allow authors to know how long
the audio stack takes to start running.
- MediaStreams that feed in/go out of a suspended graph should respectively
not buffer at the graph input, and output silence
- suspended context should not be doing much on the CPU, and we should try
to pause audio streams if we can (this behaviour is the main reason we need
this in the first place, for saving battery on mobile, and CPU on all
platforms)
- Now, the implementation:
- AudioNodeStreams are now tagged with a context id, to be able to operate
on all the streams of a given AudioContext on the Graph thread without
having to go and lock everytime to touch the AudioContext. This happens in
the AudioNodeStream ctor. IDs are of course constant for the lifetime of the
node.
- When an AudioContext goes into suspended mode, streams for this
AudioContext are moved out of the mStreams array to a second array,
mSuspendedStreams. Streams in mSuspendedStream are not ordered, and are not
processed.
- The MSG will automatically switch to a SystemClockDriver when it finds
that there are no more AudioNodeStream/Stream with an audio track. This is
how pausing the audio subsystem and saving battery works. Subsequently, when
the MSG finds that there are only streams in mSuspendedStreams, it will go
to sleep (block on a monitor), so we save CPU, but it does not shut itself
down. This is mostly not a new behaviour (this is what the MSG does since
the refactoring), but is important to note.
- Promises are gripped (addref-ed) on the main thread, and then shepherd
down other threads and to the GraphDriver, if needed (sometimes we can
resolve them right away). They move between threads as void* to prevent
calling methods on them, as they are not thread safe. Then, the driver
executes the operation, and when it's done (initializing and closing audio
streams can take some time), we send the promise back to the main thread,
and resolve it, casting back to Promise* after asserting we're back on the
main thread. This way, we can send them back on the main thread once an
operation has complete (suspending an audio stream, starting it again on
resume(), etc.), without having to do bookkeeping between suspend calls and
their result. Promises are not thread safe, so we can't move them around
AddRef-ed.
- The stream destruction logic now takes into account that a stream can be
destroyed while not being in mStreams.
- A graph can now switch GraphDriver twice or more per iteration, for
example if an author goes suspend()/resume()/suspend() in the same script.
- Some operation have to be done on suspended stream, so we now use double
for-loop around mSuspendedStreams and mStreams in some places in
MediaStreamGraph.cpp.
- A tricky part was making sure everything worked at AudioContext
boundaries. TrackUnionStream that have one of their input stream suspended
append null ticks instead.
- The graph ordering algorithm had to be altered to not include suspended
streams.
- There are some edge cases (adding a stream on a suspended graph, calling
suspend/resume when a graph has just been close()d).
The distinction between moz_malloc/moz_free and malloc/free is not
interesting. We are inconsistent in our use of one or the other, and
I wouldn't be surprised if we are mixing them anyways.
- Relevant spec text:
- http://webaudio.github.io/web-audio-api/#widl-AudioContext-suspend-Promise
- http://webaudio.github.io/web-audio-api/#widl-AudioContext-resume-Promise
- http://webaudio.github.io/web-audio-api/#widl-AudioContext-close-Promise
- http://webaudio.github.io/web-audio-api/#widl-AudioContext-state
- http://webaudio.github.io/web-audio-api/#widl-AudioContext-onstatechange
- In a couple words, the behavior we want:
- Closed context cannot have new nodes created, but can do decodeAudioData,
and create buffers, and such.
- OfflineAudioContexts don't support those methods, transitions happen at
startRendering and at the end of processing. onstatechange is used to make
this observable.
- (regular) AudioContexts support those methods. The promises and
onstatechange should be resolved/called when the operation has actually
completed on the rendering thread. Once a context has been closed, it
cannot transition back to "running". An AudioContext switches to "running"
when the audio callback start running, this allow authors to know how long
the audio stack takes to start running.
- MediaStreams that feed in/go out of a suspended graph should respectively
not buffer at the graph input, and output silence
- suspended context should not be doing much on the CPU, and we should try
to pause audio streams if we can (this behaviour is the main reason we need
this in the first place, for saving battery on mobile, and CPU on all
platforms)
- Now, the implementation:
- AudioNodeStreams are now tagged with a context id, to be able to operate
on all the streams of a given AudioContext on the Graph thread without
having to go and lock everytime to touch the AudioContext. This happens in
the AudioNodeStream ctor. IDs are of course constant for the lifetime of the
node.
- When an AudioContext goes into suspended mode, streams for this
AudioContext are moved out of the mStreams array to a second array,
mSuspendedStreams. Streams in mSuspendedStream are not ordered, and are not
processed.
- The MSG will automatically switch to a SystemClockDriver when it finds
that there are no more AudioNodeStream/Stream with an audio track. This is
how pausing the audio subsystem and saving battery works. Subsequently, when
the MSG finds that there are only streams in mSuspendedStreams, it will go
to sleep (block on a monitor), so we save CPU, but it does not shut itself
down. This is mostly not a new behaviour (this is what the MSG does since
the refactoring), but is important to note.
- Promises are gripped (addref-ed) on the main thread, and then shepherd
down other threads and to the GraphDriver, if needed (sometimes we can
resolve them right away). They move between threads as void* to prevent
calling methods on them, as they are not thread safe. Then, the driver
executes the operation, and when it's done (initializing and closing audio
streams can take some time), we send the promise back to the main thread,
and resolve it, casting back to Promise* after asserting we're back on the
main thread. This way, we can send them back on the main thread once an
operation has complete (suspending an audio stream, starting it again on
resume(), etc.), without having to do bookkeeping between suspend calls and
their result. Promises are not thread safe, so we can't move them around
AddRef-ed.
- The stream destruction logic now takes into account that a stream can be
destroyed while not being in mStreams.
- A graph can now switch GraphDriver twice or more per iteration, for
example if an author goes suspend()/resume()/suspend() in the same script.
- Some operation have to be done on suspended stream, so we now use double
for-loop around mSuspendedStreams and mStreams in some places in
MediaStreamGraph.cpp.
- A tricky part was making sure everything worked at AudioContext
boundaries. TrackUnionStream that have one of their input stream suspended
append null ticks instead.
- The graph ordering algorithm had to be altered to not include suspended
streams.
- There are some edge cases (adding a stream on a suspended graph, calling
suspend/resume when a graph has just been close()d).
This is leftover from bug 815643. In that bug,
|destinationChannelL->length()| and |destinationChannelR->length()| both
end up being |destinationBus->mDuration| and I forgot to eliminate the
common subexpression.
This patch was generated by a script. Here's the source of the script for
future reference:
find . \( -iname "*.cpp" -o -iname "*.h" \) | \
xargs -n 1 sed -i "s/nsRefPtr<nsIRunnable>/nsCOMPtr<nsIRunnable>/g"
The ogg reader makes two adjustments to the seek time - the first is to clamp it
between start and end time, which MDSM already does. The second is to subtract
SEEK_OPUS_PREROLL from the target. If we wanted to, we could return this as the
resolve value in the seek promise and handle the update in the MDSM. But I think
DropVideoUpToSeekTarget should actually handle this just fine.
On ARM, casting a negative float to an unsigned integer yields zero, whereas on
x86, it wraps. When the OscillatorNode had a negative frequency, the code would
just interpolate between the first and second value of the wave table, instead
of interpolating backward.
Ehsan Akhgari
Paul Adenot
Karl Tomlinson
Nathan Froyd
Bobby Holley
JW Wang
Matthew Gregan
Ryan VanderMeulen
Yanis Sellami
Gabor Krizsanits
Andreas Pehrson
Baptiste Emmanuel
Birunthan Mohanathas
Jean-Yves Avenard
Eric Rahm
Andrea Marchesini
Geoff Brown
Carsten "Tomcat" Book
Boris Zbarsky
Andrew McCreight
Neil Deakin
Ralph Giles
Boris Egorov
Dragana Damjanovic
Alastor Wu
Mike Hommey
Wes Kocher
Daniel Holbert
Jonathan Griffin
Phil Ringnalda
Sotaro Ikeda
Anthony Jones
Ankit Goyal
Masatoshi Kimura
Brian Smith
Chris Peterson