Bug 592833 - Update comments for new video decoder thread model. r=roc

content/media/nsBuiltinDecoder.h

@@ -37,24 +37,28 @@
  *
  * ***** END LICENSE BLOCK ***** */
 /*
-Each video element based on nsBuiltinDecoder has at least one thread
-dedicated to decoding video.
+Each video element based on nsBuiltinDecoder has a state machine to manage
+its play state and keep the current frame up to date. All state machines
+share time in a single shared thread. Each decoder also has one thread
+dedicated to decoding audio and video data. This thread is shutdown when
+playback is paused. Each decoder also has a thread to push decoded audio
+to the hardware. This thread is not created until playback starts, but
+currently is not destroyed when paused, only when playback ends.
 
-This thread (called the state machine thread owns the resources for
-downloading and reading the media file. nsDecoderStateMachine is the
-class that needs to be implemented and it gets run on the state
-machine thread.
+The decoder owns the resources for downloading the media file, and the
+high level state. It holds an owning reference to the state machine
+(a subclass of nsDecoderStateMachine; nsBuiltinDecoderStateMachine) that
+owns all the resources related to decoding data, and manages the low level
+decoding operations and A/V sync. 
 
-The state machine thread has one event that is dispatched to it (the
-implementation of nsDecoderStateMachine) and that event runs for the
-lifetime of the playback of the resource. State shared between threads
-is synchronised with the main thread via a monitor held by the
-nsBuiltinDecoder object.
-
-The state machine thread event consist of a Run method which is an
-infinite loop that performs the decoding operation and checks the
-state that the state machine is in and processes operations on that
-state.
+Each state machine runs on the shared state machine thread. Every time some
+action is required for a state machine, it is scheduled to run on the shared
+the state machine thread. The state machine runs one "cycle" on the state
+machine thread, and then returns. If necessary, it will schedule itself to
+run again in future. While running this cycle, it must not block the
+thread, as other state machines' events may need to run. State shared
+between a state machine's threads is synchronised via the monitor owned
+by its nsBuiltinDecoder object.
 
 The Main thread controls the decode state machine by setting the value
 of a mPlayState variable and notifying on the monitor based on the
@@ -85,25 +89,37 @@ SHUTDOWN
 State transition occurs when the Media Element calls the Play, Seek,
 etc methods on the nsBuiltinDecoder object. When the transition
 occurs nsBuiltinDecoder then calls the methods on the decoder state
-machine object to cause it to behave appropriate to the play state.
+machine object to cause it to behave as required by the play state.
+State transitions will likely schedule the state machine to run to
+affect the change.
 
 An implementation of the nsDecoderStateMachine class is the event
-that gets dispatched to the state machine thread. It has the following states:
+that gets dispatched to the state machine thread. Each time the event is run,
+the state machine must cycle the state machine once, and then return.
+
+The state machine has the following states:
 
 DECODING_METADATA
   The media headers are being loaded, and things like framerate, etc are
   being determined, and the first frame of audio/video data is being decoded.
 DECODING
-  The decode and audio threads are started and video frames displayed at
-  the required time. 
+  The decode has started. If the PlayState is PLAYING, the decode thread
+  should be alive and decoding video and audio frame, the audio thread
+  should be playing audio, and the state machine should run periodically
+  to update the video frames being displayed.
 SEEKING
-  A seek operation is in progress.
+  A seek operation is in progress. The decode thread should be seeking.
 BUFFERING
-  Decoding is paused while data is buffered for smooth playback.
+  Decoding is paused while data is buffered for smooth playback. If playback
+  is paused (PlayState transitions to PAUSED) we'll destory the decode thread.
 COMPLETED
-  The resource has completed decoding, but not finished playback. 
+  The resource has completed decoding, but possibly not finished playback.
+  The decode thread will be destroyed. Once playback finished, the audio
+  thread will also be destroyed.
 SHUTDOWN
-  The decoder object is about to be destroyed.
+  The decoder object and its state machine are about to be destroyed.
+  Once the last state machine has been destroyed, the shared state machine
+  thread will also be destroyed. It will be recreated later if needed.
 
 The following result in state transitions.
 
@@ -157,41 +173,41 @@ player SHUTDOWN  decoder SHUTDOWN
 The general sequence of events is:
 
 1) The video element calls Load on nsMediaDecoder. This creates the
-   state machine thread and starts the channel for downloading the
-   file. It instantiates and starts the nsDecoderStateMachine. The
+   state machine and starts the channel for downloading the
+   file. It instantiates and schedules the nsDecoderStateMachine. The
    high level LOADING state is entered, which results in the decode
-   state machine to start decoding metadata. These are the headers
-   that give the video size, framerate, etc.  It returns immediately
-   to the calling video element.
+   thread being created and starting to decode metadata. These are
+   the headers that give the video size, framerate, etc. Load() returns
+   immediately to the calling video element.
 
-2) When the metadata has been loaded by the decode thread it will call
-   a method on the video element object to inform it that this step is
-   done, so it can do the things required by the video specification
-   at this stage. The decoder then continues to decode the first frame
+2) When the metadata has been loaded by the decode thread, the state machine
+   will call a method on the video element object to inform it that this
+   step is done, so it can do the things required by the video specification
+   at this stage. The decode thread then continues to decode the first frame
    of data.
 
-3) When the first frame of data has been successfully decoded it calls
-   a method on the video element object to inform it that this step
-   has been done, once again so it can do the required things by the
-   video specification at this stage.
+3) When the first frame of data has been successfully decoded the state
+   machine calls a method on the video element object to inform it that
+   this step has been done, once again so it can do the required things
+   by the video specification at this stage.
 
    This results in the high level state changing to PLAYING or PAUSED
    depending on any user action that may have occurred.
 
-   The decode thread plays audio and video, if the correct frame time
-   comes around and the decoder play state is PLAYING.
-   
-a/v synchronisation is handled by the nsDecoderStateMachine implementation.
+   While the play state is PLAYING, the decode thread will decode
+   data, and the audio thread will push audio data to the hardware to
+   be played. The state machine will run periodically on the shared
+   state machine thread to ensure video frames are played at the 
+   correct time; i.e. the state machine manages A/V sync.
 
-The Shutdown method on nsBuiltinDecoder can spin the event loop as it
-waits for threads to complete. Spinning the event loop is a bad thing
-to happen during certain times like destruction of the media
-element. To work around this the Shutdown method does nothing but
-queue an event to the main thread to perform the actual Shutdown. This
-way the shutdown can occur at a safe time.
+The Shutdown method on nsBuiltinDecoder closes the download channel, and
+signals to the state machine that it should shutdown. The state machine
+shuts down asynchronously, and will release the owning reference to the
+state machine once its threads are shutdown.
+
+The owning object of a nsBuiltinDecoder object *MUST* call Shutdown when
+destroying the nsBuiltinDecoder object.
 
-This means the owning object of a nsBuiltinDecoder object *MUST* call
-Shutdown when destroying the nsBuiltinDecoder object.  
 */
 #if !defined(nsBuiltinDecoder_h_)
 #define nsBuiltinDecoder_h_

content/media/nsBuiltinDecoderStateMachine.h

@@ -37,11 +37,10 @@
  *
  * ***** END LICENSE BLOCK ***** */
 /*
-Each video element for a media file has two additional threads beyond
-those needed by nsBuiltinDecoder.
+Each video element for a media file has two threads:
 
   1) The Audio thread writes the decoded audio data to the audio
-     hardware.  This is done in a seperate thread to ensure that the
+     hardware. This is done in a separate thread to ensure that the
      audio hardware gets a constant stream of data without
      interruption due to decoding or display. At some point
      libsydneyaudio will be refactored to have a callback interface
@@ -49,31 +48,27 @@ those needed by nsBuiltinDecoder.
      needed.
 
   2) The decode thread. This thread reads from the media stream and
-     decodes the Theora and Vorbis data. It places the decoded data in
-     a queue for the other threads to pull from.
+     decodes the Theora and Vorbis data. It places the decoded data into
+     queues for the other threads to pull from.
 
-All file reads and seeks must occur on either the state machine thread
-or the decode thread. Synchronisation is done via a monitor owned by
-nsBuiltinDecoder.
+All file reads, seeks, and all decoding must occur on the decode thread.
+Synchronisation of state between the thread is done via a monitor owned
+by nsBuiltinDecoder.
 
-The decode thread and the audio thread are created and destroyed in
-the state machine thread. When playback needs to occur they are
-created and events dispatched to them to start them. These events exit
-when decoding is completed or no longer required (during seeking or
-shutdown).
-    
-The decode thread has its own monitor to ensure that its internal
-state is independent of the other threads, and to ensure that it's not
-hogging the nsBuiltinDecoder monitor while decoding.
+The lifetime of the decode and audio threads is controlled by the state
+machine when it runs on the shared state machine thread. When playback
+needs to occur they are created and events dispatched to them to run
+them. These events exit when decoding/audio playback is completed or
+no longer required.
 
-a/v synchronisation is handled by the state machine thread. It
-examines the audio playback time and compares this to the next frame
-in the queue of frames. If it is time to play the video frame it is
-then displayed.
+A/V synchronisation is handled by the state machine. It examines the audio
+playback time and compares this to the next frame in the queue of video
+frames. If it is time to play the video frame it is then displayed, otherwise
+it schedules the state machine to run again at the time of the next frame.
 
 Frame skipping is done in the following ways:
 
-  1) The state machine thread will skip all frames in the video queue whose
+  1) The state machine will skip all frames in the video queue whose
      display time is less than the current audio time. This ensures
      the correct frame for the current time is always displayed.
 
@@ -86,28 +81,30 @@ Frame skipping is done in the following ways:
           will be decoding video data that won't be displayed due
           to the decode thread dropping the frame immediately.
 
-YCbCr conversion is done on the decode thread when it is time to display
-the video frame. This means frames that are skipped will not have the
-YCbCr conversion done, improving playback.
+When hardware accelerated graphics is not available, YCbCr conversion
+is done on the decode thread when video frames are decoded.
 
 The decode thread pushes decoded audio and videos frames into two
 separate queues - one for audio and one for video. These are kept
 separate to make it easy to constantly feed audio data to the sound
 hardware while allowing frame skipping of video data. These queues are
-threadsafe, and neither the decode, audio, or state machine thread should
+threadsafe, and neither the decode, audio, or state machine should
 be able to monopolize them, and cause starvation of the other threads.
 
 Both queues are bounded by a maximum size. When this size is reached
 the decode thread will no longer decode video or audio depending on the
-queue that has reached the threshold.
+queue that has reached the threshold. If both queues are full, the decode
+thread will wait on the decoder monitor.
+
+When the decode queues are full (they've reaced their maximum size) and
+the decoder is not in PLAYING play state, the state machine may opt
+to shut down the decode thread in order to conserve resources.
 
 During playback the audio thread will be idle (via a Wait() on the
-monitor) if the audio queue is empty. Otherwise it constantly pops an
-item off the queue and plays it with a blocking write to the audio
+monitor) if the audio queue is empty. Otherwise it constantly pops
+sound data off the queue and plays it with a blocking write to the audio
 hardware (via nsAudioStream and libsydneyaudio).
 
-The decode thread idles if the video queue is empty or if it is
-not yet time to display the next frame.
 */
 #if !defined(nsBuiltinDecoderStateMachine_h__)
 #define nsBuiltinDecoderStateMachine_h__
@@ -122,19 +119,14 @@ not yet time to display the next frame.
 #include "nsITimer.h"
 
 /*
-  The playback state machine class. This manages the decoding in the
-  nsBuiltinDecoderReader on the decode thread, seeking and in-sync-playback on the
+  The state machine class. This manages the decoding and seeking in the
+  nsBuiltinDecoderReader on the decode thread, and A/V sync on the shared
   state machine thread, and controls the audio "push" thread.
 
-  All internal state is synchronised via the decoder monitor. NotifyAll
-  on the monitor is called when the state of the state machine is changed
-  by the main thread. The following changes to state cause a notify:
-
-    mState and data related to that state changed (mSeekTime, etc)
-    Metadata Loaded
-    First Frame Loaded
-    Frame decoded
-    data pushed or popped from the video and audio queues
+  All internal state is synchronised via the decoder monitor. State changes
+  are either propagated by NotifyAll on the monitor (typically when state
+  changes need to be propagated to non-state machine threads) or by scheduling
+  the state machine to run another cycle on the shared state machine thread.
 
   See nsBuiltinDecoder.h for more details.
 */
@@ -229,9 +221,9 @@ public:
   nsRefPtr<nsBuiltinDecoder> mDecoder;
 
   // The decoder monitor must be obtained before modifying this state.
-  // NotifyAll on the monitor must be called when the state is changed by
-  // the main thread so the decoder thread can wake up.
-  // Accessed on state machine, audio, main, and AV thread. 
+  // NotifyAll on the monitor must be called when the state is changed so
+  // that interested threads can wake up and alter behaviour if appropriate
+  // Accessed on state machine, audio, main, and AV thread.
   State mState;
 
   nsresult GetBuffered(nsTimeRanges* aBuffered);
@@ -305,7 +297,8 @@ protected:
   // wait for a specified time, and that the myriad of Notify()s we do on
   // the decoder monitor don't cause the audio thread to be starved. aUsecs
   // values of less than 1 millisecond are rounded up to 1 millisecond
-  // (see bug 651023). The decoder monitor must be held.
+  // (see bug 651023). The decoder monitor must be held. Called only on the
+  // audio thread.
   void Wait(PRInt64 aUsecs);
 
   // Dispatches an asynchronous event to update the media element's ready state.
@@ -330,9 +323,8 @@ protected:
   // machine thread, caller must hold the decoder lock.
   void UpdatePlaybackPositionInternal(PRInt64 aTime);
 
-  // Performs YCbCr to RGB conversion, and pushes the image down the
-  // rendering pipeline. Called on the state machine thread. The decoder
-  // monitor must not be held when calling this.
+  // Pushes the image down the rendering pipeline. Called on the shared state
+  // machine thread. The decoder monitor must *not* be held when calling this.
   void RenderVideoFrame(VideoData* aData, TimeStamp aTarget);
  
   // If we have video, display a video frame if it's time for display has
@@ -413,7 +405,7 @@ protected:
   // which has been pushed to the audio hardware for playback. Note that after
   // calling this, the audio hardware may play some of the audio pushed to
   // hardware, so this can only be used as a upper bound. The decoder monitor
-  // must be held when calling this. Called on the decoder thread.
+  // must be held when calling this. Called on the decode thread.
   PRInt64 GetDecodedAudioDuration();
 
   // Load metadata. Called on the decode thread. The decoder monitor
@@ -491,18 +483,18 @@ protected:
 
   // Start time of the media, in microseconds. This is the presentation
   // time of the first sample decoded from the media, and is used to calculate
-  // duration and as a bounds for seeking. Accessed on state machine and
-  // main thread. Access controlled by decoder monitor.
+  // duration and as a bounds for seeking. Accessed on state machine, decode,
+  // and main threads. Access controlled by decoder monitor.
   PRInt64 mStartTime;
 
   // Time of the last page in the media, in microseconds. This is the
   // end time of the last sample in the media. Accessed on state
-  // machine and main thread. Access controlled by decoder monitor.
+  // machine, decode, and main threads. Access controlled by decoder monitor.
   PRInt64 mEndTime;
 
   // Position to seek to in microseconds when the seek state transition occurs.
   // The decoder monitor lock must be obtained before reading or writing
-  // this value. Accessed on main and state machine thread.
+  // this value. Accessed on main and decode thread.
   PRInt64 mSeekTime;
 
   // The audio stream resource. Used on the state machine, and audio threads.