gecko-dev/dom/media/VideoStreamTrack.cpp

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

90 строки
2.8 KiB
C++
Исходник Обычный вид История

/* -*- Mode: C++; tab-width: 2; indent-tabs-mode: nil; c-basic-offset: 2 -*-*/
/* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this file,
* You can obtain one at http://mozilla.org/MPL/2.0/. */
#include "VideoStreamTrack.h"
Bug 1454998 - Rename streams to tracks. r=padenot,karlt,smaug This renames the following (in alphabetical order, non-exhaustive): AudioCaptureStream -> AudioCaptureTrack AudioNodeStream -> AudioNodeTrack AudioNodeExternalInputStream -> AudioNodeExternalInputTrack DirectMediaStreamTrackListener -> DirectMediaTrackListener MediaStream -> MediaTrack - Note that there's also dom::MediaTrack. Namespaces differentiate them. MediaStreamGraph -> MediaTrackGraph MediaStreamTrackListener -> MediaTrackListener MSG -> MTG (in comments) ProcessedMediaStream -> ProcessedMediaTrack SharedDummyStream -> SharedDummyTrack SourceMediaStream -> SourceMediaTrack StreamTime -> TrackTime TrackUnionStream -> ForwardedInputTrack - Because this no longer takes a union of anything, but only a single track as input. Other minor classes, members and comments have been updated to reflect these name changes. Differential Revision: https://phabricator.services.mozilla.com/D46146 --HG-- rename : dom/media/AudioCaptureStream.cpp => dom/media/AudioCaptureTrack.cpp rename : dom/media/AudioCaptureStream.h => dom/media/AudioCaptureTrack.h rename : dom/media/TrackUnionStream.cpp => dom/media/ForwardedInputTrack.cpp rename : dom/media/TrackUnionStream.h => dom/media/ForwardedInputTrack.h rename : dom/media/MediaStreamGraph.cpp => dom/media/MediaTrackGraph.cpp rename : dom/media/MediaStreamGraph.h => dom/media/MediaTrackGraph.h rename : dom/media/MediaStreamGraphImpl.h => dom/media/MediaTrackGraphImpl.h rename : dom/media/MediaStreamListener.cpp => dom/media/MediaTrackListener.cpp rename : dom/media/MediaStreamListener.h => dom/media/MediaTrackListener.h rename : dom/media/webaudio/AudioNodeExternalInputStream.cpp => dom/media/webaudio/AudioNodeExternalInputTrack.cpp rename : dom/media/webaudio/AudioNodeExternalInputStream.h => dom/media/webaudio/AudioNodeExternalInputTrack.h rename : dom/media/webaudio/AudioNodeStream.cpp => dom/media/webaudio/AudioNodeTrack.cpp rename : dom/media/webaudio/AudioNodeStream.h => dom/media/webaudio/AudioNodeTrack.h extra : moz-landing-system : lando
2019-10-02 13:23:02 +03:00
#include "MediaTrackGraph.h"
#include "MediaTrackListener.h"
#include "nsContentUtils.h"
#include "nsGlobalWindowInner.h"
#include "VideoOutput.h"
namespace mozilla {
namespace dom {
VideoStreamTrack::VideoStreamTrack(nsPIDOMWindowInner* aWindow,
Bug 1454998 - Rename streams to tracks. r=padenot,karlt,smaug This renames the following (in alphabetical order, non-exhaustive): AudioCaptureStream -> AudioCaptureTrack AudioNodeStream -> AudioNodeTrack AudioNodeExternalInputStream -> AudioNodeExternalInputTrack DirectMediaStreamTrackListener -> DirectMediaTrackListener MediaStream -> MediaTrack - Note that there's also dom::MediaTrack. Namespaces differentiate them. MediaStreamGraph -> MediaTrackGraph MediaStreamTrackListener -> MediaTrackListener MSG -> MTG (in comments) ProcessedMediaStream -> ProcessedMediaTrack SharedDummyStream -> SharedDummyTrack SourceMediaStream -> SourceMediaTrack StreamTime -> TrackTime TrackUnionStream -> ForwardedInputTrack - Because this no longer takes a union of anything, but only a single track as input. Other minor classes, members and comments have been updated to reflect these name changes. Differential Revision: https://phabricator.services.mozilla.com/D46146 --HG-- rename : dom/media/AudioCaptureStream.cpp => dom/media/AudioCaptureTrack.cpp rename : dom/media/AudioCaptureStream.h => dom/media/AudioCaptureTrack.h rename : dom/media/TrackUnionStream.cpp => dom/media/ForwardedInputTrack.cpp rename : dom/media/TrackUnionStream.h => dom/media/ForwardedInputTrack.h rename : dom/media/MediaStreamGraph.cpp => dom/media/MediaTrackGraph.cpp rename : dom/media/MediaStreamGraph.h => dom/media/MediaTrackGraph.h rename : dom/media/MediaStreamGraphImpl.h => dom/media/MediaTrackGraphImpl.h rename : dom/media/MediaStreamListener.cpp => dom/media/MediaTrackListener.cpp rename : dom/media/MediaStreamListener.h => dom/media/MediaTrackListener.h rename : dom/media/webaudio/AudioNodeExternalInputStream.cpp => dom/media/webaudio/AudioNodeExternalInputTrack.cpp rename : dom/media/webaudio/AudioNodeExternalInputStream.h => dom/media/webaudio/AudioNodeExternalInputTrack.h rename : dom/media/webaudio/AudioNodeStream.cpp => dom/media/webaudio/AudioNodeTrack.cpp rename : dom/media/webaudio/AudioNodeStream.h => dom/media/webaudio/AudioNodeTrack.h extra : moz-landing-system : lando
2019-10-02 13:23:02 +03:00
mozilla::MediaTrack* aInputTrack,
MediaStreamTrackSource* aSource,
MediaStreamTrackState aReadyState,
const MediaTrackConstraints& aConstraints)
Bug 1454998 - Rename streams to tracks. r=padenot,karlt,smaug This renames the following (in alphabetical order, non-exhaustive): AudioCaptureStream -> AudioCaptureTrack AudioNodeStream -> AudioNodeTrack AudioNodeExternalInputStream -> AudioNodeExternalInputTrack DirectMediaStreamTrackListener -> DirectMediaTrackListener MediaStream -> MediaTrack - Note that there's also dom::MediaTrack. Namespaces differentiate them. MediaStreamGraph -> MediaTrackGraph MediaStreamTrackListener -> MediaTrackListener MSG -> MTG (in comments) ProcessedMediaStream -> ProcessedMediaTrack SharedDummyStream -> SharedDummyTrack SourceMediaStream -> SourceMediaTrack StreamTime -> TrackTime TrackUnionStream -> ForwardedInputTrack - Because this no longer takes a union of anything, but only a single track as input. Other minor classes, members and comments have been updated to reflect these name changes. Differential Revision: https://phabricator.services.mozilla.com/D46146 --HG-- rename : dom/media/AudioCaptureStream.cpp => dom/media/AudioCaptureTrack.cpp rename : dom/media/AudioCaptureStream.h => dom/media/AudioCaptureTrack.h rename : dom/media/TrackUnionStream.cpp => dom/media/ForwardedInputTrack.cpp rename : dom/media/TrackUnionStream.h => dom/media/ForwardedInputTrack.h rename : dom/media/MediaStreamGraph.cpp => dom/media/MediaTrackGraph.cpp rename : dom/media/MediaStreamGraph.h => dom/media/MediaTrackGraph.h rename : dom/media/MediaStreamGraphImpl.h => dom/media/MediaTrackGraphImpl.h rename : dom/media/MediaStreamListener.cpp => dom/media/MediaTrackListener.cpp rename : dom/media/MediaStreamListener.h => dom/media/MediaTrackListener.h rename : dom/media/webaudio/AudioNodeExternalInputStream.cpp => dom/media/webaudio/AudioNodeExternalInputTrack.cpp rename : dom/media/webaudio/AudioNodeExternalInputStream.h => dom/media/webaudio/AudioNodeExternalInputTrack.h rename : dom/media/webaudio/AudioNodeStream.cpp => dom/media/webaudio/AudioNodeTrack.cpp rename : dom/media/webaudio/AudioNodeStream.h => dom/media/webaudio/AudioNodeTrack.h extra : moz-landing-system : lando
2019-10-02 13:23:02 +03:00
: MediaStreamTrack(aWindow, aInputTrack, aSource, aReadyState,
aConstraints) {}
void VideoStreamTrack::Destroy() {
mVideoOutputs.Clear();
MediaStreamTrack::Destroy();
}
void VideoStreamTrack::AddVideoOutput(VideoFrameContainer* aSink) {
if (Ended()) {
return;
}
auto output = MakeRefPtr<VideoOutput>(
aSink, nsGlobalWindowInner::Cast(GetParentObject())
->AbstractMainThreadFor(TaskCategory::Other));
AddVideoOutput(output);
}
void VideoStreamTrack::AddVideoOutput(VideoOutput* aOutput) {
if (Ended()) {
return;
}
for (const auto& output : mVideoOutputs) {
if (output == aOutput) {
MOZ_ASSERT_UNREACHABLE("A VideoOutput was already added");
return;
}
}
mVideoOutputs.AppendElement(aOutput);
AddDirectListener(aOutput);
AddListener(aOutput);
}
void VideoStreamTrack::RemoveVideoOutput(VideoFrameContainer* aSink) {
for (const auto& output : nsTArray<RefPtr<VideoOutput>>(mVideoOutputs)) {
if (output->mVideoFrameContainer == aSink) {
mVideoOutputs.RemoveElement(output);
Bug 1506093 - Fix DecodedStream A/V sync. r=padenot DecodedStream sends video to its video tracks by initially buffering a set of images, then appending future ones by adding them one by one. A long time ago we refactored how MediaStreamGraph sends images to the screen, i.e., to an ImageContainer. It used to send all future frames to ImageContainer::SetCurrentFrames each time it sent something. After the refactor we just forward any new frame from a direct listener to ImageContainer::SetCurrentFrames. So in case DecodedStream has already sent 10 future frames to its track, and sends another, we end up calling ImageContainer::SetCurrentFrames(frame11). However, this is not how ImageContainer works. The refactor was wrong. Even though the timestamp for frame11 is after a previously buffered frame, it will be ignored. SetCurrentFrames wipes any previously set frames. Hence the word "Current" in its name. This patch largely restores the old behaviour by adding a thin buffering layer between the MSG (in a direct listener) and the ImageContainer. This does not give 100% identical frame sync to VideoSink (how we normally render video), because VideoSink can update the timestamps of already-pushed images by pushing them again. We can't do that here because the SourceMediaStream API only allows appending. It does however get in sync for frames appended after the first frame has been rendered. Differential Revision: https://phabricator.services.mozilla.com/D22897 --HG-- extra : moz-landing-system : lando
2019-03-22 14:41:48 +03:00
RemoveDirectListener(output);
RemoveListener(output);
}
}
}
void VideoStreamTrack::RemoveVideoOutput(VideoOutput* aOutput) {
for (const auto& output : nsTArray<RefPtr<VideoOutput>>(mVideoOutputs)) {
if (output == aOutput) {
mVideoOutputs.RemoveElement(aOutput);
RemoveDirectListener(aOutput);
RemoveListener(aOutput);
}
}
}
void VideoStreamTrack::GetLabel(nsAString& aLabel, CallerType aCallerType) {
if (nsContentUtils::ResistFingerprinting(aCallerType)) {
aLabel.AssignLiteral("Internal Camera");
return;
}
MediaStreamTrack::GetLabel(aLabel, aCallerType);
}
already_AddRefed<MediaStreamTrack> VideoStreamTrack::CloneInternal() {
Bug 1454998 - Rename streams to tracks. r=padenot,karlt,smaug This renames the following (in alphabetical order, non-exhaustive): AudioCaptureStream -> AudioCaptureTrack AudioNodeStream -> AudioNodeTrack AudioNodeExternalInputStream -> AudioNodeExternalInputTrack DirectMediaStreamTrackListener -> DirectMediaTrackListener MediaStream -> MediaTrack - Note that there's also dom::MediaTrack. Namespaces differentiate them. MediaStreamGraph -> MediaTrackGraph MediaStreamTrackListener -> MediaTrackListener MSG -> MTG (in comments) ProcessedMediaStream -> ProcessedMediaTrack SharedDummyStream -> SharedDummyTrack SourceMediaStream -> SourceMediaTrack StreamTime -> TrackTime TrackUnionStream -> ForwardedInputTrack - Because this no longer takes a union of anything, but only a single track as input. Other minor classes, members and comments have been updated to reflect these name changes. Differential Revision: https://phabricator.services.mozilla.com/D46146 --HG-- rename : dom/media/AudioCaptureStream.cpp => dom/media/AudioCaptureTrack.cpp rename : dom/media/AudioCaptureStream.h => dom/media/AudioCaptureTrack.h rename : dom/media/TrackUnionStream.cpp => dom/media/ForwardedInputTrack.cpp rename : dom/media/TrackUnionStream.h => dom/media/ForwardedInputTrack.h rename : dom/media/MediaStreamGraph.cpp => dom/media/MediaTrackGraph.cpp rename : dom/media/MediaStreamGraph.h => dom/media/MediaTrackGraph.h rename : dom/media/MediaStreamGraphImpl.h => dom/media/MediaTrackGraphImpl.h rename : dom/media/MediaStreamListener.cpp => dom/media/MediaTrackListener.cpp rename : dom/media/MediaStreamListener.h => dom/media/MediaTrackListener.h rename : dom/media/webaudio/AudioNodeExternalInputStream.cpp => dom/media/webaudio/AudioNodeExternalInputTrack.cpp rename : dom/media/webaudio/AudioNodeExternalInputStream.h => dom/media/webaudio/AudioNodeExternalInputTrack.h rename : dom/media/webaudio/AudioNodeStream.cpp => dom/media/webaudio/AudioNodeTrack.cpp rename : dom/media/webaudio/AudioNodeStream.h => dom/media/webaudio/AudioNodeTrack.h extra : moz-landing-system : lando
2019-10-02 13:23:02 +03:00
return do_AddRef(new VideoStreamTrack(mWindow, mInputTrack, mSource,
Bug 1454998 - Remove the notion of streams from MediaStreamGraph. r=padenot,karlt This change mainly removes the `mTracks` member from MediaStream and moves all associated members up a level, so that a MediaStream in practice represents a single track. Classes will be renamed in a future patch to reflect this. Other changes include: The new `mEnded` member of MediaStream changes meaning to only become true when all data in the stream has been processed. It stems from StreamTracks::Track::mEnded which used to become true as soon as the last bit of data had been added to a track, and there could still be data in the track that would get processed in future iterations. We are moving towards not having any future data in tracks, which is why this change is ok to make -- keeping the old behavior will soon not make sense. TrackUnionStream is changed to no longer take a list of streams as input and forward the union of their tracks to itself. Instead it's limited to having one track as input at a time. The autofinishing functionality that TrackUnionStream had before has been transformed into an autoending functionality to allow it to defer ending until its been told that it's ok to end through the control API. This lets a single TrackUnionStream span the lifetime of multiple inputs, which will be useful for making DecodedStream spec compliant with HTMLMediaElement::CaptureStream(), and for implementing the currently discussed MediaRecorder::ReplaceTrack(), to name a few potential use cases. AudioNodeStreams used to only have a track (and thus an AudioSegment) if the EXTERNAL_OUTPUT flag was enabled on them. With all MediaStreams now representing a track, AudioNodeStreams inherently have an AudioSegment as a member. It is however only used with data if the EXTERNAL_OUTPUT flag is enabled. Differential Revision: https://phabricator.services.mozilla.com/D45821 --HG-- extra : moz-landing-system : lando
2019-10-02 13:22:53 +03:00
ReadyState(), mConstraints));
}
} // namespace dom
} // namespace mozilla