In AudioData, we use `mOriginalTime` to store the original start time before trimming, which should keep sync to `mTime` if data hasn't been trimmed before. Otherwise, if they are not equal, it would cause a problem when handling trimming.
For example.
If audio data's time is X, original time is X-1 and duration is T. When MDSM performs seeking, it would use (X + T) as trimming window's end time. However, in `AudioData::SetTrimWindow()`, original time + duration (X - 1 + T) would be less than the trimming end time, which causes an error.
Differential Revision: https://phabricator.services.mozilla.com/D100825
`AdjustForStartTime()` is only used for decoded data type in order to adjust their start time to zero, such as `VideoData` and `AudioData`.
However, some other types inherited from `MediaData` don't need that function. Eg. `MediaRawData`, so it's not necessary to put it on the `MediaData`, we should move that function to `VideoData` and `AudioData` separately.
Differential Revision: https://phabricator.services.mozilla.com/D100846
This avoids us risking an overflow when we convert encrypted media with
subsamples to AnnexB (since that conversion can grow the clear sizes of the
sample). See the test in the preceding patch for an example of how and why this
happens.
Differential Revision: https://phabricator.services.mozilla.com/D92300
An ArrayOfRemoteAudioData pack all its AudioData objects into a single Shmem.
This Shmem will be-reused by the remote decoder over and over.
When used with webaudio, this reduces the number of memory allocation from 100 to 1 for each remote decoder.
Differential Revision: https://phabricator.services.mozilla.com/D91539
Those two objects can be used to pack multiple array of objects into a minimal amount of shmem.
An ArrayOfRemoteByteBuffer will take at most a single shmem and perform a single memory allocation..
Similarly, an ArrayOfMediaRawData will pack multiple MediaRawData in at most 3 shmems (one for each array of bytes a MediaRawData contains).
They are designed to work in combination with a ShmemPool which will own each of the allocated Shmem and so can be re-used over and over.
Differential Revision: https://phabricator.services.mozilla.com/D91537
Because we store each plane's data pointer we don't need to store a separate
offset, we can just roll that offset into the pointer for that plane.
This helps remove a lot of code where we're not using the offset anyway, and in
the Chromium CDM case we simply roll the offset into our data pointer.
Differential Revision: https://phabricator.services.mozilla.com/D78320
On Android, decoded buffers need to be send back to MediaCodec in order to be
rendered and/or recycled. The current mechanism introduced in bug 1299068 only
works for playback(VideoData/VideoSink) but not WebRTC(VideoFrame/VideoOutput).
Move the callback to SurfaceTextureImage because VideoData and VideoFrame both
own that when using MediaCodec, and move the notification to VideoFrameContainer
for both VideoSink and VideoOutput pass frames there for compositing.
Differential Revision: https://phabricator.services.mozilla.com/D45771
--HG--
extra : moz-landing-system : lando
On Android, decoded buffers need to be send back to MediaCodec in order to be
rendered and/or recycled. The current mechanism introduced in bug 1299068 only
works for playback(VideoData/VideoSink) but not WebRTC(VideoFrame/VideoOutput).
Move the callback to SurfaceTextureImage because VideoData and VideoFrame both
own that when using MediaCodec, and move the notification to VideoFrameContainer
for both VideoSink and VideoOutput pass frames there for compositing.
Differential Revision: https://phabricator.services.mozilla.com/D45771
--HG--
extra : moz-landing-system : lando
When unknown, we rely on the picture height and assume that anything less than 720p is 601 and 709 otherwise. It's not perfect but it's the best we can do.
Differential Revision: https://phabricator.services.mozilla.com/D39275
--HG--
extra : moz-landing-system : lando
When unknown, we rely on the picture height and assume that anything less than 720p is 601 and 709 otherwise. It's not perfect but it's the best we can do.
Differential Revision: https://phabricator.services.mozilla.com/D39275
--HG--
extra : moz-landing-system : lando
When unknown, we rely on the picture height and assume that anything less than 720p is 601 and 709 otherwise. It's not perfect but it's the best we can do.
Differential Revision: https://phabricator.services.mozilla.com/D39275
--HG--
extra : moz-landing-system : lando
We don't want to have a sample with invalid time, duration, end time or end timecode, so add a diagnostic assertion to check. And will handle invalid sample for demuxers in next patch.
Differential Revision: https://phabricator.services.mozilla.com/D38426
--HG--
extra : moz-landing-system : lando
We already have function `GetEndTime()`, so it's good to have a similar function `GetEndTimeCode()` so that we won't have to calculate end time code by ourselves.
Differential Revision: https://phabricator.services.mozilla.com/D38425
--HG--
extra : moz-landing-system : lando
As the purpose of adjusting time is to make the start of the playback to align to zero, so the result should not be negative.
However, we can't always ensure the demuxed start time is correct, the decoded first frame sample time might be different with the time demuxer provides.
Even if we try to update the start time when we get the first decoded sample, there is still a problem because a decoder might return samples with different order, which means the first frame the decoder returns is probably not a real first frame.
Therefore, using warning to help us knows how many this situaion would be.
Differential Revision: https://phabricator.services.mozilla.com/D28432
--HG--
extra : moz-landing-system : lando
As the purpose of adjusting time is to make the start of the playback to align to zero, so the result should not be negative.
However, we can't always ensure the demuxed start time is correct, the decoded first frame sample time might be different with the time demuxer provides.
Even if we try to update the start time when we get the first decoded sample, there is still a problem because a decoder might return samples with different order, which means the first frame the decoder returns is probably not a real first frame.
Therefore, using warning to help us knows how many this situaion would be.
Differential Revision: https://phabricator.services.mozilla.com/D28432
--HG--
extra : moz-landing-system : lando
As the purpose of adjusting time is to make the start of the playback to align to zero, so the result should not be negative.
Differential Revision: https://phabricator.services.mozilla.com/D28432
--HG--
extra : moz-landing-system : lando
YUVColorSpace is inseparable from the bit depth as the matrix coefficients to be calculated need the bit depth information.
So let's put the two types together. gfx namespace also makes more sense as that's where we find IntRect, IntSize and other.
The extent of the changes highlight how much similar data structures are duplicated across the code, to the point it's scary.
Differential Revision: https://phabricator.services.mozilla.com/D25347
--HG--
extra : moz-landing-system : lando
There is no need to cast to `TimeUnit` to `int64` and then cast to `TimeUnit` again.
Differential Revision: https://phabricator.services.mozilla.com/D26758
--HG--
extra : moz-landing-system : lando
Now the caller of `AdjustForStartTime()` should check the return value to know whether the adjustment is succeeded.
If the error happens the caller should handle it and return an error.
Differential Revision: https://phabricator.services.mozilla.com/D26756
--HG--
extra : moz-landing-system : lando
There's two cases to handle.
1- A Frame isn't entirely contained between appendWindowStart and appendWindowEnd
2- A new frame is appended which overlaps partially an existing frame.
To achieve this we tweak the start time and duration of the sample added (case 1), or the frame about to be partially covered (case 2). We then set a flag that will indicate to the decoder that the decompressed frame will have to be truncated.
Differential Revision: https://phabricator.services.mozilla.com/D20172
--HG--
extra : moz-landing-system : lando
It can be determined from the size of the buffer and the number of audio frames. Additionally, it ensures that the duration of the frame is always exactly what the AudioData contains.
Differential Revision: https://phabricator.services.mozilla.com/D20170
--HG--
extra : moz-landing-system : lando
Don't re-create a new trimmed AudioData when we want to remove some content. This remove the need for some copies.
Differential Revision: https://phabricator.services.mozilla.com/D20168
--HG--
extra : moz-landing-system : lando
Jeff Gilbert
Csoregi Natalia
alwu
Paul Adenot
Bryce Seager van Dyk
Noemi Erli
Butkovits Atila
Jean-Yves Avenard
Cristina Coroiu
Simon Giesecke
Chris Peterson
John Lin
Dorel Luca
Narcis Beleuzu
Cosmin Sabou
shindli