This pull request improves support for external SIDX (Segment Index)
files in DASH manifests, particularly when the `RepresentationIndex`
uses a different `BaseURL` or `sourceURL` than the media itself. It also
enhances base64 decoding robustness and adds a new unit test to verify
correct behavior.
**DASH SIDX and Segment Reference Handling:**
* Enhanced `Mp4SegmentIndexParser.parse` to accept an `indexIsExternal`
parameter, enabling correct parsing of SIDX files that are external to
the media and may have different base URIs. The parser now adjusts the
offset logic for external indices.
[[1]](diffhunk://#diff-6435d27cfd56024b0920175aa9a6992242d18900d27f7edfaa77d89673a8dd0aR29-R37)
[[2]](diffhunk://#diff-6435d27cfd56024b0920175aa9a6992242d18900d27f7edfaa77d89673a8dd0aR54-L63)
* Addresses #6091: Updated `SegmentBase.generateSegmentIndexFromUris` to
detect when the index URI is external by comparing the base URIs, and to
pass this information to the parser. This ensures that segment
references are resolved against the correct URIs.
**Robustness Improvements:**
* Improved base64 decoding in `Uint8ArrayUtils.fromBase64` by
normalizing padding, handling cases where the input string omits
trailing `=` characters.
**Testing Enhancements:**
* Added a unit test to verify that `RepresentationIndex` with a
different `BaseURL` or `sourceURL` is correctly honored, ensuring that
segment index requests use the proper URI and range.
---------
Co-authored-by: Álvaro Velad Galván <ladvan91@hotmail.com>
Due to not setting the `boundaryEnd` for the last period, we ensure we
do not discard the wrong segments in a live to VOD situation (where a
boundaryEnd becomes available mid stream).
The same applies in live, when a period ends and a new period starts.
The previous one will now have a boundaryEnd, the new one does not.
Co-authored-by: @avelad
We had an issue where in SSAI content, 708 data was being split by ad
periods. Currently, when this happens, we reset the 708 decoder, which
means that captions are lost. Instead, we want to cache this decoder for
a later time. This change keeps track of continuous periods and caches
the 708 decoder when a period change happens to a discontinuous period.
This is so that it could be later restored if we go back to a continuous
period.
---------
Co-authored-by: Álvaro Velad Galván <ladvan91@hotmail.com>
Co-authored-by: Wojciech Tyczyński <tykus160@gmail.com>
This PR introduces support for `ProducerReferenceTime` tags in DASH.
When finding any, it emits event, similar to inband PRFTs added in
#4389.
Additionally, calculated start date from `ProducerReferenceTime` is used
as program start date, as it's more accurate value than
`MPD#availabilityStartTime` used before.
There's devices out there that are not compliant with the MSE spec. Such
as halting MSE when a secondary init segment is appended (webOS 3), or
failing to transition from a plain to encrypted init segment (Tizen
2017). While we initially prefer content workarounds, it's a time
consuming and trial & error process. For some devices it might not be
worth investing time into finding a proper workaround due to low usage.
We're giving people an alternative by resetting MSE when needed
(configurable). dash.js offers somewhat similar behavior
[here](https://github.com/Dash-Industry-Forum/dash.js/blob/a656ec709e7f92f76b392bf196ee9883da7928ce/src/streaming/controllers/StreamController.js#L672),
where MSE is reset before applying an encrypted init segment.
This PR introduces `crossBoundaryStrategy` in `StreamingConfiguration`.
It can be configured as following:
- KEEP - we're keeping MSE active, this is the default and the current
behavior.
- RESET - we'll always reset MSE when it crosses a boundary.
- RESET_TO_ENCRYPTED - we reset MSE when it crosses an encrypted
boundary, and we keep MSE afterwards. Additionally, we're not going to
reset when we're crossing a plain to plain boundary.
Each initSegmentReference now holds an `encrypted` and `boundaryEnd`
value. When configured with a different value than KEEP,
`StreamingEngine` will be instructed to fetch and append segment
references up until the boundary of the currently applied init segment.
We detect whether we're at a boundary in a few ways:
- Listening to the HTML5 MediaElement's `waiting` event, this'll
indicate that we do not have enough buffer to advance. If we're pretty
close to the boundary, we assume we're at the boundary.
- Due to subtle differences in the segment alignments, waiting wasn't
reliable. When close to a boundary, a timer is fired with the assumption
that "we'll reach the boundary at soon". I've set the threshold to 1
second, when playhead is further than the threshold, we'll skip checking
whether an MSE reset is due.
The implementation relies on the added properties in the init segment
reference, and the concept of a "Period" is avoided in StreamingEngine
to ensure it's compatible with HLS too.
---------
Co-authored-by: Álvaro Velad Galván <ladvan91@hotmail.com>
Co-authored-by: Wojciech Tyczyński <tykus160@gmail.com>
This piggybacks on the media quality observer to also check whether the
audio track changed and the change was in label, language, or role.
Currently, this new event is triggered is `observeQualityChanges` is
set. This could be moved to a separate config.
I do play to add a test or two to the integration tests.
For verifying the role/label, I was testing against this public test
stream
https://demo.unified-streaming.com/k8s/features/stable/video/tears-of-steel/tears-of-steel-desc-aud.ism/.mpd
(though, it does seem to error out every so often). There might be other
test streams. Angel One is a great one for language.
Also, something that we noticed, but won't be part of this PR. The
timing of this event is likely incorrect if the track was switched and
safeMargin was set. We have some changes for this locally that we can
PR, but there are some issues we found around devices for it, so, it
isn't a straightforward change.
A previous PR, #5950, added support for variants that contain multiple
different codecs.
It was supposed to also add support for variants with multiple
mimeTypes, but that part didn't work correctly. This reworks a lot of
#5950 and #6047, to change how they handle such complicated variants.
This has the side-effect of allowing the stream utils to differentiate
between content that has multiple codecs because of type changes, and
content that has multiple codecs because of being muxed video+audio.
Fixes#6010
---------
Co-authored-by: Álvaro Velad Galván <ladvan91@hotmail.com>
To provide feature parity with the `DOMParser.getAttribute` API which
would unescape html entities for free this PR unescapes url strings
which come from node attributes.
In a similar vain to
https://github.com/shaka-project/shaka-player/pull/6198 which unescapes
child strings.
## Background:
The native DOM Parser can perform poorly on some older devices, this
approach is faster on newer devices but is considerably better on older
devices.
This PR replaces the usage of the DOM Parser for DASH, MSS, VTT and
TTML.
The draw back of this approach that it does not include any validation
at the cost of better performance.
This adds an optional parameter to request and response filters, of a
new enum called AdvancedRequestType.
This enum describes request types that are subtypes of the basic types.
For example, INIT_SEGMENT is a type of SEGMENT.
This gives users more information about the type of the request, while
maintaining backwards compatibility.
Closes#4966
When a Period has segments outside the Period, we evict them during
manifest parsing. However, this eviction shouldn't affect the position
of segments since these never existed.
Fixes#3230
Change-Id: I0494d9c38b51857967ed4b572475ddcc37f815a0
This fixes all the license headers in the main library, which corrects
the appearance of the main license in the compiled output.
It seems that the `!` in the header forces the compiler to keep it in
the output. I believe older compiler releases did this purely based
on `@license`.
Issue #2638
Change-Id: I7f0e918caad10c9af689c9d07672b7fe9be7b2f3
This is a step along the way to flattening periods, and also
simplifies a few small things in StreamingEngine and tests.
Issue #1339 (flatten periods)
Issue #892 (refactor StreamingEngine)
Change-Id: Ie17cd5e15ed6ec9290a918c3a69c05c74581e0fc
SegmentReference used to have presentationTimeOffset, which,
subtracted from the period start time, was then _added_ to the
timestamps in the segment by MediaSource.
Now, SegmentReference has a timestampOffset field, which is exactly
what MediaSource's timestampOffset field is set to on the SourceBuffer
before this segment is appended. For DASH, this is periodStart minus
presentationTimeOffset.
This also adds append window start & end times to the
SegmentReference. Now segments can be appended to SourceBuffers
without reference to the period.
Note that start & end times of the SegmentReference in each segment
index are still relative to the period. This will change in a
follow-up.
Issue #1339 (flatten periods)
Issue #892 (refactor StreamingEngine)
Change-Id: I9d54eb2b529ec643c9475b8e9d218c3e2e826a26
This reflects changes in Google's policy on JavaScript license
headers, which should be smaller to avoid increasing the size of the
binary unnecessarily.
This also updates the company name from "Google, Inc" to "Google LLC".
Change-Id: I3f8b9ed3700b6351f43173d50c94d35c333e82b4
These are intermittently used and cannot be verified. Most of our
methods can throw a shaka.util.Error, so having it doesn't really add
anything. Plus, if we change a function to throw, we'd need to update
all their callers to ensure they have an accurate description of what
they throw; otherwise we can't trust the directive.
Change-Id: I520bd0fc4c33443e967bf5b103ca5aa9e3274884
To prepare for flattening out the manifest structure to remove
periods, this change moves initSegmentReference and
presentationTimeOffset fields into the SegmentReference object. This
way, the segments on either side of a period transition or HLS
discontinuity can have different offsets or init segments, eventually
allowing us to create a single array of SegmentReferences for
multi-period content.
Issue #1339
Change-Id: Ic7eff0483789644881247ecf8044c5fb6a48f0e6
This was broken in "Replace find/get callbacks with SegmentIndex"
because automated tests for SegmentBase did not cover multiple
Representations with different index/init ranges. Once the parsing
became async in the previous change, the index information was being
pulled from the last Representation parsed, rather than any specific
one.
This fixes the issue by making a shallow copy of the parsing context,
as was already done for SegmentTemplate.
SegmentList is synchronous, so it was not affected.
Issue #1339
Change-Id: I09c606c464b49c99d35d1031734b64872cfa6599
This replaces find/get callbacks in Stream with a SegmentIndex. With
the exception of DASH's SegmentTemplate+duration, all manifests were
already backed by SegmentIndex. Now, all manifests are backed by
SegmentIndex. This will simplify Period-flattening, in which all
tracks will be represented by a list of segments, some of which come
from different Periods.
The SegmentIndex in Stream will not be created until
createSegmentIndex is called. Prior to this change, the find/get
callbacks could be invoked without createSegmentIndex() in some cases
(excluding DASH's SegmentBase), which some lazy tests took advantage
of. Now that find/get are methods on SegmentIndex, createSegmentIndex
must be called in all cases. The tests have been updated accordingly.
Making SegmentIndex generation async in all cases exposed some issues
with the parser context being modified in-place between one
Representation and the next. So the parser now makes a shallow copy
of the context before it is bound into an async callback.
To facilitate updating the SegmentIndex for SegmentTemplate+duration
content, SegmentIndex now has a method to update its list on a timer.
Once per segment duration, the index will be updated to add and remove
SegmentReferences.
The initial expansion of SegmentTemplate+duration will be limited to a
relatively small number of segments, to avoid excessive CPU or memory
consumption. This defaults to 1000 segments, but is configurable.
Issue #1339
Change-Id: I99c007b1096c3b396d04a729750cd7b743cb899d
This is a fully automated change. The linter will fail because the
extra indentation caused line-length errors. These won't be fixed
automatically. They are fixed in a follow-up to make this one fully
automated.
Change-Id: I4d8cf9c998985add2bcd24a81c8d65495668c4f3
With the new style rule, we cannot have two statements on the same line.
So we can no longer have an "if" on a single line and we cannot have
an arrow function with a body on the same line as when it is used.
This is mostly a manual change.
Change-Id: I2285202dd5ecbad764308bc725e6d317ff2ee7f0
This is an automated change to convert use of "function" functions
to arrow functions. This doesn't change all uses of bind() that
could be converted. This also doesn't remove all "function" functions.
Change-Id: I40ac7d086bcef947a1be083359c8fd1d4499a9c3
A coming update to the Google eslint config will require using "const"
over "let". This makes that one change to isolate the big changes.
Change-Id: I7d0974c3ae15c53cc45a6b07bf9f6586e2d34aca
Before, segment timestamps were used in PresentationTimeline without
regard for the period start. This means they were not truly relative
to the presentation, but to the period.
The "isFirstPeriod" argument was also broken. It was meant to be true
for segments from the first period *ever*, but was passed true for the
first period *in the latest manifest update*. So data calculated from
that was bogus for live streams.
Now, notifySegments() is supplied with a period start time, so that
segment references can be combined with the period start to give
presentation timestamps. This fixes a major issue with the original
fix for #999.
Closes#999
Change-Id: Id0fe450f3ce4f90a2387d7103c75eb88f0c69c72
Google style guide requires adding curly braces to all block statements
even if it is only has one line. This fixes it by using eslint's
--fix flag followed by running clang-format to reformat the change.
Change-Id: Idc086c2aa8c02df5ef8b2140a11bfb9128eeb4bd
Rather than using the availability window, Playhead should use the
seek range to restrict the playhead's position.
Closes#1224
Change-Id: I8612bfafb53bbb214e32aae2d71af52d748a3aee
This is part of a change to convert all usages of 'var' with either
'let' or 'const'. This takes a conservative approach for 'const' where
it will only be used for aliases and storing the "original" values in
tests.
Change-Id: Iebba756b5d0e68c41292ecabda89503682d8d434
According to the DASH spec, the timescale in the manifest need not match the
timescale in the media. Therefore, we should be applying scaled
presentationTimeOffsets in segment index parsers, since the two scales might
differ.
Issue #1098
Change-Id: Ic191d1bba399b30a656ab5060d7bb226e659b79a
In DASH SegmentBase, we were not dividing presentationTimeOffset by
the timescale. In all other instances of presentationTimeOffset in
DASH, we were handling it correctly.
This bug was present in all v2 releases until now.
The solution is not only to divide by timescale, but to rename all
internal uses of presentationTimeOffset to either unscaled or scaled,
to differentiate between those in timescale units and those in
seconds. I believe inconsistent naming and units were a contributing
factor to the creation of the bug.
Closes#1099
Change-Id: Id561f8eb1f5bc011c606e1925c12f0d8183fd51a
This reverts commit 478fb1a473
("Detect and attempt to correct timeline sync issues") and commit
4334b6ea90 ("Add mediaSegmentReferences
to shakaExtern.Stream").
Timeline correction only addresses one class of broken live content,
while actually making it more difficult to diagnose other classes of
broken live content.
Issue #933
Change-Id: Ie71b8a45600ed0994626f53268fb90e336c3c62b
This will allow us, in many cases, to detect time sync issues in
content by comparing the media segment references to the presentation
timeline. If the references are outside the timeline, it may indicate
a common form of bad content, and we may be able to attempt to correct
it.
In some cases, the references are not available upfront. We can't
detect time sync issues in DASH with SegmentTemplate+duration, because
the references are not explicitly described by the manifest. We can't
detect time sync issues in DASH with SegmentBase, because the segment
index requires additional fetching and parsing. (SegmentBase is not
used with live content, so this should be a non-issue.)
This only introduces the new data, but does not use it for detection
or correction yet.
Issue #933
Change-Id: If70b1bdbd3b08a7c8b7ae296da209737492dfe17
The bulk of the logic for gap jumping is handled in Playhead. It
tracks the current buffered ranges and jumps over any gaps that appear.
It listens for a special browser event ('waiting') for when the video
element runs out of playable frames.
This change also removes the logic for jumping gaps at the beginning
of the timeline. This is handled by the more general gap jumping
logic and works cross-browser.
Finally, this updates the buffering logic to only count the amount of
content buffered (i.e. ignoring the gaps). This fixes some bugs where
gaps in the content can result in StreamingEngine buffering forever
since it thinks only a little is buffered.
This includes full implementation of the logic, but this doesn't close
the issue since there aren't any integration tests yet. Those will
be added next.
Issue #555
Change-Id: Id99eb9fe469e8cf2c7464a3d70c3733791e806e0
This adds a severity field to Error objects. This can be used to
detect whether an error is recoverable. All the same errors are still
reported so the field can be ignored.
There are two possible values:
* RECOVERABLE means that the Player will try to recover from the error
* CRITICAL means the Player will be unable to continue and must call
load() again
Closes#564
Change-Id: Ie2c5468340c13e7a288b99690ab65b7ecc0a6b29
We usually fit segment references to the Period for VOD content. This
ensures that a gap in the manifest at the end of the Period doesn't
cause problems. We don't do this for live because we may get more
segments. However, for multi-Period live, we should still fit the
references for all Periods except the last one.
This also removes the error about duplicate Representation IDs for
VOD content since the problem only occurs for live.
Issue #694
b/35849085
Change-Id: Ib195dc51982ca58b8d5613b37817216980b8d366
Chad Michael
Matthias
Gary Katsevman
Wojciech Tyczyński
Álvaro Velad Galván
theodab
Dave Nicholas
Aidan Ridley
Jacob Trimble
Joey Parrish
Michelle Zhuo
Sandra Lokshina