Add support for HLS com.apple.streamingkeydelivery through MSE/EME implementation.
Close#3346
## Tests
Tested on:
- Mac 11.6 Safari 15.2
- iOS 15.2 Safari 15.2
- Mac 11.6 Chrome 96 (for potential regressions on Widevine keySystem)
| Mode | DRM API | TS | CMAF (mono-key and multi-keys)
|---|---|---|---|
| file | EME | ✅ | ✅ |
| file | Legacy-prefixed | ✅ | ✅ |
| media-source | EME | **mux-js**: `encrypted` never fired<br />**real MSE**: `encrypted` event received, but with incorrect `sinf` initData (*1) | ✅ |
| media-source | Legacy-prefixed | **mux-js**: `webkitneedkey` never fired<br/>**real MSE**: TBD | 🔴 fails to append media segment to SourceBuffer (init segment ok) `(video:4) – "failed fetch and append: code=3015"` |
## Support table
| Mode | DRM API | TS | CMAF (mono-key and multi-keys)
|---|---|---|---|
| file | EME | ✅ | ✅ |
| file | Legacy-prefixed | ✅ | ✅ |
| media-source | EME | 🚫 `4040: HLS_MSE_ENCRYPTED_MP2T_NOT_SUPPORTED` | ✅ |
| media-source | Legacy-prefixed | 🚫 `4041: HLS_MSE_ENCRYPTED_LEGACY_APPLE_MEDIA_KEYS_NOT_SUPPORTED` |🚫 `4041: HLS_MSE_ENCRYPTED_LEGACY_APPLE_MEDIA_KEYS_NOT_SUPPORTED` |
⚠️ Use EME APIs with multi-keys CMAF makes the video stalling with the audio continuing alone after a short time (~3 minutes in the stream, could be shorter, could be longer). Didn't find an explanation to that yet. I've observed the same behaviour with hls.js code so I don't think this is a player issue.
* test: Switch local Chrome, Edge, Firefox, and Safari launchers to WebDriver-based
This will enable WebDriver-based screenshot tests in local test runs.
* test: Add screenshots for Edge on Mac
The screenshot review page now has options to organize screenshots either by test or by platform. Previously, it was only by test. This is useful if you want to review many anomalous screenshots for a particular device.
Instead of pixel-wise comparison with a change threshold, we now use a
structural similarity (ssim) module to decide how much a screenshot
has changed. This better tolerates small rendering differences due to
differences in GPUs across machines.
The recent changes to TTML parsing, to not inherit regions,
inadvertently ended up breaking text alignment in situations
where a region with alignment was on the p or div above a span.
Previously, we only inherited the text and display alignment
from a region on leaf nodes... which was a problem, since we
also didn't apply any styles to text nodes.
Change-Id: I62ac155bc4310a5f7da52c10ca2dd434f8015c97
This changes the TTML parser to not allow cue regions to be inherited
to the children of the element the region was originally assigned on,
except for the purposes of styles (colors, etc).
To allow regions on elements "above" the cues in TTML, such as the
<body> or <div> elements, this also changes the TTML parser to render
the full structure of the TTML file as a tree of cues. The end result
will be a single cue representing the <body>, with children
representing the <div> elements inside it, and those <divs> will have
children that represent the actual cues. Now that our text displayer
can intelligently update child cues as they enter or leave the display
window, this approach should be possible.
Closes#3850Closes#3741
Change-Id: Ia8d750daa06920610c04e9b26e29d2d304eaf8a9
Previously, we only cleared the buffer if the media state had
something buffered. This ensured that we did not pointlessly clear the
buffer when nothing was there.
However, this lead to problems if a seek was performed early during the
loading process, before the source buffer is initialized. During that
time, nothing is buffered, so we would not clear the buffer on a seek.
This lead to us accidentally fetching content from the beginning of the
presentation, rather than starting from the new clock time.
This changes the streaming engine to also clear the buffer if
mediaState.performingUpdate is true, to avoid that situation.
Closes#3299
The tb (top bitrate) property should honor bitrate/size ABR constraints and only report the top playable bitrate.
The tb property should only report the bandwidth for that segments type (audio, video, muxed).
Fix incorrect top bitrate CMCD reporting by:
Honoring the player's current bitrate/size ABR constraints.
Only reporting the bandwidth for that segments type (audio, video, muxed).
Fixes#3851
Co-authored-by: Dan Sparacio <daniel.sparacio@cbsinteractive.com>
Revert "fix: Work around override of MediaCapabilities polyfill in Apple browsers (#3668)"
This reverts commit 31c0cd4b8c.
Fixes#3843 (usage of Shaka without polyfills)
Re-opens #3530 (MediaCapabilities polyfill not working on Safari)
Change-Id: Ib5aff1b38759e1a39200332f3f07d3d6bd3bd2e1
If a rounding error in the DASH parser created an extra "phantom"
segment that extended just beyond the presentation duration, this was
causing a playback failure. Now StreamingEngine will tolerate these
small offsets and end playback at the correct time.
Fixes#3717
Change-Id: I00780a664aff4148b6b1046d8322a68da745de40
This makes the HLS parser honor more attributes for image tracks.
It also makes some changes to player.getImageTracks, so that the
returned track shows the size of a single thumbnail rather than the
entire sheet.
Closes#3840
Change-Id: I2ae096f455864201e08a85e29f0f02a3e06eb07f
Now we parse value of descriptor for urn:mpeg:mpegB:cicp:TransferCharacteristics in AdaptationSet to indicate hdr property on shaka.extern.Stream which seems equivalent to HLS definition given in VIDEO-RANGE attribute within EXT-X-STREAM-INF.
This is done by reading the supplemental / essential property in the parsing of AdaptationSet, then mutating the hdr property on shaka.extern.Stream if no hdr exists after parseRepresentation_.
The alternative considered was to parse all essential / supplemental properties on any shaka.dash.DashParser.InheritanceFrame and then carrying that through to the parseRepresentaiton_ function; this can be seen here. The problem with the alternative approach was that the essential property in the adaptation set would have had to have been "let through" on the assumption that we would parse it within the parseRepresentation_ method... But this to me seemed somewhat risky to change, and also looks messy with the empty else if condition, plus was a much larger change (with more calls to XmlUtils, though not familiar with how that is implemented, so not sure if there was any penalty because of that).
* Test expectation for HDR signalling via property
The test sets the expectation that we should be able to extract the
information of the TransferCharacteristics from the DASH
AdaptationSet. This should then get converted to the nomenclature
used in HLS for the VIDEO-RANGE attribute on EXT-X-STREAM-INF.
* Implement signalling of HDR capability through property descriptor
According to DASH-IF IOP v4.3 6.2.5.1. the transfer characteristics
can be declared at the AdaptationSet level via a supplemental or
essential property. In HLS the transfer characteristics are used to
determine the value of the VIDEO-RANGE attribute on the
EXT-X-STREAM-INF attribute, which is already used to indicate the
hdr property on the shaka.extern.Stream object. Therefore for
consistency we also add the same identification to the DASH parser.
* Added entry in AUTHORS and CONTRIBUTORS
Fixes#3726
Previously, in the updateCaptions_ method in the UI text displayer,
when trying to determine if a cue should be removed from the container,
we checked to see if it was out of the time range.
This worked in most cases, but it did have the side-effect that the
displayer would not remove the old cues if they were removed manually
via the remove() method. This was most visible if the user changed the
text language; the currently-displayed cue of the old language would
linger until it reached the point it would stop displaying normally.
This changes the UI text displayer to force an update when remove() is
called, if necessary.
Change-Id: I84a4847b67c5fb7597342a943abe13a3cc9e826e
Previously, the HLS parser would notify the presentation timeline
of the segments in the manifest before fitting the segments.
The HLS parser will also sometimes remove segments from the end of
a VOD asset if they do not fit within the playlist duration.
This could sometimes cause the presentation timeline and MediaSource
to have different opinions on how long the VOD asset was, which could
lead to the seek bar looking like the presentation stopped before the
end.
Closes#3733
Change-Id: I67fdc28a3f6eee158c9906359491fe6bb418e730
Previously, the HLS parser would notify the presentation timeline
of the segments in the manifest before fitting the segments.
The HLS parser will also sometimes remove segments from the end of
a VOD asset if they do not fit within the playlist duration.
This could sometimes cause the presentation timeline and MediaSource
to have different opinions on how long the VOD asset was, which could
lead to the seek bar looking like the presentation stopped before the
end.
Closes#3733
Change-Id: I67fdc28a3f6eee158c9906359491fe6bb418e730
Fixes#3729
If the video variant stream contains information about `hdr`, then we now use that to provide a value for `transferFunction`, when constructing the [`VideoConfiguration`](https://w3c.github.io/media-capabilities/#videoconfiguration) object used to get decoding info from media capabilities API.
The following applies:
```js
switch (video.hdr) {
case 'SDR':
mediaDecodingConfig.video.transferFunction = 'srgb';
break;
case 'PQ':
mediaDecodingConfig.video.transferFunction = 'pq';
break;
case 'HLG':
mediaDecodingConfig.video.transferFunction = 'hlg';
break;
}
```
First, the positioning of cues was incorrect. We need to explicitly
set all the position attributes when we position elements with the
"absolute" position.
Second, if we position a text <span> manually, the background will fill
the whole region. So to keep the background wrapping the text, we need
to add another <span> for the text.
Third, the background and font color should not be set on every <span>
element since it won't allow parent cues to set the inherited value.
So this moves the defaults to the top-level text element and allows
parent cue elements to override this. It also would make app CSS
easier to override. Because background color isn't inherited through
CSS, the default is set in JavaScript instead.
Fixes#3521Fixes#3600Closes#3713
Change-Id: I45fc88dcac4a0a062e1474087f24c80d98eef619
Previously, many events were being defined with a data dictionary
that used variable-type keys (e.g. {key: value}). This worked fine
in uncompiled mode, but in compiled mode it lead to those properties
being obfuscated.
This changes the FakeEvent constructor to take a map rather than an
object, so the compiler will force the keys to be strings.
Closes#3710
Change-Id: I67b1a391540a5ee21f0aaf940ae054d26f4c10a4
Add support for including Common Media Client Data (CMCD) in outgoing requests.
Fixes#3619
NOTE: The following fields have not been implemented: rtp, nrr, nor, dl
Co-authored-by: Dan Sparacio <daniel.sparacio@cbsinteractive.com>
Linear streams accumulate init data over time because the init data is pushed to
an array regardless of whether or not it already exists as part of the logic to
combine periods. This PR dedupes the init data based on keyId to help reduce
memory usage over extended playouts.
Before, the video would automatically gap jump at the start, even if the video
was not yet playing. This meant that videos with a gap at the start would jump
ahead and get rid of the video poster, even if autoplay was set to false.
Closes: #3451
A new UI test added in a recent CL involved manually setting the
currentTime value of a videoElement in tests.
For some reason that wasn't working on Safari, so this changes that
UI test to use a fake video instead of a real one.
Closes#3689
Change-Id: I412f677f6cda49bbfc850a9102edfe3b75bca302
Will only run on devices which report ClearKey support.
Related to issue #771 (ClearKey support on Tizen)
Change-Id: I1ca2458fc4bdc7b6de02a6e452bfd7e3bfe3f9a4
Previously, we added support for respecting the time constraints
of nested cues. However, the UI text displayer did not take the
time constraints of nested cues into account when determining when
and how to update the cues.
This changes the UI text displayer to also do that.
Issue #3524
Issue #3643
Change-Id: I6b643f2aa21f367a8e40a8aca2ebb62492c071c2
When trying to detect if it is appropriate to ignore a non-break
element, the ttml parser would base the decision on the presence
of timing info in the cue itself, and on the presence of
textContent within the node. If neither is present, the element
might be ignored.
This works in most cases of nested cues, as the presence of text
within a child node will show up in the textContent of the parent
node. However, not all valid ttml cues have textContent; image
cues, for example, have their contents defined as a DOM attribute.
This changes the ttml parser to not ignore any cue that has child
cues. It also changes the ttml parser to cause cues with no defined
timing to base their timing on their child cues, if present.
This also changes our UI text displayer to respect the timing info
of child cues.
Closes#3643
Change-Id: I9a017f53398bbed8dbeeeebca2cff76dd3666c64
Vincent Valot
Joey Parrish
Álvaro Velad Galván
Casey Occhialini
Theodore Abshire
Shaka Player BuildBot
Joey Parrish
Patrick Cruikshank
theRealRobG
Aidan Ridley
Jacob Trimble
Nick Michael