Add support for HLS com.apple.streamingkeydelivery through MSE/EME implementation.
Close#3346
## Tests
Tested on:
- Mac 11.6 Safari 15.2
- iOS 15.2 Safari 15.2
- Mac 11.6 Chrome 96 (for potential regressions on Widevine keySystem)
| Mode | DRM API | TS | CMAF (mono-key and multi-keys)
|---|---|---|---|
| file | EME | ✅ | ✅ |
| file | Legacy-prefixed | ✅ | ✅ |
| media-source | EME | **mux-js**: `encrypted` never fired<br />**real MSE**: `encrypted` event received, but with incorrect `sinf` initData (*1) | ✅ |
| media-source | Legacy-prefixed | **mux-js**: `webkitneedkey` never fired<br/>**real MSE**: TBD | 🔴 fails to append media segment to SourceBuffer (init segment ok) `(video:4) – "failed fetch and append: code=3015"` |
## Support table
| Mode | DRM API | TS | CMAF (mono-key and multi-keys)
|---|---|---|---|
| file | EME | ✅ | ✅ |
| file | Legacy-prefixed | ✅ | ✅ |
| media-source | EME | 🚫 `4040: HLS_MSE_ENCRYPTED_MP2T_NOT_SUPPORTED` | ✅ |
| media-source | Legacy-prefixed | 🚫 `4041: HLS_MSE_ENCRYPTED_LEGACY_APPLE_MEDIA_KEYS_NOT_SUPPORTED` |🚫 `4041: HLS_MSE_ENCRYPTED_LEGACY_APPLE_MEDIA_KEYS_NOT_SUPPORTED` |
⚠️ Use EME APIs with multi-keys CMAF makes the video stalling with the audio continuing alone after a short time (~3 minutes in the stream, could be shorter, could be longer). Didn't find an explanation to that yet. I've observed the same behaviour with hls.js code so I don't think this is a player issue.
This makes the HLS parser honor more attributes for image tracks.
It also makes some changes to player.getImageTracks, so that the
returned track shows the size of a single thumbnail rather than the
entire sheet.
Closes#3840
Change-Id: I2ae096f455864201e08a85e29f0f02a3e06eb07f
Previously, the HLS parser would notify the presentation timeline
of the segments in the manifest before fitting the segments.
The HLS parser will also sometimes remove segments from the end of
a VOD asset if they do not fit within the playlist duration.
This could sometimes cause the presentation timeline and MediaSource
to have different opinions on how long the VOD asset was, which could
lead to the seek bar looking like the presentation stopped before the
end.
Closes#3733
Change-Id: I67fdc28a3f6eee158c9906359491fe6bb418e730
Previously, many events were being defined with a data dictionary
that used variable-type keys (e.g. {key: value}). This worked fine
in uncompiled mode, but in compiled mode it lead to those properties
being obfuscated.
This changes the FakeEvent constructor to take a map rather than an
object, so the compiler will force the keys to be strings.
Closes#3710
Change-Id: I67b1a391540a5ee21f0aaf940ae054d26f4c10a4
Add support for including Common Media Client Data (CMCD) in outgoing requests.
Fixes#3619
NOTE: The following fields have not been implemented: rtp, nrr, nor, dl
Co-authored-by: Dan Sparacio <daniel.sparacio@cbsinteractive.com>
All TS packets are expected to start with the sync byte 0x47, and
TS files are expected to start at the start of a packet. Because of
that, we assumed that any TS file that did not start with the sync
byte of 0x47 is invalid.
However, it is possible that a TS file might not start on a sync
byte... perhaps the first few bytes got cut off, for some reason?
In those cases, it would be better to just drop that cut-off first
packet, instead of entirely rejecting the TS file.
This CL changes the HLS parser to keep looking forward for the
length of a few packets for the first sync byte, when parsing TS
files to try to find their start time.
Closes#3580
Change-Id: I0bace4b3a84398e09046828d749f3f025dfad26f
We should only request playlist delta updates when the lowLatencyMode is
enabled, and when the server supports the feature. We can check if the
playlist has a "CAN-SKIP-UNTIL" tag to know if the server supports that.
Issue #1525
Change-Id: I304a008aeb3a9e019f27304ba836d31538c94fbf
In live streams, we can evict segments outside the availability window
faster than they disappear from the manifest. If that happens, we used
to evict them several times (add them back in and then evict again).
This caused the eviction counter to increase beyond what it should be
and we had trouble finding segments afterwards.
Closes#3139.
Change-Id: Iafebfaf8e1e9ebb09a64cdf7e09a882115fd8eb6
With playlist delta update, the discontinuity tag in the old playlist
might be skipped in the updated playlist. The discontinuity sequence
number is for the first (skipped) segment in the playlist, and we should
not use it directly to get the timestamp offset for the first listed
segment.
Instead, we can have a map of discon sequence number to the starting
media sequence number. And for the updated list, check if we have a
larger discon sequence number in the map, and its starting Media
sequence number is smaller than the segment's media sequence number.
Change-Id: I6c2e32160791b22afb87f40af4d246c5cabb6d7f
If we have a multiplexd stream with audio and video, the video
codecs are a combined string containing both audio and video codecs.
Before sending the request to decodingInfo(), we should check which
codec is audio and which one is video.
This is a follow-up of commit 966a756.
Issue #1391
Change-Id: Ic9b1c5972a99f63a715c74ae068b85f43efac447
Previously, we fill in the variants' drmInfos with the drm
configurations of |clearKeys|, |servers| and |advanced| during
initializing the DrmEngine, before we query the media keys.
Now we need to call |MediaCapabilities.decodingInfo()| to get the
mediaKeySystemAccess of the variants after the DrmEngine is
created and configured, and the drm values are filled in for the
variants, and before |DrmEngine.queryMediaKey_()|.
The steps would be:
0. StreamUtils.setDecodingInfo() should not be called before
DrmEngine is initialized and configured.
1. Create and configure DrmEngine.
2. Fill in drm values for the variants with configurations.
3. Call StreamUtils.setDecodingInfo() to get the decodingInfo of
the variants.
4. Use the decodingInfo results to set up the Drm mediaKeys in
DrmEngine.
5. When StreamUtils.filterManifest() is called, we can reuse the
decodingInfo results instead of calling decodingInfo again.
Previously we call filterManifest() when parsing the manifest, to
filter out the unsupported streams.
Now decodingInfo can tell us if a variant is supported and its
MediaKeys at once. When initializing the DrmEngine, we get the
decodingInfo results of the variants, and only use the supported
streams to set up the MediaKeys. After that, we filter the manifest
right after DrmEngine is initialized.
Thus, we can skip filterManifest() in manifest parsers.
Using decodingInfo to get media keys will be in the next CL.
Issue #1391
Change-Id: Ieb401a1e4dfbcc958f7a14fa96df546237b0f446
PART-HOLD-BACK indicates the server-recommended minimum distance
from the end of the Playlist at which clients should begin to play
or to which they should seek when playing in Low-Latency Mode.
PART-HOLD-BACK is only required if the Playlist contains the
EXT-X-PART-INF tag.
Change-Id: Id430a38efdbc78f65e8d34ba125b02bdec6b7e59
In StreamUtils, use MediaCapabilities.decodingInfo() instead of
MediaSource.isTypeSupported() to check if the stream is supported.
MediaCapabilities.decodingInfo() takes an
MediaDecodingConfiguration object as input, and returns a Promise
with a MediaCapabilitiesInfo object. The returned object tells us
whether decoding the media is supported, smooth, and
powerefficient.
Steps:
1. Create a MediaDecodingConfiguration object for each variant as
the input.
2. Query the decodingInfo API with the config.
3. Get the 'supported' info from the decodingInfo result, to know
whether the variant is supported.
Issue #1391
Change-Id: I8fc2d3ec6a9868f38269d550d35f45c298faae98
1. Added hdr as a property in stream when constructing.
Fixes build failure from commit
71372869de .
PR #3116
Issue #2813
2. Fixed the test error from commit
d3640d1da1 .
PR #3044
Issue #3029
3. Fixed the new line with no other arguments from commit
0845843b65 .
PR #3060
Change-Id: I5833e49c1a95172742c4ec820960c9c5a7bf0cca
In our isTypeSupported polyfill for Cast, we use the HEVC profile ("hevc1.2") as
an indication of the HDR transfer function, which is not always accurate.
The polyfill should stop assuming that 10-bit color HEVC means HDR, and remove
the extra eotf="smpte2084" parameter. Instead, the manifest parser should
extract HDR profile information from the DASH manifest and pass that info along
through the Stream object.
Issue #3116 .
This enables the eslint rule requiring jsdocs on all class
declarations, function declarations, and methods.
Unfortunately, there are two problems with this:
1. We don't use class _declarations_, we use class _expressions_,
which are not covered by this rule. So it does not enforce jsdoc at
the class level.
2. We tend to document a class at the class-level, rather than at the
constructor. But a constructor counts as a method for eslint, so it
requires docs on the constructor. There is no way to configure it to
make an exception for trivial constructors.
So for all trivial (no-argument) constructors, we add empty jsdocs:
/** */
constructor() {
This was quicker and easier than setting up some alternative plugin in
eslint to make an exception for us.
The good news is that this rule caught several undocumented parameters
and places where the jsdoc comment was malformed. So fixing those
also improves the compiler's ability to enforce types.
Change-Id: Icbc46ed690c94e53d354648a883119524f8fca45
With "lowLatencyMode" enabled, "rebufferingGoal" is set to 0.001, and inaccurateManifestTolerance is set to 0 by default. However, in some cases longer rebufferingGoal helps to avoid new rebuffering.
Besides "lowLatencyMode", this code change adds the "autoLowLatencyMode" config. When "lowLatencyMode" config is disabled and "autoLowLatencyMode" config is enabled, and the manifest provides low latency features, we automatically activate the lowLatencyMode.
If "lowLatencyMode" is enabled, "autoLowLatencyMode" has no effect.
Issue #1525
The HLS content linked in #2907 uses the .mp4a extension for audio
segments, so it is helpful for us to be able to recognize the types of
these segments. This adds both .mp4a and .mp4v to the extension maps
in the HLS parser.
Change-Id: I8f2e126691e552c34ff5da027fca32ed925a2bdf
BYTERANGE-START is the byte offset of the first byte of a preload
hinted resource. This attribute is optional, and defaults to 0.
Example:
'#EXT-X-PART:DURATION=1.02,URI="fs271.mp4",BYTERANGE=18000@43000'
'#EXT-X-PRELOAD-HINT:TYPE=PART,URI="fs271.mp4",BYTERANGE-START=61000'
Issue #1525
Change-Id: Ib3f7feb6bd5d3fd09b89962c067029a448d020e7
Previously, we treated the mere presense of either the DEFAULT or
AUTOSELECT attributes as an indication of a primary language.
However, AUTOSELECT should not be used in this way at all, and both
attributes can have a value of NO, so mere presence or absence is not
enough.
Instead, we must check for DEFAULT=YES to indicate a primary language.
Fixes#2880
Change-Id: Ie107f079c06beb89ee5a6a51375af7a72fdd9021
If an HLS stream has no provided codec information in its manifest,
we previously defaulted to providing a best-guess video and audio
codec. This, however, caused problems for video-only and audio-only
streams; Chrome does not like it if an audio codec is provided for
a stream with no audio at all, for example, and fails to append the
chunks.
This modifies the HLS parser to not provide a default video codec if
the 'disableVideo' config is set, and similarly to not provide a
default audio codec if 'disableAudio' is set.
Issue #2868
Change-Id: I7826fdb69f2e5290914ed2c0d1e109f9db94f80d
If an HLS playlist contains partials segments, or a Dash playlist
has availabilityTimeOffset value, we check whether lowLatencyMode
is enabled. If not, we show a warning to remind the developer to
set the configuration.
Issue #1525
Change-Id: If7b851c32c325203870c33d868668e1a3dd3c8bf
Previously, on manifest updates, embedded captions would vanish in
single-period live DASH streams. The problem was based on the
specific point in the load order that we added dummy text streams
to indicate the presence of embedded captions.
This modifies the PlayerInterface for manifest parsers to add a
new method passed in, makeTextStreamsForClosedCaptions, which
must be called by manifest parsers for new video streams.
This also completes an unfinished feature, where new video streams
which add new closed captions would not get corresponding
text streams.
Closes#2811
Change-Id: Iee7499ec950b363cf6839765cc2bd2d01743467d
When starting playback for HLS live, we have the availability window
same as the delay duration from live edge. Thus, the playback start time
is the same as availability window start, and falls behind the window as
the window keeps move forward.
This may cause the seeked partial segments to be unavailable.
To make our system more robust, we can skip checking if the segment fits
in the availability window. Instead, we can send the request anyway. If
the segment is not available, we can handle the 404 error and try to fetch
the segment again.
This would not require us to calculate the segment availability window
in advance.
Issue #1525
Change-Id: Ib9c0eca8e9db2d93404745d87a5171fcb5fdce20
1. Move the config field from ManifestConfiguration to StreamingConfiguration, since StreamingEngine will need it.
2. In manifest parsers, we get the value of lowLatencyMode config from the StreamingConfiguration.
Issue #1525
Change-Id: Iaa961b4e6799ecc1fcf3147b2fb992e86d4b043d
1. In low latency mode, the playback start time from live edge is about
3 times partial segment duration, 3s for example. The rebufferingGoal of
2s would result in infinite buffering.
2. In low latency mode, do not allow drift for seeking. Only seek with
the presentation time.
Change-Id: Iee92727bf17dab8b52fae070522a5a2ec1621ffc
If the last segment in the media playlist contains only partial
segments, its init segment was not set, causing playback failing to
start, with 'Append: stream parsing failed' error.
Issue #1525
Change-Id: I0dab967fe55c9d792f102f4d49dda7f9437d8897
LL-HLS hints a resource that is needed to playback in the upcoming
update. It's available for request, and may not be available for
download yet.
A preload hinted resource is either a partial segment, or an init
segment. If it's an init segment, treat it the same way as the Map tag
in ManifestTextParser.
A preload hinted segment contains no duration information, so its
start time and end time are the same. It will be replaced by a partial
segment in the next update.
We should fetch and append the preload hinted segment the same way as a
partial segment.
Issue #1525
Change-Id: I1e30f216ecdc843c3cd01681629a8886383d0b22
Changed SegmentIndex and SegmentIterator to iterate through regular and
Partial SegmentReferences.
SegmentIndex:
merge(): Find all the old segments after the first new segment's start
time, and replace the old ones with new segments.
SegmentIterator:
Use the currentPosition and currentPartialPosition pointers to iterate
through the two-layer arrays.
current(): Get the current SegmentReference, and get its current Partial
SegmentReference if it has a Partial Segment list. Move to the next
regular segment if we reached the end of the current segment's partial
list.
next():
If the regular segment has a partial list, go to the next Partial
Segment. If reached the end of the current partial list, move to the
next regular segment.
If the regular segment doesn't have a partial list, move to the next
regular segment.
Issue #1525
Change-Id: Icb7f49e50314f15ea40bf3a74d008191ed1a9a6c
Player can request delta updates to reduce the transfer cost.
Client: sends a request for playlist update with '_HLS_skip=YES'
parameter.
Server: replaces the older segments in the media playlist with
'EXT-X-SKIP' tag.
Issue #1525
Change-Id: I3643641d0cb97444ba1c00f06d9e113acba7b824
For LL-HLS, use the value of 'PART-HOLD-BACK' in the Server Control tag
as the default presentation delay.
'PART-HOLD-BACK' is the server suggested min distance from the live
edge, and must be >= 3 * partial segment target duration.
It's always available when the playlist contains partial segments.
Issue #1525
Change-Id: I176188dbd39be0d038eee938d3e8358e54b8a3a8
Vincent Valot
Theodore Abshire
Patrick Cruikshank
Casey Occhialini
Álvaro Velad Galván
Michelle Zhuo
ismena
Suraj kumar (sk)
Joey Parrish
Chris Fillmore
slasktrat
Jacob Trimble