This PR fixes#3242 where for some live streams using segmented VTT, text timings are relative to segment start instead of being absolute.
The PR introduces a new setting: `manifest.segmentRelativeVttTiming: boolean` allowing such alternative timing offset calculation.
The setting is off by default, preserving the current player behaviour.
Co-authored-by: Joey Parrish <joeyparrish@users.noreply.github.com>
This makes the HLS parser read the EXT-X-PROGRAM-DATE-TIME value
on manifests, and use it to make sure that segments are inserted at
the correct place in the timeline, when in sequence mode.
Issue #2337
HLS now appends segments in sequence mode. In order to handle seeks,
we set the timestampOffset property on SourceBuffer to the startTime of
the segment. This is done after every seek, or on startup.
For non-sequence-mode content (DASH), we normally set timestampOffset
based on the Period structure. This should be suppressed in sequence
mode, though, where only the reference startTime matters.
The buffering issue was caused by two things in combination:
1. The HLS parser set meaningless timestampOffset values that would
change when a playlist was updated
2. We would use those timestampOffset values in setStreamProperties,
even though this should be skipped in sequence mode
These two things in combination would lead MediaSourceEngine to start
inserting segments near the start of the presentation, rather than at
the live edge.
This changes MediaSourceEngine so that in sequence mode, timestampOffset
is ignored in setStreamProperties. This also cleans up the HLS parser
to set each reference's timestampOffset to 0, since they should be
ignored anyway.
When running in sequence mode, we ignore the normal timestamps
of video and audio segments. This lead to problems in some Apple-
encoded webvtt content, which used the X-TIMESTAMP-MAP tag to account
for the timestamp offsets in their video. Thus, those subtitles would
end up 10 seconds offset.
This changes the webvtt parser to ignore the X-TIMESTAMP-MAP when in
sequence mode.
Issue #2337
This adds code to allow Shaka Player to play media in sequence
mode, an alternate playback mode that makes the browser ignore
media timestamps, when playing HLS media.
This is important for containerless media formats, as they do not
contain such timestamps.
Changing HLS to not require timestamps also means that we no
longer need to fetch media segments in order to get the start
time, which should lower bandwidth usage and startup delay.
In initial tests, on a simulated 3G network, load latency went down
from an average 3.16s to 2.61s on the HLS version of "Big Buck Bunny:
the Dark Truths of a Video Dev Cartoon"; an improvement of about 17%.
Issue #2337
Change-Id: I507898d74ae30ddfb1bddf8dce643780949fbd9b
When we attach closed-caption cues to the text engine, we apply the
video timestamp offset to the cues, so that they can align with the
start of the stream. However, we previously forgot to apply that
offset to any nested cues.
This did not matter beforehand, since we previously ignored the start
and end times of nested cues. However, recent changes to the UI text
displayer changed that.
So this fixes nested cues to apply video timestamp offset.
Closes#3782
Change-Id: I4c9140fcfa9bf94579f8d847e546ee4e2ec5eff4
Embedded and non-embedded captions enter the text engine through
a significantly different path. Notably, when embedded captions
are present, they are constantly being fed to the text engine even
when not turned on, as there is no extra bandwidth cost to appending
them. They are merely ignored if the correct flag is not set.
Well, it turns out that we weren't un-setting the flag when
transitioning from embedded to non-embedded captions, so the embedded
captions would continue to be displayed indefinitely in that case, on
top of the non-embedded ones.
This changes the streaming engine to clear that flag if the text media
state is detected to be non-embedded.
Issue #3199
Change-Id: Ib06d0d1628e0ce8c863ffee523db0ea4ec8c3c73
Some platforms seem to ignore EME if the first init segment appended
at the MSE level does not indicate encryption. To work around this,
we will detect such platforms, and if EME is set up, we will insert
fake encryption metadata into the init segment to convince the
platform to be ready to decrypt.
Closes#2759
Change-Id: Iccf6b8589cf9265ac9bef0d083f4a72f6f2dd628
We should call URL.revokeObjectURL after calling
URL.createObjectURL() to prevent memory leak.
createObjectURL creates a strong reference to the MediaSource object
inside the browser. Setting the src of the video then creates another
reference within the video element. revokeObjectURL will remove the
strong reference to the MediaSource object, and allow it to be
garbage-collected later.
Fixes#2953
Change-Id: If9dc18a69468dfb8a6ebe7f4992bd61c716c2104
This replaces mux.js for CEA608 decoding. Applications will no longer need to include mux.js for CEA support, and mux.js will only be necessary for TS transmuxing.
Closes#2648
During video playback, if the user switches the caption stream (e.g. CC1 to CC3 which changes languages), the first caption in the next fragment is missing.
In fragmented MP4s, the end time of a caption is determined by the start time of the next caption. Thus for the last caption in a fragment, the end time cannot be determined until the next fragment is parsed.
Before this fix: the clearing of the caption stream was being called from a chain of function calls originating from clearBuffer_() on the Media source engine. But clearing a buffer and resetting a caption stream are two independent operations. As a result, the caption parser was being reset (its buffer cleared) during video seeks, and during stream switches. This makes sense for video seeks, because the end time of the last caption in the fragment can't be determined if the entire presentation timestamp changes. However for stream switches, resetting the parser doesn't make sense. Clearing the caption parser during a stream switch would actually get rid of the last caption in that fragment (which wasn't emitted since its end time wasn't determined yet), and we would lose the data, causing the problem.
The fix is to reset (and hence clear) the caption parser during seeks, but not during stream switches.
Issue #2648
This fixes all the license headers in the main library, which corrects
the appearance of the main license in the compiled output.
It seems that the `!` in the header forces the compiler to keep it in
the output. I believe older compiler releases did this purely based
on `@license`.
Issue #2638
Change-Id: I7f0e918caad10c9af689c9d07672b7fe9be7b2f3
We used to alias static utility methods by assigning the method itself
to a local variable. This is not allowed by the new Closure Compiler
release that we are adopting, and for good reason.
The old compiler did not understand the use of "this" in static
methods, but the new one does. And it turns out that when we start
using "this" in static methods (see #2532), aliasing the method itself
can break everything.
When you refer to "this" in a static method, it refers to the class.
This is really useful in utility classes that have private static
methods they use to perform common tasks. However, just as it ruins
the value of "this" to alias an instance method, the same is true of
an ES6 class's static method.
The fix is to always alias the class instead of the method. The new
compiler will simply not let us get away with the old way any more, so
regressions after this are unlikely.
Issue #2528 (compiler upgrade)
Issue #2532 (static "this")
Change-Id: Id800d466e639c7cbcf4aa6fbb05114c772a2229f
In many places, the implicit type info was insufficient. For example,
document.createElement returns Element, but the actual return is
always a subclass of Element. In many cases, we need the compiler to
know that a specific subclass is in use, so that it can correctly
check our use of subclass-specific properties. Another common pattern
is confusion between Node and Element (which is a subclass of Node).
Almost all of the changes in the demo and UI are Element-related.
In some places, we referred to HTMLMediaElement, used in the Player
API, instead of the more specific HTMLVideoElement in use in our demo.
Since the demo uses video-specific properties, we must use the more
specific type.
Another case is the use of document.createEvent, which returns Event
according to the compiler, but in reality always returns a subclass,
like CustomEvent.
In one case in NetworkingEngine, correcting the type of an
AbortableOperation led to the discovery that we had been incorrectly
accessing a private method of that type.
In goog.Uri, there were several instances of "*" for a type, which the
newer compiler won't accept. These have all been corrected.
Finally, in some places, we had the wrong nullability on a type.
These were all caught by a compiler upgrade.
Issue #2528
Change-Id: I7f2d070e3da32fe9ff5f444315649f3cbdb5a4a5
This reflects changes in Google's policy on JavaScript license
headers, which should be smaller to avoid increasing the size of the
binary unnecessarily.
This also updates the company name from "Google, Inc" to "Google LLC".
Change-Id: I3f8b9ed3700b6351f43173d50c94d35c333e82b4
An empty list is still truthy, so the if statement should check for a
non-empty list instead. This ensures we don't try to append captions
when we play audio-only content.
Fixes#2187
Change-Id: I589a5508878ab28ad1ac69211f331ea829fa8b28
These are intermittently used and cannot be verified. Most of our
methods can throw a shaka.util.Error, so having it doesn't really add
anything. Plus, if we change a function to throw, we'd need to update
all their callers to ensure they have an accurate description of what
they throw; otherwise we can't trust the directive.
Change-Id: I520bd0fc4c33443e967bf5b103ca5aa9e3274884
This changes the network API to use BufferSource instead of ArrayBuffer,
which allows plugins to return a "view" on a buffer instead of the
whole buffer. This also adds some utilities for changing between
views and buffers.
Lastly this forbids the use of the "buffer" property of TypedArrays
since it doesn't work with partial "views". This audits and fixes the
usages of the "buffer" property to ensure correct usage.
It should be noted that both MSE and EME accept a BufferSource as input,
so we don't need to convert a "view" into an ArrayBuffer before passing
to it.
Change-Id: Iaa417773f8ce5304424e43c7372ce10ebf540d2a
Chrome has fixed their behavior involving endOfStream(), but both Tizen
and IE11 have the same broken behavior. Unfortunately these platforms
will likely never be fixed, so we need to keep this forever.
Fixes#1357
Change-Id: Id540170f8c8cb6e0ddf3a30bde5035016fa61df1
MediaSource throws if endOfStream() is called when
its readyState is equal to 'ended.' This happens if
endOfStream() has already been called.
This change ensures that we don't try to call endOfStream
on the MediaSource that has already marked as 'ended.'
Closes#2050.
Change-Id: I2672b297f0ebb7fd67afcf72ee52df6d5bacf761
This changes several classes to use the Destroyer class to handle
destroy(). This also changes the behavior to not ignore destruction
and instead throw a new error. This is more clear for callers and
ensures we propagate errors.
Change-Id: I756c085639558509c22e5c43d69ddf4acd28d46f
A recent refactoring caused a test failure on Chrome due to a Promise
getting rejected without a "catch" block and getting an unexpected
rejection error. This adds the expectation first so we don't get that
error.
This also removes the empty "catch" blocks since we use async/await,
which introduces another Promise, so the "catch" block won't suppress
the unhandled rejections like we want.
Change-Id: Ib217b4b374668d800a0eeb1a9b8069269bccf261
This sets a TextDisplayer to the native text displayer if native
controls were desired, or a custom UI text displayer otherwise.
This required adding methods to the player to switch out the current
text displayer if a new factory is provided, and a method to streaming
engine to reload the current text stream.
Change-Id: I85595e9ac9db0b60464ee7f20fa35855efb26424
This is a fully automated change. The linter will fail because the
extra indentation caused line-length errors. These won't be fixed
automatically. They are fixed in a follow-up to make this one fully
automated.
Change-Id: I4d8cf9c998985add2bcd24a81c8d65495668c4f3
With the new style rule, we cannot have two statements on the same line.
So we can no longer have an "if" on a single line and we cannot have
an arrow function with a body on the same line as when it is used.
This is mostly a manual change.
Change-Id: I2285202dd5ecbad764308bc725e6d317ff2ee7f0
This is an automated change to convert use of "function" functions
to arrow functions. This doesn't change all uses of bind() that
could be converted. This also doesn't remove all "function" functions.
Change-Id: I40ac7d086bcef947a1be083359c8fd1d4499a9c3
A coming update to the Google eslint config will require using "const"
over "let". This makes that one change to isolate the big changes.
Change-Id: I7d0974c3ae15c53cc45a6b07bf9f6586e2d34aca
Now iOS is supported for HLS through src=, so isBrowserSupported
needs to reflect that. This makes MediaSource optional for any
browser with src= playback of HLS.
This also fixes probeSupport() to reflect the capabilities of non-MSE
platforms such as iOS, which is critical for our demo app to be able
to show the correct set of playable assets.
Issue #997Closes#1857
Change-Id: Ic6e18587db90fff2b097a2038c16cc928e2b9438
This CL creates the "loaded with src=" node and its initial
implementation. Adding this node required some changed to the routing
logic as a new destination was created.
We will use src= when given mp4 content or media source is unavailable.
We detect mp4 content via the mimeType in |load| and/or the file
extension. When media source is not available on a platform, we fall
back to using src=. To do this, we check if media source is available
whenever |load| is called, and will route to the src= branch if media
source is not found.
To avoid pre-initializing media source (when it is not available) we
modify the |initializeMediaSource| flag in |attach| and |unload| when
media source is not available.
Doing this showed that we had inconsistent behaviour between |attach|
and |unload|. |attach| would default to initializing media source
whereas |unload| would not. This has been fixed.
Issue #816
Issue #997
Change-Id: I00599832b49c9079e273e65a4b827fee736479cc
To make the getBufferedInfo method simpler for when we have three
different ways of responding (not loaded, loaded with media source, and
loaded with src=), this changes media source to fill-in a buffered info
object rather than return one.
Issue #816
Issue #997
Change-Id: If9e4558ca324808a1b94e3c235f4bfb42a5df8ce
React Native introduces its own polyfill for window.URL.createObjectURL which is not compatible with Shaka. Encapsulating a reference to the original function inside of the MediaSourceEngine circumvents this issue.
Closes#1842
When CEA captions are extracted from video segments, the
presentationTimeOffset of the video segment should be applied to the
extracted captions, as well.
Fixes b/70902665
Change-Id: I446333a14b8b6374786ab594a579b6e18bc73ac1
Before, media source engine could be created with no text displayer.
However, player would always create media source engine with a
text displayer.
Just like with closed captions, this appeared to be a good time
to change it to be passed in the constructor and have media source
engine own it.
This now means that media source engine is responsible for destroying
it - lessening player's responsibilities.
Change-Id: I71cd1c28a3c22d3c765a3892fe74ae3f9abd1914
We often use the IDestroyable interface to provide us with a standardized
way to break internal references and tear-down objects, however many
objects don't need this to be async. Once using IDestroyable, everyone
must assume that you must be async.
This change introduces IReleasable, a sibling to IDestryable, which
provides the |release| method. IReleasable is the synchronous version of
IDestroyable.
This change converts EventManager from IDestroyable to IReleasable as
the first of many conversions.
Change-Id: Ic3e90e594abc1c7326eccbe2521eb71676b74a09
When we create a media source engine instance, we pass in
a closed captions parser. However a key subtlety was missing
from the parameter description; media source engine is taking
ownership over the parser.
Change-Id: Id49483ccac4cc1812c31ead9e82c09390ce2ed37
Álvaro Velad Galván
Philippe
theodab
Joey Parrish
Michelle Zhuo
Joey Parrish
Muhammad Haris
Sergei Gridasov
Sandra Lokshina
Jacob Trimble
Aaron Vaage
Kevin Scroggins