Commit Graph

280 Commits

Author SHA1 Message Date
Joey Parrish 53b6e40c6d fix(offline): Fix progress callbacks in release mode
This works around an apparent compiler bug that caused the invocation
of our progress callbacks to be removed from the compiled output.  To
test for this, the existing progress tests now use the compiled
library.

Closes #2652

Change-Id: I5698cfe0a833696e9cd5c8f8851698e1e66ef901
2020-06-16 00:09:39 +00:00
Joey Parrish f539147d48 fix: Correct license headers in compiled output
This fixes all the license headers in the main library, which corrects
the appearance of the main license in the compiled output.

It seems that the `!` in the header forces the compiler to keep it in
the output.  I believe older compiler releases did this purely based
on `@license`.

Issue #2638

Change-Id: I7f0e918caad10c9af689c9d07672b7fe9be7b2f3
2020-06-09 16:05:09 -07:00
Theodore Abshire 8d657140f1 Made ManifestParser.PlayerInterface.filter async.
This is a change in preparation for the adoption of the
MediaCapabilities API.

Related to #1391

Change-Id: If0988aedafe15aa3be794fb977fd8e04e91ca10e
2020-06-04 17:42:27 -07:00
Joey Parrish 5d2dc2daaf Update deprecation timelines (v3.1 => v4.0)
Since we are getting strict about semantic versioning, we can't remove
features in v3.1.  Any backward compatibility we offer will be
maintained until v4.0.

The exception is explicit IE 11 support, which will still be removed
in v3.1.

To help us follow the rules for removal, the Deprecate utility no
longer accepts a minor version argument.

Change-Id: I4dd94a6084e4ed72eeec410eb9aa0ad974d8dac8
2020-06-01 15:57:31 -07:00
Joey Parrish aba5e2583c Rename v2.6 to v3.0 and v2.7 to v3.1
We have decided to bump the major version number instead of the minor
number, based primarily on the fact that this release breaks
compatibility with our previous manifest structure.

Change-Id: I67e4c8267c6e103cfc7278e09daac186ae5cbbc6
2020-05-29 17:50:22 +00:00
Joey Parrish 1076b78465 Unfork Closure base and update the compiler
The Closure Library's base.js, the Closure Compiler, and the Closure
deps-writer are all now loaded via NPM instead of commiting them
directly to the repo.  This also updates both the library and compiler
to the latest version: 20200406.

We still have a fork of the Closure Library's URI parser.  The latest
upstream version of that has too many dependencies on the rest of the
library to import directly from NPM.

Some internals of the build system have been refactored, and the
"complete" set of files in the build system now includes third_party.

Our forked URI parser does not pass lint checks yet, so the linter
does not run over third_party yet.

A couple of overly-severe sets of compiler checks have been disabled,
since even the latest Closure Library's base.js doesn't pass them.

The script-loader in load.js had to be updated for compatibility with
the new Closure Library.  If you don't return "true" now, Closure's
base.js will stop loading subsequent source files.

Some local externs that we had written are now available from the
compiler, so our local copies have been deleted.

A few type-related changes have been made as well, removing casts that
were necessary with the old compiler, but not necessary with the new
one.

Finally, this corrects some type issues in the tests using the new
"typeof" annotation from the compiler.  This allows us to type a
variable as a class defined elsewhere.  For example, after loading the
compiled library, we can reference compiledShaka.Player, which has the
type "typeof shaka.Player".  The compiler can then type-check all uses
of it in the tests.

Closes #2528 (bad polyfill code generated by the old compiler)

Change-Id: I62ec61e82d4edf342b2c576c2d4f89f11562ee65
2020-04-30 19:32:30 -07:00
Joey Parrish 7917348fbe Correct out-of-date comment in lib/offline/
Change-Id: Id3087eeb897a7d4184d5a2b03f469247e1a3f9d1
2020-04-30 17:30:44 +00:00
Joey Parrish a65ef9983b Remove partial exports, broken in new compiler
The old compiler would let us export a static method on a class
without exporting the whole class and its constructor.  The new
compiler silently ignores `@export` for any methods of a non-exported
class.

This means that for the latest Closure Compiler, we must export any
class with exported static methods.  In some cases, these are
non-utility classes with constructors we'd rather not export, but the
constructor is implicitly exported by exporting the class itself.

This was initially caught by the integration tests.  The error wasn't
especially helpful, so I added a try/catch to loadShaka that makes the
error more apparent:
  TypeError: Cannot read property 'registerParserByMime' of undefined

To make sure we do not make this mistake again, I've added a check to
the extern generator, which was already able to detect these types of
classes.  I don't know a compiler-based way to do it, since the
compiler silently ignores the export annotations in these cases.

Issue #2528

Change-Id: I797c75a8098b0bb3cf837588569f878253dec2cf
2020-04-30 17:30:36 +00:00
Joey Parrish 342d35f4f9 Fix issues with nullability of numbers
Various issues with the nullability of number types led to various
fixes, including:
 - defaulting a nullable number to 0 to avoid propagating a null value
   through calculations
 - adding an assertion or runtime check that something is not null
 - moving an existing null check to before the calculation
 - returning early on null during an iteration
 - changing a nullable number to non-nullable
 - defaulting to NaN instead of null

These issues were caught by a compiler upgrade.

Issue #2528

Change-Id: I86d516c74a42ee3624c33d7513d2d4c76d3ea589
2020-04-30 16:00:00 +00:00
Joey Parrish 3ac75b04ab Fix IndexedDB types
The built-in externs for IndexedDB in Closure Compiler are missing
some type info.  This overrides them to add what's missing.

This also tweaks type information in a few places in lib/offline/ to
match.

This was caught by a compiler upgrade.

Issue #2528

Change-Id: I97096656f53b426067219e2d4e3aa16f32b87188
2020-04-30 15:59:50 +00:00
Joey Parrish 13202cecae Fix storage cell constructor args
When we changed the constructors for the V2 cells with the
introduction of V5 format, these call sites didn't get updated.  This
doesn't seem to be a user-facing bug, and did not affect any releases.

This was caught by a compiler upgrade.

Issue #2528

Change-Id: I77228d17631ef44a6b85a81b88054790461617bc
2020-04-28 20:57:59 +00:00
Theodore Abshire 087c1ba247 Allow offline downloads to be aborted.
You can either abort downloads one at a time, using the
AbortableOperation interface, or you can implicitly abort them all at
once by destroying the storage instance.

Closes #2417
Closes #1362
Issue #1301

Change-Id: I0ba102e5bf60a063f0e2f6ecd3f135445226996f
2020-04-24 20:43:54 +00:00
Álvaro Velad Galván 2807971e4e Add creationTime to offline assets (#2406) 2020-04-10 09:58:45 -07:00
Theodore Abshire a2a988d876 Allow for multiple concurrent storage operations.
In the process, this removes the in-progress flag, which should fix
issues where the storage instance can't be re-used after an error.

Closes #1432
Closes #2432

Change-Id: I51018e170fb9ab262b5c70125a03d979c8ccfb08
2020-04-10 04:33:11 +00:00
Joey Parrish ba5d062be6 Fix missing segments in offline storage
In Change-Id If885e828b4761528e40abdbc601a11cf13849a1e, I fixed a
memory leak, but broke offline storage.

The fix for the memory leak was to release SegmentIndexes and
SegmentReferences when the DASH parser stops, but offline storage was
stopping the parser before making use of the parsed manifest.

This changes offline storage to keep the parser alive until after the
storage operation is complete.

This bug did not affect any release versions.

Change-Id: I88b86d250b4407cc0740d35eaf4a7ef3d5a67798
2020-04-09 17:40:44 -07:00
Joey Parrish e24fec4b08 Flatten periods
This creates a new utility used by DashParser and old offline DB
formats to combine Streams across Periods.  This allows multi-Period
DASH content to be played without period-specific structures in the
manifest format, StreamingEngine, or Player.  This also makes the
Tracks stable across Periods.

Closes #1339 (flatten periods)
Closes #1698 (rapid period transitions issue)
Closes #856 (audio change causes bitrate change)

Change-Id: Icb04c8e47e36eacf7ac024a5063130d85a115e54
2020-04-09 19:22:16 +00:00
Joey Parrish c84676c7ee Make offline manifest conversion async
This will be used in period-flattening, where we will need to create
segment indexes before combining Streams.  Although the operation
would be synchronous for DB types, the general interface will need to
be async.

Issue #1339

Change-Id: Ibedb267118462cbaf3ca599ab6a67c8f355965f6
2020-04-09 19:22:16 +00:00
Joey Parrish 6e3dfa40ba Deduplicate key IDs with Set
When period-flattening combines Streams, key ID arrays would get very
long with duplicates.

This changes keyIds in the manifest and offline structures from Array
to Set.

Issue #1339

Change-Id: I003d23e567efafa771ecd2ad597900181604ad18
2020-04-09 19:22:16 +00:00
Joey Parrish c4a5513ca5 Rename offline StreamDB.contentType to type
This brings the field name in line with the Stream objects from the
manifest types, allowing for more general processing of Stream and
StreamDB for period-flattening.

Issue #1339

Change-Id: Ic5d0e5d69a6560d475a19f5d3ecb0b1b40b8d271
2020-04-09 19:22:16 +00:00
Joey Parrish e8ac57f36c Move drmInfos array to Stream
Period-flattening will concatenate Stream objects, so this information
should be available per-Stream instead of at the Variant level.

Issue #1339

Change-Id: I96195fea48cab1e4a349b2ab0b16064a443e928a
2020-04-09 19:22:16 +00:00
Joey Parrish 99de217c23 Remove periods from manifest structure
This removes periods from the internal manifest structure and cleans
up code and tests accordingly.  This leaves us unable to play
multi-period DASH & offline streams until the main period-flattening
algorithm is completed in shaka.util.Periods.

Three test cases have been disabled for the moment.

Multi-period playback will be restored in a smaller, more focused
follow-up commit, with disabled tests re-enabled.

Issue #1339 (flatten periods)
Issue #1698 (rapid period transitions issue)
Issue #856 (audio change causes bitrate change)
Closes #892 (refactor StreamingEngine)

Change-Id: I0cbf3b56bfdb51add15229df323b902f0b2e643a
2020-04-09 19:22:16 +00:00
Theodore Abshire ae568e0649 Fix offline picking high-bandwidth codecs.
When playing an asset, the player will first choose a codec, based
on the lowest average bandwidth.
However, the storage mechanism did not have any such filtering;
instead, all it did (if using the default track filtering plugin) was
pick the middle-bandwidth audio track. This lead to it sometimes making
different codec decisions than the player would have.
This CL breaks the codec filtering logic out into a helper function, and
calls it both for playback and storage.

It also adds headers comments to some functions in the demo page.

Closes #2390

Change-Id: I09f1a24753121afd0e7500577c0f64ccffe5a4d1
2020-04-02 22:46:10 +00:00
Joey Parrish 1ab3f9c6db Remove position from SegmentReference
As part of Period-flattening, I'm trying to remove our dependence on
the "position" field of SegmentReference.  With that eliminated, we
can more easily concatenate Arrays of SegmentReferences without
modifying them.

 - Make SegmentIndex iterable
 - Add specialized seek() and current() methods to SegmentIterator
 - Remove position from SegmentReference
 - Make positions in SegmentIndex API stable without field in
   reference
 - Remove brittle hard-coded positions in tests (except SegmentIndex
   tests, where they would be hard to avoid in testing methods
   separately)
 - Use SegmentIterator in StreamingEngine to track the next segment
   between switches

Issue #892 (refactor StreamingEngine)
Issue #1339 (period flattening)

Change-Id: I666cc21249c34ee6cbc138a59109d9f1159fa127
2020-03-23 10:09:35 -07:00
Joey Parrish 6202b9372e Revert "Move next-segment tracking to SegmentIndex"
This reverts commit 235e4e11ad.

The effort to remove SegmentReference's position field will be handled
in a different way.

Issue #892 (refactor StreamingEngine)
Issue #1339 (period flattening)

Change-Id: I62b115137abc89f498b30467e574b0401dcad05d
2020-03-23 10:09:35 -07:00
michellezhuo c82ff51d1f Add a presentationDelay config for HLS live
Closes #2373

Change-Id: I627bbd226ed9cab9bc476e2c2a3257d2e2dbc0e9
2020-03-11 18:41:15 +00:00
Joey Parrish 235e4e11ad Move next-segment tracking to SegmentIndex
As part of Period-flattening, I'm trying to reduce our dependence on
the "position" field of SegmentReference.  If it can be eliminated, we
can more easily concatenate Arrays of SegmentReferences without
modifying them.

SegmentIndex can now track the last reference you asked for and
iterate through the list of references.  This means we don't need the
"position" field of SegmentReference, which means we don't need to
know positions in advance or globally.  StreamingEngine will no longer
use position to request segments.

The old methods find(time):position and get(position):SegmentReference
have been replaced with seek(time), current(), and next(), all of
which return a SegmentReference and maintain an internal pointer to
the "current" reference.  Care has been taken to maintain that pointer
during the evict() and fit() operations.  Recent changes to merge()
made sure that the pointer does not need to change during that
operation.

All test updates are related to the SegmentIndex API change, not
changing expectations or behavior.

Issue #892 (refactor StreamingEngine)
Issue #1339 (period flattening)

Change-Id: I1682dcc2dd625c6e390711538e46d31e6eb6cea8
2020-03-11 18:14:14 +00:00
Jacob Trimble c49f24b31b Remove remaining deprecated features for v2.6.
Change-Id: I2199e3c960fe6319eb9ab342291dfd9a62d9090e
2020-02-24 15:57:05 -08:00
Álvaro Velad Galván b2672e305d Fix download contents with Widevine and Playready in the manifest. (#2400) 2020-02-21 13:12:46 -08:00
Jacob Trimble bc45564721 Remove deprecated ability to pass factories to load.
Player.load and Storage.store used to accept the manifest parser factory
directly.  But now they should only accept the MIME type string.  This
removes the deprecated functionality in preparation for v2.6.

Change-Id: I1b4c5a4a9f0b6edbea909d18111ddc87a39da331
2020-02-20 23:44:51 +00:00
Jacob Trimble 3f63021a2f Avoid using "new" with factories.
Instead of having the "factories" use "new" to construct them, now they
will be plain functions.

Closes #1521

Change-Id: Ia6151ad679a78a5c6db128d43094c82add0af348
2020-02-19 09:57:51 -08:00
Joey Parrish b1e13fa658 Fix async tracks callback
PR #2387 introduced a bug. It awaited the track selection callback,
but that was done in a loop, the results of which were not awaited.
This would cause storage to continue before the complete list of
tracks was available.

Closes #2383

Change-Id: I18a429cf0f40b829020c520c622ffdae8b12622e
2020-02-14 15:13:03 -08:00
Joey Parrish dbc9ea9772 Add externs for V1 database format
This will make it easier to ensure conversions are done correctly,
which will be increasingly important as we change the database format
as part of #1339 (period flattening).

Change-Id: I1114b1a9a0d341589f7f1026ec23ad93af38a6b0
2020-02-14 19:59:19 +00:00
Joey Parrish 0406bc0da9 Fix duplicate stream storage in offline content
Since v2.5.0, we have been storing duplicate streams for multi-period
offline content.  For example, instead of storing one stream per
period, a 3-period manifest would have 3 streams per period.  The
segment data would also be duplicated, leading to 3x the storage used
for a 3-period manifest.

This fixes the error for future content and ensures that the correct
number of variants are still shown when this broken content is loaded.

This change also tweaks the frameRate field in the existing database
dumps, which for some reason contained the wrong value for the
existing v1-v3 dumps.  This must have been a bug we fixed already.  To
make the expected results consistent across dumps, the frameRate value
in the old dumps has been updated.

Closes #2389

Change-Id: Ibf7db7543f25ad23cecd12efad1eb039630e381c
2020-02-14 19:59:12 +00:00
Joey Parrish d16865b304 Add tests for v2 and v3 offline content
Before this, we only had tests in place for loading and converting the
v1 format for offline storage.  We had database dumps already for v2
and v3, but no tests exercising them.  This adds some basic tests and
fixes a bug in the loading of expiration times for v2 and v3,
equivalent to what we already had for v1.

Note that here, v1, v2, and v3 do not refer to major versions of Shaka
Player.  Instead, they refer to IndexedDB storage formats used in v2.x
versions of Shaka Player.

Change-Id: If32d1f1131b935eb191101463c4152317e711c5c
2020-02-13 20:30:28 +00:00
Álvaro Velad Galván 059e2a9a08 Make trackSelectionCallback async (#2387)
Resolves #2383
2020-02-12 13:11:17 -08:00
Joey Parrish 4ffcbc6188 Fix pixelAspectRatio types for backward compatibility
For backward compatibility, the new pixelAspectRatio field in Stream
should be string|undefined rather than being nullable.  This ensures
that manifest parser plugins that do not output that field will still
compile against the externs.

PR #2294

Change-Id: I1aae03994a213c8ce52dc64e8a34bf179045f4fb
2020-01-15 19:04:14 +00:00
Jacob Trimble 011749e95f Standardize argument comments.
This changes the eslint rule to enforce a strict pattern for the
argument comments.  The comment must appear before the argument and
must be /* foo= */.  This still ignores line comments.

Change-Id: I3afb01c65e1088eda13facb3aeeaa7595a2f5aee
2020-01-06 19:40:52 +00:00
Álvaro Velad Galván bdca05ff4b Add pixelAspectRatio property from DASH (#2294) 2020-01-06 10:56:28 -08:00
Joey Parrish 41a8664dba Make segment references independent of periods
This is a step along the way to flattening periods, and also
simplifies a few small things in StreamingEngine and tests.

Issue #1339 (flatten periods)
Issue #892 (refactor StreamingEngine)

Change-Id: Ie17cd5e15ed6ec9290a918c3a69c05c74581e0fc
2020-01-02 19:03:18 +00:00
Joey Parrish c13830e535 Augment SegmentReference with offset and window
SegmentReference used to have presentationTimeOffset, which,
subtracted from the period start time, was then _added_ to the
timestamps in the segment by MediaSource.

Now, SegmentReference has a timestampOffset field, which is exactly
what MediaSource's timestampOffset field is set to on the SourceBuffer
before this segment is appended.  For DASH, this is periodStart minus
presentationTimeOffset.

This also adds append window start & end times to the
SegmentReference.  Now segments can be appended to SourceBuffers
without reference to the period.

Note that start & end times of the SegmentReference in each segment
index are still relative to the period.  This will change in a
follow-up.

Issue #1339 (flatten periods)
Issue #892 (refactor StreamingEngine)

Change-Id: I9d54eb2b529ec643c9475b8e9d218c3e2e826a26
2019-12-19 11:35:14 -08:00
Álvaro Velad Galván 02fa8c6876 Add audioSamplingRate property (#2290) 2019-12-10 18:00:16 -08:00
Joey Parrish 64896d70b0 Use shorter license header
This reflects changes in Google's policy on JavaScript license
headers, which should be smaller to avoid increasing the size of the
binary unnecessarily.

This also updates the company name from "Google, Inc" to "Google LLC".

Change-Id: I3f8b9ed3700b6351f43173d50c94d35c333e82b4
2019-11-22 18:18:36 +00:00
Jacob Trimble c56fe7db19 Fix and cleanup offline without init data.
The PR to add support for pulling EME init data from the segments was
incorrect and had some bugs.  This stops re-initializing DrmEngine since
this isn't needed and could cause problems with the offline licenses.
This also just uses the Pssh helper once to parse and extract the
init data.  Lastly this throws the old error if we don't have any
sessions when we are done;  if we try to store WebM content, we should
still throw if there is no init data in the manifest.

Issue #1531

Change-Id: Ie524abda993a076397e233008709f11a2e68a8f8
2019-11-05 23:33:54 +00:00
Álvaro Velad Galván 5eaca0aebd Offline license support for manifests without inline init data (#2164)
Closes #1531
2019-10-17 14:53:34 -07:00
Jacob Trimble 596c80a949 Refactor handling of ArrayBuffer.
This changes the network API to use BufferSource instead of ArrayBuffer,
which allows plugins to return a "view" on a buffer instead of the
whole buffer.  This also adds some utilities for changing between
views and buffers.

Lastly this forbids the use of the "buffer" property of TypedArrays
since it doesn't work with partial "views".  This audits and fixes the
usages of the "buffer" property to ensure correct usage.

It should be noted that both MSE and EME accept a BufferSource as input,
so we don't need to convert a "view" into an ArrayBuffer before passing
to it.

Change-Id: Iaa417773f8ce5304424e43c7372ce10ebf540d2a
2019-08-20 20:17:33 +00:00
Jacob Trimble 79aae2aeaf Fix protected member names.
According to the style guide, protected members should not end with
an underscore.  This renames them to fix it.  This also marks some
members as private that aren't used outside the class.  Lastly, this
fixes a possible bug with the use of "this".

Issue #1553

Change-Id: I6c7ecdcd228ec4e4aded4af52e2262d0bf5beaf4
2019-08-13 19:08:03 +00:00
Joey Parrish 625938e2f9 Revert offline PSSH support for manifest that are missing inline init data
This reverts commit 6e5a0797de, #2042.

The PR as it landed broke some test cases and functionality for storing persistent licenses, and it doesn't seem to work even for its intended use case.
2019-08-02 16:56:24 -07:00
Joey Parrish 7fe97b841b Move fields to SegmentReference
To prepare for flattening out the manifest structure to remove
periods, this change moves initSegmentReference and
presentationTimeOffset fields into the SegmentReference object.  This
way, the segments on either side of a period transition or HLS
discontinuity can have different offsets or init segments, eventually
allowing us to create a single array of SegmentReferences for
multi-period content.

Issue #1339

Change-Id: Ic7eff0483789644881247ecf8044c5fb6a48f0e6
2019-08-01 11:04:36 -07:00
Ante Wall 6e5a0797de feature: Offline PSSH support for manifest that are missing inline init data (#2042)
* fix: Reference issue with SegmentReference createUris

* test: Added Segment GetUris Test for timeline

* feat: Get PSSH from segements if not initdata exists

* feat: Get PSSH from segements if not initdata exists

* fix: JSDoc comments and moved drmEngine update to storage

* fix: Remove comments

* fix: Remove unused import

* fix: Lint erros and dedup initData code

* fix: remove no init data test and add check for use persistent license
2019-07-26 11:01:05 -07:00
Joey Parrish d6d9fa9250 Refactor IndexedDB storage cells into a base class
To prepare for an upcoming change to the format of the data stored in
IndexedDB, refactor the storage cells to remove common functionality.
This will make it easier to add a third cell type.

The format will change as we move fields in the manifest structure and
ultimately flatten out periods.

Issue #1339

Change-Id: Ibaeb35596180b2e2362b4d53734019599d6fff77
2019-07-09 16:36:25 +00:00