Commit Graph

208 Commits

Author SHA1 Message Date
Aaron Vaage 93bb88215b Test Progress Callback For Storage Remove
Added a new test to verify that the progress callback is called when content
is removed from storage.

Issue #1248

Change-Id: I60f1f1ab744d79214685c0060e56f4e21514948c
2018-05-10 19:37:24 +00:00
Aaron Vaage 03fc0bca99 Add Back Progress on Remove
Tracking progress when removing an asset was missing from the new storage
system. This CL adds that support back.

Issue #1248

Change-Id: Iab275cd75af817cfc34ee0888ddeea257b1ead56
2018-05-10 19:37:15 +00:00
Aaron Vaage 8040f78883 Integrate StorageMuxer and Player
This CL replaces the old storage system with the storage muxer system.
In addition to replacing the old system, this CL removes all the
code that was only used by the old system.

As of this change Storage Muxer support shaka player's V1, V2, and V2
indexeddb schemes and exposes a plug-in system for other offline storage
components.

Issue #1248
Change-Id: I1a4914477ad8db29fd0e7ad7de2f149b6497f67e
2018-05-09 18:41:21 +00:00
Michelle Zhuo 535de4db84 Refactor: Remove underscore from parameter names
Change-Id: Ie9e6fb59763f454f245175f23c6444f919ba8135
2018-05-08 19:21:51 +00:00
Aaron Vaage 272c4ba89e Create V1 Storage Cell For IndexedDB
Created the V1 storage cell for Shaka Player's original offline
data scheme.

Issue #1248

Change-Id: Ib564ec0670c56ba1b455f198133b118a21fd0a27
2018-04-25 20:18:03 +00:00
Sandra Lokshina 8065bd54a8 Change namespace from shakaExterns to shaka.externs
Change-Id: I16432351e2a266aa8fd175669aa27c44bfdffeae
2018-04-11 17:26:26 +00:00
Joey Parrish 79a6146b5f Revert change to Track and Stream structures
In the initial fix for #1013, we changed the name of the channelsCount
field in both the Track and Stream structures.  This would break
compatibility for applications.  So even though the new name was in
some ways preferable, we must revert the name to avoid more breaking
changes in v2.4.

Issue #1013

Change-Id: Ie8f3d211c42c8046039a3db9f0926c68ad1315d9
2018-04-10 23:45:53 +00:00
Joey Parrish 08af438d32 Track Download Progress With Bandwidth Estimates
Using variant bandwidth to estimate download progress. We use a constant
bandwidth estimate for audio, as most of a variant's bandwidth will be used
by video.

Since we normally don't have bandwidth for text, we use a constant bandwidth
estimate for text so that it can influence progress.

Even through we some times have the actual audio and video bandwidth (DASH)
we choose to use always used the combined amount so that we can have
consistent behaviour between manifest types.

Issue #1248

Change-Id: Ibffe95e2c8d141659855b54949a50ae84c51dded
2018-04-10 17:18:31 +00:00
Aaron Vaage 4a4050d223 Change Download Manager To Take Requests
Changed download manager to take |shakaExtern.Request| as input
instead of |shaka.media.SegmentReference| and |shaka.media.InitSegmentReference|
as it really does not need to know about those types.

Issue #1248

Change-Id: I797c437f4339cf670b5eddad14952b0526b72ea5
2018-04-09 18:13:28 +00:00
Michelle Zhuo a09ae24b85 Add audio channels count configuration
Added a preferred audio channel count to configuration, default set
to 2.
Choose the codec with the largest number of audio channels less than
or equal to the configured number of output channels. If this is not
possible, choose the smallest number of channels.

Closes #1013.
Issue #1071.

Change-Id: I6c6a956e33637cf34bd4bd79af563dc10a595e94
2018-04-06 20:51:28 +00:00
Aaron Vaage d86713cf0d Group Streams Before Downloading
To make it easier to understand, we collect all the streams within a
period using a map. This ensures that we will have a set of streams.

We then download each stream in the map. Since the map is indexed by
stream id, we can easily connect the variants and streams.

Issue #1248

Change-Id: I65ae01733b5d9dbf8b6d73c37f802320fffa03d1
2018-04-06 20:15:52 +00:00
Aaron Vaage 3e56d4ea1e Update Storage Unit Tests
Made some changes in how the unit tests were structured so
that it will be easier to identify problems with changes to
storage.

Issue #1248

Change-Id: I8331f18e9c9e360cf8057737ddfd71c7152a744b
2018-04-04 17:50:36 +00:00
Aaron Vaage 33547d071c Stubbing-in Mechanism & Cell Fields in Offline Uri
Adding "mechanism" and "cell" fields to offline uri. But since
they are not used anywhere, this change sets them to dummy values
so that all the constructors could be updated.

Moved the manifest reconstruction code into a instance class as
the mechanism and cell for the asset is needed in order to create
all the segment uris.

Change-Id: I2b5738805b17c4aa33d39b3166fddf1528aed489
2018-03-27 21:19:47 +00:00
Aaron Vaage e6302f5d73 Make Update Expiration Simpler
The only time we ever need to update a manifest value is when
we update the expiration time. To make it easier to support
updating expiration time across different version, only allow
that one value to be updated.

Change-Id: I621ec744019802edc65b059baa7f5cf0c476b0b1
2018-03-26 20:13:15 +00:00
Aaron Vaage 6ae49efc96 Move Offline Uri Functions in Class
The offline uri is going to soon contain more information (routing
information used by the storage muxer) so this change makes offline
uri a class so that all components can be accessed.

Change-Id: I5137f278e0dab059191d450876cab42745e4ed0b
2018-03-26 19:17:35 +00:00
Aaron Vaage a6c34b64b6 Create Storage Cell Tests for Memory Storage Cell
Made a copy of the storage cell tests for indexeddb but made them
work for the memory storage cell so show that we can expect the
same behavior from them.

This is to give us an extra level of confidence when we use the
memory storage cells in place of the indexeddb storage cells in
our tests.

Issue: #1248
Change-Id: I96eaf57a9d808b4c7e07f97ac2713318a889fc24
2018-03-22 15:46:41 +00:00
Aaron Vaage 798ca28131 Creating Storage Cell Tests
Issue: #1248
Change-Id: I2522ce02e8a9bdb8c6bea9f77adf91393b0265b6
2018-03-22 15:46:36 +00:00
Joey Parrish 3e8a83584d Update the compiler to v20180204
This unblocks conversion of tests to async.  We ran into issues in
compiler version v20171203 which have already been fixed upstream.

We are not, however, updating the linter.  The matching linter version
reports errors about our tests not using goog.require, which we do on
purpose.  We will wait to update the linter until we have resolved
that.

This change also fixes issues discovered by the upgraded compiler:
  1. Missing property definition in playhead_unit
  2. Wrong type name in storage_unit
  3. Wrong type name in FakeManifestParser

Change-Id: I4ba3f43e6a6471e182a69c85cfc619de3380c559
2018-03-21 13:57:46 -07:00
Jacob Trimble 064089152b Remove spacing around object definitions.
This is an automated fix to be inline with Google style guides, this
was created with eslint's --fix option.

Change-Id: I860eecbc8152603e730aa17a1393f16d26b3b6fc
2018-02-21 13:35:00 -08:00
Jacob Trimble 624acc66b8 Add curly braces to all blocks.
Google style guide requires adding curly braces to all block statements
even if it is only has one line.  This fixes it by using eslint's
--fix flag followed by running clang-format to reformat the change.

Change-Id: Idc086c2aa8c02df5ef8b2140a11bfb9128eeb4bd
2018-02-21 11:23:34 -08:00
Jacob Trimble cfbbe485e9 Use seek range in Playhead.
Rather than using the availability window, Playhead should use the
seek range to restrict the playhead's position.

Closes #1224

Change-Id: I8612bfafb53bbb214e32aae2d71af52d748a3aee
2018-02-20 23:42:12 +00:00
Jacob Trimble 4004bd42fd Convert 'var' to 'let'/'const' (7 of 9).
This is part of a change to convert all usages of 'var' with either
'let' or 'const'.  This takes a conservative approach for 'const' where
it will only be used for aliases and storing the "original" values in
tests.

Change-Id: Ib137fbfde50f1330c74c040fc062aa8e2c6d1e57
2018-02-15 21:16:33 +00:00
Joey Parrish 2f55d2a3bd Use AbortableOperation in networking
This uses AbortableOperation in all networking, from the scheme
plugins all the way to the request interface.

This also updates all default scheme plugins, docs, and sample code.

Backward compatibility is provided for scheme plugins and the
request API in NetworkingEngine.  This compatibility will be
removed in v2.5.

Two cancelation-related tests have been disabled in
player_integration until the new abort interface has been adopted
in the manifest parsers.

Issue #829

Change-Id: I91c8e6efe97798d111e8ddca5655cddc1f6bcbf3
2018-01-29 19:23:47 +00:00
Aaron Vaage 4147e77f05 Add Uris for Downloaded Content To DB Error
When DBEngine aborts because of an upgrade, include a list of the
original uris for all downloaded content.

This will allow apps to re-download the content if they choose to.

Change-Id: Ic2029d935dd8da60c0d502cd0cdbb08b38bbbe76
2018-01-26 11:07:25 -08:00
Aaron Vaage 9e89a90d3c Create DBUpgrade Fail Tests
Create tests to show that DBEngine should not open unless it has
the correct version.

Change-Id: I410e1c5d35a3a7b9e91afdd247f06ffac8d5a86d
2018-01-26 11:05:28 -08:00
Aaron Vaage 965dc8e100 Removed DBUpgrade
This removed all of DBUpgrade (lib and test) as it was not accomplishing
its requirements.

Now instead, if DBEngine detects that there needs to be an upgrade, it will
abort the upgrade and message the app with |UNSUPPORTED_UPGRADE_REQUEST|.

Change-Id: If65bc013b0482c9b0c6e71e644e9132584984414
2018-01-26 10:59:25 -08:00
Theodore Abshire be5c80eb6d Standardize feature support checks in tests
This replaces checkWhatever type functions, which are called at the
start of a test, with checkAndRun functions, which are a wrapper
around the test method. This change standardizes our tests, since
checkAndRun functions are used in the db engine tests but not
elsewhere.

Change-Id: I74c7ac6836bbaab15b57fd480dbd6105ddc5ee60
2018-01-26 16:55:44 +00:00
Joey Parrish 529e97fbf2 [ES6] Configure eslint with Google style checks
Some style rules are temporarily overridden until we can get in
compliance.

Issue #1157

Change-Id: Ie27be6ec6632b7786a1f9ebc32911ba9dcd42599
2018-01-18 00:01:05 +00:00
Joey Parrish a63352c8f3 [ES6] Enable stricter checks in eslint
This digests and organizes many of the automatic settings in eslint,
and changes several of them to be more strict.  This also fixes the
following errors:
  - array-callback-return
  - no-catch-shadow
  - no-multi-spaces
  - no-new
  - no-throw-literal
  - no-useless-call
  - no-useless-concat
  - no-useless-return

Several checks have been organized into a group of checks we should
use, but need more time to implement and fix.  Some other checks have
been delegated to the Closure compiler, which can more precisely
whitelist exceptions.

Issue #1157

Change-Id: I8fe4966959e08050f8159e6a1fee161e7d71177e
2018-01-17 21:56:41 +00:00
Aaron Vaage 624ab03e48 Stop Edge Failure During Upgrade
Edge does not like it when we delete the old stores in the
database when upgrading. Since this can't seem to be worked
around, instead of delete the store, just clear it.

This will leave two empty stores in the database, but since
they will be empty, they should not waste any space.

Change-Id: I11d5b8b105b34277d92e9c1f7f83b36fc6171960
2017-12-21 21:11:36 +00:00
Aaron Vaage c78e4a258b Close Database After Upgrade Test
On Safari it seems that deleting a database that was opened in a
previous test causes the delete call to hang. This makes sure
we close the database after each upgrade test.

Change-Id: Ie98a13515bd071cef044ee862f86d8a1a1050548
2017-12-19 09:56:36 -08:00
Aaron Vaage 1cf0a38843 Support Upgrading from Version 1 to Version 2
Added a DB Upgrade path for converting content from version 1
to the version 2 format.

Issue #1047

Change-Id: Id8dc626d8289b08ca300c40b137173c0c6ec9d35
2017-12-18 19:04:04 +00:00
Aaron Vaage 967f339934 Mark Version 2 of DB Engine
externs/shaka/offline.js now represents the data types for Shaka Player
Offline V2. All offline shaka player code uses the new version and all
V1 and transitional code has been removed.

Issue #1047

Change-Id: Ia43f8d8d11426e823629e5fcd27c4e1e0ce400d3
2017-12-16 00:11:04 +00:00
Joey Parrish 4ce963415e Fix offline storage with temporary license issues
This change:
 - Fixes the use of temporary licenses with manifests without init data
 - Re-enables an integration test which was accidentally disabled
 - Makes that test use a manifest without init data, to cover #1159
 - Makes the test less flaky
 - Reduces the minBufferTime of offline manifests, which both speeds
   up load of offline content and makes the test faster / less flaky

Closes #1159

Change-Id: I3342e5112564a607f16f5d10c457a032443c57cb
2017-12-14 21:36:08 +00:00
Aaron Vaage c927300045 Store keys instead of uris in DB objects
Instead of storing uris for SegmentDataDB in SegmentDB, store the
actual key so that we don't need to convert back-and-forth so
often.

Issue #1047

Change-Id: I2f6d1bc1e58abd7a210097dea757d1e2fc9d4cac
2017-12-14 21:34:49 +00:00
Aaron Vaage 46685fbd15 Moving Offline Uri
Moving offline uri code to avoid an upcoming circular dependency in
the upgrade code.

Change-Id: I952386f17c6d4e7806d97e2f540887b915f68de7
2017-12-14 00:00:23 +00:00
Aaron Vaage e7c5a3a8aa Fixed DBEngine Test Failures
It looks like some browsers don't do equality checks on ArrayBuffers
the same way as others. So to fix this, we checking that segments
are the same, convert each ArrayBuffer into an Uint8Array.

Change-Id: Iceb6e39e326ced4e86c52968e364bf927116c250
2017-12-13 21:44:55 +00:00
Aaron Vaage 3399292d49 Changed Storage Remove API
Changed the storage remove API to use the offline uri instead of
the stored content as that is all that is needed to remove the
content.

Updated the demo app to remove using the Uri and not stored content.

Change-Id: I72a1049838759452eb16c78c8f1f7f4170e4e841
2017-12-12 22:12:56 +00:00
Aaron Vaage dff382b9a3 Remove reserve id from StorageEngine
Removed the reserve id from storage engine. Now adding an item to a
storage engine will return a promise that resolves with the id it
was stored under.

Change-Id: Id4d2c4c51a359593f1308aeb5fe580ab460ebeb8
2017-12-12 13:28:33 -08:00
Aaron Vaage 8f45f7939d Changed Stored Content to not always need an offline uri
Before, stored content always needed an offline uri. As the id
will soon not be available before saving to storage, stored
content needed a way to represent content that was being
downloaded.

This change allows the offline uri to be null to represent content
that is being downloaded. It also allowed stored content to be
created from a Manifest (for content to be downloaded) and to be
created from a ManfiestDB (for content already downloaded).

Change-Id: I3039069995f0b061112ed0c636a53ceef5673389
2017-12-11 23:21:12 +00:00
Aaron Vaage 6a03ceb30f Isolate Progress Code
Isolated all progress code into functions in Storage to help abstract
progress from the storage logic.

Change-Id: I7a72c25dbd126cb4a5e682763fb1e46afa0703ef
2017-12-08 21:17:20 +00:00
Aaron Vaage 834d960f88 Use Manifest Duration for Offline Manifest Duration
Use the duration from the original manifest as the duration for the
offline manifest.

Change-Id: I1b31888ecf25126e4a4794f73ea3aeaf101e5820
2017-12-07 21:38:43 +00:00
Aaron Vaage 540e533e69 Correct comment in DBEngine Tests
The code in the DBEngine tests were updated but the comment explaining the test
did not get updated. This change corrects the comment to match what the test
is doing.

Change-Id: I56b290c375f15eb2fba70634d1c7d4ab841af1fe
2017-12-07 18:44:00 +00:00
Joey Parrish 8c80b6cefa Wait for close(), but timeout after 1s
An improved workaround for https://crbug.com/690583 and #1093, on top
of PR #1168.

If we don't wait at all, we end up trying to use the same session ID
too quickly, which causes a playback failure.

Change-Id: I8c9d1a0f09432054b862e033b80b0c2f47739c74
2017-12-05 16:47:54 -08:00
Aaron Vaage 2f47e6d769 Moved Legacy Variant Ids Resolution into DBEngine
Before we added variants, there was no variant ids field on
StreamDB. To support legacy downloads, we created variants
for streams with no variantIds.

This change moves that conversion into DBEngine so that outside
DBEngine, streams will always have variantIds.

Change-Id: I85d050dac0b5f82fe00947380587dd7af4401dee
2017-12-06 00:39:07 +00:00
Aaron Vaage 0e43b62f04 Fixed using wrong DBEngine name
The offline integration tests were writing to one DB Engine instance
but reading from another. This meant that it could never find what it
wrote.

Change-Id: I1d641e18401e27d906d437c70d7691f9ab448ff3
2017-12-06 00:11:20 +00:00
Aaron Vaage b98ce99f08 Add Type Structure to Storage Engine
Changed the Storage Engine interface to know about the types that it
is expected to store. This allow the implementation to do more internally
as the scope of the object has been limited.

Change-Id: I388e2bde5e7e3f99e8b28bd16e08a8f6d5fc6453
2017-12-04 22:41:23 +00:00
Aaron Vaage b68623180c Simplified Offline Schema
Before, segments required knowledge of the manifest and stream to
be created but that information was never used. This change removes
that information from the segments and breaks the circular dependency
between manifests, periods, and segments.

Change-Id: I94064220dd8e3693ff69f5c9100ec1b91aa3c34d
2017-11-27 21:17:07 +00:00
Joey Parrish e2aef50912 Disallow unknown "this"
This change enables an additional conformance check in the compiler
which prevents us from committing code where the compiler fails to
infer what "this" means.

None of these changes constituted actual bugs, just inference failures
in the compiler.

Issue #1130

Change-Id: If9b18203768e197258042a9b4339530f371ed831
2017-11-15 11:35:56 -08:00
Aaron Vaage d90821a1a5 Isolated Manifest Creation in Storage Tests
For storage tests, this change isolates the creation logic
so that the data created can more accurately reflect real data.

Change-Id: I8b8a2002c29e5c9d3a41f4de0819ad923b6b14e5
2017-11-09 16:23:52 -08:00