This adds our first screenshot-based layout tests and the infrastructure to use WebDriver for screenshots through Karma. This new kind of test will be skipped in any non-WebDriver context. There are many pieces to this system. First, we update the Karma WebDriver launcher to a newly-released version that lets us access the client spec object from the launcher. Second, we build a Karma middleware plugin to respond to HTTP requests from the tests. We handle /screenshot/isSupported and return a bool so tests can be skipped on non-WebDriver launchers. We also handle /screenshot/diff to take the screenshot and compare it to a known-good version. The screenshot is a full-page screenshot, since element screenshots don't work consistently across all the browsers in our test lab. The screenshot is then cropped to a rectangle specified in the request. This rectangle is measured to match a specific element, so in practice, we are screenshotting just one element. Browsers use sub-pixel rendering, effectively rendering at a scale larger than the "pixels" seen by JS. The screenshot comes in at this scale, so the requested cropping rectangle is scaled to match, then the cropped screenshot is scaled down to the JS-measured size. Because of sub-pixel rendering, element offsets can be non-integer numbers. Normally, Karma puts the tests in a iframe, above which is a variable-height banner showing which devices are connected to Karma and what state they are in. So this variation and the lack of integer offsets means we can run into stability issues due to rounding errors. To make offsets consistent and improve stability of the screenshots, this banner is now disabled in our Karma config. The cropping, scaling, and diffing of images is handled Karma-side by a node module called Jimp. Before we start the layout tests for UITextDisplayer, we use a node module in the browser called fontfaceonload to wait for our web fonts to be fully loaded. This module is a polyfill that polls on IE and uses a standard API in modern browsers to wait for our font to load. This is all wrapped into a new test util called waitForFont. Screenshots are stored in test/test/assets/screenshots/ and are organized into folders by platform and browser and named according to an identifier specified by each test case. The new screenshot is written to disk with the suffix "-new", and a diff image is written with the suffix "-diff". When a test fails, we can review the changes in a browser with test/test/assets/screenshots/review.html. The known-good screenshots can be updated with the new tool build/updateScreenshots.py. Change-Id: Ib477fd3c459de466c6dc91e9a60d3e2579164b12
This directory contains the scripts used to build and test Shaka Player. These scripts can run on any platform that supports python v2.7 and JRE 8+.
all.pysimply runsgendeps.py,check.py,docs.py, andbuild.py. It will forward--forcetobuild.py.build.pybuilds the compiled library. This will fail if there are syntax or type errors.check.pywill check all the files for style violations and will check the tests for type errors (but will not produce any compiled output).checkversion.pyis used internally as part of our release process.docs.pywill build the documentation. The output will be indocs/api.gendeps.pywill createdeps.jswhich is required to use the uncompiled library.shakaBuildHelpers.pyis a utility library used by these scripts.stats.pywill read the compiled library and source map to get information about the size of the compiled library.test.pywill run the unit/integration tests.
All the scripts here use two environment variables:
PRINT_ARGUMENTSif set, will print the command-line to subprocesses.RAISE_INTERRUPTif set, will raise interrupts rather than swallowing them.
$ PRINT_ARGUMENTS=1 build.py
git -C /path/to/shaka describe --tags --dirty
Compiling the library...
java -jar /path/to/shaka/node_modules/.../compiler.jar --language_in ...
Configurable Build
build.py is used to compile the library and can also be used to produce
customized builds that contain only the features that your app requires.
build.py accepts an optional argument --name which will set the name of the
build, defaulting to ui. All other arguments are treated as commands
describing what to include in the build. If nothing is given, it will use
+@complete.
A command is either an addition or a subtraction. An addition is prefixed with
a +; a subtraction with a -. An addition will add the JavaScript file (or
build file) to the resulting library while a subtraction will remove it. After
the first character, there is either the path to a JavaScript file, or a @
followed by the name of a build file.
Build files are the files found in build/types/. These files are simply a
newline separated list of commands to execute. So if the +@complete command
is given, it will open the complete file and run it (which may in turn open
other build files). Subtracting a build file will reverse all actions applied
by the given file. So -@networking will remove all the networking plugins.
# Examples:
build.py +@complete
build.py +@complete -@networking
build.py --name custom +@manifests +@networking +../my_plugin.js
Test
test.py accepts some arguments, but mostly will forward them to karma. You
can run karma start --help to get more info about the karma test runner. If
you don't have karma installed, it will be installed by npm install and will
be found in node_modules/.bin.
test.py has two arguments that it handles directly. --force will cause
the build to run even if there are no changes detected to the source code.
--no-build will not build the library even if it does not exist. Note that
some integration tests will not run without the compiled library present.
There are also several custom arguments that are handled in JavaScript by
karma.conf.js or the tests themselves (via getClientArg).
These arguments can be passed in using test.py or using karma start
directly:
--quickwill only run unit tests, skipping integration tests.--enable-loggingwill enable console logging. Logs will be printed to the console. It also accepts a value for the log level--enable-logging=1, defaulting to 3. See lib/debug/log.js for the log levels (you must pass the number).--externalwill run integration tests against external assets. This will take an extremely long time to run, and requires a fast and reliable internet connection.--drmwill run integration tests against DRM license servers. This will require a connection to the open internet.--uncompiledwill run integration tests using the uncompiled library instead of the compiled version.--randomwill run the tests in a random order to isolate test dependencies.--seedwill seed the random test order so that the same order can be reproduced across runs. Specify any value--seed=xyz.--runsallows running the tests multiple times in succession. This parameter must be specified with a positive integer value, for example--runs 5.--use-xvfbwill launch the browsers in a virtual display (only on Linux).--specFilteris used to filter a specific test or set of tests. This parameter is specified as a RegExp string--specFilter="DataUriPlugin .*\d".
The karma argument --browsers will set the browsers used to run the tests
(e.g. --browsers Chrome,Firefox). If you don't pass any arguments, test.py
will choose a defaults based on your platform. However, if you pass any
arguments to test.py, it will not choose browsers and you must pass
--browsers.
Stats
stats.py is used to print various stats about the compiled library. This is
used internally to determine dependencies and to determine the size of the
compiled library.
Before running the script, you have to compile it first. Then you need to pass
either the name of the build (e.g. ui) or the path to the .map file.
You will also need to pass some arguments to determine the output you want.
You must pass exactly one of the following:
-cor--class-depswill print dependencies between classes.-for--function-depswill print dependencies between functions.-sor--function-sizeswill print the compiled size of the functions.-tor--all-tokenswill print all the tokens in the source map.
For --class-deps and --function-deps you can also output in DOT format.
This format can be used to produce visual graphs of the dependencies. Passing
in -d or --dot-format will output in DOT format. Then the output can be
piped into another program to produce the output. For example using graphviz:
stats.py -c -d | fdb -Goverlap=prism | neato -n2 -Tsvg > out.svg