Because our release PRs are created with the default GitHub token, they
do not trigger test runs. This is a feature of GitHub that prevents
recursive loops of workflows.
This change makes it so that marking a PR "ready for review" will
trigger the tests to run again. This gives us an easy way to trigger
test runs on a release PR, by marking it a draft, then marking it
"ready" again.
We were misreading the computed ref for tests (missing "outputs" in the
variable name), which led us to check out and test main instead of the
PR's head. This also led us to set status on the latest commit in main,
instead of on the PR.
Once the variable was read correctly and the correct ref was checked
out, I found that the ref argument to set-commit-status was actually not
usable, since "git rev-parse refs/pull/4528/head" would fail. But now
that we are checking out the correct commit, the ref argument turns out
to be unnecessary. Whatever we checked out is what we should report
status on.
Player integration tests use the compiled bundle by default. This
prevents us from getting coverage data in these tests, since the
instrumentation was added to the uncompiled library only.
By collecting coverage data in uncompiled mode, we can fix this.
Since Safari 16 became the default on macos-11 images in GitHub, we are
updating our homedir installation of Safari 14 to Safari 15 instead.
This also renames that entry in the matrix from Safari-14 to Safari-old,
in recognition of the fact that we will probably have to continue
updating this.
This workflow will trigger after the nightly Selenium tests. If those tests failed, this "deflake" workflow will automatically re-run the failed jobs up to 3 times (total, including the original scheduled run).
This should help cut down on noise and help highlight actual platform-specific failures. The flakiness of our tests and infrastructure will be tracked separately and addressed at a somewhat lower priority than test greenness.
This requires multiple self-hosted runners to be set up, but will
allow us to more easily read results and to re-run individual failed
jobs per browser.
This change also includes several optimizations that were motivated by
the split:
- stop installing the GitHub CLI (now installed on newer self-hosted
images, saves ~20 seconds)
- cache node_modules (saves about ~1 minute if package-lock.json
hasn't changed)
- pre-build Shaka once in a GitHub VM instead of in parallel on
multiple runner containers hosted on the same machine (saves ~45
minutes w/ 12 runners in parallel)
The initial implementation had a bug where a class declaration
"statement" would be seen as covering all the lines of all the methods
in that class. This change fixes that case, so that the lines of
methods and inner statements are not "part" of their outer containers.
This generates a more detailed JSON coverage file containing maps of
exactly what statements were executed by tests. This also extracts
the data from this file and updates the PR with a comment showing how
much of the changed code is covered by tests.
This will attach a code coverage report to PRs and to nightly tests,
both specifically from Chrome on Linux. Reports will be attached to
workflow runs whether or not the tests pass.
The available package versions of Safari have changed. This updates
to what is currently available. This also updates to expect the
current format of the packages (pkg inside zip).
The v3.2.7 and v3.3.5 releases went out without updating the version
number in lib/player.js. This fixes the workflow to match the
release-please branch name, and changes the workflow to fail if the
branch name changes again in the future.
Having both v4.0.0 and v4.0.0-main tags refer to the same commit
confuses some of our build scripts because git describe can produce
either tag. So we should only create -main tags when the release is
from a branch.
This changes the default behavior of the test runner. Now tests that
require access to license servers will run by default. To disable DRM
tests (for example, if there is no connection to the outside
internet), use --no-drm.
These tags should be attached to the head of the main branch, so that
is what we should check out during the tagging job.
In v3.1.7, v3.2.5, and v3.3.3, we accidentally tagged the heads of the
release branches. The `-main` tags for those releases had to be
corrected manually.
If another instance of the build-and-test workflow is started for the
same PR, cancel the old one. If a PR is updated and a new test run is
started, the old test run will be cancelled automatically to conserve
resources.
Includes a YAML config file for our lab, and documentation on how to
create a custom config for another Selenium grid.
A workflow will run tests nightly in the Shaka lab, using a
self-hosted runner with access to our private grid.
The workflow can also be triggered manually by maintainers to test a
PR in the lab. This will report status back to the PR.
Rather than runtime-querying of appengine versions within the
appengine environment, we can instead generate the index at deployment
time (from git tags) and just serve static content. This simplifies
the system and avoids dependence on Google Cloud.
This was less feasible before we adopted GitHub Actions, but is now
relatively simple. The index will be regenerated when the index code
is updated or when a new release is created.
Closes#4074
In the GitHub Actions environment, it is now possible to install and
run tests on Safari 14. Though not completely straightforward, it
doesn't add much code to the workflow and shouldn't be too difficult
to maintain.
Closes#3899
This fixes the workflow job failing because of missing sources, and it
also fixes the job to run the compiler, not just build the image that
_could_ run the compiler.
Closes#4076
There is a typo in the workflow that is causing the workflow to be
rejected by GitHub.
I am not sure how I managed to merge this mistake in the first place.
The "if" fields do not need to use expression syntax (`${{ }}`), and
some conditionals will be easier to read if we omit that.
<!--
Please remember to:
1. Use Conventional Commits syntax (fix: ..., feat: ..., etc.) in commits and
PR title (https://www.conventionalcommits.org/)
2. Tag any related or fixed issues ("Issue #123", "Closes #420")
3. Sign the Google CLA if you haven't (https://cla.developers.google.com)
You may delete this comment from the PR description.
-->
Now that we are building tools to keep common workflows in sync across
repos, we should use the same names across repos. This brings Shaka
Player workflow names in line with everything else.
See https://github.com/joeyparrish/shaka-github-tools/pull/5
1. When release-please creates release PRs, it removes the -uncompiled
tag in the version number in lib/player.js. This adds a step to the
workflow to maintain the player.js version manually.
2. This also adds a new job that tags the master branch after a
release PR is merged. We maintain the -master tags on the master
branch corresponding to each release so that it is easier to manage
the range of commits for cherry-picks.
3. The "npm ci" step was failing because our package-lock.json was not
compatible with v12 of NodeJS. This updates our workflow to use v16.
The npm release workflow has now been tested in a fork up until the
publish step.
4. The tagging and GitHub release publication parts of the workflow
were previously failing due to an issue with release-please parsing
the branch name. This has now been fixed, and is awaiting an upstream
merge. For now, we can use my forked version.
Closes#3968 (branch parsing failure)
Issue #3969 (npm release is now tested up to the publish step)
Closes#3971 (master branch tags)
Closes#3974 (missing -uncompiled in lib/player.js)
Force-pushes from automated PRs (like release-please) do not seem to
trigger any of the normal PR triggers (opened, edited, synchronize).
In fact, even an exhaustive list of types will not work. So here we
add triggers for reviews, so that the validation will run after
someone approves such a PR. This is critical since this is a required
status check in most of our repos. If the workflow doesn't run, the
release PR can't be merged.
This includes a temporary config file to bootstrap the workflow for
the master branch, whose commit and branch history is too confusing
for the tool to handle with defaults. After v3.4.0 is released from
the master branch, these configs can be deleted.
Joey Parrish
Álvaro Velad Galván
Joey Parrish
Shaka Bot