This ports over internal tests suggested in #3889.
This replaces an internal tool that we have been running on a private Jenkins server.