Cross-Browser Testing at DeveloperWeek

The fine folks at Sauce Labs invited me to speak at DeveloperWeek San Francisco about browser testing at Yahoo scale. We test YUI on IE 6, 7, 8, 9, 10, & 11, iOS 6 & 7, Chrome, Firefox, and Safari, which runs over 120,000 tests on every push.

We’ve learned a lot about how to spend our time and the challenges of CI testing with multiple browsers. I began with a introduction to Yeti, the easy way to get started with web testing, which also drives over a million automated tests every week for YUI.

Check out the slides from my talk. If you were there, thanks for being a great audience! There was a great turnout with smart questions.

If you are interested in working on our unique frontend to CI testing with me at Yahoo, get in touch.

Yeti 0.2.27 — First-class code coverage

Less than a week since the last release and as promised, Yeti 0.2.27 has shipped today with code coverage support.

Yeti 0.2.27 provides first-class code coverage reporting provided by Istanbul.

Simply use Yeti with the -c or --coverage option. By default, Yeti will instrument your code on-the-fly and show a brief summary.

Yeti has ran 1,207,780 tests for YUI since Yeti 0.2.26 shipped 6 days ago.

I’m very excited to bring code coverage to yo/tests.

Yeti 0.2.26 released

I’ve released a new version of Yeti, the test runner we use here on Yahoo’s YUI team. Since August 2013, Yeti has automated 33,661,505 tests in CI for us.

Today’s release prints useful feedback to stderr when Yeti is used in CI. It also includes a fix for issue #74 (Unable to serve error) and #68 (improve DOH support).

When using Yeti to in CI, e.g. to produce JUnit XML output, previous versions of Yeti would go silent after testing began as Yeti produced XML output on stdout. This made it difficult to determine if Yeti was doing anything while tests ran. When using Yeti with --junit, today’s release prints status after every test completion to stderr while XML prints to stdout.

Big thanks to @henryqdineen for contributing the fix for DOH support!

In addition to fixing bugs, we have made some improvements to Yeti’s own tests and to Yeti’s documentation. Yeti’s website now uses Pure for your viewing pleasure on desktop and mobile devices.

Expect more updates soon. Code coverage is next on my list.

Three Talks

I’ve been busy this month speaking at HackSI, YUIConf, and Bayjax. The videos will be available soon, but you can check out the slides now:

My testing talks reveal what I’ve been working on for a few months: yo/tests, a commit-based test report for YUI that helps us ship quality releases. Compared to Jenkins’ build-based test silos and build-based CI dashboards, yo/tests actually lets engineers know what’s broken by organizing results around commits instead of build numbers and hiding flaky tests and infrastructure.