YUIConf 2012

Write Code That Works

by @reid

This is a talk about software testing

“When not to write tests”
(Alternate title)

Time is limited

Yeti
The YUI Easy Testing Interface

Test JavaScript with any browser

No driver. Only browsers.

Why not Selenium?
Real devices & Actual browser

Presenting
The Dav Glass Device Laboratory
of Southern Illinois

YetiTV

What are you building? Something.

How do you build it? With code.

What does that code do? Work.

How does it work? Not by writing tests.

Testing only helps.
Tested code ≠ Working code

Testing shows the presence, not the absence of bugs

— Edsger W. Dijkstra, J.N. Buxton and B. Randell, eds, Software Engineering Techniques, April 1970, p. 16.

Testing is a tool, not the goal

Antipatterns

Writing tests first ≠ Working code

Lots of tests ≠ Working code

100% line coverage ≠ Working code
(but it helps)

You are not paid to write tests.

Listen to a smart guy.

I get paid for code that works, not for tests, so my philosophy is to test as little as possible to reach a given level of confidence
If I don't typically make a kind of mistake (like setting the wrong variables in a constructor), I don't test for it.
When coding on a team, I modify my strategy to carefully test code that we, collectively, tend to get wrong.

Kent Beck wrote that. Known for advancing Test Driven Development.

He also was one of the 17 original signatories of the Agile Manifesto.

What’s agile again?

The Agile Manifesto (2001)

We are uncovering better ways of developing software by doing it and helping others do it.

Through this work we have come to value:

  • Individuals and interactions over processes and tools
  • Working software over comprehensive documentation
  • Customer collaboration over contract negotiation
  • Responding to change over following a plan

That is, while there is value in the items on the right, we value the items on the left more.

Working software > comprehensive docs

Working software > talking about it

Working software > metrics about it

Working software > green blinkenlights

Over-testing

security screening at denver airport

Writing tests has a cost.
Time to write, to update, to read and understand, and to actually run!

Quality theater.

Total test count. Line coverage. Branch coverage. Statement coverage. Speed of test runs.

We forgot about some other metrics.

Amount of test code. Amount of time writing tests. High coverage by useless tests.

Be sensible.

Consider the unseen metrics.

We forgot about our own productivity.

More wisdom from Kent Beck

Different people will have different testing strategies based on this philosophy,
but that seems reasonable to me given the immature state of understanding of how tests can best fit into the inner loop of coding.
Ten or twenty years from now we'll likely have a more universal theory of which tests to write, which tests not to write, and how to tell the difference.
In the meantime, experimentation seems in order.

Test Driven Development & Agile & your creative process is for experimenting.

We experiment to make great code.

If you can’t test everything, what should you test?

Prioritize your testing strategy.

What code is changing?

Semantic Versioning
Project-level stability contract

X.Y.Z

?
Section-level stability contract

Stability Index
Section-level stability contract

Throughout the documentation, you will see indications of a section’s stability. The Node.js API is still somewhat changing, and as it matures, certain parts are more reliable than others.
Some are so proven, and so relied upon, that they are unlikely to ever change at all. Others are brand new and experimental, or known to be hazardous and in the process of being redesigned.

— The Node.js Stability Index documentation.

“How reliable is this?”
Helps your customers

Priortize your testing strategy
Helps your team

Test what matters most.

...my philosophy is to test as little as possible to reach a given level of confidence
If I don't typically make a kind of mistake (like setting the wrong variables in a constructor), I don't test for it.
When coding on a team, I modify my strategy to carefully test code that we, collectively, tend to get wrong.

Device Laboratory Reprise

YetiTV

Visit yeti.cx to get started

Yeti is a tool to help build working code.

Your code is ultimately up to you.

In the meantime, experimentation seems in order.

More on my blog.

Go experiment.

Follow @reid
An Upstage presentation
Built with YUI