Info Espressos: Testing

Unit Testing Practices
=========================
from Osherove, Sanderson, Lorenz, Bob Martin and others
See also videos at the bottom of this page.

Qualities of good Unit Tests (in priority order)
– Trustworthy
– Readable
– Maintainable

Trustworthiness
—————–
What is it?
– Need to trust a test to pass when it should.
(i.e. When the tested expectation is met.)
– Need to trust a test to fail when it should.
(i.e. When the tested expectation is not met.)

Test Isolation
– No dependency on influences between tests!!!
– Don’t run one test from another.
– Run alone or all together – expect the same result
– In any order
– Otherwise leads to nasty bugs

Readability
—————
– Test naming:
@Test
MethodName_Scenario_ExpectedResultOrBehaviour_index()
e.g. Add_LessThanZero_ThrowsException()
or Add_TwoPositiveNumbers_ReturnsPositiveSum_0001()

– No “magic” values (numbers, Strings etc). Name these as constants.
e.g.:
bad: assertEqual(1003, calc.parse(“-1”))

better: assertEqual(ILLEGAL_RETURN_CODE,
calc.parse(ILLEGAL_NEGATIVE_NUMBER_STRING));

best: int returnCode = calc.parse(ILLEGAL_NEGATIVE_NUMBER_STRING);
assertEqual(ILLEGAL_RETURN_CODE, returnCode);

Basic Unit Test Structure
– SETUP – Runs once before each test.
– Arrange – Given these preconditions
– Act – When I do this
– Assert – Then I get this expected result.
– Clean-up (sometimes)
– TEARDOWN – Runs once before each test.
Roll Back DB, Clean File System etc.
Usually means you have an integration test.

Maintainability
—————-
– In general go for readability over maintainability.
– Small simple tests.
– Separation of unit and integration tests.
– Not too many unnecessary tests.
– Reusable code in test Setup method (run once for every test).
– For creating objects that are used by some tests but not all,
create reusable factory methods to create those objects
(through refactoring).

TDD (Test First)
——————
– Cycle (Test – Code – Refactor – Repeat):
Write a Failing Test,
Make it Pass,
Refactor(changing code structure not function)
Repeat
– Don’t add functionality without a failing test.
– TDD forces you to test the test:
You must first see it failing and then make it pass.
– The failing test must still be trustworthy – not stupid such as:
assertEquals(calc.sum(1,2), 4)

Sometimes your choices are limited:
– When writing a new system it is much easier to do Test First
– With legacy/existing code use more of an incremental approach.
– Incremental work is the key in any place where you are not in
control of the design choices.

Unit Testing Guidelines
————————-
– Test the contract of a class through public methods only:
If you feel you should test a private or protected method,
this might be a “design smell” suggesting that the method
should be public, or static, or moved to another class.
This falls into the domain of Test Driven Design, as your tests are telling you that there is a potential design flaw.
– Write very simple unit tests
Test only one expectation (one assertion) at a time.
– Distinguish Unit tests from Integration tests
(ones that require config, interactions etc.)
– Don’t put logic in tests at all if possible.
No loops, conditions, random number generators, concatenations
and other String manipulations, mathematical operations.
Definitely don’t reproduce tested logic in test.
Rather use constants for expected values.
– Get “good” code coverage (>90%).
This is only meaningful if the tests are good.

Removing or Changing tests:
– As far as possible don’t remove or change tests.
Tracks in the sand – you want to know how the requirements evolved.
– Bad reason:
The test is broken because of configuration or refactoring.
– Good reasons:
To improve clarity (no functional change)
The test is invalid due to changing requirements
if semantics of using the tested requirement changes

Test Reviews
– Makes sure that every test gets respect.
– Check manually.
– Change logic of tested code (one thing at a time)
and run all tests.
Example replace “if (a==b)” with “if (true)”
Some tests must fail.

Separate Unit and Integration Tests
The following distinguish Integration tests from true Unit tests:
– Require configuration
– Don’t just run in memory
– Touch external resources which can be faked:
(database, network, email, file system)
– Use threads.
– Test more than one expectation

Test only Public Methods
– Testing privates makes your tests more brittle
– Makes you think about the design and usability of a feature.
– Test-First leads to privates after refactoring, but they are
all tested.

Avoid Multiple Asserts
– Like having multiple tests
– But if the first fails, it kills the others
– You don’t get the big picture – some symptoms are silently ignored
– Hard to name (multiple scenarios and expectations)
– Can lead to more debugging

Note: User Stories are included here as they are the source of Test Cases

Agile User Stories by Mark Shead

User Story in details for Agile Software Development by Agile Digest

Scrum Fundamentals and Advanced Training: Mapping User Stories to Test Cases by Tommy Norman

BDD by Gary Straughan (energising presenter) from Development That Pays in just over 4 minutes

How to write effective test cases QUICKLY? by Inder P Singh in under 20 minutes:

Jim Coplien and (Uncle) Bob Martin Debate TDD (20 minutes)