Mutant on Steroids

Markus Schirp

23.03.2019

Mutation testing

Strong form of coverage

Old technique

Derived check

Automated

Code and tests are inputs

Unspecified semantics are outputs

Example report

Human decision required

If redundant remove If required test

Nomenclature

Naming is fun

Subject

Thing that is covered by tests that can be mutated

Match expression

For subject discovery

MyApp*
MyApp::SomeClass
MyApp#some_instance_method
MyApp.some_class_method

Selection

Subset of all tests relevant for the subject

Uses metadata

RSpec explicit and implicit

Minitest explicit

Operator

Class of changes to a subject

Reduction / Orthogonal

Mutation

Result of applying an operator against a subject

Run against selection of tests for classification

Alive-Mutation

Bad

You saw one in the example report

Remove redundant semantics

Add a test as last resort

Preconditions

Discoverable Subjects

Tests are green

Tests are idempotent

Tests are randomizable

Tests are concurrency hard (optional)

Tests have selection metadata

Rails

A precondition “challenge”

Starting now

Incremental is the key

--since to the rescue!

Thanks