I am trying to write characterization tests, in order to quickly put a legacy system to a test harness.
I was not able to find many examples with production code. The examples I find are small programs.
An example by an author of Working Effectively with Legacy Code implements them as unit tests.
It’s nice because it’s human-readable and help you understand the code. However, I think it would work for small programs in the example, but the system I am trying to test is a pretty complex API client.
Other examples store the inputs and outputs in files and read them to compare the results with the ‘snapshots’.
I feel like this allows the test input to be generated, and suitable for testing a large set of inputs.
However, for some reason it feels more high-level than unit testing, and not very human-readable.
Are these tests supposed to be part of your unit test suite, or are they supposed to be complements to unit tests?
In other words, should I sample some characteristic test inputs and write characterization tests as unit tests, but using the actual output from the code to ‘lock down’ the existing behaviors, and trying to make it readable? Or should I treat characterization tests as a complement to unit tests? If so, what should I focus on in my unit tests?
Generally, characterisation tests are not an end state. They're a way to pin the current behaviour of untested legacy code, so that you can start making changes towards maintainability and testability with a reasonable level of confidence that the overall picture hasn't changed. They're not good tests, though, they don't:
tell you any of the behaviour is actually correct;
tell you where the problem is when a test starts failing (they're just detecting changes); or
tell you which parts of the current implementation are important.
So readability isn't particularly important in this case; once you have explored the behaviour more thoroughly, found the overlaps between the test cases (and which ones shouldn't have passed) and written higher quality tests that give you real confidence, you can get rid of most if not all of them.
The other use for characterisation tests is where the alternatives would be less readable, e.g. when you have code that outputs a report and you don't want to have to exhaustively check each part of it. Instead you generate the report, manually check it and then compare future reports to it to make sure nothing changes. Although this "snapshot"-style testing can be harmful at lower levels, enforcing "change detection" that actually prevents refactoring, at this level it's useful.
Answered by jonrsharpe on October 30, 2020
1 Asked on December 23, 2020 by anand
7 Asked on December 13, 2020 by nikhita-jalan
1 Asked on December 7, 2020 by belovedfool
1 Asked on December 5, 2020 by annushri
4 Asked on November 26, 2020 by justin
1 Asked on October 30, 2020 by user25409
3 Asked on September 29, 2020 by yob-v-u
4 Asked on September 18, 2020 by reema
1 Asked on September 5, 2020 by chathud
0 Asked on August 25, 2020 by slava-klevchenia
1 Asked on August 12, 2020 by beti
1 Asked on August 10, 2020 by udit-choudhary
18 Asked on July 21, 2020 by coybit
Get help from others!