See "Hard-to-Test Code" on xUnit Patterns and chapter 9 ("I can't get this class into a test harness") of "Working Effectively with Legacy Code".
Cause: Highly Coupled Code
Functions/methods that mostly only get other objects as arguments and call methods on them.
It's hard to setup. May need a lot of mock objects, which behave correctly. It may also require a lot of trickery to mock right, e.g. methods that require careful/proper initialization (__init__).
Micheal Feathers lists most prominent examples of highly coupled code.
Parameter of a type that we don't really want to instantiate, e.g. because it is slow or causes side-effects (or both). In Python we can either:
- pass None, if the parameter isn't used on the code path we're testing, or
- create a stub that simulates behavior of real object.
Function or method creates objects that we shouldn't really use in the test code. In Python we can override a global object or resource reference with a stub only for the duration of a test and restore it back to the original value after that. For a presentation of using this technique see the first example in "Mock testing examples and resources".
Code constructs a large number of objects internally, and maybe uses some of them to construct even more objects. Automatic tool can handle 50 parameters as easily as 5, but we'll have a problem with clarity of generated test cases. We'll need a way to automatically extract and build custom creation methods.
Irritating Global Dependency
Code uses a global variable or resource, which doesn't work well in testing environment. We can use the same technique as for Hidden Dependency.
Horrible Include Dependencies
Dependencies between header files are very complicated. Python import and namespaces don't seem to be affected by this problem.
To create an object we need to pass other objects to it, which require even more objects to be created. By using mocks as parameters we won't have to create anything else and as a bonus we'll be able to control their behavior better. Solution the same as for Irritating Parameter.
When Extract Interface would be to much work we may try Subclass and Override Method. Since we may use the latter using stubs anyway, it's not an issue in Python.
Cause: Asynchronous Code
Either threaded or event-driven applications. The canonical solution to this is to separate application logic from the asynchronous execution model, which isn't all that straightforward. This topic certainly needs more research, and we should start with finding real-world usage patterns.
Although we may work around it, we should discourage a use of isinstance. Following the advice in chapter 7 of "Working effectively…" we should discourage use of dependencies on concrete classes and encourage interface dependencies based on duck-typing.
One way to find out if given piece of code requires special setup is to, using the information from the dynamic run&trace, try running it on its own, feeding the same input as gathered. If we get an exception, or a different output, it means the setup is needed. If it works, we're happy. :-)