Unit Tests
Unit tests assumes that the everything outside the Unit Under Test (UUT) is fixed and must not depend on other systems work correctly. The expected data are checked in the Version Control System (vcs, e.g. git) and the responses from external systems (like databases) or other components of the code are mocked (simulating their correct behaviour). Unit tests must run fast because they are run very often. Unit tests must never fail.
Python Unit tests for flatland are in the tests
folder and run with tox -e py36,py37
.
Integration Tests
Integration tests do not make the strong assumption of isolation. Integration tests should not produce too many false negatives nor false positives; they may fail, but not for too long. For the time being, our integration tests are running the notebooks and examples through tox -e notebooks,examples
without failure. Of course, integration tests should also test the actual result against precise expectations, running without failure is not enough in the long run.