Roger Almeida

A blog about tech

Black and White box testing

Q: Should these tests live together with the application or on a different repo?

This a pretty common question on software projects. Here is my answer to it:

A: BlackBox tests should live apart from the app source code. WhiteBox tests should live together with the app source code

🖤 BlackBox Testing

By BlackBox testing I refer to tests that do not and SHOULD not depend on the internal of an application. These tests see the application as black box. The only way for theses tests to interact with the application is through its published interfaces. Some examples of published interfaces are:

A Blackbox test should then continue to work even if the internal of the application are refactored. A common example of such tests is an Postman test suite that exercise an rest API via its end-points.

🤍 WhiteBox Testing

A WhiteBox test on the other hand intentionally knows and cares about the internals of an application. An example of such test is an Unit Test that mocks dependencies of an specific method/class.

Another common scenario for WhiteBox testing are tests that depends on frameworks used internally by the application. For instance

These frameworks offer fantastic tooling to facilitate for instance isolating part of the application from the rest and testing just a subset of the whole application. It requires deep understanding of the internal design of the application and depending on the level of refactoring you are planning on doing it will force you to also refactor the testing code.

These test must live together with the source code (same repo) so that the feedback loop between introducing a breakage change under discovering it can be reduced. Hopefully you will even be able to get the feedback during compilation/transpilation phase, or better yet your code editor might be able to inspect and report the error to you near real-time.

🤔 When should I use BlackBox and WhiteBox testing?

This is not a situation of one or the other, both types of tests are complementary. In general BlackBox tests are slower, it is important to do not rely on them to catch all the possible test scenarios. But WhiteBox tests only cannot exercise all the moving parts together, so in my experience it is important to mix BlackBox and WhiteBox tests to have a good balance on regression and enabling refactoring for the long term as well.

05 Sep 2023