Even though httptest provides a way to avoid testing against external services, you should still write integration tests, automated tests that connect to other services. These validate that your understanding of the service’s API is correct. The challenge is figuring out how to group your automated tests; you want to run integration tests only when the support environment is present. Also, integration tests tend to be slower than unit tests, so they are usually run less frequently.
In the related discussion, I covered build tags, which the Go compiler uses to control when a file is compiled. While they are primarily intended to allow developers to write code that’s intended only for a specific operating system, CPU, or Go version, you can take advantage of the ability to specify custom build tags to control when integration tests are compiled and run.
Let’s try this out with the math-solving project. Use Docker to download a server implementation with docker pull jonbodner/math-server and then run the server locally on port 8080 with docker run -p 8080:8080 jonbodner/math-server. (The examples/solver directory has a Docker Compose file, so if you have cloned it locally, you can run docker compose up.)
Note
If Docker is unavailable, use a local test server that implements the same API.
You need to write an integration test to make sure that your Resolve method properly communicates with the math server. The examples/solver/remote_solver_integration_test.go file has a complete test in the TestRemoteSolver_ResolveIntegration function. The test looks like every other table test that you’ve written. The interesting thing is the first line of the file, separated from the package declaration by a newline:
//go:build integration
To run your integration test alongside the other tests you’ve written, use this:
$ go test -tags integration -v ./...
Some people in the Go community prefer to use environment variables rather than build tags to create groups of integration tests. Peter Bourgon describes the concept in a blog post with the unsubtle title “Don’t Use Build Tags for Integration Tests”. His argument is that it’s hard to find out what build tags to set in order to run integration tests. An explicit check for an environment variable in each integration test, combined with a detailed message in a t.Skip method call, makes it clear that tests aren’t being run and how to run them. The downside to this approach is that it is far more verbose than using a build tag, since it requires the developer to include a preamble in every integration test.
Yet another way to group tests is to use go test with the -short flag. If you want to skip over tests that take a long time, label your slow tests by placing the following code at the start of the test function:
if testing.Short() {
t.Skip("skipping test in short mode.")
}
When you want to run only short tests, pass the -short flag to go test.
There are a few problems with the -short flag. It has the same verbosity problem as using environment variables, but doesn’t give you any flexibility for grouping your tests, since it only allows for two levels of testing: short tests and all tests. Both build tags and environment variables let you group your integration tests, specifying which services should run. Another argument against using the -short flag to indicate integration tests is philosophical: build tags indicate a dependency, while the -short flag is meant to indicate only that you don’t want to run tests that take a long time. Those are different concepts. Finally, I find the -short flag unintuitive. You should run short tests all the time. It makes more sense to require a flag to include long-running tests, not to exclude them.