Testing multiple microservices together
Microservice architecture solves many problems, especially around deployability and modifiability. An area in which it makes our life as developers quite often more complex is testing. Especially at the top of the testing pyramid.
I decided to build c14r.io using microservice architecture. After deployment, I typically ingest a new image and verify that it shows up in the UI. Wouldn’t it be nice if this could be done even before the deployment is executed?
There is just one problem. c14r is composed up of many microservices working together, relying on various infrastructure components.
Enter testcontainers. A tool that lets me boot up disposable, temporary database instances and release candidate versions of an upcoming deployment during a software build. It is triggered automatically whenever a pull request is issued.
With testcontainers, we can recreate the microservice interaction shown above using code.
Network network = Network.newNetwork(); mongoDBContainer = new MongoDBContainer(DockerImageName.parse("mongo:4.0.10")); neo4jContainer = new GenericContainer(DockerImageName.parse("docker.io/bitnami/neo4j:4-debian-10")).withExposedPorts(7687); kafka = new KafkaContainer(DockerImageName.parse("confluentinc/cp-kafka:5.4.3")); crawlerMicroservice = new GenericContainer(DockerImageName.parse("docker.io/library/dexter:1.0")).withNetworkMode("host");readAndPrepareMicroservice = new GenericContainer(DockerImageName.parse("docker.io/library/ridik:1.0")).withNetworkMode("host");indexJobMicroservice = new GenericContainer(DockerImageName.parse("docker.io/library/jobbie:1.0")).withNetworkMode("host");
In a static block we can define the startup order of our containers and inject the connection details into the microservices:
static {
mongoDBContainer.start();
neo4jContainer.start();
kafka.start();
addEnv("KAFKA_BROKERS", kafka.getBootstrapServers());
addEnv("MONGO_CONNECTION", mongoDBContainer.getReplicaSetUrl());
addEnv("NEO4J_CONNECTION", getNeo4jURL());
addEnv("NEO4J_USERNAME", "neo4j");
addEnv("NEO4J_PASSWORD", "bitnami");
indexJobMicroservice.start();
crawlerMicroservice.start();
readAndPrepareMicroservice.start();
}Now we have a disposable instance of our release candidate software running. We can use it to play through various test scenarios like sending data to the indexJob microservice and validating if the crawling microservice push the data correctly to the data services, where it can be picked up by the readAndPrepare microservice.
# Send a index request
HttpResponse indexRequestBase = indexImage("{\"imageName\":\"debian\", \"imageTag\": \"stretch-20191014\", \"repositoryName\": \"library\"}");
assertEquals(202, indexRequestBase.statusCode());[...]# Validate if the image is indexed correctly
HttpResponse indexedBase = getDetailForImageImage("library/debian:stretch-20191014"); String responseBase = indexedBase.body().toString(); assertEquals(200, indexedBase.statusCode()); assertNotEquals("{\"variants\":[],\"tags\":[]}", responseBase);
The second part of the code above will only be green if the interaction worked end2end. After we executed the test, we are therefore sure that no contract was violated and the microservices do still play nicely with each other. Which saved me from at least one faulty production deployment already.
The end2end test can be easily integrated into a GitHub action (to embed into Pull Request process):
e2e:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Set up JDK 11
uses: actions/setup-java@v2
with:
java-version: '11'
distribution: 'adopt'
- name: Build Dexter Microservice Container Image
run: mvn spring-boot:build-image
working-directory: 'dexter/'
- name: Build Ridik Microservice Container Image
run: mvn spring-boot:build-image
working-directory: 'ridik/'
- name: Build Jobbie Microservice Container Image
run: mvn spring-boot:build-image
working-directory: 'jobbie/'
- name: Execute End2End Test
run: mvn test
working-directory: 'e2e/'The code for the end2end test is available on github.
