www.praqma.
com
Testing Microservices
Techniques for Automation
European Testing Conference 2018
Emily Bache
@emilybache
www.praqma.com
Emily Bache
Practice Lead for Test Automation
Consultant
Author of
“The Coding Dojo Handbook”
@emilybache
emily.bache@praqma.com
www.praqma.com
Pagero Online
Cloud Service
Invoice Invoice
Issuer Recipient
Some numbers…
• We were around 45 developers, in 7 teams
• We had around 60 microservices
• Our monolith was in production for nearly 10
years before we started with microservices,
3-4 years ago
www.praqma.com
Pagero Online’s architecture
www.praqma.com
A Microservice
• Provides one business
capability
• Independently Deployable
• Lightweight API
• Not too big: fits in my head Docker is used to containerise the service
www.praqma.com
Testing a Microservices
Architecture
In my experience
www.praqma.com
Test Pyramid: Mike Cohn
End-To-End tests
End-to-End exercise the whole
system
“Service Tests”
Unit
Unit tests are for
individual functions
https://www.mountaingoatsoftware.com/blog/the-forgotten-layer-of-the-test-automation-pyramid
www.praqma.com
Testing Strategy for the Monolith
Unit tests and end-to-end
GUI tests still valid for
Microservices
architecture
www.praqma.com
Testing Strategy
Testing a monolith Testing a bunch of microservices
www.praqma.com
Testing a Microservice
www.praqma.com
Microservice Tests
µSUT
• Deploy your service on localhost (and
the database if it has one) Test Case
• Test uses the public API to trigger
functionality
• Mock a response to any calls made to
other services or queues
µSDB
collaborator µS
www.praqma.com
Faster execution
Deploy in Same Process
µSUT
• In-process testing: Deploy your
service with an in-memory database Test Case Internal api
and in-memory api
• ‘internal api’ to set up test data and
query internals
µSDB
collaborator µS
www.praqma.com
More production-like
Deploy in Different
Processes
µSUT
• Out-of-process testing: Deploy your
service in one process, and the test Test Case
case in another
• Security needed for ‘internal api’ so
only test/non-production code can
access
• Easier to performance test
µSDB
collaborator µS
www.praqma.com
Business-Facing Feature
Tests
Test a whole scenario across several microservices
www.praqma.com
Testing a whole Feature
Test Case
This is where it starts getting really complicated…
www.praqma.com
Example End-to-End Test Strategy
• Test crucial workflows, not every detail
• Use APIs instead of the GUI for most tests
• Techniques:
Keep testing costs
as low as possible
• Selective Deployment
• Approval Testing
• Event Monitoring
www.praqma.com
Service-layer tests in a monolith
Monolith
• Test cases are for
different business-
Test Case A
facing workflows
Test Case B
• Will exercise different
components within the
monolith
Database
www.praqma.com
Feature tests in microservices
Microservices • Test cases are for
different business-facing
Test Case A workflows
• Will exercise different
Test Case B groups of microservices
• Need not deploy
everything for every test
case
www.praqma.com
Selective Deployment
• Need not deploy
everything for every test
Test Case A case
• But: all test cases
needed a few essential
infrastructure services.
www.praqma.com
Pagero Online
Cloud Service
Invoice Invoice
Issuer Recipient
Parts of a test case
Invoice Document Invoice Recipient
Test Workflow
Issuer Recipient Presentation
Arrange Act Assert
(Almost) all the tests involve these elements
www.praqma.com
Approval Testing
Invoice Document Invoice Recipient
Test Workflow
Issuer Recipient Presentation
Arrange Act Assert
Approval Testing specifies the Assert: compare against an ‘Approved’
result
www.praqma.com
The “Approved” Result as Text
Recipient
Recipient Presentation
Presentation in plain text
pdf-to-text utility
• Find the important outputs. Convert them to plain text.
• Use textual diff to decide if the actual output matches the approved version
www.praqma.com
Elements of a Test Case
Invoice Document Invoice Recipient Approved
Test Workflow
Issuer Recipient Presentation Output
common to to determine
for reference
Input data varies by test case many tests pass/fail
• Minimise test creation work
• Minimise test maintenance work
• Maximise serendipity - find bugs you didn’t anticipate
www.praqma.com
Debugging a failing test?
Test Case
Which part broke?
www.praqma.com
Correlation ID
Pick out all events with ‘123’
www.praqma.com
Event log recorded in a test case
Helps you to debug what happened when the test fails
www.praqma.com
Test Case Elements
Invoice Document Invoice Test Workflow Event Logs Approved
Issuer Recipient Output
common to to debug to determine
Input data varies by test case many tests the test pass/fail
• Recorded traffic is stored with the Recipient
Presentation
approved output for reference
www.praqma.com
Independent Microservices
& Teams
Organizing Testing efforts
www.praqma.com
Testing in the pipeline
version
unit
microservice
system
manual
production
control test test test test
Pair
Programming,
TDD
more production-like environment
test larger pieces of code together
monitoring
Test costs increase
www.praqma.com
Multi-team development
Component B
Infrastructure & Architecture
Component A
Components C&D
Components E&F
Testing in the pipeline
version
unit
component
system
manual
production
control test test test test
Pair
Programming,
TDD
monitoring
www.praqma.com
Multi-Team CD
Team A
micro- system
manual
version
unit
production
service
test test
control test
test
These tests are
often broken!
monitoring
Team B
Team pipelines
Team A
micro- team-
manual
version
unit
system
service
system
test
production
control test test
test test
One environment
shared by all teams
monitoring
Team B
Teams Deploy Independently
Team A incremental
micro- team-
team-
roll-out to
version
unit
service
system
manual
production
control test
test test test
monitoring &
manual testing
Team B
Summary
@emilybache
monitoring
Testing in a
Microservices Architecture team-
system
test
System Tests
Summary
@emilybache
monitoring
Testing in a
Microservices Architecture team-
system
test
System Tests
The Continuous Delivery and DevOps Company
www.praqma.com
