This repository contains automated integration tests for eCAL. It is designed to run and validate communication scenarios between multiple processes using eCAL middleware.
-
View status and logs of last tests:
-
Test report (available after completion of the tests):
The main goal of this project is to provide:
- Automated test runs on push, pull request, or schedule
- Reliable feedback on message communication and process stability
- Visual test reports for easy inspection
-
Supports multiple eCAL transport layers (SHM, UDP, TCP)
-
Covers common scenarios such as:
- Publisher to Subscriber communication
- Multi-publisher and multi-subscriber setups
- Fault injection (e.g., crashing subscribers or publishers)
- Network failure simulations
-
Test results are published to GitHub Pages
-
Integration in eCAL repositori via
repository_dispatch
Each test case is implemented using Robot Framework, with Docker-based isolation.
| Test Case | Description | README |
|---|---|---|
| basic_pub_sub | Simple 1:1 publisher-subscriber communication | README |
| multi_pub_sub | Multiple publishers and subscribers (N:N) | README |
| network_crash | Local UDP works after network disconnect | README |
| pub_crash | One publisher crashes mid-test | README |
| sub_crash | Subscriber crashes during message receive | README |
| sub_send_crash | Subscriber crashes during send/zero-copy test | README |
| rpc_ping_test | Simple RPC test with one client calling one server | README |
| rpc_reconnect_test | RPC client disconnects and reconnects during the test | README |
| rpc_n_to_n_test | Multiple RPC clients call multiple RPC servers (N:N) | README |
Test results are automatically deployed to: 👉 https://emirtutar.github.io/ECAL_Test_Framework
Each run is timestamped and archived, with access to:
log.htmlreport.htmloutput.xml
Tests can be triggered in three ways:
- Manually via GitHub Actions
- Automatically from the eCAL repository using
repository_dispatch(Create a PR or Push to Master)
If you are working on a fork of the official eCAL repository (e.g., UserName/ecal) and you create a pull request or push from develop to master within your fork, you need to allow GitHub Actions to trigger the integration tests in this repository.
To do so, follow these steps:
- Go to your GitHub Developer Settings → Personal Access Tokens.
- Click on “Generate new token (classic)”.
- Set a name like ECAL Test Trigger Token.
- Set an expiration (e.g., 90 days or custom).
- Under Scopes, enable:
-
repo (for private forks)
or
-
public_repo (for public forks only)
- Click Generate token and copy it immediately. You won't see it again.
- Go to your forked repository on GitHub (e.g., UserName/ecal).
- Navigate to: Settings → Secrets and variables → Actions → New repository secret.
- Name the secret exactly:
TEST_FRAMEWORK_TOKEN- Paste the copied token as the value.
- Click Add secret.
- Once added, GitHub Actions in your fork can use this token to trigger the test framework via repository_dispatch.
After the test run finishes, the result (pass/fail) is reported back to the Pull Request in the eCAL repository as a commit status.
- eCAL (If you build without Docker)
- Docker
- Python 3
- Robot Framework
.
├── integration_tests/ # Main directory for all test cases and supporting infrastructure
│ ├── basic_pub_sub/ # Test case: simple 1:1 publisher-subscriber communication
│ ├── multi_pub_sub/ # Test case: multiple publishers and subscribers (N:N)
│ ├── network_crash/ # Test case: test resilience after network disconnect
│ ├── pub_crash/ # Test case: publisher crashes mid-transmission
│ ├── sub_crash/ # Test case: subscriber crashes during reception
│ ├── sub_send_crash/ # Test case: subscriber crashes during send (zero-copy stress)
│ ├── rpc_ping_test/ # Test case: basic RPC ping between client and server
│ ├── rpc_reconnect_test/ # Test case: RPC reconnects after temporary disconnect
│ ├── lib/ # Shared helper libraries used by multiple test cases
│ │ ├── EcalConfigHelper/ # C++ helper to configure eCAL transport modes
│ │ ├── DockerLibrary.py # Robot Framework library to manage Docker containers
│ │ └── GlobalPathsLibrary.py # Library to manage paths, container names, and image tags
│ └── docker/ # Base Dockerfile used to build test environments
│ └── Dockerfile.ecal_base # Base image with all dependencies (eCAL, build tools)
│
├── create_new_test/ # Generator to create new test case folder structure
│ ├── create_new_test.py # Python script to auto-generate new test case folders
│ └── templates/ # Jinja2 templates used by the generator
│
├── .devcontainer/ # Devcontainer setup for local development in VS Code
│ ├── devcontainer.json # Container definition for VS Code Remote Containers
│ └── README.md # Instructions for using the dev container setup
│
├── .github/ # GitHub Actions CI configuration
│ └── workflows/
│ └── run-tests.yml # Main workflow file to build, test, and publish reports
└── README.md # Main documentation file of the repository
If you want to develop and run the tests locally in VS Code without getting header errors (e.g. eCAL or TCLAP not found), we recommend using the included .devcontainer setup.
- Make sure Docker and VS Code are installed.
- Install the Dev Containers extension in VS Code.
- Open this repository in VS Code.
- When prompted, click "Reopen in Container".
- VS Code will build and open a full-featured development environment inside Docker.
This container is based on the same Docker image used for testing, so all dependencies like eCAL and TCLAP are pre-installed. You can develop and run Robot Framework tests directly without needing to configure anything on your host system.
For more details, see: .devcontainer/README.md
To create a new test case folder automatically, use the generator in create_new_test/.