Write real-time functions that compute over temporal data streams with full historical context.
Homepage
·
Documentation
·
Roadmap
The hardware requirement for running is relatively low. All you need is a machine with at least 1 GB of RAM, a CPU with at least 1 free core, and a place to store your data (data ratio is about 8.7MB/1M events).
To build Slung from source is pretty simple:
- Make sure you have Zig, and Nix installed.
- Clone the repository locally.
- Then run the following commands.
nix develop -c zig build --release=fast
cp ./zig-out/bin/slung .See our docs for more information.
Warning
This project is currently WIP and is no where near ready for production use.
| # | Objective | Status |
|---|---|---|
| 1 | Stream pipeline [#1] | |
| 2 | TSM tree [#2] | ✅ |
| 3 | Parallel data query engine [#3] | |
| 4 | Wasm execution | ✅ |
| 5 | Gateway | 📅 |
| 6 | Object storage sync | 📅 |
| 7 | Write-ahead log (WAL) | 🚫 |
| 8 | WebTransport | 🚫 |
- ✅ Done: Feature is implemented and verified.
⚠️ In-progress: Active development or stabilization phase.- 📅 Planned: Queued for development; architecture defined.
- 🚫 Deferred: Not currently in scope for the current milestone.
- Non-blocking websocket channel [#4]
- Stream rx/tx pool manager [#5]
- More streaming primitives (WebTransport, NATS, Kafka). Via Gateway
- Multi-threaded networking (config). Note: this is implicitly supported.
- Cache (Skip list)
- Disk entry
- AMQ Filter (Bloom filter)
- Encoding (Gorilla/delta)
- Multi-level compaction
Optimisation: ReleaseFast. i5-10210U. SAMSUNG MZVLB256HAHQ-000H1:
...
ingested 1 billion in 772.57s
write latency per item: 772ns
write speed: 1295336 WPS
peak mem: 575 MiB
--- Storage ---
Total: 9116360690 bytes (8.49 GB)
Bytes per point: 9.12
--- Query Benchmark ---
Querying h-9 range 999000000-999999999 (1M points)
avg: 49.9944
Run 1: 168.19ms
Run 2: 166.62ms
Run 3: 157.24ms
Run 4: 158.76ms
Run 5: 159.87ms
...
Optimisation: ReleaseFast:
TODO: benchmark
- Host functions
- Life cycle manager
- HTTP write back