[FEATURE REQUEST] Making CodeGraphContext scalable #714
Description
Enable multiple CodeGraphContext instances to run in parallel (indexers + query API), share a common datastore/state (sharded index or central DB + cache), and coordinate work so the system can be deployed and scaled in Kubernetes for high throughput and availability.
Why
- Support larger codebases and higher query throughput.
- Improve availability and enable rolling upgrades.
- Allow cloud-native deployments (Kubernetes, Nomad) and easier ops.
- Allows us to use CodeGraphContext for a lot of repositories by a lot of developers at the same time.
Acceptance criteria
- Multiple indexer and API instances can run concurrently against the same store without corrupting indexes or returning inconsistent query results.
- Sharding or coordination mechanism for index building (e.g., work queue + lease/lock) is implemented.
- Distributed cache (Redis/ElastiCache) or search layer (Elasticsearch/Opensearch) integration for fast query scaling.
- Health/readiness/liveness endpoints and metrics (requests, queue length, shard status).
Implementation notes?
- Use a shared object store or DB for raw data;
- Coordinate index jobs via durable queue (Redis streams, Kafka) and use leases/locks for worker exclusivity.
- Make components stateless where possible; externalize state to DB/cache.