Over a hangout today, @MoritzMuehlenhoff, @Joe, @faidon discussed different ways we could segment the cluster network to make extracting sensitive data harder for an attacker. In the short term, there are other projects with more security benefit. But documenting this here so we can revisit in the future.
Currently, setting up a vlan is labor intensive, so it isn't practical to put each new service in its own vlan We could probably setup an untrusted or trusted vlan within each datacenter.
- Trusted: We would move sensitive data services to access that data to an isolate network. Currently, that would likely be Password hashes, CheckUser data, and maybe logs.
- Untrusted: We would move services like Citoid, Mathoid, which don't need access to the rest of the cluster to work, into their own network so a compromise there limits the attacker to those services.
A prerequisite for setting up another vlan in the cluster is moving network configuration to an orchestration / code review system.