As promised here are the prototypes for dumping and loading Server Configuration Profiles for the previous design. The idea would be to implement these in the redfish module in spirerack, but I have not had an opportunity to complete that.

In T247364#7008439, @MoritzMuehlenhoff wrote:

@crusnov Could you please take modules/raid/files/check-raid.py with precedence? It's part of a Bullseye base install and thus affected by it's lack of Python 2.

In T271136#6999009, @crusnov wrote:

In T271136#6999003, @elukey wrote:

@crusnov if you have time let's do it this week or the next!

Yes (thank you for the ping), let's do it first thing tomorrow, I'll ping on IRC when I add the DNS and deploy it.

In T271136#6999003, @elukey wrote:

@crusnov if you have time let's do it this week or the next!

I have discussed moving forward on the production side with @ayounsi and we've talked about the requirements for production from a network perspective. Basically it'd be good to ensure nothing bad will happen by having the mgmt->dhcp holes open to the install servers and keep track of that to some extent. It'd be good, if possible, to monitor this also.

• added my full working todo list for this project

In T271136#6954616, @elukey wrote:

We have a special setting in commons.yaml, kafka_brokers_main, that it is used IIRC to instruct zookeeper about what connections to accept, and I see that it already includes kafka-main200[1-3]'s ipv6 addresses, so in theory we should be good. kafka-main200[4,5] have role insetup so they can be done anytime.

Tried a simple telnet and it worked via ipv6:

elukey@kafka-main2001:~$ telnet conf2001.codfw.wmnet 2181
Trying 2620:0:860:101:10:192:0:143...
Connected to conf2001.codfw.wmnet.

The other side effect will be that kafka clients may start using the ipv6 addresses to send traffic to, but I can't think about any problem on this side. Since it is a delicate cluster I'd do a slow rollout (one host at the time), and also inform ServiceOps about it as FYI :)

In T271142#6960492, @akosiaris wrote:

rdb looks straight forward.

Yes it does. ferm configuration is just opening up the ports there so it's IPv4/IPv6 agnostic and redis isn't listening on IPv6 there (it is capable though). OS stacks will fallback to IPv4 so everything should be fine. That being said, let's coordinate on this, ping me online before publishing the DNS entries so I can make sure nothing breaks.

Since the maps servers are being replaced? I think? Perhaps we can cross them off for this project. Am I right in that this is happening?

@fgiunchedi Is there any process we should follow to test/make sure everything is okay if we add ipv6 DNS for ms-be and ms-fe?

• Moved dbproxy to T271138

In T271142#6957794, @crusnov wrote:

Sounds good, if you'd like to ping me on IRC when you want to do this project we can do at least the first stage.

In T271142#6957437, @ArielGlenn wrote:

In T271142#6957026, @crusnov wrote:

In T271142#6956040, @ArielGlenn wrote:

Having been duly poked, may I ask which services you are looking at, both for dumpsdata* and snapshot*, that listen only on IPv4? Then we can talk about whether they should also handle IPv6 and if so, how to get there. Thanks!

The base question is if it is safe under current circumstances to add AAAA DNS. Just eyeballing the list of services on the boxes it looked like they all are already on IPv6, and the request for information is if this assessment is correct (and thus it is safe to add AAAA DNS) and if it isn't what do we need to do so to make it correct (and if that work is worth it).

THanks :)

I don't think it will make a difference but let's start with a testbed host, snapshot1005, and after it's got the entry I'll run some tests there. If that checks out, as I expect, we can do all the snapshots. I'll have to think about safe tests for the dumpsdata hosts, but I have an idea there too for after the snapshots.

A thing we discovered today that should also be imported from Netbox to puppet is the PDU list which is used to produce monitoring, stored in modules/facilities/manifests/init.pp

In T271142#6956040, @ArielGlenn wrote:

Having been duly poked, may I ask which services you are looking at, both for dumpsdata* and snapshot*, that listen only on IPv4? Then we can talk about whether they should also handle IPv6 and if so, how to get there. Thanks!

In T271142#6955077, @akosiaris wrote:

Hi!

TL;DR: Aside from snapshot and dumpsdata that @ArielGlenn is better equipped to answer for, maybe we can do scandium and rdb* but the rest are either high risk/just not worth it/both

Here's a quick survey of the hosts listed above and maybe some potential problems in just adding AAAA DNS records to these clusters:

Thanks for following up.

A quick survey of the clusters above:

It appears the kafka-main2* cluster is indeed listening on ipv6, it just seems to need DNS (especially in the face of the eqiad ones already having this DNS). Is there any particular care that's needed here?

Thank you for the extra explanation. Given that these will have the same problems as the db hosts, we will mark them as skipping for now / on back burner.

In T271144#6950499, @crusnov wrote:

The point of the project is to get as many hosts to have an IPv6 address (and, obviously, to be functional on that address) as we can, and, in general, for it to be default to have IPv6 addresses in DNS. If it's not appropriate for a particular cluster, that's a valid outcome.

In this case the load balancer servers listed above indeed do not have IPv6 DNS. This ticket requests any information needed to add these hosts's IPv6 addresses to our DNS or to prompt the actions required to do so.

Yes, apologies for the ambiguity, this is specifically about AAAA records. The assumption we made back when we imported DNS to Netbox was that if there was not AAAA record for the box we should not add one in Netbox for risk of something unexpected happening (such as firewall rules not applying to/being open to IPv6 and services not listening on the IPv6 address). Basically, will anything bad happen if we add the DNS entries?

I guess given that the ganeti clusters will wait for Buster, the Kafka cluster is the only one remaining. What needs to be done for this?

Thanks! I'm closing this ticket and updating the parent to reflect these changes.

The point of the project is to get as many hosts to have an IPv6 address (and, obviously, to be functional on that address) as we can, and, in general, for it to be default to have IPv6 addresses in DNS. If it's not appropriate for a particular cluster, that's a valid outcome.

In T265904#6945448, @Volans wrote:

We should fix the script IMHO and once we're sure it would not re-add any SLAAC IP just delete all of them from Netbox. Thoughts?

So the idea is we'd like overall for all of our clusters to have IPv6 reachability. This is not terribly urgent, just a state that has remained a long time and we'd like to rectify.

In T277705#6923085, @faidon wrote:

@crusnov maybe you can have a look?

So we discussed this at the automation meeting, and it turns out we've all agreed that the current code and patches need to be thrown out entirely and the project redone with the django-auth-cas-ng solution because of the Logout Problem. Basically this is an unrealized problem involving mod_cas keeping sessions separately from CAS or Django. Each of these keep their own sessions, and even if the django or the CAS sessions get invalidated, the mod_cas session lives on, and re-creates the Django session. Rather than kludging it to invalidate this session also, removing the apache layer seems like a better fix.

In T244849#6915834, @Volans wrote:

In T244849#6915810, @crusnov wrote:

After reviewing the LDAP authentication library I can say that you're correct, it does do this in its backend. If this is a required functionality it can be added to our current authentication backend in a way similar to how LDAP does it.

I think that it's required to avoid the security issue of a user removed from an LDAP group keeping the previous access and the usability issue of a user that was added to a more privileged group that will not gain the expected privileges.

In T244849#6915745, @Volans wrote:

IIRC the LDAP auth re-syncs the groups all the time because needs to be sure the user reflects their current groups, to ensure that their access is consistent with their real groups and doesn't grant/revoke permissions because of a missing/stale group. But please double check this assumption.

In T244849#6915715, @Volans wrote:

@crusnov @jbond
I did some quick testing on netbox-next and I have some questions:

1. Where are the logging messages saved?

@BBlack Thanks!

In T276585#6902476, @BBlack wrote:

@crusnov - It's been followed up offline from phab in general with some meetings, the output of which aren't (yet) reflected in phab, if you're just looking for whether it's being ignored or not! :)

Has this been followed up with an NDA ticket? @MNadrofsky

Update on progress: I discussed the possibilities and situation with @jbond, with the idea that adapting RemoteUserBackend was the general consensus of the above discussion.

I've experimented with SSO on netbox-next and been reading a lot of code, and this is an update on all of that.

There is also https://djangocas.dev