I think we need to be able to pass to the release script a set of base images for mediawiki and web, and build the final images for each of those pairs.

Right now the mediawiki-deployments.yaml file that scap reads has a series of simple dictionaries, structured as follows:

- namespace: my-mw-namespace
  release: main
  canary: canary
  mw_flavour: publish
  web_flavour: webserver
  debug: false

So in theory we already have a way to indicate the mediawiki and web flavours to build for there, we just need another configuration that tells the build process what base image corresponds to what flavour?

There's also more improvements that we'll need to make to scap, but that's probably matter of another task.

Agreed with @Joe's assessment above: for each image type (e.g., mediawiki), scap would need to support a configurable set of base images from which the image will be built ("flavors").

To expand on some of the additional improvements that will be necessary to support the migration to 8.1:

1. For a given namespace in mediawiki-deployments.yaml, scap should support additional / arbitrary helmfile releases - i.e., not just a primary release like "main" plus a canary release.
2. In addition, scap should offer the ability to map those releases in to specific image flavors in a sensible way (again presumably in mediawiki-deployments.yaml), in turn used to render the files in /etc/helmfile-defaults/mediawiki/release/.

This would support both the 8.1 migration and any future migration that uses a similar pattern.

Thanks for chatting earlier today @dduvall.

To summarize, the core requirement is that scap gains the ability to build multiple "flavors" for each image "kind" (mw, web), where flavor in this case corresponds to which base image is used, and publish that information for consumption by helmfile.

The first part is relatively straightforward - e.g., introducing a mapping (for each image kind) from flavor to base image, and parameterizing make-container-image et al. by base image / flavor.

The second part - publishing the resulting image IDs for consumption by helmfile - is a bit more open ended. What I'd propose is generalizing our deployments mapping [0] (i.e., what ends up in mediawiki-deployments.yaml) to use helmfile release as the "API" of sorts, meaning we would need to:

1. support an arbitrary set of releases per k8s namespace, rather than a singular primary release plus one optional canary, and
2. allow releases to be mapped to a specific flavor for to each image kind.

For example, this might end up looking like (in "maximally verbose" form - e.g., one could add a release defaults field or allow specifying certain fields at the top-level rather than per-release [1], either of which would deflate this significantly):

- namespace: mw-web
  releases:
    main:
      mw_flavour: publish-7.4
      web_flavour: webserver
    next:
      mw_flavour: publish-8.1
      web_flavour: webserver
    canary:
      stage: canaries
      mw_flavour: publish-7.4
      web_flavour: webserver
  debug: false

(Edited slightly to switch from a canary boolean to a stage association)

Then, nothing really changes about the "API" between scap and helmfile - it's all just new /etc/helmfile-defaults/mediawiki/release/ files, structured the same way as before.

This has the nice property that it retains the model we use now in our mediawiki chart, where release is ~ 1:1 with k8s deployment, which is pretty much exactly what we want for the purposes of a migration like this one. Isolating by release also allows us to easily deploy "new thing" independently and layer value overrides easily, which we'll need to do extensively in the context of the migration.

[0] https://gerrit.wikimedia.org/g/operations/puppet/+/df0ce14049768faff2c1738e7cad793fda53e71e/hieradata/role/common/deployment_server/kubernetes.yaml#261

[1] In this case, the above could collapse to:

- namespace: mw-web
  releases:
    main: {}
    next:
      mw_flavour: publish-8.1
    canary:
      stage: canaries
  mw_flavour: publish-7.4
  web_flavour: webserver
  debug: false

swfrench opened https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/462

kubernetes: generalize deployments configuration

dancy merged https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/462

kubernetes: generalize deployments configuration

Mentioned in SAL (#wikimedia-operations) [2024-10-02T15:22:44Z] <dancy@deploy2002> Started scap sync-world: Testing T370934

Mentioned in SAL (#wikimedia-operations) [2024-10-02T15:26:04Z] <dancy@deploy2002> Finished scap sync-world: Testing T370934 (duration: 03m 19s)

Change #1077479 had a related patch set uploaded (by Scott French; author: Scott French):

[operations/puppet@production] types: extend Profile::Mediawiki_deployment

https://gerrit.wikimedia.org/r/1077479

Change #1077480 had a related patch set uploaded (by Scott French; author: Scott French):

[operations/puppet@production] hieradata: use 'releases' in mw-debug mw_releases

https://gerrit.wikimedia.org/r/1077480

Change #1077482 had a related patch set uploaded (by Scott French; author: Scott French):

[operations/puppet@production] hieradata: convert remaining mw_releases entries

https://gerrit.wikimedia.org/r/1077482

Change #1077483 had a related patch set uploaded (by Scott French; author: Scott French):

[operations/puppet@production] types: remove older Mediawiki_deployment variant

https://gerrit.wikimedia.org/r/1077483

Change #1077479 merged by Scott French:

[operations/puppet@production] types: extend Profile::Mediawiki_deployment

https://gerrit.wikimedia.org/r/1077479

Change #1077480 merged by Scott French:

[operations/puppet@production] hieradata: use 'releases' in mw-debug mw_releases

https://gerrit.wikimedia.org/r/1077480

Mentioned in SAL (#wikimedia-operations) [2024-10-03T17:11:07Z] <swfrench@deploy2002> Started scap sync-world: Testing after mediawiki-deployments.yaml format change - T370934

Mentioned in SAL (#wikimedia-operations) [2024-10-03T17:13:57Z] <swfrench@deploy2002> Finished scap sync-world: Testing after mediawiki-deployments.yaml format change - T370934 (duration: 02m 50s)

Test deployment looks good. I'll wait at least a day (so, effectively, until Monday) to make any changes that critically depend on the new format, in case any surprise consumers of mediawiki-deployments.yaml are broken and need fixed.