Sam/15 16 nix package, build and test of 15.8 and 16.3 #1266
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
olirice
approved these changes
Oct 9, 2024
* fix: reformat ec2 cleanup commands (#1267) Co-authored-by: Sam Rose <samuel@supabase.io> * feat: build and flake check of pg 16.3 with exts/wrappers * pg_partman test 15/16 compat * merge sql interface test * tests: build test and cache both versions * chore: run checks individually * feat: realease 15 and 16 to staging * chore: update versions * chore: make yq available * chore: run yq from nix * chore: more setup for staging AMI * fix: yq usage * chore: shell vars * fix: When --init none is used, only users who can elevate to sudo privileges can run Nix * fix: no -i * fix: quote correction * fix: newline extra quotes * fix: no need for pg major version on packer * fix: postgresql_major * fix: ql * fix: no ansible args in stage to invocation * fix: unique val * fix: adjustments to build scripts * chore: env var handling * fix: bump to build * chore: set up more required vars * chore: bump var * feat: pg 16 debug symbols * feat: matrix pg versions build on testinfra * feat: matrix on Test Database * chore: running nix in the right context * feat: just use existing Dockerfile + pg version * chore: refer to var * fix: read name without including quotes * chore: try format function * fix: strip quotes from version number * chore: env var * fix: pg client * fix * fix: try to use psql from our own corresponding pkg * fix: try psql from ppa * fix: dbmate per pg version * build dbmate and then install client * fix: account for architecture * chore: limit changes detection migrations/schema.sql * missing docker compose call * ore: drop tests while investigating * test: try on pg15 only * chore: schema needs update * chore: now run on all versions in matrix * test: trying a version of schema per major pg version as there are type diffs --------- Co-authored-by: Sam Rose <samuel@supabase.io> Co-authored-by: Oliver Rice <github@oliverrice.com>
|
How do these workflow changes impact the workflows which are tightly bound to the i.e. https://github.com/supabase/postgres/blob/develop/.github/workflows/publish-nix-pgupgrade-scripts.yml The current workflow generates a git tag for every AMI release - do these changes maintain this behaviour? |
Yes they do as demonstrated when I ran a staging release they are visible at https://github.com/supabase/postgres/releases (those with -staging-) |
create a matrix build for 15 16 (or how ever many versions there are)
|
Going to address the docker work in a soon to be completed follow up PR to try to keep this work moving along, and break up into smaller chunks |
|
@samrose that works great, thanks! Only nit I'd have is that we'd want to preserve the overrides (the |
|
Looks good, thanks! Approved. |
pcnc
approved these changes
Oct 17, 2024
damonrand
pushed a commit
to cepro/postgres
that referenced
this pull request
Jun 15, 2025
* feat: build and flake check of pg 16.3 with exts/wrappers * pg_partman test 15/16 compat * merge sql interface test * tests: build test and cache both versions * chore: run checks individually * pg 15 and 16 packer/ansible/ghactions (supabase#1268) * fix: reformat ec2 cleanup commands (supabase#1267) Co-authored-by: Sam Rose <samuel@supabase.io> * feat: build and flake check of pg 16.3 with exts/wrappers * pg_partman test 15/16 compat * merge sql interface test * tests: build test and cache both versions * chore: run checks individually * feat: realease 15 and 16 to staging * chore: update versions * chore: make yq available * chore: run yq from nix * chore: more setup for staging AMI * fix: yq usage * chore: shell vars * fix: When --init none is used, only users who can elevate to sudo privileges can run Nix * fix: no -i * fix: quote correction * fix: newline extra quotes * fix: no need for pg major version on packer * fix: postgresql_major * fix: ql * fix: no ansible args in stage to invocation * fix: unique val * fix: adjustments to build scripts * chore: env var handling * fix: bump to build * chore: set up more required vars * chore: bump var * feat: pg 16 debug symbols * feat: matrix pg versions build on testinfra * feat: matrix on Test Database * chore: running nix in the right context * feat: just use existing Dockerfile + pg version * chore: refer to var * fix: read name without including quotes * chore: try format function * fix: strip quotes from version number * chore: env var * fix: pg client * fix * fix: try to use psql from our own corresponding pkg * fix: try psql from ppa * fix: dbmate per pg version * build dbmate and then install client * fix: account for architecture * chore: limit changes detection migrations/schema.sql * missing docker compose call * ore: drop tests while investigating * test: try on pg15 only * chore: schema needs update * chore: now run on all versions in matrix * test: trying a version of schema per major pg version as there are type diffs --------- Co-authored-by: Sam Rose <samuel@supabase.io> Co-authored-by: Oliver Rice <github@oliverrice.com> * chore: rm dead code * chore: resolve versions from ansible/vars.yml instead of packer file create a matrix build for 15 16 (or how ever many versions there are) * fix: don not lose if statements used for hotfix when needed * chore: cleanup --------- Co-authored-by: Sam Rose <samuel@supabase.io> Co-authored-by: Oliver Rice <github@oliverrice.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What kind of change does this PR introduce?
This makes all the necessary changes to package, build and test in the nix/pg_regress context our psql bundles with 15.8 and 16.3
We attempt to change no versions, and as much as possible to keep 1 pg_regress test suite, and one set of extensions and wrappers.
There will be a follow up PR to address the packer/ansible/gh action work for integration tests and releasing images.
It includes the changes to packer/ansible/gh actions to allow for a matrix build of pg versions on release.
Significant changes
This PR moves the source of truth for data about postgres major and semver back to ansible/vars.yml, which are then read at time of release in the ami-release gh action. The common-vars.nix.yml file is no longer under source control, but instead is generated duing the ci run for ami release or testinfra runs. The version is now controlled in this allows us to avoid making too many changes to our release and test tooling for AMI)
To bump the versions of postgres it is proposed to update these values that are now in
anisble/vars.ymlThese versions cover both release and testinfra