pg 15 and 16 packer/ansible/ghactions #1268
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Co-authored-by: Sam Rose <samuel@supabase.io>
…vileges can run Nix
soedirgo
reviewed
Oct 9, 2024
samrose
added a commit
that referenced
this pull request
Oct 17, 2024
* feat: build and flake check of pg 16.3 with exts/wrappers * pg_partman test 15/16 compat * merge sql interface test * tests: build test and cache both versions * chore: run checks individually * pg 15 and 16 packer/ansible/ghactions (#1268) * fix: reformat ec2 cleanup commands (#1267) Co-authored-by: Sam Rose <samuel@supabase.io> * feat: build and flake check of pg 16.3 with exts/wrappers * pg_partman test 15/16 compat * merge sql interface test * tests: build test and cache both versions * chore: run checks individually * feat: realease 15 and 16 to staging * chore: update versions * chore: make yq available * chore: run yq from nix * chore: more setup for staging AMI * fix: yq usage * chore: shell vars * fix: When --init none is used, only users who can elevate to sudo privileges can run Nix * fix: no -i * fix: quote correction * fix: newline extra quotes * fix: no need for pg major version on packer * fix: postgresql_major * fix: ql * fix: no ansible args in stage to invocation * fix: unique val * fix: adjustments to build scripts * chore: env var handling * fix: bump to build * chore: set up more required vars * chore: bump var * feat: pg 16 debug symbols * feat: matrix pg versions build on testinfra * feat: matrix on Test Database * chore: running nix in the right context * feat: just use existing Dockerfile + pg version * chore: refer to var * fix: read name without including quotes * chore: try format function * fix: strip quotes from version number * chore: env var * fix: pg client * fix * fix: try to use psql from our own corresponding pkg * fix: try psql from ppa * fix: dbmate per pg version * build dbmate and then install client * fix: account for architecture * chore: limit changes detection migrations/schema.sql * missing docker compose call * ore: drop tests while investigating * test: try on pg15 only * chore: schema needs update * chore: now run on all versions in matrix * test: trying a version of schema per major pg version as there are type diffs --------- Co-authored-by: Sam Rose <samuel@supabase.io> Co-authored-by: Oliver Rice <github@oliverrice.com> * chore: rm dead code * chore: resolve versions from ansible/vars.yml instead of packer file create a matrix build for 15 16 (or how ever many versions there are) * fix: don not lose if statements used for hotfix when needed * chore: cleanup --------- Co-authored-by: Sam Rose <samuel@supabase.io> Co-authored-by: Oliver Rice <github@oliverrice.com>
samrose
added a commit
that referenced
this pull request
Oct 22, 2024
* fix: reformat ec2 cleanup commands (#1267) Co-authored-by: Sam Rose <samuel@supabase.io> * feat: build and flake check of pg 16.3 with exts/wrappers * pg_partman test 15/16 compat * merge sql interface test * tests: build test and cache both versions * chore: run checks individually * feat: realease 15 and 16 to staging * chore: update versions * chore: make yq available * chore: run yq from nix * chore: more setup for staging AMI * fix: yq usage * chore: shell vars * fix: When --init none is used, only users who can elevate to sudo privileges can run Nix * fix: no -i * fix: quote correction * fix: newline extra quotes * fix: no need for pg major version on packer * fix: postgresql_major * fix: ql * fix: no ansible args in stage to invocation * fix: unique val * fix: adjustments to build scripts * chore: env var handling * fix: bump to build * chore: set up more required vars * chore: bump var * feat: pg 16 debug symbols * feat: matrix pg versions build on testinfra * feat: matrix on Test Database * chore: running nix in the right context * feat: just use existing Dockerfile + pg version * chore: refer to var * fix: read name without including quotes * chore: try format function * fix: strip quotes from version number * chore: env var * fix: pg client * fix * fix: try to use psql from our own corresponding pkg * fix: try psql from ppa * fix: dbmate per pg version * build dbmate and then install client * fix: account for architecture * chore: limit changes detection migrations/schema.sql * missing docker compose call * ore: drop tests while investigating * test: try on pg15 only * chore: schema needs update * chore: now run on all versions in matrix * test: trying a version of schema per major pg version as there are type diffs --------- Co-authored-by: Sam Rose <samuel@supabase.io> Co-authored-by: Oliver Rice <github@oliverrice.com>
samrose
added a commit
that referenced
this pull request
Nov 25, 2024
* feat: build and flake check of pg 16.3 with exts/wrappers * pg_partman test 15/16 compat * merge sql interface test * tests: build test and cache both versions * chore: run checks individually * pg 15 and 16 packer/ansible/ghactions (#1268) * fix: reformat ec2 cleanup commands (#1267) Co-authored-by: Sam Rose <samuel@supabase.io> * feat: build and flake check of pg 16.3 with exts/wrappers * pg_partman test 15/16 compat * merge sql interface test * tests: build test and cache both versions * chore: run checks individually * feat: realease 15 and 16 to staging * chore: update versions * chore: make yq available * 341A chore: run yq from nix * chore: more setup for staging AMI * fix: yq usage * chore: shell vars * fix: When --init none is used, only users who can elevate to sudo privileges can run Nix * fix: no -i * fix: quote correction * fix: newline extra quotes * fix: no need for pg major version on packer * fix: postgresql_major * fix: ql * fix: no ansible args in stage to invocation * fix: unique val * fix: adjustments to build scripts * chore: env var handling * fix: bump to build * chore: set up more required vars * chore: bump var * feat: pg 16 debug symbols * feat: matrix pg versions build on testinfra * feat: matrix on Test Database * chore: running nix in the right context * feat: just use existing Dockerfile + pg version * chore: refer to var * fix: read name without including quotes * chore: try format function * fix: strip quotes from version number * chore: env var * fix: pg client * fix * fix: try to use psql from our own corresponding pkg * fix: try psql from ppa * fix: dbmate per pg version * build dbmate and then install client * fix: account for architecture * chore: limit changes detection migrations/schema.sql * missing docker compose call * ore: drop tests while investigating * test: try on pg15 only * chore: schema needs update * chore: now run on all versions in matrix * test: trying a version of schema per major pg version as there are type diffs --------- Co-authored-by: Sam Rose <samuel@supabase.io> Co-authored-by: Oliver Rice <github@oliverrice.com> * chore: rm dead code * chore: resolve versions from ansible/vars.yml instead of packer file create a matrix build for 15 16 (or how ever many versions there are) * feat: WIP orioledb 16 * feat: intro into CI building psql16 bundle oriole version * test: working through build compat for oriole * working orioledb build * fix: update value * chore: cut staging only release for oriole16 * narrow down to just oriole staging ami here * fix: make sure name lines up with gh action convention * fix: get correct naming convention in here too for oriole bundle * chore: bump to cut release * fix: make sure src and debug builds available for oriole too * chore: bump to staging release * fix: when oriole16 rm timescale from supautils conf * fix: also remove from postgresql conf * fix: find all instances and remove in this condition for tiemscaledb * fix: maybe broke pgsodium with too many spaces * fix: create a symlink for missing path * fix: try a more universal way to deactivate pgsodium at this stage * chore: when oriole16 rm timescale plv8 postgis pgvector pgrouting * fix: more handling of oriole16 differences * chore more oriole16 handling * fix: refine sed regex * fix: also cond rm actual files * fix: correct dir * chore: bump to release * feat: parse and handle oropledb-16 in start-server * feat: start-client for oriole16 * wip for orioledb push to build on linux * fix: running server * chore: consolidate nix code that handles building of postgres versions * fix: update name for bundle * chore: small cleanup * chore: bump version release * chore: cleanup names * chore: correct names * fix: var name * fix: one more var name * chore: bump to release * fix: reformatting metadata for clikchouse dep as git was not able to fetch * feat: wip refactor to pg 17 for orioledb * feat: working orioledb-17 * fix: increase role duration to avoid expiration * fix: correct version * fix: clickhouse needs git in build inputs * fix: install and then remove git for clickhouse dep * fix: correct rules for version * fix: db_user_namespace was deprecated starting in pg 16.4 and higher * apply fix for wrappers build, deactivate ext in unit test * chore: deactivate more tests due to deactivated extensions * fix: also remove from sql file * fix: regex * fix: also remove ref here * fix: graphql_public schema too * chore: correct dir * chore: staging release * feat: re-introduce wal2json rum and pgvector * fix: also restore pgvector in start-server * chore: bump staging release * feat: oriole specific default settings * feat: use icu for locales only if building orioledb * fix: need var in stage 1 too * fix: settings must be in db init * fix: ansible args * fix: PostgreSQL uses ICU for all locale-related operations, so we don't need to specify the collation and character type settings separately. * chore: add a migration for orioledb activation by default * feat: turn on oriole if oriole ext exists * test: conditional on test for orioledb * fix: salt changed repo and method for adding apt package * fix: udpate salt minion setup * fix: silent skip if oriole not in the install * test: check if oriole is available and if so then enable * chore: build and cache src and debug pkgs as well * fix: tmp supress build of src and debug * chore: tmp disable src and debug * fix: activate oriole first in the oriole context * fix: pgroonga build on macos * fix: restore flake check for now * fixing flake check for darwin + passing check until pg_net * fix: start-server macos * fix: src yq from apt * fix: all instances yq * fix: jq invoke * fix: nix profile install * fix: nix run * fix: sudo nix run * chore: cut staging release * chore: rm debugging * fix: still need CURRENT_SYSTEM * test: re-introduce flake checks for 15 and 16 * chore: bump versions of patched pg for oriole to 17_4 and ext to beta6 * chore: bump var to release ami * chore: break down into functions * fix: skip x86 darwin for oriole nix builds * chore: cleanup repo for final review, rebase and merge * fix: try to properly init db in non oriole context * chore: restore installation of debug and src for all versions` * chore: newline * chore: cleaning up and restoring missing code * chore: cleanup * chore: fix gh action conditonal for oriole * fix: filter orioledb-17 from test on read --------- Co-authored-by: Oliver Rice <github@oliverrice.com>
damonrand
pushed a commit
to cepro/postgres
that referenced
this pull request
Jun 15, 2025
* feat: build and flake check of pg 16.3 with exts/wrappers * pg_partman test 15/16 compat * merge sql interface test * tests: build test and cache both versions * chore: run checks individually * pg 15 and 16 packer/ansible/ghactions (supabase#1268) * fix: reformat ec2 cleanup commands (supabase#1267) Co-authored-by: Sam Rose <samuel@supabase.io> * feat: build and flake check of pg 16.3 with exts/wrappers * pg_partman test 15/16 compat * merge sql interface test * tests: build test and cache both versions * chore: run checks individually * feat: realease 15 and 16 to staging * chore: update versions * chore: make yq available * chore: run yq from nix * chore: more setup for staging AMI * fix: yq usage * chore: shell vars * fix: When --init none is used, only users who can elevate to sudo privileges can run Nix * fix: no -i * fix: quote correction * fix: newline extra quotes * fix: no need for pg major version on packer * fix: postgresql_major * fix: ql * fix: no ansible args in stage to invocation * fix: unique val * fix: adjustments to build scripts * chore: env var handling * fix: bump to build * chore: set up more required vars * chore: bump var * feat: pg 16 debug symbols * feat: matrix pg versions build on testinfra * feat: matrix on Test Database * chore: running nix in the right context * feat: just use existing Dockerfile + pg version * chore: refer to var * fix: read name without including quotes * chore: try format function * fix: strip quotes from version number * chore: env var * fix: pg client * fix * fix: try to use psql from our own corresponding pkg * fix: try psql from ppa * fix: dbmate per pg version * build dbmate and then install client * fix: account for architecture * chore: limit changes detection migrations/schema.sql * missing docker compose call * ore: drop tests while investigating * test: try on pg15 only * chore: schema needs update * chore: now run on all versions in matrix * test: trying a version of schema per major pg version as there are type diffs --------- Co-authored-by: Sam Rose <samuel@supabase.io> Co-authored-by: Oliver Rice <github@oliverrice.com> * chore: rm dead code * chore: resolve versions from ansible/vars.yml instead of packer file create a matrix build for 15 16 (or how ever many versions there are) * fix: don not lose if statements used for hotfix when needed * chore: cleanup --------- Co-authored-by: Sam Rose <samuel@supabase.io> Co-authored-by: Oliver Rice <github@oliverrice.com>
damonrand
pushed a commit
to cepro/postgres
that referenced
this pull request
Jun 15, 2025
* feat: build and flake check of pg 16.3 with exts/wrappers * pg_partman test 15/16 compat * merge sql interface test * tests: build test and cache both versions * chore: run checks individually * pg 15 and 16 packer/ansible/ghactions (supabase#1268) * fix: reformat ec2 cleanup commands (supabase#1267) Co-authored-by: Sam Rose <samuel@supabase.io> * feat: build and flake check of pg 16.3 with exts/wrappers * pg_partman test 15/16 compat * merge sql interface test * tests: build test and cache both versions * chore: run checks individually * feat: realease 15 and 16 to staging * chore: update versions * chore: make yq available * chore: run yq from nix * chore: more setup for staging AMI * fix: yq usage * chore: shell vars * fix: When --init none is used, only users who can elevate to sudo privileges can run Nix * fix: no -i * fix: quote correction * fix: newline extra quotes * fix: no need for pg major version on packer * fix: postgresql_major * fix: ql * fix: no ansible args in stage to invocation * fix: unique val * fix: adjustments to build scripts * chore: env var handling * fix: bump to build * chore: set up more required vars * chore: bump var * feat: pg 16 debug symbols * feat: matrix pg versions build on testinfra * feat: matrix on Test Database * chore: running nix in the right context * feat: just use existing Dockerfile + pg version * chore: refer to var * fix: read name without including quotes * chore: try format function * fix: strip quotes from version number * chore: env var * fix: pg client * fix * fix: try to use psql from our own corresponding pkg * fix: try psql from ppa * fix: dbmate per pg version * build dbmate and then install client * fix: account for architecture * chore: limit changes detection migrations/schema.sql * missing docker compose call * ore: drop tests while investigating * test: try on pg15 only * chore: schema needs update * chore: now run on all versions in matrix * test: trying a version of schema per major pg version as there are type diffs --------- Co-authored-by: Sam Rose <samuel@supabase.io> Co-authored-by: Oliver Rice <github@oliverrice.com> * chore: rm dead code * chore: resolve versions from ansible/vars.yml instead of packer file create a matrix build for 15 16 (or how ever many versions there are) * feat: WIP orioledb 16 * feat: intro into CI building psql16 bundle oriole version * test: working through build compat for oriole * working orioledb build * fix: update value * chore: cut staging only release for oriole16 * narrow down to just oriole staging ami here * fix: make sure name lines up with gh action convention * fix: get correct naming convention in here too for oriole bundle * chore: bump to cut release * fix: make sure src and debug builds available for oriole too * chore: bump to staging release * fix: when oriole16 rm timescale from supautils conf * fix: also remove from postgresql conf * fix: find all instances and remove in this condition for tiemscaledb * fix: maybe broke pgsodium with too many spaces * fix: create a symlink for missing path * fix: try a more universal way to deactivate pgsodium at this stage * chore: when oriole16 rm timescale plv8 postgis pgvector pgrouting * fix: more handling of oriole16 differences * chore more oriole16 handling * fix: refine sed regex * fix: also cond rm actual files * fix: correct dir * chore: bump to release * feat: parse and handle oropledb-16 in start-server * feat: start-client for oriole16 * wip for orioledb push to build on linux * fix: running server * chore: consolidate nix code that handles building of postgres versions * fix: update name for bundle * chore: small cleanup * chore: bump version release * chore: cleanup names * chore: correct names * fix: var name * fix: one more var name * chore: bump to release * fix: reformatting metadata for clikchouse dep as git was not able to fetch * feat: wip refactor to pg 17 for orioledb * feat: working orioledb-17 * fix: increase role duration to avoid expiration * fix: correct version * fix: clickhouse needs git in build inputs * fix: install and then remove git for clickhouse dep * fix: correct rules for version * fix: db_user_namespace was deprecated starting in pg 16.4 and higher * apply fix for wrappers build, deactivate ext in unit test * chore: deactivate more tests due to deactivated extensions * fix: also remove from sql file * fix: regex * fix: also remove ref here * fix: graphql_public schema too * chore: correct dir * chore: staging release * feat: re-introduce wal2json rum and pgvector * fix: also restore pgvector in start-server * chore: bump staging release * feat: oriole specific default settings * feat: use icu for locales only if building orioledb * fix: need var in stage 1 too * fix: settings must be in db init * fix: ansible args * fix: PostgreSQL uses ICU for all locale-related operations, so we don't need to specify the collation and character type settings separately. * chore: add a migration for orioledb activation by default * feat: turn on oriole if oriole ext exists * test: conditional on test for orioledb * fix: salt changed repo and method for adding apt package * fix: udpate salt minion setup * fix: silent skip if oriole not in the install * test: check if oriole is available and if so then enable * chore: build and cache src and debug pkgs as well * fix: tmp supress build of src and debug * chore: tmp disable src and debug * fix: activate oriole first in the oriole context * fix: pgroonga build on macos * fix: restore flake check for now * fixing flake check for darwin + passing check until pg_net * fix: start-server macos * fix: src yq from apt * fix: all instances yq * fix: jq invoke * fix: nix profile install * fix: nix run * fix: sudo nix run * chore: cut staging release * chore: rm debugging * fix: still need CURRENT_SYSTEM * test: re-introduce flake checks for 15 and 16 * chore: bump versions of patched pg for oriole to 17_4 and ext to beta6 * chore: bump var to release ami * chore: break down into functions * fix: skip x86 darwin for oriole nix builds * chore: cleanup repo for final review, rebase and merge * fix: try to properly init db in non oriole context * chore: restore installation of debug and src for all versions` * chore: newline * chore: cleaning up and restoring missing code * chore: cleanup * chore: fix gh action conditonal for oriole * fix: filter orioledb-17 from test on read --------- Co-authored-by: Oliver Rice <github@oliverrice.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What kind of change does this PR introduce?
This PR is a child PR of #1266
It includes the changes to packer/ansible/gh actions to allow for a matrix build of pg versions on release.
Significant changes
This PR moves the source of truth for data about postgres major and semver back to ansible/vars.yml, which are then read at time of release in the ami-release gh action. The common-vars.nix.yml file is no longer under source control, but instead is generated duing the ci run for ami release or testinfra runs. The version is now controlled in this allows us to avoid making too many changes to our release and test tooling for AMI)
To bump the versions of postgres it is proposed to update these values that are now in
anisble/vars.ymlThese versions cover both release and testinfra