Pull Request Test Coverage Report for Build 10247045417

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

• For more information on this, see Tracking coverage changes with pull request builds.
• To avoid this issue with future PRs, see these Recommended CI Configurations.
• For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

• 0 of 0 changed or added relevant lines in 0 files are covered.
• No unchanged relevant lines lost coverage.
• Overall coverage increased (+0.3%) to 87.946%

Totals
Change from base Build 9916528000:	0.3%
Covered Lines:	2590
Relevant Lines:	2945

💛 - Coveralls

Quickly looking at the logs, when it starts with the musllinux docker image, I see:

Running before_all...
  
      + /opt/python/cp38-cp38/bin/python -c 'import sys, json, os; json.dump(os.environ.copy(), sys.stdout)'
      + sh -c ./ci/install_geos.sh
  /project /project
  Using cached install /host/home/runner/work/_temp/geos-3.11.4
  /project

So it's not rebuilding GEOS in the musllinux image, which seems problematic. We probably need some logic in install_geos.sh to avoid doing that (include the architecture in the path?)

The primary issue that I see that libgeos is built (and cached) using the github runner image, while cibuildwheel uses a different image. These images have different libraries, and are incompatible with each other.

I'm having some good success using custom linux images that I've published as packages for the shapely org (although currently private; for public versions look here). For example a Dockerfile to create a custom image is:

# docker build -t ghcr.io/shapely/musllinux_1_1_x86_64_geos:3.12.2 --build-arg GEOS_VERSION=3.12.2 musllinux_1_1_x86_64
# docker run -it --rm ghcr.io/shapely/musllinux_1_1_x86_64_geos:3.12.2

FROM quay.io/pypa/musllinux_1_1_x86_64:latest

ARG GEOS_VERSION=local
ENV GEOS_VERSION=${GEOS_VERSION}

RUN curl -OL --retry 5 https://download.osgeo.org/geos/geos-$GEOS_VERSION.tar.bz2
RUN tar xfj geos-$GEOS_VERSION.tar.bz2 && rm geos-$GEOS_VERSION.tar.bz2
RUN cmake -DCMAKE_BUILD_TYPE=Release -S geos-$GEOS_VERSION -B build
RUN cmake --build build -j 4
RUN cmake --install build
RUN cd build && ctest --output-on-failure
RUN rm -rf build geos-$GEOS_VERSION

I also have success with aarch64 via emulation, so have updated the circleCI configuration.

I'm able to push these images into GitHub packages/containers here although they are currently "private", as I don't have necessary permissions to make them public. @jorisvandenbossche (or @sgillies) are you able to review and enable? The Dockerfiles for the images can be put into a subrepo too, which would be linked to the packages. But again, I don't have the credentials to create new repos (no wait, I can make these just fine; see shapely/manylinux_geos).

Good news, the linux binary wheels look good! I'm using my personal containers at ghcr.io/mwtoews for now, but I'd prefer to see ghcr.io/shapely be used for this PR when ready.

Bad news, some unrelated issue appears to have broken the macos-14 builds.

@mwtoews @jorisvandenbossche I'll make the images public and then tick the "public" option here if that's okay with you.

Update: seems like we'll have to check that box first and then you can update your package settings to make them public @mwtoews .

@mwtoews will try to take a closer look tomorrow, but just a quick note that in the pyogrio wheel building workflow, we build and cache custom images in github actions using docker/build-push-action, without the need to manually push some images somewhere:

https://github.com/geopandas/pyogrio/blob/4aea0909c7371db9528ed999689e672f7af6f0da/.github/workflows/release.yml#L101-L113

Thanks for pointing me to pyogrio's approach @jorisvandenbossche ! I see a similar approach could be used, which is much simpler with only libgeos to bundle.

Yes, indeed for pyogrio the dockerfiles are quite complicated because it needs to build GDAL, as you showed above a customized manylinux dockerfile for adding GEOS can be simpler.