8000 Attempt to bring in builds for python 3.13 by JacobCallahan · Pull Request #5 · JacobCallahan/ssh2-python · GitHub
[go: up one dir, main page]
Skip to content

Attempt to bring in builds for python 3.13 #5

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
JacobCallahan wants to merge 2 commits into from
Closed

Attempt to bring in builds for python 3.13 #5

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
da6ab86
Attempt to bring in builds for python 3.13
JacobCallahan Nov 25, 2024
530530b
Change how libssh2 is vendored
JacobCallahan Nov 25, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
79 changes: 27 additions & 52 deletions .github/workflows/build_and_release.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,73 +35,41 @@ jobs:
name: sdist
path: ./dist/*

- name: Sanity check sdist files
run: |
ls ./dist

- name: Output sdist name
id: save-path
shell: bash -el {0}
run: echo "sdist_name=$(ls ./dist)" >> "$GITHUB_OUTPUT"

build_wheels:
needs: build_sdist
name: Build wheels on ${{ matrix.os }}
name: Build Wheels
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
fail-fast: true
matrix:
# macos-13 is an intel runner, macos-14 is an arm runner
# windows-latest is currently not working
os: [ubuntu-latest, macos-13, macos-14]

os: [ubuntu-latest, macos-latest, windows-latest]
cibw_python: ["3.10", "3.11", "3.12", "3.13"]
steps:
- name: Checkout ssh2-python
uses: actions/checkout@v4
with:
fetch-depth: 0

- name: Download sdist
uses: actions/download-artifact@v4
with:
name: sdist
path: ./dist

- name: Output sdist name
id: save-path
shell: bash -el {0}
run: echo "sdist_name=$(ls ./dist)" >> "$GITHUB_ENV"

- name: Unzip sdist (macOS)
if: startsWith(matrix.os, 'macos')
run: |
tar -xzf ./dist/${{ env.sdist_name }} -C ./dist

- name: Output sdist name (macOS)
id: save-path2
shell: bash -el {0}
run: echo "sdist_name=$(cd ./dist && ls -d */)" >> "$GITHUB_ENV"

- name: Set up Python ${{ matrix.cibw_python }}
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.cibw_python }}

- name: Install python dependencies
run: |
pip install --upgrade pip
pip install -r requirements_dev.txt
pip install --upgrade pip
pip install -r requirements_dev.txt

- name: Build normal wheels
uses: pypa/cibuildwheel@v2.17
with:
package-dir: ./dist/${{ startsWith(matrix.os, 'macos') && env.sdist_name || needs.build_sdist.outputs.sdist_file }}
package-dir: ./dist/${{ startsWith(matrix.os, 'macos') && env.sdist_name || needs.build_sdist.outputs.sdist_file }}
env:
CIBW_SKIP: "*musllinux*"
CIBW_BUILD: "cp310-* cp311-* cp312-*"
CIBW_PRERELEASE_PYTHONS: True
CIBW_BUILD: "cp310-* cp311-* cp312-* cp313-*"
CIBW_PRERELEASE_PYTHONS: true
CIBW_BEFORE_ALL_LINUX: "yum install -y cmake openssh-server openssl-devel"
CIBW_BEFORE_ALL_MACOS: "brew install cmake libssh2"
CIBW_BEFORE_ALL_WINDOWS: "choco install cmake openssh"

- uses: actions/upload-artifact@v4
with:
Expand All @@ -113,16 +81,23 @@ jobs:
runs-on: ubuntu-latest
environment: pypi
permissions:
id-token: write
contents: read
packages: write
steps:
- uses: actions/download-artifact@v4
- name: Download sdist
uses: actions/download-artifact@v2
with:
name: sdist
path: ./dist

- name: Download wheels
uses: actions/download-artifact@v2
with:
# unpacks all CIBW artifacts into dist/
pattern: cibw-*
path: dist
merge-multiple: true
- uses: pypa/gh-action-pypi-publish@release/v1
name: cibw-wheels-${{ matrix.os }}-${{ strategy.job-index }}
path: ./wheelhouse

- name: Upload to PyPI
uses: pypa/gh-action-pypi-publish@v1.4.2
with:
user: __token__
password: ${{ secrets.PYPI_TOKEN }}
skip_existing: true
password: ${{ secrets.pypi_token }}
134 changes: 87 additions & 47 deletions .github/workflows/test_build.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,10 +15,10 @@ jobs:
outputs:
sdist_file: ${{ steps.save-path.outputs.sdist_name }}
steps:
- name: Checkout ssh2-python
uses: actions/checkout@v4
- uses: actions/checkout@v4
with:
fetch-depth: 0
submodules: true

- name: Set up Python
uses: actions/setup-python@v5
Expand All @@ -33,77 +33,117 @@ jobs:
- uses: actions/upload-artifact@v4
with:
name: sdist
path: ./dist/*

- name: Sanity check sdist files
run: |
ls ./dist
path: dist/*

- name: Output sdist name
id: save-path
shell: bash -el {0}
run: echo "sdist_name=$(ls ./dist)" >> "$GITHUB_OUTPUT"
shell: bash
run: echo "sdist_name=$(ls dist)" >> "$GITHUB_OUTPUT"

build_wheels:
name: Build ${{ matrix.os }} - ${{ matrix.python-version }}
needs: build_sdist
name: Build wheels on ${{ matrix.os }}
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
# macos-13 is an intel runner, macos-14 is an arm runner
# windows-latest is currently not working
os: [ubuntu-latest, macos-13, macos-14]

os: [ubuntu-latest, macos-latest, windows-latest]
python-version: [ '3.10', '3.11', '3.12', '3.13' ]
steps:
- name: Checkout ssh2-python
uses: actions/checkout@v4
- uses: actions/checkout@v4
with:
fetch-depth: 0
submodules: true

- name: Download sdist
uses: actions/download-artifact@v4
with:
name: sdist
path: ./dist
path: dist

- name: Output sdist name
id: save-path
shell: bash -el {0}
run: echo "sdist_name=$(ls ./dist)" >> "$GITHUB_ENV"
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}

- name: Unzip sdist (macOS)
if: startsWith(matrix.os, 'macos')
- name: Install system dependencies (Linux)
if: runner.os == 'Linux'
run: |
tar -xzf ./dist/${{ env.sdist_name }} -C ./dist
sudo apt-get update
sudo apt-get install -y cmake openssh-server libssh2-dev

- name: Output sdist name (macOS)
id: save-path2
shell: bash -el {0}
run: echo "sdist_name=$(cd ./dist && ls -d */)" >> "$GITHUB_ENV"
- name: Install system dependencies (macOS)
if: runner.os == 'macOS'
run: brew install cmake
# run: brew install cmake libssh2

- name: Set up Python ${{ matrix.cibw_python }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.cibw_python }}
- name: Install system dependencies (Windows)
if: runner.os == 'Windows'
run: choco install -y cmake openssl
# run: choco install -y cmake openssl libssh2

- name: Install python dependencies
- name: Install build requirements
run: |
pip install --upgrade pip
pip install -r requirements_dev.txt
pip install --upgrade pip
pip install -r requirements_build.txt

- name: Build normal wheels
uses: pypa/cibuildwheel@v2.17
with:
package-dir: ./dist/${{ startsWith(matrix.os, 'macos') && env.sdist_name || needs.build_sdist.outputs.sdist_file }}
env:
CIBW_SKIP: "*musllinux*"
CIBW_BUILD: "cp310-* cp311-* cp312-*"
CIBW_PRERELEASE_PYTHONS: True
CIBW_BEFORE_ALL_LINUX: "yum install -y cmake openssh-server openssl-devel"
CIBW_BEFORE_ALL_MACOS: "brew install cmake libssh2"
- name: Build libssh2
shell: bash
run: python _setup_libssh2.py

- name: Check contents of the libssh2 install directory
run: ls -R build_dir/libssh2_install

- name: Set environment variables for libssh2
run: |
echo "CFLAGS=-I$(pwd)/build_dir/libssh2_install/include" >> $GITHUB_ENV
echo "LDFLAGS=-L$(pwd)/build_dir/libssh2_install/lib" >> $GITHUB_ENV

- name: Build wheel
run: |
python setup.py build_ext --inplace
python -m build --wheel

# - name: Test
# run: pytest tests/

- uses: actions/upload-artifact@v4
with:
name: cibw-wheels-${{ matrix.os }}-${{ strategy.job-index }}
path: ./wheelhouse/*.whl
name: dist-${{ matrix.os }}
path: dist/*.whl

# publish:
# name: Publish to PyPI
# runs-on: ubuntu-latest
# needs: build_wheels
# if: startsWith(github.ref, 'refs/tags/')
# steps:
# - uses: actions/checkout@v4

# - uses: actions/download-artifact@v4
# with:
# name: sdist
# path: dist

# - uses: actions/download-artifact@v4
# with:
# name: dist-ubuntu-latest
# path: dist

# - uses: actions/download-artifact@v4
# with:
# name: dist-macos-latest
# path: dist

# - uses: actions/download-artifact@v4
# with:
# name: dist-windows-latest
# path: dist

# - name: Publish to PyPI
# env:
# TWINE_USERNAME: __token__
# TWINE_PASSWORD: ${{ secrets.PYPI_TOKEN }}
# run: |
# pip install twine
# twine upload --skip-existing dist/*
3 changes: 3 additions & 0 deletions .gitmodules
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
[submodule "libssh2"]
path = libssh2
url = https://github.com/libssh2/libssh2.git
63 41DA changes: 48 additions & 15 deletions _setup_libssh2.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,30 +14,63 @@
# License along with this library; if not, write to the Free Software
# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
import os

import platform
from sys import stderr
from subprocess import check_call
from glob import glob
from shutil import copy2


def build_ssh2():
if bool(os.environ.get('SYSTEM_LIBSSH', False)):
stderr.write("Using system libssh2..%s" % (os.sep))
build_dir = os.environ.get('BUILD_DIR', 'build_dir')
libssh2_path = os.path.abspath('libssh2')

if bool(os.environ.get('SYSTEM_LIBSSH2', False)):
print("Using system libssh2..")
return
if os.path.exists('/usr/local/opt/openssl'):
os.environ['OPENSSL_ROOT_DIR'] = '/usr/local/opt/openssl'

if not os.path.exists('build_dir'):
os.mkdir('build_dir')
if not os.path.exists(build_dir):
os.makedirs(build_dir)

cmake_args = [
f'cmake {libssh2_path}',
'-DBUILD_SHARED_LIBS=ON',
'-DENABLE_ZLIB_COMPRESSION=ON',
'-DENABLE_CRYPT_NONE=ON',
'-DENABLE_MAC_NONE=ON',
f'-DCMAKE_INSTALL_PREFIX={os.path.join(build_dir, "libssh2_install")}'
]

# Platform specific configurations
if platform.system() == 'Darwin':
if os.path.exists('/usr/local/opt/openssl'):
openssl_root = '/usr/local/opt/openssl'
cmake_args.extend([
f'-DOPENSSL_ROOT_DIR={openssl_root}',
f'-DOPENSSL_INCLUDE_DIR={openssl_root}/include',
f'-DOPENSSL_CRYPTO_LIBRARY={openssl_root}/lib/libcrypto.dylib',
f'-DOPENSSL_SSL_LIBRARY={openssl_root}/lib/libssl.dylib'
])
elif platform.system() == 'Windows':
cmake_args.extend([
'-DCMAKE_BUILD_TYPE=Release',
'-DCRYPTO_BACKEND=WinCNG'
])
else: # Linux
cmake_args.append('-DCRYPTO_BACKEND=OpenSSL')

os.chdir('build_dir')
check_call('cmake ../libssh2/libssh2 -DBUILD_SHARED_LIBS=ON \
-DENABLE_ZLIB_COMPRESSION=ON -DENABLE_CRYPT_NONE=ON \
-DENABLE_MAC_NONE=ON -DCRYPTO_BACKEND=OpenSSL',
shell=True, env=os.environ)
check_call('cmake --build . --config Release', shell=True, env=os.environ)
os.chdir(build_dir)
print(f"************* Building libssh2 in {os.getcwd()} *************")
print(f"cmake arguments: {cmake_args}")
check_call(' '.join(cmake_args), shell=True)

if platform.system() == 'Windows':
check_call('cmake --build . --config Release --target install', shell=True)
else:
check_call('cmake --build . --target install', shell=True)
os.chdir('..')
print("************* Finished building libssh2 *************")


for src in glob('build_dir/src/libssh2.so*'):
copy2(src, 'ssh2/')
if __name__ == '__main__':
build_ssh2()
18 changes: 18 additions & 0 deletions dev/update_project.sh
View file
Open in desktop
41DA
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
#!/bin/bash
# This should be ran from the root of the project

# Update the libssh2 submodule
echo "Updating libssh2 submodule..."
git submodule update --init --recursive
cd libssh2
git fetch
git checkout master
git pull origin master
cd ..

# Rebuild the project with Cython
echo "Rebuilding the project with Cython..."
pip install -Ur requirements_dev.txt
python setup.py build_ext --inplace

echo "Update and rebuild complete."
1 change: 1 addition & 0 deletions libssh2
Open in desktop
Submodule libssh2 added at 780bf1
Loading
Loading
0