8000 Release 2019.10.06 (#1998) · jhay06/python-for-android@b1f6064 · GitHub
[go: up one dir, main page]
Skip to content

Commit b1f6064

Browse files
inclementopacamtshirtmanzworkbZen-CODE
authored
Release 2019.10.06 (kivy#1998)
* [recipes] Fix compilation for regex recipe The error was: build/other_builds/hostpython3/desktop/hostpython3/Include/Python.h:39:19: fatal error: crypt.h: No such file or directory * [recipes] Update regex's recipe * put the sdk manager command in a code block (kivy#1956) to avoid quotes being replaced by fancy ones * Updated version number for develop branch following 2019.08.09 release (kivy#1960) * Fix build for case-insensitive FS and add CI test for OSX (kivy#1951) * [python] Fix build for case-insensitive FS 🍎 It turns out that the generated python binary for some builds are named `python.exe` instead of `python`. This depends on the File System where the build happens. It will be named `python.exe` when the FS is case-insensitive (Mac OSX and Cygwin), but it will be named `python` when the FS is case-sensitive (most GNU Linux distributions). The proposed solution consists in make a copy of the generated python binary with a given name (python3 or python2) so this way To achieve this goal we refactor a little our `HostPythonRecipe`:   - add private property `HostPythonRecipe._exe_name` (the name of the python executable based on major version)   - add public property `HostPythonRecipe.python_exe` (the full path of the python executable)   - implement `HostPythonRecipe.should_build` And also it's affected the `GuestPythonRecipe`, because we need to use the generated python executable by `HostPythonRecipe`:   - add private property `GuestPythonRecipe._libpython` (python's library name with extension...hardcoded for now...)   - implement `GuestPythonRecipe.should_build`... to check the library instead of the executable so we avoid conflicts with case-insensitive FS We also need: - fix `PythonRecipe.real_hostpython_location` because the name of our host python executable will depend on major version   - fix python2 interpreter (fix-interpreter-version.patch) Note: the variation of the name of the python's executable is mentioned at python's build docs (https://github.com/python/cpython/blob/3.7/README.rst#build-instructions) Note: @TheSin- , ¡¡¡thanks for your debugging sessions!!! * [ci] Add Mac OSX CI's test & refactor android's NDK/SDK installation To do so we:   - create a makefile to install the necessary dependencies for Mac OS X: `ci/makefiles/osx.mk`   - create a makefile to install android's SDK/NDK: `ci/makefiles/android.mk`   - refactor docker files: make use of android's makefile - change OS, from `linux` to `osx`, for CI test `Python 3 armeabi-v7a`, so we don't increase the overall build time and jobs   - rename the `Python 2` test to `Python 2 armeabi-v7a (with numpy)` to reflect the build arch and numpy's build. * customizability (kivy#1869) * customizability options - added compile-options * apptheme option for all bootstraps * Initial migration to NDK r19 (Part I - The core) (kivy#1722) * [ndk19] Rework of the arch environment to work with ndk r19+ Things done here: - Modifies target so we can use the toolchains `in-place` (as if we were using a standalone toolchain) - Removes some cflags that we don't need anymore - Move `macros` and `includes` from `CFLAGS` to `CPPFLAGS` (because this is the way it's supposed to be) - Move libraries from `LDFLAGS` to `LDLIBS` (again, the same as above) - Refactor `Arch`'s variables into class attributes, to make `get_env` smaller and to be more readable/maintainable: + class attributes for flags: * common_cflags * common_cppflags * common_ldflags * common_ldlibs * common_ldshared * arch_cflags + class attributes for compiler: * clang_path * clang_exe * clang_exe_cxx - Remove hardcoded cpu count in `MAKE` - Remove `gcc` of environment (because it has been partially removed in ndk r18) - Shorten long line to be pep8 friendly (line where we define env['BUILDLIB_PATH']) - Commented lines for `ArchAarch_64` that it seems tha we don't need anymore (not removed because I'm not sure if we need them) Note: This will force us to use a minimal ndk version of 19 See also: https://developer.android.com/ndk/guides/other_build_systems * [ndk19] Update `test_archs` to ndk's r19+ build system Also add some more tests. Now we check if some of the functions we mock, `glob` and `find_executable`, are called with the expected arguments * [ndk19] Update `test_bootstrap` to ndk's r19+ build system Because `clang` is the new default compiler and the affected tests were made thinking in `gcc` compiler * [ndk19] Remove `clang` of `get_recipe_env` Because as per ndk r19 the default compiler is clang and gcc has been partially removed * [ndk19] Adapt python recipes to ndk's r19+ build system We move python recipe to use a minimal ndk version of 19 because it will simplify the python's cross compilation (this way we can remove some environment variables that we don't need anymore). The only side effect detected is that the python's `zlib` module isn't built anymore. This is caused because python3 and python2 build systems contains libz's hardcoded version, which does not match the version used in ndk 19 (may be this the cause of some `zip/unzip` errors reported in python-for-android's issues?). We will take care later of this zlib problem, or we will get a runtime crash...so it seems that zlib is an `essential module` to successfully run a python-for-android app * [ndk19] Update libffi to version post `v3.3RC0` and adapt it to be ndk r19+ compatible * [ndk19] Fix build of python's zlib module Because python has a hardcoded zlib version which does not match android's ndk r19, and that prevents the build of zlib's module Note: it seems that we have to point to the right lib and headers, otherwise we won't succeed in building python's zlib module * [ndk19] Fix build of python2's _ctypes module for ndk r19+ * [ndk19] Fix numpy for ndk r19+ Note: because of the new build system recently introduced, we can safely remove some of the patches that we needed because it seems that it builds and runs fine without them * [ndk19] Update docs with the new ndk requirements * [ndk19] Set min/max/recommended android's NDK * [ndk19] Make CI tests with android's NDK `r19b` Because it's our minimum supported android NDK * [ndk19] Fix png recipe for NDK r19+ We remove `get_recipe_env` because we don't need anymore to specify the flags set in there. Note: As per new NDK r19+ build system implemented in p4a, the removed `flags` should be automatically set by the android's NDK * [ndk19] Add `-fPIC` to `armeabi-v7a`'s arch cflags... ...force `Position-Independent Code` To solve the kivy's build error detected in travis `OSX` test (https://travis-ci.org/kivy/python-for-android/jobs/576204088#L1105). But also detected the same error in my local tests (linux os in this case) for `freetype` and `harfbuzz` recipes...so I have the feeling that this is better to set as global 🤞..is it? * [LIBS - PART I] Initial refactor of library recipes (kivy#1944) This commit will introduce a new cls attribute: `built_libraries`. This `built_libraries` attribute must be a dictionary. The keys are the full library name, e.g.: `libffi.so`. And the values must be the relative path of the library, e.g: `.libs`. So the this cls attribute for libffi recipe it would look like: ```built_libraries = {'libffi.so': '.libs'}``` This new cls attribute will allow us to detect library recipes and to refactor common operations that we do for those kind of recipes: - copy library into right location - implement `should_build` for library recipes Those two things will make that our rebuilds to be more consistent and I hope that it will be easier to maintain. **So, once explained a little this PR, in here we do:** - [x] Add attribute `built_libraries` - [x] Add methods to refactor common library operations: - [x] install_libraries - [x] get_libraries - [x] implement `should_build` for library recipes - [x] implement basic tests for the newly introduced `Recipe` methods - [x] Make use of those Library attributes/methods for some basic set of recipes: - [x] libffi - [x] openssl - [x] png - [x] jpeg - [x] freetype - [x] harfbuzz - [x] libcurl - [x] libzbar - [x] libiconv - [x] libexpat - [x] libogg - [x] libxml2 - [x] libxslt - [x] libshine - [x] libx264 - [x] libglob - [x] libsodium - [x] libsecp256k1 **Things to do in separate PRs**, to be easy to review and because those points will start to conflict with the `NDK r19 migration` thing just merged: - Make use of Library attributes/methods introduced in this PR for un covered library recipes in here. Here we have two situations: - library recipes that depends on android's STL library (almost all the work done in here will depend of the `NDK r19 migration` thing) - all remaining library recipes, which are not STL lib dependent and that are not implemented in here (because I was unable to perform a successful build with them using arch `arm64-v8a`...so I think it would be better to deal with those recipes in separate PRs and later...with the `NDK r19 migration` thing merged) **Notes about changed recipes:** all the recipes touched in here almost have the same changes: - remove `should_build` method (to make use of the one implemented in base class for library recipes) - remove the copy of the library (because that will be done automatically by the method `install_libraries` implemented in base class) - fixed the imports due to refactoring * [recipe-lib] Add attribute Recipe.built_libraries so we can refactor some essential operations performed for library recipes (like copy the library into the right location, or decide if we should build the library recipe again) * [recipe-lib] Add method Recipe.get_libraries This function will allows us to get a list of the built/context libraries. This will be useful to check if we have to built a library recipe or which library should be rebuild * [recipe-lib] Make `Recipe.should_build` decide the library build In case that the attribute built_libraries has been set, we will check the need of the recipe build, otherwise we will act as usual, forcing the build unless overwrote in subclass * [recipe-lib] Add method `Recipe.install_libraries` So we can: - control the copy of the library recipe - refactor the install of the lib for all library recipes * [recipe-lib] Make libffi a library recipe * [recipe-lib] Make openssl a library recipe and ... also make the imports from the right module * [recipe-lib] Make png a library recipe * [recipe-lib] Make jpeg a library recipe * [recipe-lib] Make freetype a library recipe and ... a 8000 lso make the imports from the right module * [recipe-lib] Make harfbuzz a library recipe and ... also make the imports from the right module * [recipe-lib] Make libcurl a library recipe and ... also make the imports from the right module * [recipe-lib] Make libzbar a library recipe and ... also make the imports from the right module * [recipe-lib] Make libiconv a library recipe and ... also make the imports from the right module * [recipe-lib] Make libexpat a library recipe and ... also: - make the imports from the right module - remove hardcoded arch * [recipe-lib] Make libogg a library recipe * [recipe-lib] Make libxml2 a library recipe and ... also make the imports from the right module * [recipe-lib] Make libxslt a library recipe and ... also make the imports from the right module * [recipe-lib] Make libshine a library recipe and ... also: - make the imports from the right module - remove the hardcoded cpu count when compiling * [recipe-lib] Make libx264 a library recipe and ... also:   - make the imports from the right module   - remove the hardcoded cpu count when compiling * [recipe-lib] Make libglob a library recipe * [recipe-lib] Make libsodium a library recipe and ... also:   - make the imports from the right module   - fix hardcoded arch   - enable cpu count when compiling * [recipe-lib] Make libsecp256k1 a library recipe and ... also make the imports from the right module * [tests] Add tests for library recipe * [NDK19] Fix libglob for android NDK r19 - change the `ARG_MAX` define, because it's already defined at `sysroot/usr/include/linux/limits.h` - Replaced `size_t` by Including the <stddef.h> header. Because found the solution at stackoverflow: As per C99, §7.17, size_t is not a builtin type but defined in <stddef.h> See also: - https://travis-ci.org/kivy/python-for-android/jobs/576392711#L5992-L6013 - https://stackoverflow.com/questions/26410466/gcc-linaro-compiler-throws-error-unknown-type-name-size-t * [NDK19] Remove `--sysroot` from LDFLAGS for cffi and pymunk (kivy#1965) Because `--sysroot` flag is not needed anymore and make it fails the build since we migrated to new android's build system (implemented in NDK r19) See also: - https://developer.android.com/ndk/guides/other_build_systems * Added libffi headers troubleshooting note to doc (kivy#1972) * [LIBS - PART II] Part II of NDK r19 migration - Initial STL lib migration (kivy#1947) * [recipe-stl] Add android's STL lib support to `Recipe` To allow us to refactor some common operations that we use in our recipes that depends on android's STL library. Note: This commit will allow us to begin the migration to `c++_shared`. This is a must when we move to android's NDK r19+, because as for android NDK >= 18 is the only one supported STL library. * [recipe-stl] Make CppCompiledComponentsPythonRecipe use Recipe's STL support * [recipe-stl] Make icu a library recipe with STL support (rework) Also done here:   - Remove hardcoded version in url   - Disable versioned shared libraries   - Make it to be build as a shared libraries (instead of static)   - Disable the build of static libraries (because we build them as shared ones, so we avoid to link with them without our consents)   - Shorten long lines to be pep8's friendly   - Remove icu from ci/constants   - Remove `RuntimeError` about the need to use NDK api <= 19 (because that is not the case anymore)   - consider host's number of cpu to perform the build * [recipe-stl] Rework pyicu recipe to match latest icu changes Also done here:   - Remove icu.patch because now we don't have the version in our icu libraries   - Shorten long lines to be pep8's friendly * [tests] Add tests for recipe with STL support * [tests] Add tests for icu recipe * [tests] Add tests for pyicu recipe * [WIP][LIBS - PART III] Rework of pyleveldb, leveldb and snappy (kivy#1966) * [recipe-lib] Make snappy a library recipe * [recipe-stl] Make leveldb a library recipe and... make it work with the reworked snappy recipe * [recipe-stl] Fix pyleveldb for reworked leveldb/snappy * [WIP][LIBS - PART IV] Rework of shapely and libgeos (kivy#1967) * [recipe-stl] Make libgeos a library recipe * [recipe-stl] Fix shapely for reworked libgeos * [recipe-stl] Make libzmq a library recipe (kivy#1968) * [recipe-stl] Rework of libtorrent and boost (kivy#1971) which: - fix build for android's NDK r19+ - allow us to build `boost+libtorrent` for any arch - Update boost to version `1.69.0` - update libtorrent to version `1.2.1` The build method needs to be changed because one of the scripts used in `boost+libtorrent` build (make-standalone-toolchain.sh), has been removed from android's NDK. [This is because](https://developer.android.com/ndk/guides/standalone_toolchain): `As of r19, the NDK's default toolchains are standalone toolchains, which renders this process unnecessary.` Note: since 3887d2b, `python-for-android` uses android's NDK r19 as the minimum supported and also changed the build method used by p4a as [described in here](https://developer.android.com/ndk/guides/other_build_systems) * Also copy the service/main.py when building with setup.py (kivy#1936) * [recipe-stl] Rework of protobuf_cpp recipe (kivy#1969) In here we do: - inherit from CppCompiledComponentsPythonRecipe Because depends on android's STL library - make use of the base class methods for library recipes - Split build_arch into proper methods - Shorten some long lines (to be PEP8 friendly) - make generated library shared - remove recipe from CI/constants * [CI] Fix CI's test for `arm64-v8a` (kivy#1977) * Updated README.md to clarify NDK versions (kivy#1981) * 📚 Updated README.dm to clarify NDK versions * 🔧 Change back to r19b * A bunch of tests for library recipes (kivy#1982) The main idea is to test as many recipes as we can with the less code possible and without creating any file/directory so our tests can be performed as fast as possible (all this tests will only add between 2 and 3 seconds to our CI tests and will cover almost 100% of the code for each tested recipe) To do so, we create a couple of modules: tests.recipe_ctx: allow us to create a proper Context to test our recipes tests.recipe_lib_test: which holds some base classes to be used to test a recipe depending on the build method used. For starters we introduce two kind of base classes: BaseTestForMakeRecipe: To test an standard library build (this will iinclude the recipes which requires the classical build commandsconfigure/make) BaseTestForCmakeRecipe: To test an library recipe which is compiled with cmake We also refactor the existing recipes tests, so we can remove some lines in there...the ones that creates a Context. * [test] Add module `tests.recipe_ctx` A helper module to test recipes. Here we will initialize a `Context` to test recipes. * [test] Refactor `setUp/tearDown` for `test_icu` * [test] Refactor `setUp/tearDown` for `test_pyicu` * [test] Refactor `setUp` for `test_reportlab` * [test] Refactor `setUp` for `test_gevent` * [test] Add module `tests.recipe_lib_test` A helper module to test recipes which will allow to test any recipe using `configure/make` commands. * [test] Add test for `libffi` * [test] Add test for `libexpat` * [test] Add test for `libcurl` * [test] Add test for `libiconv` * [test] Add test for `libogg` * [test] Add test for `libpq` * [test] Add test for `libsecp256k1` * [test] Add test for `libshine` * [test] Add test for `libvorbis` * [test] Add test for `libx264` * [test] Add test for `libxml2` * [test] Add test for `libxslt` * [test] Add test for `png` * [test] Add test for `freetype` * [test] Add test for `harfbuzz` * [test] Add test for `openssl` * [test] Add `BaseTestForCmakeRecipe` * [test] Add test for `jpeg` and clean code We can remove the `get_recipe_env` because the environment that we use is already using a clang as default compiler (not the case when we migrated the jpeg recipe to use `cmake`)...so we can do a little refactor :) * [test] Add test for `snappy` * [test] Add test for `leveldb` * [test] Add test for `libgeos` * [test] Add test for `libmysqlclient` * [test] Add test for `openal` * [test] Make the `super` calls Python3 style * [test] Move mock tests outside context manager... Because there is no need to do it there. Also rewrote the inline comments. * Recipes tests enhancements (kivy#1984) This is a follow up of kivy#1982 In here we do: - Move from `__new__` to `__init__` for `BaseTestForMakeRecipe` ([discussed at here](kivy#1982 (comment))) - Convert cls attributes to instance attributes ([discussed at here](kivy#1982 (comment))) - Remove `mock` dependency for py3 tests (Because, as per Python 3.3+, `mock` it's a built-in-module). (Unfortunately we still will have a couple of `import mock` entries that we cannot remove until we completely remove python2 from our tests) * [test] From `__new__` to `__init__` for `BaseTestForMakeRecipe` * [test] Remove mock dependency for py3 tests Because, as per Python 3.3+, `mock` it's a built-in-module * [test] Convert cls attributes to instance attributes Because we may have some side effects without willing, considering that cls attributes are shared between instances. * Fixes test_virtualenv and test_venv failing, closes kivy#1994 (kivy#1995) Recent pep517 release seems to break tests. * Made p4a use per-arch dist build dirs (kivy#1986) * Made p4a use per-arch dist build dirs * Fixed up check on existing folder when overwriting distributions * Improved Distribution.get_distribution api * Fixed test_archs.py * Fixed test_bootstrap.py * Fixed test_distribution.py * Fixed recipes tests * Fixed formatting * Made sure whole build process accesses dist_dir correctly * Fixed apk output name formatting by moving version to end of string * Further test fixes following dist_dir changes * Removed unnecessary variable * Fixes libiconv & libzbar configure host Having `--host=x86` was making the configure script fail at enabling shared library build on `x86`. This was resulting in failing to copy `libiconv.so` and `libzbar.so` on `x86` arch. * Updates Java version troubleshooting (kivy#1991) Adds the NoClassDefFoundError XmlSchema case and groups related cases together. Gives instructions for both Ubuntu and macOS to fix. * Made on-device unit tests app use the develop branch by default (kivy#1985) * Updated version to 2019.10.06 Co-authored-by: Pol Canelles <canellestudi@gmail.com> Co-authored-by: Gabriel Pettier <gabriel.pettier@gmail.com> Co-authored-by: Philipp Auersperg-Castell <phil@bluedynamics.com> Co-authored-by: Richard Larkin <zenkey.zencode@gmail.com> Co-authored-by: Andre Miras <AndreMiras@users.noreply.github.com>
1 parent dd69749 commit b1f6064
  • openssl
  • png
  • protobuf_cpp
  • pycrypto
  • pyicu
  • pyleveldb
  • pymunk
  • python2
  • python3
  • regex
  • shapely
  • snappy
  • recommendations.py
  • toolchain.py
  • setup.py
  • testapps/on_device_unit_tests
  • tests
  • tox.ini

    • Some content is hidden

    Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

    120 files changed

    +2754
    -1580
    lines changed

    ‎.travis.yml

    Lines changed: 28 additions & 11 deletions
    Original file line numberDiff line numberDiff line change
    @@ -16,11 +16,6 @@ before_install:
    1616
    - git remote set-branches --add origin develop
    1717
    - git fetch
    1818

    19-
    env:
    20-
    global:
    21-
    - ANDROID_SDK_HOME=/opt/android/android-sdk
    22-
    - ANDROID_NDK_HOME=/opt/android/android-ndk
    23-
    2419
    jobs:
    2520
    include:
    2621
    - &linting
    @@ -65,19 +60,41 @@ jobs:
    6560
    # case that the travis log doesn't produce any output for more than 10 minutes
    6661
    - while sleep 540; do echo "==== Still running (travis, don't kill me) ===="; done &
    6762
    script:
    68-
    - docker run -e CI -e TRAVIS_JOB_ID="$TRAVIS_JOB_ID" -e TRAVIS_BRANCH="$TRAVIS_BRANCH" p4a /bin/sh -c "$COMMAND"
    63+
    - >
    64+
    docker run
    65+
    -e CI
    66+
    -e TRAVIS_JOB_ID
    67+
    -e TRAVIS_BRANCH
    68+
    -e ANDROID_SDK_HOME="/home/user/.android/android-sdk"
    69+
    -e ANDROID_NDK_HOME="/home/user/.android/android-ndk"
    70+
    p4a /bin/sh -c "$COMMAND"
    6971
    after_script:
    7072
    # kill the background process started before run docker
    7173
    - kill %1
    72-
    name: Python 3 armeabi-v7a
    74+
    name: Python 3 arm64-v8a
    7375
    # overrides requirements to skip `peewee` pure python module, see:
    7476
    # https://github.com/kivy/python-for-android/issues/1263#issuecomment-390421054
    75-
    env: COMMAND='. venv/bin/activate && cd testapps/ && python setup_testapp_python3_sqlite_openssl.py apk --sdk-dir $ANDROID_SDK_HOME --ndk-dir $ANDROID_NDK_HOME --requirements libffi,sdl2,pyjnius,kivy,python3,openssl,requests,sqlite3,setuptools' --arch=armeabi-v7a
    77+
    env:
    78+
    COMMAND='. venv/bin/activate && cd testapps/ && python setup_testapp_python3_sqlite_openssl.py apk --sdk-dir $ANDROID_SDK_HOME --ndk-dir $ANDROID_NDK_HOME --requirements libffi,sdl2,pyjnius,kivy,python3,openssl,requests,sqlite3,setuptools --arch=arm64-v8a'
    7679
    - <<: *testing
    77-
    name: Python 3 arm64-v8a
    78-
    env: COMMAND='. venv/bin/activate && cd testapps/ && python setup_testapp_python3_sqlite_openssl.py apk --sdk-dir $ANDROID_SDK_HOME --ndk-dir $ANDROID_NDK_HOME --requirements libffi,sdl2,pyjnius,kivy,python3,openssl,requests,sqlite3,setuptools' --arch=arm64-v8a
    80+
    name: Python 3 armeabi-v7a
    81+
    os: osx
    82+
    osx_image: xcode11 # since xcode1.3, python3 is the default interpreter
    83+
    before_script:
    84+
    # installs java 1.8, android's SDK/NDK and p4a
    85+
    - make -f ci/makefiles/osx.mk
    86+
    - export JAVA_HOME=/Library/Java/JavaVirtualMachines/adoptopenjdk-8.jdk/Contents/Home
    87+
    # Run a background process (like we do with linux tests)
    88+
    - while sleep 540; do echo "==== Still running (travis, don't kill me) ===="; done &
    89+
    script:
    90+
    - >
    91+
    cd testapps && python3 setup_testapp_python3_sqlite_openssl.py apk
    92+
    --sdk-dir $HOME/.android/android-sdk
    93+
    --ndk-dir $HOME/.android/android-ndk
    94+
    --requirements libffi,sdl2,pyjnius,kivy,python3,openssl,requests,sqlite3,setuptools
    95+
    --arch=armeabi-v7a
    7996
    - <<: *testing
    80-
    name: Python 2 basic
    97+
    name: Python 2 armeabi-v7a (with numpy)
    8198
    env: COMMAND='. venv/bin/activate && cd testapps/ && python setup_testapp_python2_sqlite_openssl.py apk --sdk-dir $ANDROID_SDK_HOME --ndk-dir $ANDROID_NDK_HOME --requirements sdl2,pyjnius,kivy,python2,openssl,requests,sqlite3,setuptools,numpy'
    8299
    - <<: *testing
    83100
    name: Rebuild updated recipes

    ‎Dockerfile.py2

    Lines changed: 11 additions & 51 deletions
    Original file line numberDiff line numberDiff line change
    @@ -37,64 +37,17 @@ ENV RETRY="retry -t 3 --"
    3737
    RUN curl https://raw.githubusercontent.com/kadwanev/retry/1.0.1/retry \
    3838
    --output /usr/local/bin/retry && chmod +x /usr/local/bin/retry
    3939

    40-
    ENV ANDROID_NDK_HOME="${ANDROID_HOME}/android-ndk"
    41-
    ENV ANDROID_NDK_VERSION="17c"
    42-
    ENV ANDROID_NDK_HOME_V="${ANDROID_NDK_HOME}-r${ANDROID_NDK_VERSION}"
    43-
    44-
    # get the latest version from https://developer.android.com/ndk/downloads/index.html
    45-
    ENV ANDROID_NDK_ARCHIVE="android-ndk-r${ANDROID_NDK_VERSION}-linux-x86_64.zip"
    46-
    ENV ANDROID_NDK_DL_URL="https://dl.google.com/android/repository/${ANDROID_NDK_ARCHIVE}"
    47-
    48-
    # download and install Android NDK
    49-
    RUN ${RETRY} curl --location --progress-bar --insecure \
    50-
    "${ANDROID_NDK_DL_URL}" \
    51-
    --output "${ANDROID_NDK_ARCHIVE}" \
    52-
    && mkdir --parents "${ANDROID_NDK_HOME_V}" \
    53-
    && unzip -q "${ANDROID_NDK_ARCHIVE}" -d "${ANDROID_HOME}" \
    54-
    && ln -sfn "${ANDROID_NDK_HOME_V}" "${ANDROID_NDK_HOME}" \
    55-
    && rm -rf "${ANDROID_NDK_ARCHIVE}"
    56-
    57-
    58-
    ENV ANDROID_SDK_HOME="${ANDROID_HOME}/android-sdk"
    59-
    60-
    # get the latest version from https://developer.android.com/studio/index.html
    61-
    ENV ANDROID_SDK_TOOLS_VERSION="4333796"
    62-
    ENV ANDROID_SDK_BUILD_TOOLS_VERSION="28.0.2"
    63-
    ENV ANDROID_SDK_TOOLS_ARCHIVE="sdk-tools-linux-${ANDROID_SDK_TOOLS_VERSION}.zip"
    64-
    ENV ANDROID_SDK_TOOLS_DL_URL="https://dl.google.com/android/repository/${ANDROID_SDK_TOOLS_ARCHIVE}"
    65-
    66-
    # download and install Android SDK
    67-
    RUN ${RETRY} curl --location --progress-bar --insecure \
    68-
    "${ANDROID_SDK_TOOLS_DL_URL}" \
    69-
    --output "${ANDROID_SDK_TOOLS_ARCHIVE}" \
    70-
    && mkdir --parents "${ANDROID_SDK_HOME}" \
    71-
    && unzip -q "${ANDROID_SDK_TOOLS_ARCHIVE}" -d "${ANDROID_SDK_HOME}" \
    72-
    && rm -rf "${ANDROID_SDK_TOOLS_ARCHIVE}"
    73-
    74-
    # update Android SDK, install Android API, Build Tools...
    75-
    RUN mkdir --parents "${ANDROID_SDK_HOME}/.android/" \
    76-
    && echo '### User Sources for Android SDK Manager' \
    77-
    > "${ANDROID_SDK_HOME}/.android/repositories.cfg"
    78-
    79-
    # Download and accept Android licenses (JDK necessary!)
    80-
    RUN ${RETRY} apt -y install -qq --no-install-recommends openjdk-8-jdk \
    81-
    && apt -y autoremove
    82-
    RUN yes | "${ANDROID_SDK_HOME}/tools/bin/sdkmanager" "build-tools;${ANDROID_SDK_BUILD_TOOLS_VERSION}" > /dev/null
    83-
    RUN yes | "${ANDROID_SDK_HOME}/tools/bin/sdkmanager" "platforms;android-27" > /dev/null
    84-
    85-
    # Set avdmanager permissions (executable)
    86-
    RUN chmod +x "${ANDROID_SDK_HOME}/tools/bin/avdmanager"
    87-
    88-
    8940
    ENV USER="user"
    9041
    ENV HOME_DIR="/home/${USER}"
    42+
    ENV ANDROID_HOME="${HOME_DIR}/.android"
    9143
    ENV WORK_DIR="${HOME_DIR}" \
    9244
    PATH="${HOME_DIR}/.local/bin:${PATH}"
    9345

    9446
    # install system dependencies
    9547
    RUN ${RETRY} apt -y install -qq --no-install-recommends \
    9648
    python virtualenv python-pip wget lbzip2 patch sudo \
    97-
    && apt -y autoremove
    49+
    && apt -y autoremove \
    50+
    && apt -y clean
    9851

    9952
    # build dependencies
    10053
    # https://buildozer.readthedocs.io/en/latest/installation.html#android-on-ubuntu-16-04-64bit
    @@ -113,6 +66,10 @@ RUN ${RETRY} apt -y install -qq --no-install-recommends \
    11366
    && apt -y autoremove \
    11467
    && apt -y clean
    11568

    69+
    # Install Java, set JAVA_HOME (to accept android's SDK licenses) and clean
    70+
    RUN ${RETRY} apt -y install -qq --no-install-recommends openjdk-8-jdk \
    71+
    && apt -y autoremove && apt -y clean
    72+
    ENV JAVA_HOME /usr/lib/jvm/java-8-openjdk-amd64
    11673

    11774
    # prepare non root env
    11875
    RUN useradd --create-home --shell /bin/bash ${USER}
    @@ -124,9 +81,12 @@ RUN echo "%sudo ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers
    12481

    12582
    WORKDIR ${WORK_DIR}
    12683
    COPY --chown=user:user . ${WORK_DIR}
    127-
    RUN chown --recursive ${USER} ${ANDROID_SDK_HOME}
    84+
    RUN mkdir ${ANDROID_HOME} && chown --recursive ${USER} ${ANDROID_HOME}
    12885
    USER ${USER}
    12986

    87+
    # Download and install android's NDK/SDK
    88+
    RUN make -f ci/makefiles/android.mk target_os=linux
    89+
    13090
    # install python-for-android from current branch
    13191
    RUN virtualenv --python=python venv \
    13292
    && . venv/bin/activate \

    ‎Dockerfile.py3

    Lines changed: 9 additions & 52 deletions
    Original file line numberDiff line numberDiff line change
    @@ -17,8 +17,6 @@
    1717

    1818
    FROM ubuntu:18.04
    1919

    20-
    ENV ANDROID_HOME="/opt/android"
    21-
    2220
    # configure locale
    2321
    RUN apt update -qq > /dev/null && apt install -qq --yes --no-install-recommends \
    2422
    locales && \
    @@ -37,57 +35,9 @@ ENV RETRY="retry -t 3 --"
    3735
    RUN curl https://raw.githubusercontent.com/kadwanev/retry/1.0.1/retry \
    3836
    --output /usr/local/bin/retry && chmod +x /usr/local/bin/retry
    3937

    40-
    ENV ANDROID_NDK_HOME="${ANDROID_HOME}/android-ndk"
    41-
    ENV ANDROID_NDK_VERSION="17c"
    42-
    ENV ANDROID_NDK_HOME_V="${ANDROID_NDK_HOME}-r${ANDROID_NDK_VERSION}"
    43-
    44-
    # get the latest version from https://developer.android.com/ndk/downloads/index.html
    45-
    ENV ANDROID_NDK_ARCHIVE="android-ndk-r${ANDROID_NDK_VERSION}-linux-x86_64.zip"
    46-
    ENV ANDROID_NDK_DL_URL="https://dl.google.com/android/repository/${ANDROID_NDK_ARCHIVE}"
    47-
    48-
    # download and install Android NDK
    49-
    RUN ${RETRY} curl --location --progress-bar --insecure \
    50-
    "${ANDROID_NDK_DL_URL}" \
    51-
    --output "${ANDROID_NDK_ARCHIVE}" \
    52-
    && mkdir --parents "${ANDROID_NDK_HOME_V}" \
    53-
    && unzip -q "${ANDROID_NDK_ARCHIVE}" -d "${ANDROID_HOME}" \
    54-
    && ln -sfn "${ANDROID_NDK_HOME_V}" "${ANDROID_NDK_HOME}" \
    55-
    && rm -rf "${ANDROID_NDK_ARCHIVE}"
    56-
    57-
    58-
    ENV ANDROID_SDK_HOME="${ANDROID_HOME}/android-sdk"
    59-
    60-
    # get the latest version from https://developer.android.com/studio/index.html
    61-
    ENV ANDROID_SDK_TOOLS_VERSION="4333796"
    62-
    ENV ANDROID_SDK_BUILD_TOOLS_VERSION="28.0.2"
    63-
    ENV ANDROID_SDK_TOOLS_ARCHIVE="sdk-tools-linux-${ANDROID_SDK_TOOLS_VERSION}.zip"
    64-
    ENV ANDROID_SDK_TOOLS_DL_URL="https://dl.google.com/android/repository/${ANDROID_SDK_TOOLS_ARCHIVE}"
    65-
    66-
    # download and install Android SDK
    67-
    RUN ${RETRY} curl --location --progress-bar --insecure \
    68-
    "${ANDROID_SDK_TOOLS_DL_URL}" \
    69-
    --output "${ANDROID_SDK_TOOLS_ARCHIVE}" \
    70-
    && mkdir --parents "${ANDROID_SDK_HOME}" \
    71-
    && unzip -q "${ANDROID_SDK_TOOLS_ARCHIVE}" -d "${ANDROID_SDK_HOME}" \
    72-
    && rm -rf "${ANDROID_SDK_TOOLS_ARCHIVE}"
    73-
    74-
    # update Android SDK, install Android API, Build Tools...
    75-
    RUN mkdir --parents "${ANDROID_SDK_HOME}/.android/" \
    76-
    && echo '### User Sources for Android SDK Manager' \
    77-
    > "${ANDROID_SDK_HOME}/.android/repositories.cfg"
    78-
    79-
    # Download and accept Android licenses (JDK necessary!)
    80-
    RUN ${RETRY} apt -y install -qq --no-install-recommends openjdk-8-jdk \
    81-
    && apt -y autoremove
    82-
    RUN yes | "${ANDROID_SDK_HOME}/tools/bin/sdkmanager" "build-tools;${ANDROID_SDK_BUILD_TOOLS_VERSION}" > /dev/null
    83-
    RUN yes | "${ANDROID_SDK_HOME}/tools/bin/sdkmanager" "platforms;android-27" > /dev/null
    84-
    85-
    # Set avdmanager permissions (executable)
    86-
    RUN chmod +x "${ANDROID_SDK_HOME}/tools/bin/avdmanager"
    87-
    88-
    8938
    ENV USER="user"
    9039
    ENV HOME_DIR="/home/${USER}"
    40+
    ENV ANDROID_HOME="${HOME_DIR}/.android"
    9141
    ENV WORK_DIR="${HOME_DIR}" \
    9242
    PATH="${HOME_DIR}/.local/bin:${PATH}"
    9343

    @@ -114,6 +64,10 @@ RUN ${RETRY} apt -y install -qq --no-install-recommends \
    11464
    && apt -y autoremove \
    11565
    && apt -y clean
    11666

    67+
    # Install Java and set JAVA_HOME (to accept android's SDK licenses)
    68+
    RUN ${RETRY} apt -y install -qq --no-install-recommends openjdk-8-jdk \
    69+
    && apt -y autoremove && apt -y clean
    70+
    ENV JAVA_HOME /usr/lib/jvm/java-8-openjdk-amd64
    11771

    11872
    # prepare non root env
    11973
    RUN useradd --create-home --shell /bin/bash ${USER}
    @@ -127,9 +81,12 @@ RUN pip2 install --upgrade Cython==0.28.6
    12781

    12882
    WORKDIR ${WORK_DIR}
    12983
    COPY --chown=user:user . ${WORK_DIR}
    130-
    RUN chown --recursive ${USER} ${ANDROID_SDK_HOME}
    84+
    RUN mkdir ${ANDROID_HOME} && chown --recursive ${USER} ${ANDROID_HOME}
    13185
    USER ${USER}
    13286

    87+
    # Download and install android's NDK/SDK
    88+
    RUN make -f ci/makefiles/android.mk target_os=linux
    89+
    13390
    # install python-for-android from current branch
    13491
    RUN virtualenv --python=python3 venv \
    13592
    && . venv/bin/activate \

    ‎README.md

    Lines changed: 19 additions & 15 deletions
    Original file line numberDiff line numberDiff line change
    @@ -92,29 +92,33 @@ Please refer to the LICENSE file.
    9292

    9393
    ## History
    9494

    95-
    In 2015 these tools were rewritten to provide a new, easier to use and
    96-
    extend interface. If you'd like to browse the old toolchain, its
    95+
    In 2015 these tools were rewritten to provide a new, easier-to-use and
    96+
    easier-to-extend interface. If you'd like to browse the old toolchain, its
    9797
    status is recorded for posterity at at
    9898
    https://github.com/kivy/python-for-android/tree/old_toolchain.
    9999

    100-
    In the last quarter of 2018 the python recipes has been changed, the
    101-
    new recipe for python3 (3.7.1) has a new build system which has been
    100+
    In the last quarter of 2018 the python recipes were changed. The
    101+
    new recipe for python3 (3.7.1) had a new build system which was
    102102
    applied to the ancient python recipe, allowing us to bump the python2
    103-
    version number to 2.7.15. This change, unifies the build process for
    104-
    both python recipes, and probably solve some issues detected over the
    105-
    years. Also should be mentioned that this **unified python recipes**
    106-
    require to target to a **minimum api level of 21**,
    107-
    *Android 5.0 - Lollipop*, so in case the you need to build targeting an
    108-
    api level below 21, you must use an old version of python-for-android
    109-
    (<=0.7.1). All this work has been done using android ndk version r17c,
    110-
    and your build should success with that version...but be aware that the
    111-
    project is in constant development so...the ndk version will change at
    112-
    some time.
    103+
    version number to 2.7.15. This change unified the build process for
    104+
    both python recipes, and probably solved various issues detected over the
    105+
    years. It should also be mentioned that these **unified python recipes**
    106+
    require a **minimum target api level of 21**,
    107+
    *Android 5.0 - Lollipop*, so in the case that you need to build targeting an
    108+
    api level below 21, you should use an older version of python-for-android
    109+
    (<=0.7.1).
    110+
    111+
    Be aware that this project is in constant development so, as per time of writing,
    112+
    you should use a minimum on Android's NDK r19, and ``we recommend using NDK r19b``.
    113+
    This is because the toolchains installed by
    114+
    default with the NDK can be used *in-place* and the python-for-android project
    115+
    has been adapted for that feature. Also be aware that more recent versions of the
    116+
    Android's NDK may not work.
    113117

    114118
    Those mentioned changes has been done this way to make easier the transition
    115119
    between python3 and python2. We will slowly phase out python2 support
    116120
    towards 2020...so...if you are using python2 in your projects you should
    117-
    consider to migrate it into python3.
    121+
    consider migrating it into python3.
    118122

    119123
    ## Contributors
    120124

    ‎ci/constants.py

    Lines changed: 0 additions & 2 deletions
    Original file line numberDiff line numberDiff line change
    @@ -65,10 +65,8 @@ class TargetPython(Enum):
    6565
    # IndexError: list index out of range
    6666
    'secp256k1',
    6767
    'ffpyplayer',
    68-
    'icu',
    6968
    # requires `libpq-dev` system dependency e.g. for `pg_config` binary
    7069
    'psycopg2',
    71-
    'protobuf_cpp',
    7270
    # most likely some setup in the Docker container, because it works in host
    7371
    'pyjnius', 'pyopenal',
    7472
    # SyntaxError: invalid syntax (Python2)

    ‎ci/makefiles/android.mk

    Lines changed: 68 additions & 0 deletions
    Original file line numberDiff line numberDiff line change
    @@ -0,0 +1,68 @@
    1+
    # Downloads and installs the Android SDK depending on supplied platform: darwin or linux
    2+
    3+
    # We must provide a platform (darwin or linux) and we need JAVA_HOME defined
    4+
    ifndef target_os
    5+
    $(error target_os is not set...aborted!)
    6+
    endif
    7+
    8+
    # Those android NDK/SDK variables can be override when running the file
    9+
    ANDROID_NDK_VERSION ?= 19b
    10+
    ANDROID_SDK_TOOLS_VERSION ?= 4333796
    11+
    ANDROID_SDK_BUILD_TOOLS_VERSION ?= 28.0.2
    12+
    ANDROID_HOME ?= $(HOME)/.android
    13+
    ANDROID_API_LEVEL ?= 27
    14+
    15+
    ANDROID_SDK_HOME=$(ANDROID_HOME)/android-sdk
    16+
    ANDROID_SDK_TOOLS_ARCHIVE=sdk-tools-$(target_os)-$(ANDROID_SDK_TOOLS_VERSION).zip
    17+
    ANDROID_SDK_TOOLS_DL_URL=https://dl.google.com/android/repository/$(ANDROID_SDK_TOOLS_ARCHIVE)
    18+
    19+
    ANDROID_NDK_HOME=$(ANDROID_HOME)/android-ndk
    20+
    ANDROID_NDK_FOLDER=$(ANDROID_HOME)/android-ndk-r$(ANDROID_NDK_VERSION)
    21+
    ANDROID_NDK_ARCHIVE=android-ndk-r$(ANDROID_NDK_VERSION)-$(target_os)-x86_64.zip
    22+
    ANDROID_NDK_DL_URL=https://dl.google.com/android/repository/$(ANDROID_NDK_ARCHIVE)
    23+
    24+
    $(info Target install OS is : $(target_os))
    25+
    $(info Android SDK home is : $(ANDROID_SDK_HOME))
    26+
    $(info Android NDK home is : $(ANDROID_NDK_HOME))
    27+
    $(info Android SDK download url is : $(ANDROID_SDK_TOOLS_DL_URL))
    28+
    $(info Android NDK download url is : $(ANDROID_NDK_DL_URL))
    29+
    $(info Android API level is : $(ANDROID_API_LEVEL))
    30+
    $(info Android NDK version is : $(ANDROID_NDK_VERSION))
    31+
    $(info JAVA_HOME is : $(JAVA_HOME))
    32+
    33+
    all: install_sdk install_ndk
    34+
    35+
    install_sdk: download_android_sdk extract_android_sdk update_android_sdk
    36+
    37+
    install_ndk: download_android_ndk extract_android_ndk
    38+
    39+
    download_android_sdk:
    40+
    curl --location --progress-bar --continue-at - \
    41+
    $(ANDROID_SDK_TOOLS_DL_URL) --output $(ANDROID_SDK_TOOLS_ARCHIVE)
    42+
    43+
    download_android_ndk:
    44+
    curl --location --progress-bar --continue-at - \
    45+
    $(ANDROID_NDK_DL_URL) --output $(ANDROID_NDK_ARCHIVE)
    46+
    47+
    # Extract android SDK and remove the compressed file
    48+
    extract_android_sdk:
    49+
    mkdir -p $(ANDROID_SDK_HOME) \
    50+
    && unzip -q $(ANDROID_SDK_TOOLS_ARCHIVE) -d $(ANDROID_SDK_HOME) \
    51+
    && rm -f $(ANDROID_SDK_TOOLS_ARCHIVE)
    52+
    53+
    54+
    # Extract android NDK and remove the compressed file
    55+
    extract_android_ndk:
    56+
    mkdir -p $(ANDROID_NDK_FOLDER) \
    57+
    && unzip -q $(ANDROID_NDK_ARCHIVE) -d $(ANDROID_HOME) \
    58+
    && ln -sfn $(ANDROID_NDK_FOLDER) $(ANDROID_NDK_HOME) \
    59+
    && rm -f $(ANDROID_NDK_ARCHIVE)
    60+
    61+
    # updates Android SDK, install Android API, Build Tools and accept licenses
    62+
    update_android_sdk:
    63+
    touch $(ANDROID_HOME)/repositories.cfg
    64+
    yes | $(ANDROID_SDK_HOME)/tools/bin/sdkmanager --licenses > /dev/null
    65+
    $(ANDROID_SDK_HOME)/tools/bin/sdkmanager "build-tools;$(ANDROID_SDK_BUILD_TOOLS_VERSION)" > /dev/null
    66+
    $(ANDROID_SDK_HOME)/tools/bin/sdkmanager "platforms;android-$(ANDROID_API_LEVEL)" > /dev/null
    67+
    # Set avdmanager permissions (executable)
    68+
    chmod +x $(ANDROID_SDK_HOME)/tools/bin/avdmanager

    ‎ci/makefiles/osx.mk

    Lines changed: 23 additions & 0 deletions
    Original file line numberDiff line numberDiff line change
    @@ -0,0 +1,23 @@
    1+
    # installs java 1.8, android's SDK/NDK, cython and p4a
    2+
    3+
    # The following variable/s can be override when running the file
    4+
    ANDROID_HOME ?= $(HOME)/.android
    5+
    6+
    all: install_java upgrade_cython install_android_ndk_sdk install_p4a
    7+
    8+
    install_java:
    9+
    brew tap adoptopenjdk/openjdk
    10+
    brew cask install adoptopenjdk8
    11+
    /usr/libexec/java_home -V
    12+
    13+
    upgrade_cython:
    14+
    pip3 install --upgrade Cython==0.28.6
    15+
    16+
    install_android_ndk_sdk:
    17+
    mkdir -p $(ANDROID_HOME)
    18+
    make -f ci/makefiles/android.mk target_os=darwin JAVA_HOME=`/usr/libexec/java_home -v 1.8`
    19+
    20+
    install_p4a:
    21+
    # check python version and install p4a
    22+
    python3 --version
    23+
    pip3 install -e .

    0 commit comments

    Comments
     (0)
    0