Description
Abstract
Ticket to keep a track and find possible solutions for flaky tests we saw on Travis.
Known errors
curl: (56) GnuTLS recv error
Happens during CrystaX NDK download, the error is:
curl: (56) GnuTLS recv error (-9): A TLS packet with unexpected length was received.
Example log: https://travis-ci.org/kivy/python-for-android/jobs/397824994
Possible solution, do some download retry:
https://stackoverflow.com/questions/10568497/how-does-curl-retry-max-time-seconds-work
curl: (35) gnutls_handshake() failed
Also happens during CrystaX NDK download, error:
curl: (35) gnutls_handshake() failed: Error in the pull function.
Example logs:
- https://travis-ci.org/kivy/python-for-android/jobs/381502437
- https://travis-ci.org/kivy/python-for-android/jobs/381297644
- https://travis-ci.org/kivy/python-for-android/jobs/381499077
Possible solution also some retries.
curl: (7) Failed to connect to eu.crystax.net port 443: Connection timed out
Again during CrystaX NDK download, error:
curl: (7) Failed to connect to eu.crystax.net port 443: Connection timed out
The command '/bin/sh -c curl --location --progress-bar "${CRYSTAX_NDK_DL_URL}" --output "${CRYSTAX_NDK_ARCHIVE}" --insecure && bsdtar -xf "${CRYSTAX_NDK_ARCHIVE}" --directory "${ANDROID_HOME}" --exclude=crystax-ndk-${CRYSTAX_NDK_VERSION}/docs --exclude=crystax-ndk-${CRYSTAX_NDK_VERSION}/samples --exclude=crystax-ndk-${CRYSTAX_NDK_VERSION}/tests --exclude=crystax-ndk-${CRYSTAX_NDK_VERSION}/toolchains/renderscript --exclude=crystax-ndk-${CRYSTAX_NDK_VERSION}/toolchains/x86_64-* --exclude=crystax-ndk-${CRYSTAX_NDK_VERSION}/toolchains/llvm-* --exclude=crystax-ndk-${CRYSTAX_NDK_VERSION}/toolchains/aarch64-* --exclude=crystax-ndk-${CRYSTAX_NDK_VERSION}/toolchains/mips64el-* && ln -sfn "${CRYSTAX_NDK_HOME_V}" "${CRYSTAX_NDK_HOME}" && rm -rf "${CRYSTAX_NDK_ARCHIVE}"' returned a non-zero code: 7
Example log: https://travis-ci.org/kivy/python-for-android/jobs/415163201 (rebuilt since then).
TypeError: unsupported operand type(s) for -=: 'Retry' and 'int'
Happens when PyPI or whichever index server is down.
Error log: https://travis-ci.org/kivy/python-for-android/builds/375908909
Possible solution, retry or fallback on some mirrors.
error occurred while preparing SDK package Android SDK
Happens during SDK update using sdkmanager, error is:
Warning: An error occurred while preparing SDK package Android SDK Platform 19: Connection timed out (Connection timed out).
Example log: https://travis-ci.org/kivy/python-for-android/jobs/389474611
Implementing a retrying procedure may also help here.
UnknownHostException: services.gradle.org
Exception in thread "main" java.net.UnknownHostException: services.gradle.org
Happened during gradle (https://services.gradle.org/distributions/gradle-3.3-all.zip) download.
See full log attached:
gradle-download-error-log.txt
E: Failed to fetch
This one is more rare, but happened recently in the before_install section while running the apt update, see log extract below:
$ sudo apt update -qq
W: http://ppa.launchpad.net/couchdb/stable/ubuntu/dists/trusty/Release.gpg: Signature by key 15866BAFD9BCC4F3C1E0DFC7D69548E1C17EAB57 uses weak digest algorithm (SHA1)
W: An error occurred during the signature verification. The repository is not updated and the previous index files will be used. GPG error: https://packagecloud.io/github/git-lfs/ubuntu trusty InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 6B05F25D762E3157
W: Failed to fetch https://packagecloud.io/github/git-lfs/ubuntu/dists/trusty/InRelease The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 6B05F25D762E3157
E: Failed to fetch http://security.ubuntu.com/ubuntu/dists/trusty-security/main/source/Sources.gz Hash Sum mismatch
E: Failed to fetch http://security.ubuntu.com/ubuntu/dists/trusty-security/main/binary-amd64/Packages.gz Hash Sum mismatch
W: Some index files failed to download. They have been ignored, or old ones used instead.
The command "sudo apt update -qq" failed and exited with 100 during
See complete log: https://travis-ci.org/AndreMiras/python-for-android/jobs/486983237
Using travis_retry would probably be an easy workaround for that one.
Android SDK Platform 19: archive is not a ZIP archive.
During the docker build:
Step 21/36 : RUN "${ANDROID_SDK_HOME}/tools/bin/sdkmanager" "platforms;android-19" && "${ANDROID_SDK_HOME}/tools/bin/sdkmanager" "platforms;android-27" && "${ANDROID_SDK_HOME}/tools/bin/sdkmanager" "build-tools;${ANDROID_SDK_BUILD_TOOLS_VERSION}" && chmod +x "${ANDROID_SDK_HOME}/tools/bin/avdmanager"
---> Running in d32f2dfeef3f
Warning: An error occurred while preparing SDK package Android SDK Platform 19: archive is not a ZIP archive.
The command '/bin/sh -c "${ANDROID_SDK_HOME}/tools/bin/sdkmanager" "platforms;android-19" && "${ANDROID_SDK_HOME}/tools/bin/sdkmanager" "platforms;android-27" && "${ANDROID_SDK_HOME}/tools/bin/sdkmanager" "build-tools;${ANDROID_SDK_BUILD_TOOLS_VERSION}" && chmod +x "${ANDROID_SDK_HOME}/tools/bin/avdmanager"' returned a non-zero code: 1
The command "docker build --tag=p4a --file Dockerfile.py3 ." exited with 1.
Remark / current conclusions
So far the weakest point seems to be the NDK download, so that may probably be the first one we want to tacle.
Possible workaround / solutions
Edit
Recently the CrystaX NDK was dropped in profit for pure Python builds. However this ticket is still relevant for other hiccup. I'll update later with new errors we see the most often.