PythonActivityUtil helper for unpacking data #2189

AndreMiras · 2020-05-10T21:59:50Z

This helper class shares code to be reused between the PythonActivity
classes of the different bootstraps.
Also adds few minor adjustments to existing classes.

opacam · 2020-05-11T09:31:30Z

I tested the artifacts for arm64-v8a and armeabi-v7a and both apps crashes before unpacking files:

adb logcat

05-11 11:20:54.197 18372 18372 V PythonActivity: PythonActivity onCreate running 05-11 11:20:54.197 18372 18372 V PythonActivity: About to do super onCreate 05-11 11:20:54.198 18372 18372 V SDL : Device: hydrogen 05-11 11:20:54.198 18372 18372 V SDL : Model: Mi Max 05-11 11:20:54.198 18372 18372 V SDL : onCreate() 05-11 11:20:54.198 18372 18372 V PythonActivity: Did super onCreate 05-11 11:20:54.198 18372 18372 V SDL : getting identifier 05-11 11:20:54.198 18372 18372 V SDL : kind is drawable and name presplash 05-11 11:20:54.198 18372 18372 V SDL : result is 2130771970 05-11 11:20:54.236 18372 18372 V SDL : asked to get string presplash_color 05-11 11:20:54.236 18372 18372 V SDL : getting identifier 05-11 11:20:54.236 18372 18372 V SDL : kind is string and name presplash_color 05-11 11:20:54.236 18372 18372 V SDL : result is 2130968577 05-11 11:20:54.262 18372 18396 V PythonActivity: Ready to unpack 05-11 11:20:54.276 18372 18372 V PythonActivity: onResume() 05-11 11:20:54.276 18372 18372 V SDL : onResume() 05-11 11:20:54.281 18372 18396 E AndroidRuntime: FATAL EXCEPTION: AsyncTask #1 05-11 11:20:54.281 18372 18396 E AndroidRuntime: Process: org.test.unit_tests_app, PID: 18372 05-11 11:20:54.281 18372 18396 E AndroidRuntime: java.lang.RuntimeException: An error occurred while executing doInBackground() 05-11 11:20:54.281 18372 18396 E AndroidRuntime: at android.os.AsyncTask$4.done(AsyncTask.java:399) 05-11 11:20:54.281 18372 18396 E AndroidRuntime: at java.util.concurrent.FutureTask.finishCompletion(FutureTask.java:383) 05-11 11:20:54.281 18372 18396 E AndroidRuntime: at java.util.concurrent.FutureTask.setException(FutureTask.java:252) 05-11 11:20:54.281 18372 18396 E AndroidRuntime: at java.util.concurrent.FutureTask.run(FutureTask.java:271) 05-11 11:20:54.281 18372 18396 E AndroidRuntime: at android.os.AsyncTask$SerialExecutor$1.run(AsyncTask.java:289) 05-11 11:20:54.281 18372 18396 E AndroidRuntime: at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167) 05-11 11:20:54.281 18372 18396 E AndroidRuntime: at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641) 05-11 11:20:54.281 18372 18396 E AndroidRuntime: at java.lang.Thread.run(Thread.java:919) 05-11 11:20:54.281 18372 18396 E AndroidRuntime: Caused by: java.lang.RuntimeException: Can't create handler inside thread Thread[AsyncTask #1,5,main] that has not called Looper.prepare() 05-11 11:20:54.281 18372 18396 E AndroidRuntime: at android.os.Handler.<init>(Handler.java:207) 05-11 11:20:54.281 18372 18396 E AndroidRuntime: at android.os.Handler.<init>(Handler.java:119) 05-11 11:20:54.281 18372 18396 E AndroidRuntime: at android.app.Activity.<init>(Activity.java:880) 05-11 11:20:54.281 18372 18396 E AndroidRuntime: at org.kivy.android.PythonActivityUtil.<init>(PythonActivityUtil.java:22) 05-11 11:20:54.281 18372 18396 E AndroidRuntime: at org.kivy.android.PythonActivity$UnpackFilesTask.doInBackground(PythonActivity.java:104) 05-11 11:20:54.281 18372 18396 E AndroidRuntime: at org.kivy.android.PythonActivity$UnpackFilesTask.doInBackground(PythonActivity.java:99) 05-11 11:20:54.281 18372 18396 E AndroidRuntime: at android.os.AsyncTask$3.call(AsyncTask.java:378) 05-11 11:20:54.281 18372 18396 E AndroidRuntime: at java.util.concurrent.FutureTask.run(FutureTask.java:266) 05-11 11:20:54.281 18372 18396 E AndroidRuntime: ... 4 more

AndreMiras · 2020-05-11T09:59:57Z

Thank you for checking @opacam so I could eventually break it 🎉 😄
From my local tests it was working perfect with the tests apps, but maybe then I haven't cleaned things up properly before building again.
I'll take a look later. Thanks again for the heads up

AndreMiras · 2020-05-11T18:12:08Z

Incredible I've tested both artifacts and they are working on my device 🤔
For instance one output:

05-11 20:10:27.657 24939 25018 I python  : Initializing Python for Android
05-11 20:10:27.657 24939 25018 I python  : Setting additional env vars from p4a_env_vars.txt
05-11 20:10:27.658 24939 25018 I python  : Changing directory to the one provided by ANDROID_ARGUMENT
05-11 20:10:27.658 24939 25018 I python  : /data/user/0/org.test.unit_tests_app/files/app
05-11 20:10:27.701 24939 25018 I python  : Preparing to initialize python
05-11 20:10:27.701 24939 25018 I python  : _python_bundle dir exists
05-11 20:10:27.701 24939 25018 I python  : calculated paths to be...
05-11 20:10:27.701 24939 25018 I python  : /data/user/0/org.test.unit_tests_app/files/app/_python_bundle/stdlib.zip:/data/user/0/org.test.unit_tests_app/files/app/_python_bundle/modules
05-11 20:10:27.738 24939 25018 I python  : set wchar paths...
05-11 20:10:27.960 24939 25018 I python  : Initialized python
05-11 20:10:27.960 24939 25018 I python  : AND: Init threads
05-11 20:10:27.963 24939 25018 I python  : testing python print redirection
05-11 20:10:27.965 24939 25018 I python  : Android path ['.', '/data/user/0/org.test.unit_tests_app/files/app/_python_bundle/stdlib.zip', '/data/user/0/org.test.unit_tests_app/files/app/_python_bundle/modules', '/data/user/0/org.test.unit_tests_app/files/app/_python_bundle/site-packages']
05-11 20:10:27.966 24939 25018 I python  : os.environ is environ({'PATH': '/sbin:/vendor/bin:/system/sbin:/system/bin:/system/xbin:/system/vendor/bin:/vendor/xbin:/system/vendor/xbin:/product/bin:/product/xbin', 'ANDROID_BOOTLOGO': '1', 'ANDROID_ROOT': '/system', 'ANDROID_ASSETS': '/system/app', 'ANDROID_DATA': '/data', 'ANDROID_STORAGE': '', 'EXTERNAL_STORAGE': '/sdcard', 'ASEC_MOUNTPOINT': '/mnt/asec', 'BOOTCLASSPATH': '/system/framework/core-oj.jar:/system/framework/core-libart.jar:/system/framework/conscrypt.jar:/system/framework/okhttp.jar:/system/framework/core-junit.jar:/system/framework/bouncycastle.jar:/system/framework/ext.jar:/system/framework/framework.jar:/system/framework/telephony-common.jar:/system/framework/voip-common.jar:/system/framework/ims-common.jar:/system/framework/apache-xml.jar:/system/framework/org.apache.http.legacy.boot.jar:/system/framework/hwEmui.jar:/system/framework/hwTelephony-common.jar:/system/framework/hwframework.jar:/system/framework/org.simalliance.openmobileapi.jar:/system/framework/org.ifaa.android.manager.jar:/system/framework/hwaps.jar:/system/framework/hwcustEmui.jar:/system/framework/hwcustTelephony-common.jar:/system/framework/hwcustframework.jar', 'SYSTEMSERVERCLASSPATH': '/system/framework/services.jar:/system/framework/ethernet-service.jar:/system/framework/wifi-service.jar:/system/framework/hwServices.jar:/system/framework/hwWifi-service.jar:/system/framework/hwcustServices.jar:/system/framework/hwcustwifi-service.jar', 'OEM_ROOT': '/hw_oem', 'CUST_POLICY_DIRS': '/system/emui/base:/system/emui/oversea:/system/emui/oversea_honor:/system/emui/lite:/system/global:/vendor/etc:/product/etc:/hw_oem:/cust/hw/normal:/cust_comm:/version/region_comm/oversea:/cust_spec', 'ANDROID_SOCKET_zygote': '14', 'ANDROID_ENTRYPOINT': 'main.pyc', 'ANDROID_ARGUMENT': '/data/user/0/org.test.unit_tests_app/files/app', 'ANDROID_APP_PATH': '/data/user/0/org.test.unit_tests_app/files/app', 'ANDROID_PRIVATE': '/data/user/0/org.test.unit_tests_app/files', 'ANDROID_UNPACK': '/data/user/0/org.test.unit_tests_app/files/app', 'PYTHONHOME': '/data/user/0/org.test.unit_tests_app/files/app', 'PYTHONPATH': '/data/user/0/org.test.unit_tests_app/files/app:/data/user/0/org.test.unit_tests_app/files/app/lib', 'PYTHONOPTIMIZE': '2', 'P4A_BOOTSTRAP': 'SDL2', 'PYTHON_NAME': 'python', 'P4A_IS_WINDOWED': 'False', 'P4A_ORIENTATION': 'sensor', 'P4A_NUMERIC_VERSION': 'None', 'P4A_MINSDK': '21', 'LC_CTYPE': 'C.UTF-8'})
05-11 20:10:27.966 24939 25018 I python  : Android kivy bootstrap done. __name__ is __main__
05-11 20:10:27.966 24939 25018 I python  : AND: Ran string
05-11 20:10:27.966 24939 25018 I python  : Run user program, change dir and execute entrypoint

05-11 20:10:28.409 24939 25018 I python  : Imported unittest
05-11 20:10:28.411 24939 25018 I python  : App requirements are:  {'numpy', 'python3', 'libffi', 'pyjnius', 'requests', 'sqlite3', 'kivy', 'openssl', 'setuptools', 'sdl2'}
05-11 20:10:28.414 24939 25018 I python  : Defined test case
05-11 20:10:28.416 24939 25018 I python  : Adding Testcase:  tests.test_requirements.NumpyTestCase
05-11 20:10:28.419 24939 25018 I python  : Adding Testcase:  tests.test_requirements.LibffiTestCase
05-11 20:10:28.420 24939 25018 I python  : Adding Testcase:  tests.test_requirements.PyjniusTestCase
05-11 20:10:28.421 24939 25018 I python  : Adding Testcase:  tests.test_requirements.RequestsTestCase
05-11 20:10:28.422 24939 25018 I python  : Adding Testcase:  tests.test_requirements.Sqlite3TestCase
05-11 20:10:28.423 24939 25018 I python  : Adding Testcase:  tests.test_requirements.KivyTestCase
05-11 20:10:28.424 24939 25018 I python  : Adding Testcase:  tests.test_requirements.OpensslTestCase
05-11 20:10:28.426 24939 25018 I python  : Tests to perform are:  {'numpy': 'tests.test_requirements.NumpyTestCase', 'libffi': 'tests.test_requirements.LibffiTestCase', 'pyjnius': 'tests.test_requirements.PyjniusTestCase', 'requests': 'tests.test_requirements.RequestsTestCase', 'sqlite3': 'tests.test_requirements.Sqlite3TestCase', 'kivy': 'tests.test_requirements.KivyTestCase', 'openssl': 'tests.test_requirements.OpensslTestCase'}
05-11 20:10:28.506 24939 25018 I python  : [WARNING] [Config      ] Older configuration version detected (0 instead of 21)
05-11 20:10:28.506 24939 25018 I python  : [WARNING] [Config      ] Upgrading configuration in progress.
05-11 20:10:28.513 24939 25018 I python  : [INFO   ] [Logger      ] Record log in /data/user/0/org.test.unit_tests_app/files/app/.kivy/logs/kivy_20-05-11_0.txt
05-11 20:10:28.513 24939 25018 I python  : [INFO   ] [Kivy        ] v1.11.1
05-11 20:10:28.513 24939 25018 I python  : [INFO   ] [Kivy        ] Installed at "/data/user/0/org.test.unit_tests_app/files/app/_python_bundle/site-packages/kivy/__init__.pyc"
05-11 20:10:28.514 24939 25018 I python  : [INFO   ] [Python      ] v3.8.1 (default, May 10 2020, 22:11:20) 
05-11 20:10:28.514 24939 25018 I python  : [Clang 8.0.2 (https://android.googlesource.com/toolchain/clang 40173bab62ec7462
05-11 20:10:28.514 24939 25018 I python  : [INFO   ] [Python      ] Interpreter at ""

05-11 20:10:29.427 24939 25018 I python  : [INFO   ] [Factory     ] 184 symbols loaded

05-11 20:10:29.935 24939 25018 I python  : [INFO   ] [Image       ] Providers: img_tex, img_dds, img_sdl2, img_gif (img_pil, img_ffpyplayer ignored)
05-11 20:10:30.004 24939 25018 I python  : [INFO   ] [Text        ] Provider: sdl2
05-11 20:10:30.110 24939 25018 I python  : [INFO   ] [Window      ] Provider: sdl2

05-11 20:10:30.912 24939 25018 I python  : [INFO   ] [GL          ] Using the "OpenGL ES 2" graphics system
05-11 20:10:30.917 24939 25018 I python  : [INFO   ] [GL          ] Backend used <sdl2>
05-11 20:10:30.920 24939 25018 I python  : [INFO   ] [GL          ] OpenGL version <b'OpenGL ES 3.2 v1.r12p1-04bet0.1bb9662be2ebee934dcbd7265c794a91'>
05-11 20:10:30.921 24939 25018 I python  : [INFO   ] [GL          ] OpenGL vendor <b'ARM'>
05-11 20:10:30.923 24939 25018 I python  : [INFO   ] [GL          ] OpenGL renderer <b'Mali-T830'>
05-11 20:10:30.924 24939 25018 I python  : [INFO   ] [GL          ] OpenGL parsed version: 3, 2
05-11 20:10:30.926 24939 25018 I python  : [INFO   ] [GL          ] Texture max size <8192>
05-11 20:10:30.928 24939 25018 I python  : [INFO   ] [GL          ] Texture max units <16>
05-11 20:10:31.037 24939 25018 I python  : [INFO   ] [Window      ] auto add sdl2 input provider
05-11 20:10:31.047 24939 25018 I python  : [INFO   ] [Window      ] virtual keyboard not allowed, single mode, not docked
05-11 20:10:31.105 24939 25018 I python  : on_tests_to_perform:  dict_keys(['numpy', 'libffi', 'pyjnius', 'requests', 'sqlite3', 'kivy', 'openssl'])
05-11 20:10:31.390 24939 25018 I python  : [INFO   ] [GL          ] NPOT texture support is available

05-11 20:10:31.649 24939 25018 I python  : [WARNING] [Base        ] Unknown <android> provider
05-11 20:10:31.650 24939 25018 I python  : [INFO   ] [Base        ] Start application main loop

05-11 20:10:32.432 24939 25018 I python  : Imported unittest
05-11 20:10:32.432 24939 25018 I python  : loading tests...
05-11 20:10:32.453 24939 25018 I python  : running unittest...


05-11 20:10:35.678 24939 25018 I python  : unittest result is:
05-11 20:10:35.678 24939 25018 I python  : ..............
05-11 20:10:35.678 24939 25018 I python  : ----------------------------------------------------------------------
05-11 20:10:35.678 24939 25018 I python  : Ran 14 tests in 3.223s
05-11 20:10:35.678 24939 25018 I python  : 
05-11 20:10:35.678 24939 25018 I python  : OK
05-11 20:10:35.678 24939 25018 I python  : 
05-11 20:10:35.678 24939 25018 I python  : Ran tests

I'll try to understand what's going on by looking at your stacktrace.

opacam · 2020-05-11T19:39:51Z

I will try again, it could be a transfer issue with my phone

AndreMiras · 2020-05-11T19:47:14Z

That seems very unlikely to be a transfer issue to me. The stacktrace you shared is close to things I've touched so it could really be a regression on certain device. If you can reproduce on this PR, could you also check if current develop testapps are also crashing or if it's just this pull request?
In the meantime I'll try to see if I can reproduce with other devices. I may have idea how to fix, but first I want to be able to reproduce

opacam · 2020-05-11T19:50:26Z

Yea, I will try with latest artifact of develop branch...

BTW, the same error:

05-11 21:44:57.881 13760 13783 V PythonActivity: Ready to unpack
05-11 21:44:57.886 13760 13760 V PythonActivity: onResume()
05-11 21:44:57.886 13760 13760 V SDL     : onResume()
05-11 21:44:57.889 13760 13783 E AndroidRuntime: FATAL EXCEPTION: AsyncTask #1
05-11 21:44:57.889 13760 13783 E AndroidRuntime: Process: org.test.unit_tests_app, PID: 13760
05-11 21:44:57.889 13760 13783 E AndroidRuntime: java.lang.RuntimeException: An error occurred while executing doInBackground()
05-11 21:44:57.889 13760 13783 E AndroidRuntime:        at android.os.AsyncTask$4.done(AsyncTask.java:399)
05-11 21:44:57.889 13760 13783 E AndroidRuntime:        at java.util.concurrent.FutureTask.finishCompletion(FutureTask.java:383)
05-11 21:44:57.889 13760 13783 E AndroidRuntime:        at java.util.concurrent.FutureTask.setException(FutureTask.java:252)
05-11 21:44:57.889 13760 13783 E AndroidRuntime:        at java.util.concurrent.FutureTask.run(FutureTask.java:271)
05-11 21:44:57.889 13760 13783 E AndroidRuntime:        at android.os.AsyncTask$SerialExecutor$1.run(AsyncTask.java:289)
05-11 21:44:57.889 13760 13783 E AndroidRuntime:        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
05-11 21:44:57.889 13760 13783 E AndroidRuntime:        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
05-11 21:44:57.889 13760 13783 E AndroidRuntime:        at java.lang.Thread.run(Thread.java:919)
05-11 21:44:57.889 13760 13783 E AndroidRuntime: Caused by: java.lang.RuntimeException: Can't create handler inside thread Thread[AsyncTask #1,5,main] that has not called Looper.prepare()
05-11 21:44:57.889 13760 13783 E AndroidRuntime:        at android.os.Handler.<init>(Handler.java:207)
05-11 21:44:57.889 13760 13783 E AndroidRuntime:        at android.os.Handler.<init>(Handler.java:119)
05-11 21:44:57.889 13760 13783 E AndroidRuntime:        at android.app.Activity.<init>(Activity.java:880)
05-11 21:44:57.889 13760 13783 E AndroidRuntime:        at org.kivy.android.PythonActivityUtil.<init>(PythonActivityUtil.java:22)
05-11 21:44:57.889 13760 13783 E AndroidRuntime:        at org.kivy.android.PythonActivity$UnpackFilesTask.doInBackground(PythonActivity.java:104)
05-11 21:44:57.889 13760 13783 E AndroidRuntime:        at org.kivy.android.PythonActivity$UnpackFilesTask.doInBackground(PythonActivity.java:99)

For the record: I tested with Android 10 device

opacam · 2020-05-11T19:56:30Z

Develop is working, tested arm64-v8a artifact

AndreMiras · 2020-05-11T20:06:27Z

OK thank you good to know it's only this PR and develop is still OK.
I'll try to find a device where to reproduce, there's no rush to merge this.
Thanks for the help 🍻

opacam · 2020-05-11T20:18:29Z

No problem, ping me when you need me 😉
Btw, I think that is a very good refactoring, thanks!!

AndreMiras · 2020-05-11T20:20:39Z

Great I could reproduce on another device!
Will try to work on the fix now

AndreMiras · 2020-05-11T21:10:03Z

Got a working fix I think, but linting errors started popping.
So let's first merge #2191 and rebase

opacam

Awesome work!!, thanks!!

Note: tested the latest artifact for arm64-v8a without any issue 😄

🔖 v2020.06.02 * Adds missing requests sub dependencies (kivy#2221) * Bumps to Gradle 6.4.1 (kivy#2222) * Bumps to Cython==0.29.19 (kivy#2220) * Updates install and troubleshooting docs (kivy#2219) * Bumps to Ubuntu 20.04 (kivy#2218) * Attempt to improve the issue template (kivy#2217) * Add `opencv_extras` recipe (kivy#2209) * Split logic for build modes & debug symbols (kivy#2213) * Troubleshoot SSL error (kivy#2205) * Remove superfluous recipes fixes (kivy#2202) * Add tests for hostpython3 recipe (kivy#2196) * Fix for 'cannot find setuptools module' (kivy#2195) * Rename `Hostpython3Recipe` class to camel case (kivy#2194) * Fix `test_should_build` (kivy#2193) * Add initial tests for python3 recipe (kivy#2192) * PythonActivityUtil helper for unpacking data (kivy#2189) * Fixes flake8 errors post update (kivy#2191) * Share PythonUtil.java between bootstraps (kivy#2188) * Java code linting using PMD 6.23.0 (kivy#2187) * Deletes deprecated renpy Python{Activity,Service}.java (kivy#2186) * Removes java concurrency/ folder (kivy#2185) * Reuse common AssetExtract.java (kivy#2182) * Use common Hardware.java (kivy#2183) * Moves kamranzafar/ java directory to common/ (kivy#2184) * Updates release documentation (kivy#2177) * Fixes service only unittest loading (kivy#2181) * Narrows some context manager scopes (kivy#2179) * Downgrades to SDL2 2.0.9 (kivy#2180) * Bump to SDL2 2.0.10 & extract .java from SDL2 tarball (kivy#2113) * Adds pygame recipe (kivy#2164) * Adds macOS install instructions (2165) * Removed python2 support mention from README (kivy#2162) * Adding more assets (kivy#2132) * Get --add-source working for dirs in Gradle builds (kivy#2156) * Fixes python build with macOS venv (kivy#2159)

AndreMiras force-pushed the feature/share_unpack_data_java_code branch from e486766 to dfea735 Compare May 11, 2020 20:50

AndreMiras force-pushed the feature/share_unpack_data_java_code branch from dfea735 to 5312595 Compare May 12, 2020 07:20

opacam approved these changes May 12, 2020

View reviewed changes

AndreMiras merged commit 0a76c4a into kivy:develop May 12, 2020

AndreMiras deleted the feature/share_unpack_data_java_code branch May 12, 2020 11:32

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

PythonActivityUtil helper for unpacking data #2189

PythonActivityUtil helper for unpacking data #2189

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

PythonActivityUtil helper for unpacking data #2189

PythonActivityUtil helper for unpacking data #2189

Uh oh!

Conversation

Uh oh!

Uh oh!

Uh oh!

Uh oh!