Android

Build on Android using Termux

Termux is a method to execute llama.cpp on an Android device (no root required).

apt update && apt upgrade -y
apt install git make cmake

It's recommended to move your model inside the ~/ directory for best performance:

cd storage/downloads
mv model.gguf ~/

Get the code & follow the Linux build instructions to build llama.cpp.

Building the Project using Android NDK

Obtain the Android NDK and then build with CMake.

Execute the following commands on your computer to avoid downloading the NDK to your mobile. Alternatively, you can also do this in Termux:

$ mkdir build-android
$ cd build-android
$ export NDK=<your_ndk_directory>
$ cmake -DCMAKE_TOOLCHAIN_FILE=$NDK/build/cmake/android.toolchain.cmake -DANDROID_ABI=arm64-v8a -DANDROID_PLATFORM=android-23 -DCMAKE_C_FLAGS=-march=armv8.4a+dotprod ..
$ make

Install termux on your device and run termux-setup-storage to get access to your SD card (if Android 11+ then run the command twice).

Finally, copy these built llama binaries and the model file to your device storage. Because the file permissions in the Android sdcard cannot be changed, you can copy the executable files to the /data/data/com.termux/files/home/bin path, and then execute the following commands in Termux to add executable permission:

(Assumed that you have pushed the built executable files to the /sdcard/llama.cpp/bin path using adb push)

$cp -r /sdcard/llama.cpp/bin /data/data/com.termux/files/home/
$cd /data/data/com.termux/files/home/bin
$chmod +x ./*

Download model llama-2-7b-chat.Q4_K_M.gguf, and push it to /sdcard/llama.cpp/, then move it to /data/data/com.termux/files/home/model/

$mv /sdcard/llama.cpp/llama-2-7b-chat.Q4_K_M.gguf /data/data/com.termux/files/home/model/

Now, you can start chatting:

$cd /data/data/com.termux/files/home/bin
$./llama-cli -m ../model/llama-2-7b-chat.Q4_K_M.gguf -n 128 -cml

Here's a demo of an interactive session running on Pixel 5 phone:

llama-interactive2.mp4

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!