Optional non-cross-compilation support?

[robertkirkman]

I have recently personally discovered/created a technique that creates an NDK that can run inside Termux and compile Application.mk NDK projects that fits in a zip of only 200 MB or possibly even less, compared to the 500 MB zip that comes from this page.

The major tradeoff that achieves this can be explained as, the resulting NDK loses cross-compilation support and therefore what that means is that it becomes only capable of building .so binaries that can be packaged into apps that run on the same device where Termux is running, and loses the ability to build .so files for apps for other devices - but with the potential benefit that the amount of downloading and storage needed is reduced significantly. Unpacking the release from this repo, removing some of the files and repacking it might achieve similar results, but my method also involves several other space-saving optimizations and download speed optimizations that cannot be achieved solely by editing the release zip from here.

I can add that, contrary to what one might expect, my technique does not require a separate github release file for each Termux app architecture; hypothetically, my technique could exist as a single github release file download for all architectures and still keep the space optimization despite the removal of cross-compilation support. If that sounds very confusing, it will be made clear when I show all the details.

The reason I opened this issue is because I would like to find out whether anyone wants this option so, whether anyone only needs to build apps for their own device and would gladly take that tradeoff to save space, or on the other hand, maybe everyone absolutely always needs all four .so files, the x86 one, the x86_64 one, the armeabi-v7a one and the arm64-v8a one, so maybe nobody would ever need this.

If many people want to know how to do this then I will polish and test my method more and then post all of the code here so that people can see how to do it, but if nobody wants this and it's a terrible idea, then this issue can be closed.

Also I forgot to mention that this method also works when you change the package name of Termux app from com.termux to something else, so it is involved with what I was talking about in #163 and if a cross-compiler is also built a similar way, it will have a similar capability

[robertkirkman]

I went ahead and finished my demo enough to post the proof of concept since it is sort of working.

Build official NDK sample with cross-compilation disabled

Total Termux storage use: 1.29 GB

pkg install wget git
wget https://github.com/user-attachments/files/17145163/ndk_27b_all.deb.txt -O ndk_27b_all.deb
apt install ./ndk_27b_all.deb
git clone -b android-mk https://github.com/android/ndk-samples.git
cd ndk-samples/hello-jni/jni/
ndk-build

Build and run lzhiyong/termux-ndk/build-app/GL3 with cross-compilation disabled

Total Termux storage use: 3.35 GB

cd
pkg install openjdk-17 gradle zip
# if i don't do this with the specific number 26.1.10909125 it doesn't work and I don't know why, especially since it is based on NDK r27b, like lzhiyong's termux-ndk, not 26. That's why I haven't placed this in the package yet.
echo "Pkg.Desc = Android NDK" >> $PREFIX/lib/ndk/source.properties
echo "Pkg.Revision = 26.1.10909125" >> $PREFIX/lib/ndk/source.properties
# I have not modified the SDK release
wget https://github.com/lzhiyong/termux-ndk/releases/download/android-sdk/android-sdk-aarch64.zip 
unzip -d opt android-sdk-aarch64.zip 
git clone https://github.com/lzhiyong/termux-ndk.git
cd termux-ndk/
# change ndk.dir, abiFilters and doNotStrip
wget https://github.com/user-attachments/files/17144835/termux-ndk-GLES3-disable-cross-compilation.patch.txt
patch -p1 < termux-ndk-GLES3-disable-cross-compilation.patch.txt
cd build-app/GL3/
gradle assembleDebug
# variant of the aapt2 thing
for i in $(find ~/.gradle/ | grep aapt2$); do cp ~/android-sdk/build-tools/35.0.0/aapt2 $i; done
gradle assembleDebug
cd app/build/outputs/apk/debug/
unzip app-debug.apk 
rm app-debug.apk
# for some reason gradle build is not doing this part properly while ndk-build invoked directly does
# (basically, force c++_shared for now until i figure out how to fix c++_static)
cd ../../../../src/main/
mv cpp jni
cd jni
ndk-build
cp ../libs/arm64-v8a/* ../../../build/outputs/apk/debug/lib/arm64-v8a/
cd ../../../build/outputs/apk/debug/
zip -0 -r GLES3.apk ./*
# this is the most convenient way I know to resign any app for testing purpose
wget https://github.com/patrickfav/uber-apk-signer/releases/download/v1.3.0/uber-apk-signer-1.3.0.jar
wget https://github.com/termux/termux-app/raw/refs/heads/master/app/testkey_untrusted.jks
export PATH="$HOME/opt/android-sdk/build-tools/35.0.0/:$PATH"
java -jar uber-apk-signer-1.3.0.jar -a GLES3.apk --ks testkey_untrusted.jks --ksAlias alias --ksKeyPass xrj45yWGLbsO7W0v --ksPass xrj45yWGLbsO7W0v
termux-setup-storage
cp GLES3-aligned-signed.apk /storage/emulated/0/
# Install and test apk in SurfaceFlinger now to make sure it is working for you; on my device it works

Build the ndk_27b_all.deb so you can see how it came to exist

Total Termux storage use: 6.11 GB

git clone https://github.com/termux/termux-packages.git
cd termux-packages/
wget https://github.com/user-attachments/files/17145162/termux-ndk-package.patch.txt
git apply -v termux-ndk-package.patch.txt
scripts/setup-termux.sh
./build-package.sh -I -f ndk
ls output/

Current suspected limitations of this implementation:

• probably CMake support not finished
• probably does not include all patches from lzhiyong/termux-ndk/patches, only some
• APP_STL := c++_static probably not working; I tried to get c++_static to also work but encountered a problem, so it is hardcoded to c++_shared. I could try to fix that. It will require writing the equivalent of a working libc++-static package which I tried to make but could not get to fully work yet.
• Projects that link to some libraries like libz.so using -lz may experience deceptively successful build, but library permission conflict at runtime between /system/lib64/libz.so that SurfaceFlinger apps have permission to access, and /data/data/com.termux/files/usr/lib/libz.so that SurfaceFlinger apps do not have permission to access! I will try to figure out some solution but it is hard. I tried to link directly to /system/lib64/libz.so but it did not work yet. In one of the ancestor projects I made previously, guide to build mesa for SurfaceFlinger with cross-compilation disabled (but meson build system not ndk-build build system), I worked around this by manually forcing the use of custom statically linkable libz.a. I feel like that kind of approach is probably insufficient here and I will have to learn how to fix libz.so properly.

termux-ndk-GLES3-disable-cross-compilation.patch
termux-ndk-package.patch
termux-ndk-package-27c.patch
ndk_27b_all.deb

The above file ndk_27b_all.deb makes this tangentially related to several things people were talking about in #156

• built from termux-packages ./build-package.sh
• invoke preexisting Termux package build-essential for C and C++ compilers
• potential for use on TUR if cross-compilation is not necessary, OR if cross-compilation support is re-added back in by recompiling the clang package to support cross-compilation and changing the package dependency ndk-sysroot (non cross compilation) to ndk-multilib (cross compilation equivalent)

[robertkirkman]

Unfinished libc++-native-static package

I tried to make this to enable support for APP_STL := c++_static, and it builds and installs, but I have not been able to get ndk-build to link to it correctly yet, there are errors. I will show it here in case someone already knows how to fix it or wants to try. (also this is only written for building with scripts/run-docker.sh [x86 build host] and that is how i was testing it so far, once working when built from the x86 build host, it would also need to be expanded to build ondevice)

--- a/packages/libc++/build.sh
+++ b/packages/libc++/build.sh
@@ -21,10 +21,18 @@ termux_step_get_source() {
 		local lib_path="toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib/${TERMUX_HOST_PLATFORM}"
 		mkdir -p "$TERMUX_PKG_SRCDIR"/"$lib_path"
 		cp "$NDK"/"$lib_path"/libc++_shared.so "$TERMUX_PKG_SRCDIR"/"$lib_path"
+		cp "$NDK"/"$lib_path"/libc++_static.a "$TERMUX_PKG_SRCDIR"/"$lib_path"
+		cp "$NDK"/"$lib_path"/libc++abi.a "$TERMUX_PKG_SRCDIR"/"$lib_path"
+		cp "$NDK"/"$lib_path"/libc++experimental.a "$TERMUX_PKG_SRCDIR"/"$lib_path"
+		cp "$NDK"/"$lib_path"/libstdc++.a "$TERMUX_PKG_SRCDIR"/"$lib_path"
 	fi
 }
 
 termux_step_post_make_install() {
-	install -m700 -t "$TERMUX_PREFIX"/lib \
-		toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib/"${TERMUX_HOST_PLATFORM}"/libc++_shared.so
+	local lib_path="toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib/${TERMUX_HOST_PLATFORM}"
+	install -m700 -t "$TERMUX_PREFIX"/lib "$lib_path"/libc++_shared.so
+	install -m700 -t "$TERMUX_PREFIX"/lib "$lib_path"/libc++_static.a
+	install -m700 -t "$TERMUX_PREFIX"/lib "$lib_path"/libc++abi.a
+	install -m700 -t "$TERMUX_PREFIX"/lib "$lib_path"/libc++experimental.a
+	install -m700 -t "$TERMUX_PREFIX"/lib "$lib_path"/libstdc++.a
 }
--- /dev/null
+++ b/packages/libc++/libc++-native-static.subpackage.sh
@@ -0,0 +1,8 @@
+TERMUX_SUBPKG_DESCRIPTION="Install native static C++ from NDK"
+TERMUX_SUBPKG_PLATFORM_INDEPENDENT=false
+TERMUX_SUBPKG_INCLUDE="
+lib/libc++_static.a
+lib/libc++abi.a
+lib/libc++experimental.a
+lib/libstdc++.a
+"
\ No newline at end of file

[robertkirkman]

• CMake

I forgot to mention this earlier but recently I have tested cmake 3.30.4 unpatched from Termux repos on my modified toolchain and it did work, despite not having the patches from this repo. Like my custom ndk-build command, it did also compile a .so for SurfaceFlinger successfully (as long as libc++_shared.so was installed appropriately into the apk as well).

# (after installing the ndk_27b_all.deb from above)
echo "Pkg.Desc = Android NDK" >> $PREFIX/lib/ndk/source.properties
echo "Pkg.Revision = 27.1.0" >> $PREFIX/lib/ndk/source.properties
pkg install cmake
cd ExampleCMakeProject
mkdir build
cd build
cmake -DCMAKE_TOOLCHAIN_FILE=$PREFIX/lib/ndk/build/cmake/android.toolchain.cmake \
             -DANDROID_NDK=$PREFIX/lib/ndk/ \
             -DANDROID_ABI=$(getprop ro.product.cpu.abi) \
             -DANDROID_STL=c++_shared \
             -DCMAKE_LIBRARY_OUTPUT_DIRECTORY=$HOME \
             -DCMAKE_C_COMPILER=$PREFIX/bin/clang \
             -DCMAKE_CXX_COMPILER=$PREFIX/bin/clang++ \
             ..
make
cp $PREFIX/lib/libc++_shared.so $HOME # Remeber to place this into your .apk's lib subdirectory for the same name you see in your getprop ro.product.cpu.abi command, next to your other .so files, when you build your .apk file, until some time when there is a way to use c++_static!

[knyipab]

Looks really great! Ever consider to make a PR to TUR?
Then we could start considering about Android Studio on TUR.

[robertkirkman]

Looks really great! Ever consider to make a PR to TUR? Then we could start considering about Android Studio on TUR.

I think that, since many Android NDK projects may specify APP_STL := c++_static and my workaround for those projects is just to force them to APP_STL := c++_shared mode instead, and also because of the similar problem remaining with projects that use -lz ,
this might not be ready for general users yet without causing confusion, since the compatibility can be hit or miss, basically.
I'm going to try to think of solutions for those problems and if I can, then I think it would be ready.

In my example here, I used the Java-based portion of the SDK unmodified from lzhiyong, but I think for the full potential of this technique to be realized, it needs to be refactored to be fully combined with the way you use the SDK from the other issue. The way I use the NDK here is basically similar to how you use the SDK, so further refactoring and repackaging can probably merge them together into a cohesive dependency tree.