Beginnings of native Android build #110471
Conversation
ghost
added
the
dotnet-policy-service
bot
added
the
community-contribution
steveisok
added
os-android
and removed
community-contribution
|
Tagging subscribers to 'arch-android': @vitek-karas, @simonrozsival, @steveisok, @akoeplinger |
grendello
force-pushed
the
dev/grendel/android-build-with-ndk
branch
from
c25fbd9 to
3d9da10
Compare
…ndk' into dev/grendel/android-build-with-ndk
…stem.Security.Cryptography.Native.Android. It will incorrectly try to install in the coreclr build
…em.native library for the singlefilehost
… R2R that acts like linux
…rg, and exclude asm routines on x64 build
…true when dealing with android
|
ok, I think this is ready for another review. @kotlarmilos can you do another official build pass to make sure we don't break / regress? |
eng/Subsets.props
Outdated
| <PropertyGroup> | ||
| <CrossBuild Condition="'$(CrossBuild)' == '' and '$(TargetsAndroid)' == 'true'">true</CrossBuild> | ||
| </PropertyGroup> |
There was a problem hiding this comment.
What is the expected way to invoke build.sh for android? Do we need to pass -cross? If we do need to pass -cross this property should already be set by that.
And if we don't pass -cross: does the code path added in eng/build.sh in this PR kick in? (It seems to be conditioned on -cross being passed to build.sh.)
(So far we have not been passing -cross when building for Android, there is a discussion about that at #56622).
There was a problem hiding this comment.
For me, locally, this change breaks the build (and so does passing -cross):
$ ./build.sh clr+libs --os android --arch arm64 -c release --restore --build
...
Commencing CoreCLR Repo build
Error: rootfsDir has been passed, but the location is not valid.
@steveisok are you absolutely sure it's necessary to enable cross build? Using the NDK cmake toolchain definition automatically makes the build a cross one, using the sysroot from the NDK and the appropriate target API+platform libraries and headers.
There was a problem hiding this comment.
it's necessary to enable cross build
In coreclr, we consider crossbuild to be anything that's not targeting the current architecture / OS for unix based systems. So this should be cross build too. I think there are subtle parts of the build scripts that I think could misbehave if it is not done that way. I am not 100% sure though.
But, I am not a fan of feeding in the -cross option automatically. That would make the build for Android experience different from other targets where we always specify the -cross on the command line explicitly.
@grendello looking at your command line, have you set the ROOTFS_DIR env variable too? That's what's necessary for cross builds in general.
There was a problem hiding this comment.
In coreclr, we consider crossbuild to be anything that's not targeting the current architecture / OS for unix based systems. So this should be cross build too. I think there are subtle parts of the build scripts that I think could misbehave if it is not done that way. I am not 100% sure though.
This is what #56622 discusses - the existing Android builds (Mono Android/Bionic, native AOT Bionic) do not pass -cross. The build infrastructure doesn't think it's a cross build (it's not considered as a cross build in libs build or in runtime build). It gets away with it because including the Android toolchain definition will make things happen.
The issue discusses switching Android build to the -cross plan but that didn't happen. I don't know if we even have Android build machines that would have the correct ROOTFS_DIR set up. We never built Android as -cross except for the community CoreCLR Android port attempt long time ago (none of the things that actually shipped and created official build specified -cross).
There was a problem hiding this comment.
it's necessary to enable cross build
In coreclr, we consider crossbuild to be anything that's not targeting the current architecture / OS for unix based systems. So this should be cross build too. I think there are subtle parts of the build scripts that I think could misbehave if it is not done that way. I am not 100% sure though. But, I am not a fan of feeding in the -cross option automatically. That would make the build for Android experience different from other targets where we always specify the -cross on the command line explicitly.
@grendello looking at your command line, have you set the ROOTFS_DIR env variable too? That's what's necessary for cross builds in general.
No, I have not - to what should I point it? The NDK's toolchains/llvm/prebuilt/${OS}/sysroot directory?
There was a problem hiding this comment.
we've historically never built Android/iOS/Browser with -cross since they all bring a toolchain with custom cmake and rootfs handling, we've only used -cross for "linux" builds. I'm not sure how hard it will be to untangle that.
That's what I mean by not being convinced we need to explicitly set ROOTFS_DIR in the coreclr build. Only passing the toolchain as a cmake argument will do.
There was a problem hiding this comment.
Is passing -cross needed only to select a different crossgen binary or do we need it for something else?
I'm asking because -cross does a lot more than that and passing both -cross and setting up Android cmake toolchain file will cause many things to be set twice. It feels like it will cause maintenance pain.
We could make it so proper crossgen gets picked for Android even without -cross and clean that up when/if the build gets switched to the real -cross plan.
There was a problem hiding this comment.
Is passing -cross needed only to select a different crossgen binary or do we need it for something else?
As far as I can tell, we only need it for crossgen. On osx (and presumably windows), the crossgen/ilcompiler builds will fail because it'll try to link against libjitinterface_arm64.so instead of libjitinterface_arm64.dylib.
There was a problem hiding this comment.
I validated we don't need -cross when building from linux hosts. I pushed a commit to only enable that when on osx for now so that we can move the PR forward.
There was a problem hiding this comment.
Looks like _BuildAnyCrossArch is the property we need with a HostOS != TargetOS condition addition.
|
Please add in the description of the PR which host->target builds have been successfully verified, and what build command was used |
…operly set when HostOS != TargetOS
There was a problem hiding this comment.
We should exclude official builds of coreCLR on Android for arm and x86 as they are still failing (referenced in the description of the PR and in the #111491) and I think this can be merged.
Thanks everyone for your contributions!
They are already excluded by this: |
* main: (22 commits) Clean up Stopwatch a bit (dotnet#111834) JIT: Fix embedded broadcast simd size (dotnet#111638) Revert potential UB due to aliasing + more WB removals (dotnet#111733) re-enable acceleration of Vector512<long>.op_Multiply (dotnet#111832) Handle OSSL 3.4 change to SAN:othername formatting JIT: Fix stack allocated arrays for NativeAOT (dotnet#111827) JIT: enhance RBO inference for similar compares to constants (dotnet#111766) JIT: Don't run optSetBlockWeights when we have PGO data (dotnet#111764) [Android] Make sure RuntimeFlavor=CoreCLR when clr subset is specified (dotnet#111821) Change empty subject test certificate to include a critical SAN. Fix reversed code offsets in GcInfo (dotnet#111792) Swap some libraries areas between leads (dotnet#111816) Add left-handed spherical and cylindrical billboards (dotnet#109605) JIT: revise `optRelopImpliesRelop` to always set `reverseSense` (dotnet#111803) Fix Zip64ExtraField handling (dotnet#111802) Add build support for Android+CoreCLR (dotnet#110471) arm64: Add bic(s) compact encoding (dotnet#111452) JIT: Ensure `BBF_PROF_WEIGHT` flag is set when we have PGO data (dotnet#111780) Add support for AVX10.2, Add AVX10.2 API surface and template tests (dotnet#111209) JIT: Preliminary for enabling inlining late devirted calls (dotnet#111782) ...
Tracking issue: #111491
Builds that work (linux and OSX):
Builds that don't (blocked by #111665)
Build commands that can be used:
CoreCLR itself
./build.sh -Subset clr.runtime+clr.alljits+clr.corelib+clr.nativecorelib+clr.tools+clr.packages+libs -c Debug|Release -os android -arch <arm64|arm|x64|x86>No subset (CoreCLR+Mono)
./build.sh -arch <arm64|arm|x64|x86> -os android -c Debug|Release