[release/9.0] Building runtime failed with ILCompiler.CodeGenerationFailedException

[ellahathaway]

The Ubuntu2404Arm64_Offline_MsftSdk_arm64 leg hit the following exception:

/vmr/src/runtime/artifacts/bin/Crossgen2Tasks/Release/net9.0/Microsoft.NET.CrossGen.targets(469,5): error : Unhandled exception. ILCompiler.CodeGenerationFailedException: Code generation failed for method '[System.Net.Quic]Interop+Crypto..cctor()' [/vmr/src/runtime/src/installer/pkg/sfx/Microsoft.NETCore.App/Microsoft.NETCore.App.Runtime.sfxproj]
/vmr/src/runtime/artifacts/bin/Crossgen2Tasks/Release/net9.0/Microsoft.NET.CrossGen.targets(469,5): error :  ---> System.NullReferenceException: Object reference not set to an instance of an object. [/vmr/src/runtime/src/installer/pkg/sfx/Microsoft.NETCore.App/Microsoft.NETCore.App.Runtime.sfxproj]
/vmr/src/runtime/artifacts/bin/Crossgen2Tasks/Release/net9.0/Microsoft.NET.CrossGen.targets(469,5): error :    at Internal.JitInterface.CorInfoImpl._beginInlining(IntPtr thisHandle, IntPtr* ppException, CORINFO_METHOD_STRUCT_* inlinerHnd, CORINFO_METHOD_STRUCT_* inlineeHnd) in /_/src/runtime/src/coreclr/tools/Common/JitInterface/CorInfoImpl_generated.cs:line 154 [/vmr/src/runtime/src/installer/pkg/sfx/Microsoft.NETCore.App/Microsoft.NETCore.App.Runtime.sfxproj]
/vmr/src/runtime/artifacts/bin/Crossgen2Tasks/Release/net9.0/Microsoft.NET.CrossGen.targets(469,5): error :    --- End of inner exception stack trace --- [/vmr/src/runtime/src/installer/pkg/sfx/Microsoft.NETCore.App/Microsoft.NETCore.App.Runtime.sfxproj]
/vmr/src/runtime/artifacts/bin/Crossgen2Tasks/Release/net9.0/Microsoft.NET.CrossGen.targets(469,5): error :    at Internal.JitInterface.CorInfoImpl.CompileMethodInternal(IMethodNode methodCodeNodeNeedingCode, MethodIL methodIL) in /_/src/runtime/src/coreclr/tools/Common/JitInterface/CorInfoImpl.cs:line 381 [/vmr/src/runtime/src/installer/pkg/sfx/Microsoft.NETCore.App/Microsoft.NETCore.App.Runtime.sfxproj]
/vmr/src/runtime/artifacts/bin/Crossgen2Tasks/Release/net9.0/Microsoft.NET.CrossGen.targets(469,5): error :    at Internal.JitInterface.CorInfoImpl.CompileMethod(MethodWithGCInfo methodCodeNodeNeedingCode, Logger logger) in /_/src/runtime/src/coreclr/tools/aot/ILCompiler.ReadyToRun/JitInterface/CorInfoImpl.ReadyToRun.cs:line 810 [/vmr/src/runtime/src/installer/pkg/sfx/Microsoft.NETCore.App/Microsoft.NETCore.App.Runtime.sfxproj]
/vmr/src/runtime/artifacts/bin/Crossgen2Tasks/Release/net9.0/Microsoft.NET.CrossGen.targets(469,5): error :    at ILCompiler.ReadyToRunCodegenCompilation.<>c__DisplayClass50_0.<ComputeDependencyNodeDependencies>g__CompileOneMethod|5(DependencyNodeCore`1 dependency, Int32 compileThreadId) in /_/src/runtime/src/coreclr/tools/aot/ILCompiler.ReadyToRun/Compiler/ReadyToRunCodegenCompilation.cs:line 898 [/vmr/src/runtime/src/installer/pkg/sfx/Microsoft.NETCore.App/Microsoft.NETCore.App.Runtime.sfxproj]
/vmr/src/runtime/artifacts/bin/Crossgen2Tasks/Release/net9.0/Microsoft.NET.CrossGen.targets(469,5): error :    at ILCompiler.ReadyToRunCodegenCompilation.<>c__DisplayClass50_0.<ComputeDependencyNodeDependencies>g__CompileOnThread|4(Int32 compilationThreadId) in /_/src/runtime/src/coreclr/tools/aot/ILCompiler.ReadyToRun/Compiler/ReadyToRunCodegenCompilation.cs:line 833 [/vmr/src/runtime/src/installer/pkg/sfx/Microsoft.NETCore.App/Microsoft.NETCore.App.Runtime.sfxproj]
/vmr/src/runtime/artifacts/bin/Crossgen2Tasks/Release/net9.0/Microsoft.NET.CrossGen.targets(469,5): error :    at ILCompiler.ReadyToRunCodegenCompilation.<>c__DisplayClass50_0.<ComputeDependencyNodeDependencies>g__CompilationThread|3(Object objThreadId) in /_/src/runtime/src/coreclr/tools/aot/ILCompiler.ReadyToRun/Compiler/ReadyToRunCodegenCompilation.cs:line 811 [/vmr/src/runtime/src/installer/pkg/sfx/Microsoft.NETCore.App/Microsoft.NETCore.App.Runtime.sfxproj]
/vmr/src/runtime/artifacts/bin/Crossgen2Tasks/Release/net9.0/Microsoft.NET.CrossGen.targets(469,5): error : Unhandled exception. System.ArgumentNullException: Value cannot be null. (Parameter 'array') [/vmr/src/runtime/src/installer/pkg/sfx/Microsoft.NETCore.App/Microsoft.NETCore.App.Runtime.sfxproj]
/vmr/src/runtime/artifacts/bin/Crossgen2Tasks/Release/net9.0/Microsoft.NET.CrossGen.targets(469,5): error :    at System.Array.Clear(Array array, Int32 index, Int32 length) [/vmr/src/runtime/src/installer/pkg/sfx/Microsoft.NETCore.App/Microsoft.NETCore.App.Runtime.sfxproj]
/vmr/src/runtime/artifacts/bin/Crossgen2Tasks/Release/net9.0/Microsoft.NET.CrossGen.targets(469,5): error :    at Internal.JitInterface.CorInfoImpl.CompileMethodCleanup() in /_/src/runtime/src/coreclr/tools/Common/JitInterface/CorInfoImpl.cs:line 700 [/vmr/src/runtime/src/installer/pkg/sfx/Microsoft.NETCore.App/Microsoft.NETCore.App.Runtime.sfxproj]
/vmr/src/runtime/artifacts/bin/Crossgen2Tasks/Release/net9.0/Microsoft.NET.CrossGen.targets(469,5): error :    at Internal.JitInterface.CorInfoImpl.CompileMethod(MethodWithGCInfo methodCodeNodeNeedingCode, Logger logger) in /_/src/runtime/src/coreclr/tools/aot/ILCompiler.ReadyToRun/JitInterface/CorInfoImpl.ReadyToRun.cs:line 826 [/vmr/src/runtime/src/installer/pkg/sfx/Microsoft.NETCore.App/Microsoft.NETCore.App.Runtime.sfxproj]
/vmr/src/runtime/artifacts/bin/Crossgen2Tasks/Release/net9.0/Microsoft.NET.CrossGen.targets(469,5): error :    at ILCompiler.ReadyToRunCodegenCompilation.<>c__DisplayClass50_0.<ComputeDependencyNodeDependencies>g__CompileOneMethod|5(DependencyNodeCore`1 dependency, Int32 compileThreadId) in /_/src/runtime/src/coreclr/tools/aot/ILCompiler.ReadyToRun/Compiler/ReadyToRunCodegenCompilation.cs:line 898 [/vmr/src/runtime/src/installer/pkg/sfx/Microsoft.NETCore.App/Microsoft.NETCore.App.Runtime.sfxproj]
/vmr/src/runtime/artifacts/bin/Crossgen2Tasks/Release/net9.0/Microsoft.NET.CrossGen.targets(469,5): error :    at ILCompiler.ReadyToRunCodegenCompilation.<>c__DisplayClass50_0.<ComputeDependencyNodeDependencies>g__CompileOnThread|4(Int32 compilationThreadId) in /_/src/runtime/src/coreclr/tools/aot/ILCompiler.ReadyToRun/Compiler/ReadyToRunCodegenCompilation.cs:line 833 [/vmr/src/runtime/src/installer/pkg/sfx/Microsoft.NETCore.App/Microsoft.NETCore.App.Runtime.sfxproj]
/vmr/src/runtime/artifacts/bin/Crossgen2Tasks/Release/net9.0/Microsoft.NET.CrossGen.targets(469,5): error :    at ILCompiler.ReadyToRunCodegenCompilation.<>c__DisplayClass50_0.<ComputeDependencyNodeDependencies>g__CompilationThread|3(Object objThreadId) in /_/src/runtime/src/coreclr/tools/aot/ILCompiler.ReadyToRun/Compiler/ReadyToRunCodegenCompilation.cs:line 811 [/vmr/src/runtime/src/installer/pkg/sfx/Microsoft.NETCore.App/Microsoft.NETCore.App.Runtime.sfxproj]
/vmr/src/runtime/artifacts/bin/Crossgen2Tasks/Release/net9.0/Microsoft.NET.CrossGen.targets(357,5): error NETSDK1096: Optimizing assemblies for performance failed. You can either exclude the failing assemblies from being optimized, or set the PublishReadyToRun property to false. [/vmr/src/runtime/src/installer/pkg/sfx/Microsoft.NETCore.App/Microsoft.NETCore.App.Runtime.sfxproj]

https://dev.azure.com/dnceng/internal/_build/results?buildId=2522028&view=logs&j=b1caa8c7-837c-5a0e-fa2b-360b4dfa6cb4&t=5a647af6-2fef-58c2-4caf-850dc1bc163c (internal microsoft link)

It's important to note that this build is using 9.0.100-preview.7.24407.12 SDK and that this error has not been hit since this run. I suspect that this may be the same issue as dotnet/dnceng#3305 and dotnet/runtime#104123.

[dotnet-issue-labeler]

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[dotnet-issue-labeler]

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[ellahathaway]

This is likely a dupe - will close once I get confirmation that this is just another expression of dotnet/dnceng#3305 and dotnet/runtime#104123

[ellahathaway]

Hit this issue in another run: https://dev.azure.com/dnceng/internal/_build/results?buildId=2522556&view=results

[ellahathaway]

I'm continuing to investigate this issue. This is the exact same error as described in dotnet/runtime#105441. In that issue, this comment was left saying that the issue is being left open until they switch to a new enough SDK with the fix, but I am unsure which PR contains the fix.

@jakobbotsch - do you have a status update on that issue? I'm hesitant to close this source-build issue until I can confirm that the fix has flowed in with an SDK newer than 9.0.100-preview.7.24407.12

[jakobbotsch]

@ellahathaway the fix was made in dotnet/runtime#105832.