GenerateBundle Task fails writing to singlefilehost.exe

[dsplaisted]

Build Information

Build: https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=1146095
Build error leg or test failing: Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution
Pull request: #50648

Error Message

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": "",
  "ErrorPattern": "The process cannot access the file '.*singlefilehost\\.exe' because it is being used by another process",
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=1146095
Error message validated: [The process cannot access the file '.*singlefilehost\.exe' because it is being used by another process]
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 9/12/2025 3:27:55 PM UTC

Report

Build	Definition	Test	Pull Request
1162252	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51075
1162215	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51074
1162101	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.5.WorkItemExecution	#51071
1162125	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51068
2805635	dotnet-sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#53833
1159610	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51020
1161981	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51068
1161907	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51069
1161906	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50999
1160239	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51042
1161644	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51063
1160876	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51054
1161099	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50864
1161516	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51039
1161431	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51031
1161295	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51058
1161249	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51036
1161229	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50999
1161230	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.5.WorkItemExecution	#51050
1161015	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50999
1160940	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50999
1160906	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51031
1160756	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51053
1160740	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51052
1160652	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51050
1159928	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51039
1160074	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51032
1160469	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50999
1160285	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51031
1160278	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51036
1160283	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51044
1160277	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51043
1159681	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51031
1160093	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.5.WorkItemExecution	#51017
1160083	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.5.WorkItemExecution	#51019
1160022	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51036
1159860	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51039
1159639	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50999
1159565	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50992
1158694	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50901
1159023	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51020
1159428	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50992
1159285	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50992
1159255	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50999
1159211	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51027
1158830	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50983
1158684	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51014
1158808	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51019
1157762	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50901
1157823	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50974
1158448	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50992
1157799	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50975
1158291	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50992
1158265	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50983
1157845	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51004
1157923	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#51006
1157881	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50999
1157814	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50496
1157558	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50974
1157804	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#49617
1157775	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50977
1157737	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50998
1157755	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50999
1157560	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50901
1157588	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50997
1157189	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50974
1156987	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.5.WorkItemExecution	#42266
1157487	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50983
1157462	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50946
1157424	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50983
1157262	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50983
1155810	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50963
1157177	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50977
1156801	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50901
1157136	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50951
1157135	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50983
1157152	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50980
1157149	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50982
1157039	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50977
1156886	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50975
1156955	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50977
1156737	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50974
1156153	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50971
1156830	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#47772
1156846	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.5.WorkItemExecution	#50975
1156782	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.5.WorkItemExecution	#50948
1156426	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50946
1154766	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50954
1156177	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50972
1154298	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50844
1156155	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50970
1156096	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50937
1155916	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50885
1155811	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50967
1155126	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50960
1155866	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50948
1154474	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50948
1155226	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50937
1155743	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50963
1155679	dotnet/sdk	Microsoft.NET.Publish.Tests.dll.6.WorkItemExecution	#50901
Displaying 100 of 184 results

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
14	83	184

[dsplaisted]

@dotnet/illink @agocke We are seeing failures where the GenerateBundle task is failing because it cannot access the singlnefilehost.exe file. We added retry logic because we thought maybe the issues were caused by defender. I thought that helped, but now the issue is happening pretty consistently, so it may be a product issue.

Interestingly, it seems to only fail on Mac OS and Linux when bundling a Windows app. I can't remember if it was failing on Windows before we added the retry logic or not.

Can someone investigate this?

I'm also wondering whether we should merge this codeflow PR which is blocked by the failure. That's where I've seen the failure most consistently recently, so it's possible there's a code change coming in that made the problem worse.

[agocke]

@jtschuster can you take a look?

[dsplaisted]

Ping @jtschuster, any progress on this? I just noticed it in #50876

[agocke]

I'm going to load balance a bit. @sbomer can you look instead?

[jtschuster]

Still investigating, but I am able to repro locally on a mac, but only on the first run of the test. Subsequent runs use the same path for the tmp project and don't repro. It's also interesting that it's failing on a read and not a write.

[dsplaisted]

@jtschuster Any update on this? This is impacting a bunch of our CI runs.

[jtschuster]

Still investigating exactly what's going on, but my hunch is that there's an async memory map flush that keeps the apphost mapped from CreateAppHost. In particular it looks like this todo might fix the issue: https://github.com/dotnet/runtime/blob/b8c39cc4d102518f8608df70e6e3cb7a5c10d48d/src/installer/managed/Microsoft.NET.HostModel/AppHost/HostWriter.cs#L172
The ResourceUpdater.Update() maps the destination apphost back into memory, and if there's async flushing after Dispose(), we could run into issues.

As a workaround, it looks like adding --disable-build-servers to the PublishCommand.Execute() method does something to make the issue fail to repro for both me and @sbomer. I'm not sure why, though.