Skip to content

Conversation

Numpsy
Copy link
Contributor

@Numpsy Numpsy commented Jan 9, 2025

Description

As #2818, but updated on top of the latest changes and with a couple of extra commits to try to get more tests passing - see further comments for details

let scenarioPath = resolvePath scenario ""
// dotnet tool install --version 5.19.0-alpha.local.1 fake-cli --add-source /e/Projects/FAKE/release/dotnetcore/

// Work around https://github.com/dotnet/sdk/issues/40655 by specifying the tool manifest explicitly
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some of tests in previous runs seemed to be failing with dotnet/sdk#40655, whereby installing the dotnet tool in a subdirectory that contains a tools config file will ignore that file and use the one in the repo root, but then trying to use that tool will use the local config file and then fail.
That issue is marked as resolved, but it still seemed to be occuring in the SDK version used by the tests

|> Async.RunSynchronously
|> List.ofSeq
|> List.find (fun product -> product.ProductVersion.Equals("6.0"))
|> List.find (fun product -> product.ProductVersion.Equals("8.0"))
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure what the logic for this (or cachedDotnetSdkReleases.json) should be when updating .NET versions.
The current cachedDotnetSdkReleases.json only contains the .NET 6 releases and that seemed to cause test failures when built as .NET 8 and looking for the v8 SDK versions.
Changing this to 8.0 fixed that, but that might then not be right if fake-cli is multi-targetted at .NET 6 and .NET 8 as the .NET 6 version might still need the list of v6 releases?

Copy link
Contributor

github-actions bot commented Jan 9, 2025

Test Results

  7 files   -   5    7 suites   - 5   18m 27s ⏱️ - 34m 15s
449 tests  -   2  444 ✅  -   6  1 💤 ±0  4 ❌ +4 
507 runs   - 780  496 ✅  - 788  3 💤 ±0  8 ❌ +8 

For more details on these failures, see this check.

Results for commit 62084b1. ± Comparison against base commit 73fc961.

This pull request removes 5 and adds 3 tests. Note that renamed tests count towards both.
[Fake.DotNet.FxCop.Tests; Test failure on non-Windows platforms]
[Fake.DotNet.ILMerge.Tests; Test failure on non-Windows platforms]
[Fake.DotNet.sdkAssemblyResolverTests; Runner run script with 6.0.300 SDK version assemblies and resolve runtime version from cached file]
[Fake.DotNet.sdkAssemblyResolverTests; Runner run script with 6.0.300 SDK version assemblies]
[Fake.DotNet.sdkAssemblyResolverTests; Runner run script with 6.0.301 SDK version assemblies]
[Fake.DotNet.sdkAssemblyResolverTests; Runner run script with 8.0.400 SDK version assemblies and resolve runtime version from cached file]
[Fake.DotNet.sdkAssemblyResolverTests; Runner run script with 8.0.400 SDK version assemblies]
[Fake.DotNet.sdkAssemblyResolverTests; Runner run script with 8.0.401 SDK version assemblies]

♻️ This comment has been updated with latest results.

@Numpsy
Copy link
Contributor Author

Numpsy commented Jan 9, 2025

The remaining failures are in the TemplateIntegrationTests, with errors like

25h[11:48:10 ERR] Fake.DotNet.Cli.IntegrationTests.TemplateTests.can install and run the template.fails to build a target that doesn't exist failed in 00:00:09.6230000. 
Expected the message 'Target Nonexistent is not defined' but got: No exact match of product releases 8.0.401 found.
No product release found for 8.0.401. Maybe a pre-release? Returning all the versions.
No exact match of product releases 8.0.404 found.
No product release found for 8.0.404. Maybe a pre-release? Returning all the versions.
There was a problem while setting up the environment:
-> Could not find referenced assemblies in path: '[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.36\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.35\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.33\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.32\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.31\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.30\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.29\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.28\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.27\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.26\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.25\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.24\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.23\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.22\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.21\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.20\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.19\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.18\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.16\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.15\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.14\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.13\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.12\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.11\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.10\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.9\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.8\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.7\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.6\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.5\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.4\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.3\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.2\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.1\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.0\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.0-rc.2.21480.5\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.0-rc.1.21451.13\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.0-preview.7.21377.19\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.0-preview.6.21352.12\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.0-preview.5.21301.5\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.0-preview.4.21253.7\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.0-preview.3.21201.4\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.0-preview.2.21154.6\ref\net6.0],[C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.0-preview.1.21102.12\ref\net6.0]', please check installed SDK and runtime versions

and I'm not sure if this is really a problem with the new stuff, or if it's because it's trying to use the 6.1.3 release to test the templates, and failing when running as .NET 8? (I suppose it shou;dn't be looking at the 6.0 assemblies when running as 8.0)

@Thorium
Copy link
Member

Thorium commented Jul 17, 2025

I expect that there is a chicken-and-egg problem, and the integration tests are actually restoring the .NET6 fake to try compiling .NET8, instead of referring to the just built .NET 8 version.

I tried this branch locally (Numpsy/new8-build-bump-roll-tests-2) on my machine, and my observations are here:

src\app\fake-cli\bin\Release\ has now net6.0 and net8.0 paths. Can a dotnet tool even have multiple target frameworks? Or should we just accept that FAKE 6.1.3 is working well for .NET6 projects and from now on everything else is .NET 8 only?

I run on my Windows machine dotnet build, and then from the actions:
dotnet tool install fake-cli --add-source "./src/app/fake-cli/bin/Debug" --add-source "./src/app/fake-cli/bin/Release" --version 1.0.0-1

It turned out that this didn't work properly. So I removed the Clean-task and called fake manually:

C:\git\FAKE>src\app\fake-cli\bin\Debug\net8.0\fake-cli -v build -e FAKE_DETAILED_ERRORS=true

And behold, all the integration tests went through, after painfully long half-an-hour later:

image

Where it did finally fail is:

dotnet.exe  deb --runtime linux-x64 --framework net8.0 --configuration Release --output "C:\git\FAKE\release\dotnetcore" --no-restore`

error NETSDK1045: The current .NET SDK does not support targeting .NET 8.0.  Either target .NET 6.0 or lower, or use a version of the .NET SDK that supports .NET 8.0. [C:\git\FAKE\src\app\fake-cl

So basically, dotnet deb tool has to be updated from 0.1.220 to version 0.1.232

"version": "0.1.220",

...and that's it, I think we could consider merging this and releasing a beta or new major version.

@xperiandri
Copy link
Collaborator

Can a dotnet tool even have multiple target frameworks?

@Thorium yes it can.
But there are some challenges with that.
You can see my changes in this PR to test that the older framework works fsprojects/FSharpLint#748

@Numpsy
Copy link
Contributor Author

Numpsy commented Jul 17, 2025

well for .NET6 projects and from now on everything else is .NET 8 only?

Whether there's any benefit to supporting .NET 6 in fake-cli any more is another question, and given the amount of pain in testing it all, just doing 8 seems simpler for a new major release?

.NET 10 is on the horizon, so making sure it works for that going forward seems more important than keeping 6 working in new versions. (My personal opinion anyway).

I'll see about rebasing it and fixing the conflicts anyway

@Thorium
Copy link
Member

Thorium commented Jul 17, 2025

Many projects use a separate non-FAKE fsproj as a build-project and reference the FAKE libraries, and there I see value in having .NET6 support kept.

But the fake-cli tool, I think, could target only the NET8 (or even 10 later).

@xperiandri
Copy link
Collaborator

xperiandri commented Jul 17, 2025

Then it will be impossible to run it unless you have the latest SDK installed

@Numpsy
Copy link
Contributor Author

Numpsy commented Jul 17, 2025

Then it will be impossible to run it unless you have the latest SDK installed

The 'DNX' stuff that MS are talking about with .NET 10 (https://github.com/dotnet/core/blob/main/release-notes/10.0/preview/preview6/sdk.md#platform-specific-net-tools) could maybe help if the tool could be self contained, but that's a work in progress so it remains to be seen

@Numpsy
Copy link
Contributor Author

Numpsy commented Jul 17, 2025

On the subject of targetting multiple .NET versions in fake-cli, re-reading the old comments reminded me of the situation with cachedDotnetSdkReleases.json - i.e., if the tool i smulti-targetted, you might need multiple versions of that file for different .NET versions?

Thorium and others added 11 commits July 17, 2025 17:57
Change global.json .NET 6 to 8
Search and replace targetframeworks from fsproj files to add net8.0
Add net8.0 to paket.dependencies
dotnet paket install to find .NET8 compatible dependencies
Expecto had to be hardcoded for now, because some tests are running on netstandard2.0 library (hopefully we can update this separately later)
MSBuild.StructuredLogger problem: DisableInternalBinLog = true had to be added to build.fsx NuGet commands (hopefully we can update this separately later)
A few places of code had new overrides so had to explicitly type to strings
SdkAssemblyResolver to default .NET 8 as well
Readme update
GitHub pipeline configs: Add .NET 8 install.
@Numpsy Numpsy force-pushed the new8-build-bump-roll-tests-2 branch from 0364d81 to 233081b Compare July 17, 2025 17:31
@Numpsy
Copy link
Contributor Author

Numpsy commented Jul 17, 2025

@Thorium I didn't see your other PR until after i'd rebased the changes to fix the merge conflicts, which caused conflicts in that, so I've cherry picked your dotnet-deb update straight into this PR

@Numpsy Numpsy marked this pull request as ready for review July 17, 2025 19:07
@Thorium
Copy link
Member

Thorium commented Jul 18, 2025

I think this file https://github.com/fsprojects/FAKE/blob/master/src/template/fake-template/Content/.config/dotnet-tools.json
says "use the nuget version of FAKE in the integration test templates", which causes any major changes to FAKE (like .NET update) to break the integration tests, because the NuGet version doesn't have these changes yet.

But I'm not sure if there is any workaround possible.

@Thorium
Copy link
Member

Thorium commented Jul 18, 2025

One more thing: Because the system uses FSharp.Compiler.Service v43.8, the fsdocs-tool has to be upgraded from 16 to 20, to make GenerateDocs task pass.

Edit: Updated my PR, but you can also just do dotnet tool update fsdocs-tool instead.

After this, my local build does pass:
image

Thorium and others added 2 commits July 18, 2025 11:31
…l has to be upgraded from 16 to 20, to make GenerateDocs task pass.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants