Fitnesse Slim runner cannot load .net 5 assembly: Error extracting manifest import from file (hr = 0x80131018)

Running a test in Fitnesse gives:

Could not complete testing: fitnesse.slim.SlimError: Error SLiM server died before Header Message could be read.

When using Slim RunnerW.exe to debug my test I get an exception:

System.BadImageFormatException: Could not load file or assembly 'file:///c:\path\assemby.exe' or one of its dependencies. The module was expected to contain an assembly manifest.

I used ProcessExplorer to check and RunnerW.exe was running in 64bit mode. My code is compiled with "Any CPU", the only difference with another working project is that it is a .net 5 (core) project using FitSharp 2.8.2.1 NuGet package.

After enabling the FusionLog it was clear that it could not load my main test assembly. This is part of the log:

LOG: This bind starts in LoadFrom load context.
WRN: Native image will not be probed in LoadFrom context. Native image will only be probed in default load context, like with Assembly.Load().
LOG: No application configuration file found.
LOG: Using host configuration file: 
LOG: Using machine configuration file from C:\Windows\Microsoft.NET\Framework64\v4.0.30319\config\machine.config.
LOG: Attempting download of new URL file:///C:/path/assembly.exe.
LOG: Assembly download was successful. Attempting setup of file: C:\path\assembly.exe
LOG: Entering run-from-source setup phase.
ERR: Error extracting manifest import from file (hr = 0x80131018).
ERR: Run-from-source setup phase failed with hr = 0x80131018.
ERR: Failed to complete setup of assembly (hr = 0x80131018). Probing terminated.

I think it is trying to load a .Net 5 assembly into a .Net 4 process?

Any ideas how to resolve this? I thought of adding an app.config file and specifying a runtime, but I'm not sure, it may just be a rabbit hole since it seems we need to be mixing frameworks here?