How to run unit tests on VSTS from a different Outdir than default?

-1

The problem:

My tests from Project.Tests.dll are not running anymore.

Story:

My 79 unit tests were correctly running on Azure Pipeline before the change I made. I had to modify OutDir parameter which points to the different folder now: /p:Outdir="$(build.artifactstagingdirectory)\\published"

After this change no unit test were running. I guessed that it was because regex from the TestAssemblies step wasn't modified. So I modified it properly. However now only 39 of my tests is executed and for some assemblies I have an error:

NUnit failed to load d:\a\1\a\published\Project.Tests.dll

Here is what I had before in the pipeline:

Full build arguments are following following:

/p:DeployOnBuild=true /p:WebPublishMethod=Package /p:PackageLocation="$(build.artifactstagingdirectory)" /verbosity:detailed

And test regex:

**\$(BuildConfiguration)\*test*.dll
!**\obj\**

Here is what I have currently in the pipeline:

Full build arguments are following following:

/p:DeployOnBuild=false /p:Outdir="$(build.artifactstagingdirectory)\\published" /p:SkipInvalidConfigurations=true /p:PackageLocation="$(build.artifactstagingdirectory)\\"

And test regex:

$(build.artifactstagingdirectory)\published\*test*.dll
!**\obj\**

Output from the build:

...
vstest.console.exe 
"d:\a\1\a\published\Project.Tests.dll"
"d:\a\1\a\published\Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.dll"
"d:\a\1\a\published\Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices.Interface.dll"
"d:\a\1\a\published\Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices.dll"
"d:\a\1\a\published\Microsoft.VisualStudio.TestPlatform.TestFramework.Extensions.dll"
"d:\a\1\a\published\Microsoft.VisualStudio.TestPlatform.TestFramework.dll"
"d:\a\1\a\published\NUnit3.TestAdapter.dll"
/logger:"trx"
/TestAdapterPath:"d:\a\1\s"
...
Passed   ValidateUrl_TestToFail("")
Running all tests in d:\a\1\a\published\Project.Tests.dll
   NUnit failed to load d:\a\1\a\published\Project.Tests.dll

[Edited] Full error after enabling logs:

Exception NUnit.Engine.NUnitEngineException, Exception thrown executing tests
An exception occurred in the driver while loading tests.
   at NUnit.Engine.Runners.DirectTestRunner.LoadDriver(IFrameworkDriver driver, String testFile, TestPackage subPackage)
   at NUnit.Engine.Runners.DirectTestRunner.LoadPackage()
   at NUnit.Engine.Runners.TestDomainRunner.LoadPackage()
   at NUnit.Engine.Runners.DirectTestRunner.EnsurePackageIsLoaded()
   at NUnit.Engine.Runners.DirectTestRunner.Explore(TestFilter filter)
   at NUnit.Engine.Runners.MasterTestRunner.Explore(TestFilter filter)
   at NUnit.VisualStudio.TestAdapter.NUnit3TestExecutor.RunAssembly(String assemblyPath, TestFilter filter) in D:\repos\NUnit\nunit3-vs-adapter\src\NUnitTestAdapter\NUnit3TestExecutor.cs:line 310
   at NUnit.VisualStudio.TestAdapter.NUnit3TestExecutor.RunTests(IEnumerable`1 sources, IRunContext runContext, IFrameworkHandle frameworkHandle) in D:\repos\NUnit\nunit3-vs-adapter\src\NUnitTestAdapter\NUnit3TestExecutor.cs:line 103
Innerexception: System.IO.FileLoadException: Could not load file or assembly 'nunit.framework, Version=3.5.0.0, Culture=neutral, PublicKeyToken=2638cd05610744eb' or one of its dependencies. The located assembly's manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040)
File name: 'nunit.framework, Version=3.5.0.0, Culture=neutral, PublicKeyToken=2638cd05610744eb'
   at System.Reflection.RuntimeAssembly._nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, RuntimeAssembly locationHint, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
   at System.Reflection.RuntimeAssembly.nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, RuntimeAssembly locationHint, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
   at System.Reflection.RuntimeAssembly.InternalLoadAssemblyName(AssemblyName assemblyRef, Evidence assemblySecurity, RuntimeAssembly reqAssembly, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
   at System.Activator.CreateInstance(String assemblyString, String typeName, Boolean ignoreCase, BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes, Evidence securityInfo, StackCrawlMark& stackMark)
   at System.Activator.CreateInstance(String assemblyName, String typeName, Boolean ignoreCase, BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes, Evidence securityInfo)
   at System.AppDomain.CreateInstance(String assemblyName, String typeName, Boolean ignoreCase, BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes, Evidence securityAttributes)
   at System.AppDomain.CreateInstanceAndUnwrap(String assemblyName, String typeName, Boolean ignoreCase, BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes, Evidence securityAttributes)
   at System.AppDomain.CreateInstanceAndUnwrap(String assemblyName, String typeName, Boolean ignoreCase, BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes, Evidence securityAttributes)
   at NUnit.Engine.Drivers.NUnit3FrameworkDriver.CreateObject(String typeName, Object[] args)
   at NUnit.Engine.Drivers.NUnit3FrameworkDriver.Load(String testAssemblyPath, IDictionary`2 settings)
   at NUnit.Engine.Runners.DirectTestRunner.LoadDriver(IFrameworkDriver driver, String testFile, TestPackage subPackage)

Output from the build (BEFORE):

vstest.console.exe 
"d:\a\1\s\MVT.Core.Tests.EFExample\bin\Release\NUnit3.TestAdapter.dll"
"d:\a\1\s\Project.Tests\bin\Release\Project.Tests.dll"
"d:\a\1\s\Project.Tests\bin\Release\NUnit3.TestAdapter.dll"
"d:\a\1\s\Services\Project2.Tests\bin\Release\Project2.Tests.dll"
"d:\a\1\s\Services\Project2.Tests\bin\Release\Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.dll"
"d:\a\1\s\Services\Project2.Tests\bin\Release\Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices.Interface.dll"
"d:\a\1\s\Services\Project2.Tests\bin\Release\Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices.dll"
"d:\a\1\s\Services\Project2.Tests\bin\Release\Microsoft.VisualStudio.TestPlatform.TestFramework.Extensions.dll"
"d:\a\1\s\Services\Project2.Tests\bin\Release\Microsoft.VisualStudio.TestPlatform.TestFramework.dll"
"d:\a\1\s\Services\Project2.Tests\bin\Release\NUnit3.TestAdapter.dll"
"d:\a\1\s\Services\Project3.Tests\bin\Release\Project3.Tests.dll"
/logger:"trx"
/TestAdapterPath:"d:\a\1\s"
...
Running all tests in d:\a\1\s\Project.Tests\bin\Release\Project.Tests.dll
NUnit3TestExecutor converted 28 of 28 NUnit test cases
...
unit-testing
azure-pipelines
asked on Stack Overflow Sep 27, 2019 by codeedward • edited Oct 1, 2019 by codeedward

0 Answers

Nobody has answered this question yet.


User contributions licensed under CC BY-SA 3.0