UPDATE: Previous version of this had an error in the OpenCover code where failed tests did not result in a failed AppVeyor build.
AppVeyor is a really cool CI (continuous integration) service that I am currently playing with, so far its been very easy and powerful for both continuous build/tests and deployment (plus they do outstanding support). However, it is missing a couple of features that I would like, one of them being showing code coverage results with each build.
So, until the wonderful people at AppVeyor implement a way to do this natively, here is a bit of a hack to see some nicely formatted coverage results when running automated builds on AppVeyor.
Or at least these are the steps I will be using, let me know if you are doing anything similar or if you have improvements on these!
(The examples below are using the SampleSln files that are included in the OpenCover nugget package).
Before starting lets assume that you have a working build and tests running on AppVeyor.
Also, make sure that you have all the required nuget packages:
Unless you include the nuget packages in your repo, you may need to tell AppVeyor to install these nugget packages by adding a “Before Build Script” (via AppVeyor settings):
nuget restore SampleSln\BomSample.sln
First, create a bat script to run your unit tests (using the native nUnit console runner on AppVeyor), I called mine “run tests-AppVeyor.bat” and saved it in the project root. Using this unit runner (rather than one in our repo) means that tests appear in the AppVeyor tabs as expected (which means we do not have to use Rest end point to send the test results to AppVeyor’s “Tests” tab)
nunit-console-x86 /noshadow .\BomTest\bin\Debug\BomTest.dll
You can, of course, use any of the standard command arguments to nUnit.
Then push this up to your GIT repo, so that AppVeyor grabs a copy.
Now, in the AppVeyor settings change the “Test” option to run a “script” and put the OpenCover (& ReportGenerator) scripts in here. EG:
cd SampleSln packages\OpenCover.4.5.3522\OpenCover.Console.exe -register:user -returntargetcode "-filter:+[Bom]* -[*Test]*" "-target:runtests_appveyor.bat" packages\ReportGenerator.1.9.1.0\ReportGenerator.exe "-reports:results.xml" "-targetdir:.\coverage"
NB: I cd to the project directory just to make paths easier and you will obviously need to tweak the OpenCover filter params for your project.
Now when AppVeyor builds your project and runs the tests you will have a folder with some nicely formatted coverage results AND the test results in your AppVeyor tabs as normal.
To view these lovely OpenCover/ReportGenerator results we need to make them available on a web page somewhere (they are just HTML/CSS & a little JS). I managed this by making the coverage folder available as an AppVeyor artefact. You can do this in the project Settings in AppVeyor.
I then use an inline project deployment to FTP just these files to a simple (free) AWS site where the team can view them. By naming the artefact you can have a deployment for just the coverage files and keep that separate from any other AppVeyor deployment(s).
Currently, I’m putting a copy of the results for each build into a different remote folder:
/site/wwwroot/SampleSln/$(appveyor_build_version)/
(That probably is not sustainable, but it will do for now).
And, finally, I updated our Slack notifications to include a link to the coverage results, making it easy for everyone to view them. This is done with a custom “Message Template” that links to the relevant htm for this build.
<{{buildUrl}}|Build {{projectName}} {{buildVersion}} {{status}}>
Commit <{{commitUrl}}|{{commitId}}> by {{commitAuthor}} on {{commitDate}}:
_{{commitMessage}}_
<http://mysite.azurewebsites.net/SampleSln/{{buildVersion}}/SampleSln/coverage/index.htm|Coverage>
I am all ears if you have a different or better way of implementing simple code coverage on AppVeyor.
References:
– http://www.codeproject.com/Articles/677691/Getting-code-coverage-from-your-NET-testing-using
– http://www.appveyor.com/docs/running-tests
– http://www.appveyor.com/docs/deployment/ftp
– http://www.palmmedia.de/OpenSource/ReportGenerator
– http://www.appveyor.com/docs/notifications#slack