Monday, May 6, 2013

How to Automate Unit Testing with TeamCity (Part 3)

Please notice that this is a multi-part, step-by-step guide, on how to automate builds in TeamCity CI Server. Follow the links for Part 1 and Part 2 of this series.

Part 3: Configuring Automated Testing

Now that we have our MVC application and our corresponding test project building successfully, we can proceed to create our TEST step, where TeamCity will run our tests. Let's add now the Test step (here is the link to TeamCity MSTest short documentation).

First, access the build configuration for the correct project (either the new MVC 2 app or the updated MVC app we created at the beginning).

  1. Login into your TeamCity server administration portal.
  2. Click on the Projects link, located in the top navigation menu. This will display the list of projects and the build status.
  3. Click on the build configuration you are adding the new build step. This will take you to the build's overview page.
  4. Once in the overview page, look for the "Edit Configuration Settings" on the top right corner, below the search box. Click on it to go to the build configuration page.TC_build_web_84
  5. We are going to add a new build step, so click on the Build Step section on the right menu.

    TC_build_web_85
  6. You will see the existing build steps for this configuration, which should display the "BUILD" step we created before to compile the solution. Now click on the "Add build step" to display the build step configuration wizard, were we will add the Test step.

    TC_build_web_86
  7. Select MSTest from the Runner Type drop box.
  8. Enter the step name, in this case "Test".
  9. Then you need to specify the path to MSTest.exe. You can choose from the default system paths or specify the path yourself. Try with the defaults depending on the Visual Studio version you are using. If that doesn't work, then try specifying the path yourself. Again, if the MSTest.exe file is not available on your TeamCity server, you might need to copy it from a machine who has it.
  10. Enter the list of assemblies that contain your tests. You can either add the relative path here, or you can actually enter a parameter, that we can then add in the build parameters, as we did for the build. The DLL file should be "MvcApplication2.Tests.dll" and should be located in the Big/Release folder of the compiled project. For now let's enter the name of the variable we will use. I used "%env.TestAssemblies%".
  11. Leave the other spaces empty for now, and click the save button.

TC_build_web_87

You will be taken to the list of build steps, and now the new "Test" step should be there. Now, we need to enter the values of the new parameters we need for the Test step.

  1. Click on the Build Parameters section on the right panel.

    TC_build_web_88
  2. Notice that TeamCity already added the parameters we specified in the new build step, and now we are required to enter the values.

    TC_build_web_89
  3. Let's add first the path to the test binary file. Click on the edit link for the "env.TestAssemblies" parameter, and add the assembly relative path. In my case it is " web\MvcApplication2\MvcApplication2.Tests\bin\%env.Configuration%\MvcApplication2.Tests.dll " because of my solution folder structure. Notice that I'm using the %env.Configuration% which is actually another parameter that we can configure. For this build we already set that parameter to "Release".
  4. Click save to add the new value.

    TC_build_web_90
  5. Now do the same for the MSTest.exe path variable. Locate the app on the correct server path that matches the Visual Studio version. In my case, I'm using Visual Studio 2012. On my dev machine, the app is located in the following path: C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE
    On my current TeamCity version, the app is not available, so I need to copy it. Copy the files if needed.
  6. Now enter the correct server path to the MSBuild.test file and save the parameter value.

    TC_build_web_91
  7. Now we are ready to test our test step.

You can actually run the build configuration manually from TeamCity, which is handy when creating and testing the configurations. Notice the "Run" button at the top right corner, below the search box. Let's run the configuration now.

TC_build_web_105

Once you have you MSTest step setup and running, you can take advantage of TeamCity test views to get the most out of it.

The summary page will tell you if tests are executed or ignored:

TC_build_web_97

The overview summary will also quickly tell you if tests passed or not, and the status of the build.


TC_build_web_105

The output log of the build has detailed information for each step, including the run of the unit tests. If there are issues with the tests or the MSTest installation, you will see them here.

TC_build_web_106

You can also check how long is each test taking.

TC_build_web_108

On top of that, you can see detailed information for each particular test, which might help in more advanced scenarios.

TC_build_web_110

If you have any issues running the unit tests, chances are that some dependency or binary associated with MSTest is not configured correctly.

TC_build_web_79

Make sure to check my post on the subject. It will save you tons of time and wall-banging!

1 comment:

  1. Thank you so much for this post. It has been really helpful!

    ReplyDelete