VS2010: Testing tools
By Tim Anderson
Visual Studio 2010 includes much to facilitate code testing.
HardCopy Issue: 52 | Found In: Visual Studio 2010 | Published: 01/05/2011 | Last Revision: 29/06/2011
Software testing comes in many forms. Unit tests are dedicated test code written so as to verify that you are getting the correct results from small units of application code. In test-driven development, unit tests have a primary role. Coverage is a measurement of how much of an application is actually being covered by unit tests. User Interface (UI) tests exercise the finished application by simulating user input.
Load tests stress an application, often by simulating multiple users and large databases. A related idea is performance testing which verifies acceptable performance of a system under load.
Continuous Integration is the practice of testing the quality of a complete build frequently, so that if developers write code that breaks some part of the system, it is discovered early. Continuous Integration for large applications is a complex process as it requires the deployment of a complete build, possibly to a variety of different systems, before it can be tested.
The test tools you get with Visual Studio vary according to the edition. Unit testing is in all editions from Professional upward. The Premium Edition adds code coverage, test impact analysis and coded UI tests. Ultimate and the dedicated Test Professional add performance and load testing, Test Manager for planning and executing tests and analysing the results, and Lab Manager for using virtual test environments spanning multiple machines.
Unit testing is deeply integrated with Visual Studio and Team Foundation Server (TFS). You can link tests to work items such as bugs or tasks, analyse test results, and specify tests that must succeed before check-in of new code is allowed.
Testing in Visual Studio 2010 is enhanced in Feature Pack 2, free to Visual Studio with MSDN subscribers. After installing the Feature Pack, you can edit coded user interface tests in a new visual editor. Coded UI tests are also extended to cover Silverlight 4 applications. Finally, you can play back Web tests using Mozilla Firefox as well as Internet Explorer.
Advanced load testing for Web applications is built into Visual Studio 2010. You can create load test scenarios, specifying the number of users, how the load varies, which tests to run for each virtual user, the connection speed so you can have some users on slow and some on fast connections, the mix of browsers, and for how long to run the tests. Load testing is accomplished with test agents running on one or more machines, managed by Visual Studio Test Controller.
Visual Studio was limited to 250 virtual users for a load or performance test, but Microsoft lifted this limit in April 2011 for users of Visual Studio Ultimate with MSDN. You can now test with as many virtual users as you like.
Lab Management
Lab Management automates what developers frequently do anyway – namely testing on virtual machines (VMs) – although there is also an option to use physical machines with controller agents installed.
In order to set this up, you need one or more Windows servers running Hyper-V, Microsoft’s VM host, together with System Center Virtual Machine Manager (SCVMM) to manage your VMs.
The next step is to integrate Team Foundation Server with SCVMM, and configure Lab Management for a TFS project. This enables workflow integration, where you can automatically reset the virtual lab to a clean state, install your application and run execution tests.
Microsoft Lab Center, part of Test Manager, is where you import virtual machine templates and create environments formed of one or more VMs. You can build up a library of OS images configured to match typical production machines. You can then deploy and start an environment, ready for use. At any time you can snapshot the environment.
How Lab Management integrates with the rest of Visual Studio.
The Testing Center lets you define and run tests against a virtual environment and collect the resulting logs. A great feature is the ability to link bug reports to environment snapshots. You or other team members can open the bug report, click Connect to Snapshot, and observe the bug in a live running system.
You can also link builds to test plans. When creating a definition for an automated build, a process wizard lets you select a Lab environment, scripts to run on deployment, and a suite of tests to run.
Another interesting feature is environment cloning which allows several people to work simultaneously on different copies of the same environment snapshot. One scenario is that developers can work on fixing a bug while testers concentrate on further tests.
Although some effort is involved in setting up Lab Management, it is a huge feature and worlds away from the days of continually reinstalling Windows in order to test a setup problem or replicate different application combinations.