My last post documented my first venture into working with Visual Studio's Code Analysis output with PowerShell to find classes that need more testing. Since then I've taken the idea further to analyse how coverage has changed over a series of builds from TFS.

What resulted is a PowerShell script that takes, as a minimum, the name of a TFS Project and the name of a Build Definition within that project. The script will then, by default, locate the two most recent successful builds, grab the code coverage data from the test runs of each of those builds and output a list of classes whose coverage has changed between those builds, citing the change in the number of blocks not covered.

Additional parameters to the script allow partially successful or failed builds to be considered and also to analyse coverage change over a span of several builds rather just two consecutive builds.

The primary motivator behind developing this script was to be able to identify more accurately where coverage was lost when a new build has an overall coverage percentage lower than the last. This then helps to locate, among other things, where new code has been added without testing or where existing tests have been deleted or disabled.

A code base with a strong commitment to the Single Responsibility Principle should find class-level granularity sufficient but extending the script to support method-level reporting should be trivial given that the Coverage Analysis DataSet already includes all the required information.

The script requires the Team Foundation Server PowerShell Snapin from the TFS October 2008 Power Tools and the Visual Studio Coverage Analysis assemblies presumably available in Visual Studio 2008 Professional or higher. These dependencies only support 32-bit PowerShell so my script unfortunately suffers the same constraint.

Download the script here, and use it something like this:

PS C:\TfsWorkspace\> C:\Scripts\Compare-TfsBuildCoverage.ps1 -project Foo -build FooBuild | sort DeltaBlocksNotCovered

Visual Studio 2008 Team System's built-in Code Coverage is nice but the standard results window only allows you to drill down through each assembly, then namespace, class, and finally method. You can't easily find the class with the least blocks covered, something I needed to do the other day.

I found John Cunningham's blog about "off-road" code coverage and was pleased to see that Microsoft had provided an assembly in Visual Studio that can be used to parse the *.coverage file output by a test run. I followed his example to write a PowerShell script to provide basic access to the data.

You can download my script here.

Then you can use it like this:

$CoverageDS = ./Get-CodeCoverageDataSet.ps1 "data.coverage"
$CoverageDS.Class `
  | Sort-Object -Property BlocksNotCovered -Descending `
  | Select-Object `
    -First 25 `
    -Property `
      BlocksNotCovered, `
      @{ 
        Name = "Namespace"; 
        Expression = { 
          $CoverageDS.NamespaceTable.FindByNamespaceKeyName($_.NamespaceKeyName).NamespaceName
        }
      }, `
      ClassName

The coverage file is typically found in the TestResults\[TestRunName]\In\[ComputerName]\ folder. You can easily perform queries over methods or lines rather than classes by using the other tables in the returned dataset. You can also use the ConvertTo-Html cmdlet to easily create a report for your team.

In a sad attempt to get blogging again, I am going to start by responding to this blatant prod. I agree with some of Jim's answers but to keep it interesting I've selected a disjoint set and knocked it up a notch.

I am:

Pragmatic about dogmatism.

Practicing consumerism.

Prone to defeatism.

Parodying athleticism.

Proof of determinism.

To keep it going, the same rule applies: if you've read this then follow suit blogging your own answers, even if you are busy writing your own blog engine.

As I have recently accepted a new job interstate, I will no longer be organising dinners for fellow developers in Adelaide. However, rather than allow such a fun event to disappear, I am passing the awesome responsibility of organising the dinners to my friend and colleague, Jim Burger, whom many of you have met on previous occasions.

I will be giving Jim a hard time to make sure he continues to arrange a dinner at least once each quarter and I believe it will also help to ensure he remembers to attend them himself ;). Names and contact details of previous attendees and people who have registered their interest will be given to Jim so he can send you the next invite but let him or I know if you'd rather be removed from the list.

If you've never been invited to an Adelaide Geek Dinner before but want to be on the list, just contact Jim or I and you'll surely be invited to the next one.

I was recently inspired by a question about property names in strings on Stack Overflow. The problem is that Windows Forms data binding takes the names of properties to bind as string parameters and if these properties change or are removed, errors won't surface until the bound control is exercised at run-time. So, using Paul's strongly typed property name ideas, I propose this alternative syntax for programmatic data binding in Windows Forms:

nameTextBox.Bind(t => t.Text, aBindingSource, (Customer c) => c.FirstName);
// Binds the Text property on nameTextBox to the FirstName property
// of the current Customer in aBindingSource, no string literals required.

And the following code to implement support for it:

public static class ControlExtensions
{
    public static Binding Bind<TControl, TDataSourceItem>(this TControl control, Expression<Func<TControl, object>> controlProperty, object dataSource, Expression<Func<TDataSourceItem, object>> dataSourceProperty) where TControl: Control
    {
        return control.DataBindings.Add(PropertyName.For(controlProperty), dataSource, PropertyName.For(dataSourceProperty));
    }
}

public static class PropertyName
{
    public static string For<T>(Expression<Func<T, object>> property)
    {
        var member = property.Body as MemberExpression;
        if (null == member)
        {
            var unary = property.Body as UnaryExpression;
            if (null != unary) member = unary.Operand as MemberExpression;
        }
        return null != member ? member.Member.Name : string.Empty;
    }
}

Thoughts? Overkill?

A note to myself, and to others who may find it useful:

Use System.Collections.ObjectModel.Collection<T> (or a subclass) as the return type for properties exposing collections of related items on your presentation model classes if you intend to bind to the relationships with Windows Forms BindingSources.

Collection<T> appears to be the lowest-level type in Framework 3.5 that provides both the non-generic IList implementation as required by the binding system and also provides strongly-typed programmatic access to the elements of the collection. It is used as the base for BindingList<T>, ObservableCollection<T> and many others and you can easily inherit from it yourself.

Using a return type of IList<T> or ICollection<T> instead, which may be preferred for being less concrete, is not sufficient as neither of these inherit from the non-generic IList and will fail to bind correctly (sometimes throwing exceptions) at run-time when the parent list of the bound relationship is empty.

As promised in May, it is time for the August Geek Dinner, specifically the time will be 6:30pm on Saturday 16th August. Everyone seemed to enjoy the food and atmosphere of Caffe Amore on the corner of Pulteney and Pirie Streets last time so we'll be going there again this time.

If you're already on the geek dinner mailing list from previous times expect an invitation shortly. If you've never been invited before or you think your invite may be lost in the mail, just contact me and I'll make sure you get included.

Please RSVP before Wednesday 13th August so I can finalise the booking with the restaurant. Hope to see you all there!

The original TFS Deployer attempted to access various restricted resources than effectively required it to run as an administrator user. Since some changes were made in February, Deployer can now be executed under a standard user account with some additional considerations which follow.

Team Foundation Server Permissions

The TFS Deployer service account must be a member of the Team Foundation Valid Users server-level group on the server specified by the TeamFoundationServerUrl config setting and also a member of the Readers project-level group for the Team Projects that Deployer will be monitoring.

If you intend to make use of the Retain Build feature, the Deployer service account will also need the Write To Operational Build Store project-level permission on applicable Team Projects.

PowerShell Execution Policy

TFS Deployer can execute Windows Command Scripts if you really need it to but PowerShell scripts are the preferred option. However, PowerShell is built to be secure-by-default and installs with script-running capabilities disabled. It is left for the user to enable scripts after PowerShell has been installed and can be performed by opening the PowerShell console as an administrator and executing either:

Set-ExecutionPolicy AllSigned ; # or
Set-ExecutionPolicy RemoteSigned

The former choice will require you to sign all your deployment scripts in source control with a certificate trusted by the Deployer. Scott Hanselman has a comprehensive post on his blog describing the signing process. The latter choice will not impose this requirement because files retrieved from source control by Deployer are not marked with a zone identifier.

HTTP Namespace Reservation

TFS Deployer utilises self-hosted WCF over HTTP to subscribe to Build Quality Change events from TFS. This technique relies on the Windows HTTP Server API which requires the Deployer service account to have been granted rights to listen on the portion of the HTTP URL namespace it uses for event notification. By default only local administrators have access to the entire namespace and permissions must be granted using the httpcfg tool (on Windows XP and Server 2003) or the netsh tool (on Vista and Server 2008).

Using the netsh tool, from an administrator command prompt, the following command (as a single line) will grant rights to the appropriate URL for TFS Deployer (assuming you are using the default port number):

netsh http add urlacl url=http://+:8881/BuildStatusChangeEvent user=DOMAIN\TfsDeployerUser

Using the httpcfg tool is more difficult as it requires an ACL string in Security Descriptor Definition Language (SDDL). However, Dominick Baier has created a tool to simplify this process and I can recommend it.

Additional Permissions

All deployment scripts executed by TFS Deployer will be executed under the context of the service account unless the scripts explicitly contain alternate credentials. If the scripts copy files to a network share, configure a web site in IIS, or even upgrade the schema of a SQL database, ensure that the service account also has the minimum privileges to perform those actions too.

Any problems, questions, or suggestions... let me know.