Monday, December 10, 2007

AusGPS.com - Australia's new online navigation community

Back in July, after watching my good friend Lewis enjoy the benefits of in-car satellite navigation for a some time, I decided to buy a TomTom One XL for myself.

Since then, Lewis and I have had our fair share of both good and bad experiences with our chosen GPS devices and their respective feature support in Australia. While the devices sold locally are much the same, if not identical, to those sold worldwide, the quality of the maps and the availability of real-time data suffers in our sparsely populated nation.

In order to bring together the GPS users of Australia to share our experiences, our combined knowledge, and a form a central voice to the organisations responsible for improving our overall sat-nav situation, Lewis has started a new website just for us.

AusGPS.com is live now and currently consists of a forum and a review of the popular current-model TomTom ONE v3. As the community grows the website will offer regular GPS-relevant news updates, more reviews on other devices and services available to Australians, and even a central download location for popular GPS accessories.

If you currently own a navigation device, or are thinking of purchasing one, check out Lewis' new site, register on the forum and say hello.

 Thursday, November 29, 2007

Upgrading to Team Foundation Server 2008 Workgroup Edition

This week I took upon myself the challenge to upgrade our TFS 2005 SP1 single-server installation to the newly released 2008 version. When I was first introduced to our Team Foundation server it was running Beta 3 of TFS 2005. Since then it has mysteriously become my duty to upgrade to 2005 RTM, 2005 SP1, and now 2008 RTM.

I have gathered much needed experience on my previous upgrades to know how hard it would probably be. Beta 3 was originally running on SQL Server 2005 Developer Edition and the TFS beta wasn't the Workgroup Edition. Needing to upgrade the beta database structures, switch the edition of SQL Server, and even migrate the whole lot to another physical machine has given me the confidence to deal with any TFS install.

I prepared for the 2008 upgrade by backing everything up, running the Best Practices Analyzer, then following the pre-upgrade checklist to the letter. So far everything is great. I inserted the TFS 2008 media, and began the setup. The setup prescan said everything was ok. It detected the existing installation, asked me for a few more settings (account passwords and mail server) then performed the upgrade.

Smooth. No errors. Database schema upgrade succeeded. And it was quick too. Again, I followed all the post-upgrade checklist items one-by-one and I was almost giddy at how easy it all was. I went back to my developer workstation and played around with work items and checked out and in some files. Perfect! I was so happy I went and upgraded our build servers too.

The next day my work colleague tried to undo a pending change and received a nasty error. The undo was failing and crashing the version control component on the server. I chased it down to the prc_iiUndoPendingChanges stored procedure experiencing a collation mismatch. The proc is encrypted but with SQL Profiler I was able to verify that it was a clashing collation between the TfsVersionControl database and SQL's tempdb database.

How do you change the collation on the tempdb database to match the others? You don't... you change the collation on the master database. How do you change the collation on the master database? You drop all your databases, rebuild the master using setup.exe on the SQL install media, then put all your databases back.

Still not that easy though. TFS 2008 requires SQL Server SP1. If you rebuild the master database with the RTM media you get an RTM master database. You can't get or make SP1 slipstreamed install media and you can't reapply SP1 after rebuilding master because all the other parts of SQL are already patched.

So, in the end you backup all your databases, uninstall all of SQL Server, reinstall it with the right collation, reapply the service pack, restore all your databases, rebuild the data warehouse and generally do all the rest of the necessary steps.

It was almost the perfect upgrade. Unfortunately collation settings were missed by the TFS team on one of their temp table definitions, it wasn't picked up by testing, dogfooding, or the beta program, and I get all this hassle.

Still, all the new features in TFS 2008 make it worth it and I'd do it again. And will probably have to.

 Sunday, November 18, 2007

Organising the dinner with Preparty

In October I started thinking about organising a dinner to bring Adelaide's .NET developers together. I'd never done something like that before and I wasn't sure about the best way to manage it.

Luckily, I remembered meeting Jonathon Kresner at the Wagga Wagga Code Camp in March. Jon had told me he was developing a website in .NET to organise parties. I decided to send him an email and ask how it was going.

Coincidentally, Jon had reached the final beta testing stages of his website, Preparty.com, and when I explained that I was interested in organising a dinner for .NET developers, he was kind enough to grant me a login if I don't mind assisting with the testing.

Preparty.com basically provides you with everything you need to organise a party of any kind (dinner, music gig, or wedding reception) from beginning to end with a smooth user experience.

You start by finding an appropriate venue by choosing from many popular places already registered on Preparty, or by entering details of your own. You then enter the names and email addresses of all your guests and complete the details of your event such as date and time, theme, and other relevant information.

Preparty then handles the creation and sending of e-Invitations with all the details and even a map of the venue location and follows up with a place to track RSVPs, You can even organise entertainment via Preparty and send SMS reminders to your guests as the event draws near. And when it's all over you can invite guests to upload photos from the event to create a slideshow.

I exercised almost all the features of the Preparty website and only found a couple of usability issues that Jon corrected very quickly. For a website supposedly in beta testing, it was very stable.

Preparty is free to register but charges a very affordable rate to purchase email and sms credits. I know now that the website is out of beta and has been released to the public I'll be looking to Preparty.com to organise all my future events and I recommend you take a look too.

Inaugural Adelaide Geek Dinner Success

Tonight saw the first (to my knowledge) of hopefully many more evenings with some of Adelaide's top .NET software developers.

While others were invited but were unfortunately unable to attend, a very friendly and interesting eight people ultimately arrived. In no particular order:

The evening started at 6:30pm (for most of us ;) and eventually came to a close at around 10:30pm after everyone was quite full from a delicious dinner and meaty conversation. I've uploaded a couple of photos to my Flickr account here.

I was originally uncertain about trying to organise such an event especially as I had only briefly met some of the guys before and some not at all. I am very glad I did though because I learned a lot, had some good laughs, and made some new friends too.

The response was so good that I plan to arrange another dinner, with a few more people who couldn't make it this time (eg Candy ;). An evening in January 2008 is my goal but this time I'll invite suggestions for venue, date, and anything else that you think might be a good idea.

Watch this blog for more information on the next dinner as it becomes available and if you'd like to be invited, just send me an email or leave a comment here on the blog.

 Wednesday, October 17, 2007

Adelaide Geek Dinner Approaches

I announced earlier that I was planning a dinner get-together for developers in Adelaide. I have since finalised the time and venue and have established an initial guest list.

The dinner will be held on Saturday November 17th starting 6:30pm at Cafe Buongiorno in the city. The cafe is at 187 Rundle Street, near the corner of Pulteney Street and the entrance to U-Park (for convenient parking). They serve familiar Italian meals - pizza and pasta starting at around $15.

I have found and contacted several local developer-bloggers and asked them to attend, some have committed, some have tentatively accepted but the response has been positive. If you are, or know, a local developer-blogger who hasn't been personally invited, send me an email and I'll add you to the guest list and ensure I book a large enough table at the cafe.

I look forward to seeing you there.

 Tuesday, October 16, 2007

PowerShell Resources

Ever since I heard of the concept of PowerShell (or Monad as it was known then) I was excited. Now that is been RTM for some time and I have had an opportunity to work with it in a production environment I love it even more.

While PowerShell could be summarised as a cross between a *nix shell and the .NET Framework there is still a lot unique to PowerShell alone and learning how it works and finding efficient tools to work with it is still necessary to make the most of PowerShell.

To learn PowerShell I purchased Bruce Payette's book Windows PowerShell In Action. It is a good-size detailed book at over 500 pages and receiving both the soft-cover book and the searchable PDF was excellent value. Pretty much every aspect of PowerShell is described including why certain design decisions were made. My only issue with the book, and it's not a big issue, is that I was itching to write some PowerShell scripts but script files aren't explained until Chapter 8 and security for script files isn't fully explained until Chapter 13 (the last chapter).

When I started to write my scripts I had the PowerShell console open on one monitor and Notepad open on the other. I would try certain commands in the console window and when they worked and gave the results I wanted I would copy them to the script in Notepad, save, and switch back to the console window to test the script. I still pretty much work like this today but I've replaced Notepad.

Considering I spend most of my time in Visual Studio I really wanted the same Intellisense and Syntax Highlighting experience when writing PowerShell scripts. The first PowerShell "IDE" I encountered was PowerShell Analyzer but I felt overwhelmed by the UI given that all I wanted was to edit .ps1 files. It seems very capable but just didn't feel right. More recently I have tried PowerGUI and it is very close to "Notepad for PowerShell". I have used it to develop the scripts for the last two PowerShell posts and I recommend it.

Do you have any resources you feel have been invaluable for getting the most out of PowerShell?

 Sunday, October 14, 2007

Report Services Automation With PowerShell

In late September Paul Stovell wrote about a set of VB.NET scripts he prepared to help deploy reports to SQL Server Reporting Services. If you've ever had the displeasure of deploying SSRS reports without Visual Studio then you'll understand how much it sucks.

Paul went to the effort to write individual scripts for creating folders and data sources on the server and uploading report definitions and configuring permissions. With Paul's work simple command scripts can then be used deploy reports.

However these command scripts still need to be written and they end up containing much of the same information as can be found in the .rptproj project file and the .rds data source files. I despise the idea of maintaining any sort of configuration information in more than one place so adding to the deploy command script whenever I add a report to the project in Visual Studio just makes me cringe.

Additionally, as Paul briefly mentions, MSBuild (and therefore Team Build) does not support Report Services projects so, once again, to deploy your reports as part of Continuous Integration you need to have separate tools.

Today I constructed a lengthy PowerShell script to take a Report Services .rptproj project file and output a command script that utilises Paul's VB.NET scripts to deploy the reports as per the project settings. Due to the size of the script rather than publishing it inline, you can download it here.

The script accepts three parameters. ProjectFile is the path to .rptproj file for the reports you want to deploy. If you omit this parameter the script uses the first report project file it finds in the current directory. The second parameter, ConfigurationName tells the script which project configuration to use for the target server URL and destination folders. If you omit this parameter the script uses the first configuration defined in the project. The last parameter SearchPaths is a list of paths for the script to search when locating both rs.exe and Paul's .rss files. The SearchPaths parameter is automatically combined with the environment PATH variable and may be omitted.

Here is an example usage:

PS C:\Users\Jason\Dev\MyReports> .\Deploy-SqlReports.ps1 `
    -ProjectName MyReports.rptproj `
    -ConfigurationName Release `
    -SearchPaths "C:\Tools\Report Services\" `
    | Out-File deploy.cmd -Encoding ASCII;

As always, my PowerShell skills are slowly improving and this script is not necessarily perfect in either robustness or efficient use of PowerShell. Hopefully it will be as useful to you as it has been to me and any changes you need should be easily made. Please leave a comment with your thoughts and suggestions.

 Saturday, October 13, 2007

Find Duplicate Files With PowerShell

I have pieced together a simple PowerShell script to recursively locate all duplicate files (by content, not name) below a chosen directory. It is not the most elegant code but for my purposes it works and hopefully you will be able to tweak it to suit your needs.

Firstly, it filters out any zero-length files. Zero-length files are naturally duplicates of each other and can be found quite trivially without my script. Secondly it groups all files by their length because if the length doesn't match, they can't have the same content. The script then excludes the length-groups with only one entry and calculates the MD5 hash of the remaining files. Groups of files with both matching size and hash are then returned in the results.

The hashing function was taken from the Duplicate Files post on the Windows PowerShell team blog. It simply uses the .NET cryptography namespace to compute the hash. From here you could easily exchange the MD5 algorithm for SHA1 or any other preferred algorithm.

Due to the need to read the entire contents of potentially matching files to compute the hash this can cause the script to take a long time against larger files. Executing the script against deep directory structures with many files will take longer too. The script could be easily modified to take a filtered input of files to only find, for example, duplicate photos.

Here is the script:

param ([string] $Path = (Get-Location))

function Get-MD5([System.IO.FileInfo] $file = $(throw 'Usage: Get-MD5 [System.IO.FileInfo]'))
{
    # This Get-MD5 function sourced from:
    # http://blogs.msdn.com/powershell/archive/2006/04/25/583225.aspx
    $stream = $null;
    $cryptoServiceProvider = [System.Security.Cryptography.MD5CryptoServiceProvider];
    $hashAlgorithm = new-object $cryptoServiceProvider
    $stream = $file.OpenRead();
    $hashByteArray = $hashAlgorithm.ComputeHash($stream);
    $stream.Close();

    ## We have to be sure that we close the file stream if any exceptions are thrown.
    trap
    {
        if ($stream -ne $null) { $stream.Close(); }
        break;
    }

    return [string]$hashByteArray;
}

$fileGroups = Get-ChildItem $Path -Recurse `
    | Where-Object { $_.Length -gt 0 } `
    | Group-Object Length `
    | Where-Object { $_.Count -gt 1 };

foreach ($fileGroup in $fileGroups)
{
    foreach ($file in $fileGroup.Group)
    {
        Add-Member NoteProperty ContentHash (Get-MD5 $file) -InputObject $file;
    }

    $fileGroup.Group `
        | Group-Object ContentHash `
        | Where-Object { $_.Count -gt 1 };
}

Once you have the output of the script you could use it delete the unnecessary files:

$dupes = Get-DuplicateItems;
$dupes | % { ($null, $rest) = $_.Group; $rest; } `
| Remove-Item -WhatIf;

As always, if you have any suggestions or improvements don't hesitate to leave a comment here.

BlogML Contribution

I was hit with trackback spam on my blog some time ago and decided the easiest way to stop it was to disable trackbacks on the site all together. The unfortunate downside to this is that I'm not automatically notified when someone blogs in response to one of my posts. I have to go looking.

Today I found a post by Doron Yaacoby written in August (sorry I took so long) about the Live Space support I added to the BlogML project. As I originally mentioned when I posted about this feature, I was too lazy to code a GUI for it. Thankfully, Doron has written one for us, and in WPF too!

It has been a while since I actually used my Live Space BlogML code to convert my own blog but, in response to Doron's note about comment support, from memory the API exposed by the Live Spaces website for querying blog data does not expose comments. Maybe they've changed the API since. Maybe there's a clever way to screen-scrape it.

Thanks for contributing Doron, and thanks for your kind words about my code too.

 Sunday, September 30, 2007

Adelaide Geek Dinner

I am organising a dinner get-together for local developers to meet, socialise, and learn from each other. I am still finalising details of the time and venue (which will depend on the numbers of guests) but I am currently thinking it will be an evening in the middle of November at a restaurant in the Adelaide CBD.

Also, to avoid being overwhelmed by numbers and to ensure the concept will work here before scaling up, I am going to start by only inviting .NET developers who also publish a blog. I already know a few such people in South Australia and have found a few more on Google and will ensure they receive an invitation but I'm sure I've missed some.

So, if you are, or know someone who is, a .NET developer and a blogger and will be in Adelaide in November, send me an email or post a comment here, and I will make sure you receive an invitation when the details have been settled.

If you are a .NET developer but not a blogger and would really like to attend, just grab a free Blogger or Live Spaces account and post one article about your .NET experiences then let me know.

If the evening is a success then we can plan to have these dinners on a regular basis and open the guest list to a larger range of people. Looking forward to meeting you...

 Friday, September 28, 2007

Streaming Large Objects with ADO.NET, Properly

It has been a while since I last worked with storing files in a SQL database and I decided to Google around to remind myself of the best way to do it. I was very disappointed with most of the approaches I found. Unfortunately, my Google-Fu didn't return the MSDN articles I've linked to below, and I had to find out the hard way.

To begin, all solutions I found dealt only with reading a BLOB from a SQL Server image or varbinary(max) column in a streaming fashion. Worst of all very few actually understood what streaming should do, and that is not load the entire object into an array in memory.

My whinging aside, streaming a file out of a SQL table is easy. You start by using a DataReader created by passing CommandBehavior.SequentialAccess to a DbCommand's ExecuteReader function. I also find that selecting only the blob column and only the desired row(s) from the table is the most effective.

When you have the DataReader positioned on the appropriate record you repeatedly call the GetBytes method in a loop, retrieving a small chunk each time and writing it to the output stream. The output can be any IO.Stream like a file or even your ASP.NET response. This MSDN article has a good description of the situation with the SequentialAccess enumeration and some sample code.

Writing a stream of data into a SQL table turned out to be slightly less obvious. I'm only working with SQL Server 2005 so I didn't consider supporting older versions but the approach is similar. SQL 2005 provides a Write "method" on the large value data types in the UPDATE statement.

My solution was to first insert the new row into the table providing values for all columns except the blob. Then I had a stored procedure that would take the row's primary key values, an offset, and a chunk of the data to insert and use the UPDATE .Write method to update the row.

Similar to the reading code, my writing code would read a small chunk from the incoming IO.Stream and pass it to the stored procedure, incrementing the offset each time. Once again, there is another MSDN article that describes the process well but their code looks like it will also work with SQL versions prior to 2005.

In both cases tweaking the size of the chunk used in each iteration of the loop will require some testing and measuring to find the best performance but now you can read and write files of almost 2GB into SQL Server without trying to allocate a similarly sized array in memory first.