Sunday, May 06, 2007

Preaching To The Converter

During the past week I have managed to transfer my posts from my Live Space to this new Das Blog site. It was pleasantly trouble free, due mostly to the hard work having been already done by the BlogML team.

The biggest headache was the limitation of the Live Space MetaWeblog API implementation. The API will only allow you to retrieve the most recent 20 posts or, if you know the postId, one post at a time. Unfortunately there is no supported method for retrieving the postIds beyond the most recent 20.

Luckily, upon inspection of the nature of the postIds on my Live Space, and inspection of the Live Space Team's blog for confirmation, I discovered a predictable pattern to the postId. It consists of a unique hex string for the space, and an exclamation mark followed by a decimal number that increments with each new post.

With this information I was able to write a Live Space to BlogML converter that, while not exhibiting amazing performance, is able to retrieve all posts on a Live Space much easier than doing it manually or by screen-scraping.

My currently one-way Live Space converter is available in ChangeSet 21935 and later of the BlogML project on CodePlex. Hopefully the project coordinators will build a new release soon and it will then be available as part of the main package.

To use the converter, start by enabling email publishing on your Live Space. Then create a new Visual Studio project and reference the LiveSpace.BlogML assembly. Construct an instance of the LiveSpaceBlogMLWriter passing the name of your Live Space and your email publishing secret word. Optionally set the PostCount property to a number ideally no greater than the number of posts on your site. Finally, call the Write method passing a preconfigured XmlWriter as the destination.

Please submit any issues you have with the converter to the BlogML Discussion pages and I'll endeavour to solve them. For those without the resources or inclination to write a program as mentioned above, let me know and I'll find some time to create a user interface for it.

UPDATE: Doron Yaacoby has created a GUI for the Live Space BlogML converter.

 Saturday, April 28, 2007

Welcome

Welcome to my new blog domain.

I recently decided that I would commit to a more regular blogging schedule. I've kept to my plan for three weeks so far and decided I wanted some more control over my blog than Live Spaces would allow. This site is the result.

There may be lull in the posts while I investigate the Live Spaces MetaWeblog API to export my old posts at Jason's Thoughts to BlogML then import them to this new site. Hopefully it won't be very long at all and will generate some material worth blogging in the process.

In the mean time, links to my scattering of other content is in the Navigation side bar and I highly recommend my favourite bloggers listed in the Blogroll sidebar.

 Sunday, April 22, 2007

The Horse That Will Not Die

I am a VB.NET developer, I have been for a long time thanks to my background with old VB3 and BASIC before that. However, I like C# and I use it whenever an opportunity arises but due to business requirements the primary language with both my last employer and my current employer is VB.NET. It shouldn't matter much though because, due to IL, the Framework library and the Common Language Runtime, both VB.NET and C# compile down to very similar results.

Sadly, the VB.NET developers experience differs from the C# developers experience even though they now share the same IDE. It seems more and more regularly that I find myself making the comment to my colleagues at work that some particular issue would be easier to deal with if we were using C#. Here are some that come to mind:

C# has the ternary operator (<condition> ? <trueresult> : <falseresult>). This is a minor difference but can simplify some otherwise verbose code. VB.NET has the IIF function but it always returns Object instead of the type of the result, thereby requiring a cast, and it is not short-circuited, making it less effective at avoiding NullReferenceExceptions. Apparently, in Orcas, the IIF function gets the same behaviour as C#'s ternary operator.

C# has automated refactoring included. Thankfully, Microsoft arranged for DevExpress to provide a free version of Refactor! for VB.NET so we don't miss out completely. Unfortunately, it doesn't completely bridge the gap. A notable omission is the "Generate Method Stub" refactor that C# includes.

At work, we are slowly adopting Unit Testing where we can to help detect issues sooner in the development cycle. Exposing your production code for unit testing also helps to refactor it for better maintainability. Sometimes you can't easily test internal code but luckily the framework includes the  InternalsVisibleToAttribute which can allow a separate (unit testing) project to call Friend/Internal methods in another project. You can apply this attribute to any VB or C# code but only the C# compiler looks for it meaning we would need to write our unit tests in C#. Again, apparently Orcas has updated the VB compiler to fully support this attribute to.

As well as adopting Unit Testing, we are trying to establish a Continuous Integration system to discover the full impact of changes as they are checked-in. Unfortunately, while the VB IDE is smart enough to infer references beyond the first level, the MsBuild tool used by most CI systems does not, and therefore all builds fail. We could manually add the second level references to each of our projects but any well-meaning developer who uses the Remove Unused References button will undo the hard work. C# requires the second level references to be specified for even the IDE build so it doesn't have this issue. Apparently Orcas changes the VB IDE behaviour to match C# and MsBuild.

VB doesn't completely lose out though. VB has background compilation which makes on-the-fly error detection and Intellisense much more productive. It can slow things down in large solutions and it tends to be the cause of most of the IDE's crashes but in C#, where there is no background compiling, it is sorely missed.

In Visual Studio 2003, VB was behind C# in other ways. The VB team at Microsoft ensured VB received all the missing bits in Visual Studio 2005, however the C# team bounded forward with even more features for C#, thereby still leaving VB behind. With Orcas in Beta 1, we can see that VB has caught up again with the features C# already had. And once again, C# in Orcas has even newer features.

No Sir, I don't like it.

 Friday, April 13, 2007

Where's The Kaboom?

My new home office machine has arrived and I am setting myself up for the most pain possible with my chosen configuration. Firstly I am installing Visual Studio 2005 on Vista Business Edition. Microsoft has only just released a patch for Visual Studio on Vista and it isn't perfect. Add to that running as a restricted user in Vista which introduces more problems. Finally I've chosen the 64-bit edition of Vista which complicates working with Visual Studio and software in general.

Reading the many articles and forum posts about problems with Vista and 64-bit will discourage most people from trying but so far I've found the experience to be quite pleasant. The first road-block is the need for new Vista 64-bit signed hardware drivers but Windows detected everything but the onboard sound and I was able to very easily find drivers on the HP web site for that. Two vital utilities, Daemon Tools and UltraMon, are already available with Vista x64 versions.

Vista has streamlined .zip file handling so I don't need any archiving software and the usual problems with new OS burning software are gone because Vista writes to CD and DVD out of the box. The small problem of burning disc images is solved by the free ISO Recorder, updated for Vista x64. I don't need MakeMeAdmin anymore because Vista's UAC temporary privilege elevation features have solved the problem. And, of course, Microsoft Office works beautifully.

I could rant for hours about the wonders of this new system but there were some issues. I managed to crash Visual Studio while setting the options for the first time but this is an easily avoidable documented issue (thanks Jim). I upgraded SQL Server 2005 to SP2 via Vista's built in AutoUpdate tool so I needed to run the User Provisioning Tool manually afterward to be able to connect. Also, a minor annoyance, Windows x64 has a separate Program Files folder for 32-bit and 64-bit applications and some installers were defaulting to the wrong folder.

I'm keeping a list of issues I encounter with Windows in general and developing with Visual Studio and SQL Server. I'll write about my experiences further on this blog as time passes and as I start pushing the boundaries of compatibility. At this point I have no regrets moving to Vista and 64-bit. I sure don't want to go back to either 32-bit or Windows XP and I'd recommend anyone with similar needs to my own to do the same.

 Friday, April 06, 2007

Backwards Compatibility

At work we are currently developing a suite of applications that are expected to work on Windows XP and, I assume, eventually Windows Vista. The entire development team refreshed their workstations just after Vista went RTM and was available to MSDN subscribers. At the time Visual Studio 2005 wasn't fully supported in Vista and there were too many unknowns so an executive decision was made to stay with XP on the dev machines. I feel it was the right decision at the time.

However, there was one particular reason for staying with XP that I wasn't sure I could agree with. There was a concern that by developing under Vista we might lose touch with XP and find ourselves developing a product that has accidentally used facilities only available in Vista or that differ in Vista. The result, of course, is that we may lose compatibility with XP and discover the problem too late in development to solve it without large time, effort, or cost consequences.

I have decided to use my blog as a place of discussion for reasons why this concern is, or is not, valid. My bias is that the concern is unnecessary but I invite comments on anything I haven't considered.

Acceptance Testing

For each iteration of a product that we produce it goes to our testers to find bugs and usability issues before it can be considering for release. Our testers utilise a clean Windows XP environment inside Virtual PC to perform the suite of tests against the product. Any problems in our software that are specific to XP should be picked up here.

Visual Studio Limitations

Visual Studio 2005 was released well before Windows Vista was close to completion. As a result the IDE itself is not aware of any new features in Vista. In fact, it doesn't even fully support all the features introduced in Windows XP, one aspect that comes to mind is correct rendering of 32-bit icons. Any Vista specific features that are used will have be done so explicitly and presumably with prior consideration.

Framework 3.0

The new .NET Framework 3.0 features are included out of the box in Vista. Updates have been released for Visual Studio 2005 to provide project items and designers for the new WPF, WCF, and WWF components. However, just as we need to ensure that our XP clients need to install .NET Framework 2.0, version 3.0 is supported on XP and Server 2003, and is redistributable via the same methods as 2.0.

Features Lost

One of the primary focuses of Windows Vista has been to improve security. Many popular applications have required (or still require) an update to support Vista due to a change in the operating system's default behaviour. I believe there is a greater risk of utilising a Windows XP feature that will break under Vista because security has been tightened or the feature has been removed altogether (anyone still using WinHelp?).

Practice

With these points in mind I am in the process of building a Vista 64-bit development machine for the home office. With this configuration I intend to experience first hand what issues there are in terms of application compatibility and problems with the development process itself. Anything I find along the way that supports or disproves my beliefs above will help to decide when the development team should migrate to Vista, if at all.

If you have any experiences yourself with this situation I would appreciate your input.

 Friday, March 02, 2007

Self-executing SQL scripts

I am currently preparing our deployment processes for our most recent software product at work. A fair portion of it involves executing SQL scripts to install and update SQL Server 2005 databases. The deployment will ultimately be performed to several sites by staff who are less familiar with SQL Server than the development team.
 
At the moment it involves executing the SQLCMD tool from the command line with appropriate parameters to connect to the server, process the script and output a log. However, I feel that this is just one more error-prone step that should be avoided. Some time ago I read about Polyglots on Wikipedia and I was inspired. I thought I would try to write a batch file that also contained a SQL script. The ultimate goal would be a single-file that could be double-clicked and the script would run and the log would be created.
 
This meant it must be written so the batch command interpreter would ignore the SQL and the SQLCMD tool would ignore the batch commands. The trick was finding the keywords and structures in each language that had similar syntax. After several attempts I settled on GOTO, proving that it isn't totally harmful.
 
Here is a base example of my solution that you can use to create your own self executing SQL scripts. Just put it in a file with a ".cmd" extension and change as appropriate:
 
:setvar NUMBEROFROWS 15
 
GOTO startofpolyglotsqlbatch /*
:startofpolyglotsqlbatch
@echo off
sqlcmd.exe -S MySqlServer -E -e -i "%~f0" -o "%~f0.log"
more "%~f0.log"
goto endofpolyglotsqlbatch
::
*/
startofpolyglotsqlbatch:

USE MyDb;
GO
SELECT TOP $(NUMBEROFROWS) * FROM MyTable;
GO

/*
:endofpolyglotsqlbatch
::
*/
 
I have added some colour to highlight how the code is interpretted. The initial GOTO is parsed by both the batch command processor and SQLCMD but goes to a different destination for each. The green text is only seen and processed by the batch parser and the blue text is only seen and processed by SQLCMD. The grey text can be replaced with content relevant to your script. This is designed to only work with SQL Server 2005 and only on Windows XP, Windows Server 2003, or later.
 Saturday, February 24, 2007

What rolls down stairs, alone or in pairs?

I recently discovered that our databases in SQL Server 2005 using the Bulk Logged recovery model were not getting their logs backed up by the Log Backup maintenance plan. I did some searching to no avail and ending up posting a question to the MSDN forums. I was told this is a known bug in SQL 2005 RTM and SP1 and will be fixed in SP2. SP2 is now available and the problem is fixed... but not very well.
 
The Log Backup maintenance plan allows you to choose "all databases" when asked what to back up. In pre-SP2 it would select all databases on the server, filter out the ones with inappropriate recovery models, then execute a BACKUP LOG statement against the remaining databases. The problem was that instead of only excluding Simple recovery models databases, it excluded Bulk Logged also. I imagined the fix in SP2 would be to correct the filter. Not so.
 
Now, in SP2, the maintenance plan selects all databases on the server and attempts to execute a BACKUP LOG statement against each of them. Those with appropriate recovery models succeed, the others fail. Unfortunately the failures are then treated by SQL Agent as a job error and the maintenance plan reports as failed even though all the right databases were backed up successfully.
 
The benefit in SP2 is that the Bulk Logged databases actually get backed up now because you couldn't back them up at all with a maintenance plan previously even if you explicitly selected the Bulk Logged databases only. The downside is that I have to update my maintence plan everytime a new database is added or the recovery model changes. I appreciate that you should give more time to getting backups right on a production database, but on the development servers this is tedious.
 Tuesday, August 15, 2006

Components need good ISite

Accessing a Form from inside a Component is surprisingly convoluted in the .NET Framework. I created a new class that inherits from Component that would provide extra functionality to any Form I dropped it on. Ideally, it would search it's parent Form for various types of controls and add handlers to certain control events to provide special behaviour. Unfortunately, neither the Component.Site or .Container properties provide access to the parent Form or ContainerControl.

Some of the components included with the Framework are interacting with their parents so I know it's possible. I dropped a ErrorProvider onto a new Form and checked the designer code to see how it was instantiated. I then opened up Reflector, headed directly for the ErrorProvider component and followed this rather bizarre code path:

  1. The designer code creates a private member "components" initialised as a new ComponentModel.Container.  
  2. It then creates the ErrorProvider passing the new Container to its constructor.  
  3. The ErrorProvider constructor simply calls the Container.Add method passing itself.  
  4. The Container.Add method (the trick bit), after extensive validation and array sizing, creates a new ComponentModel.Site and assigns it to the (ErrorProvider) component's Site property.  
  5. The ErrorProvider.Site property stores a reference to the incoming Site then proceeds to request a ComponentModel.Design.IDesignerHost via a call to Site.GetService.  
  6. The ErrorProvider then retrieves the IDesignerHost.RootComponent property, casts it to a ContainerControl and assigns the value to it's own public ContainerControl property.  
  7. The designer then serializes the value of this property to the designer code of the parent Form as "Me.ErrorProvider1.ContainerControl = Me".

The ErrorProvider, and probably many other components included with the Framework, use the behaviour of the designer's code generator to ensure it will have access to it's container at run-time. This feels tightly-coupled yet fragile all at the same time with these odd interdependencies.

Sadly, a Component cannot find trace its roots any other way, so my custom component was forced to borrow this ugly technique. Like the ErrorProvider, my component too must be instantiated differently if created programmatically instead of in the Visual Studio designer.

At least now I, and hopefully you, understand how the Component, Site, and Container classes interact with each other and if you want to access the Form from your own Component, know you can.