windbg SOS CLR

This post is a reminder to myself, and I hope it will help others. About once a year I find a .NET program will run fine in development but crash and vanish when deployed to production with the generic message "this program has stopped working".

It's usually caused by some subtle environmental difference between the testing and live environments, such as a missing file or a bad configuration setting. Following are the steps I can never remember to help in this situation:

1. Get a copy of "Debugging Tools for Windows (x64)" (which was needed in my case) -- This is easier than it sounds, as there are many downloads available for a wide variety of platforms. I can't remember exactly how I found the right one for me, so proceed carefully.

2. Run windbg.exe and enter these commands:

.load SOS
sxe clr

3. Either launch the program to be debugged or attach to the process of a program that's already running. Start doing things in the program.

4. If the program throws an Exception the debugger will pause and you can issue this command to hopefully see a CLR stack trace to know where the problem is in managed code:

!clrstack

I won't waste space on the technical details here as this is just a reminder post. There are lots of online articles that explain what these windbg commands actually do. And there a lot more interesting commands and tricks to peek into the internals of what managed code is doing. Look for tutorials or cheat-sheets on this subject.

Orthogonal Programming

Windows 10 Open command window here

NOTE (Sep 2019) - Some recent Windows update has broken the following instructions. The registry change no longer returns the missing context menu. If anyone knows of a simple low-risk fresh workaround, please let me know.

Before the most recent major update to Windows 10 you could shift + right-click in a blank part of the Windows Explorer file list and get a context menu "Open command window here". I've been using that feature dozens of times daily for a decade. The menu was recently replaced with "Open PowerShell window here".

PowerShell is certainly an advanced scripting language, but it has irritatingly different behaviour for someone who just wants to run a few dir or pkzip commands quickly in the folder that is currently selected in Windows Explorer. Many old commands and /switch combinations are invalid and have to be prefixed with cmd /c to work in the PowerShell window, which wastes time.

There are many articles on how to get the old menu back, but a lot of them are dangerous or overkill. One article led me to the simplest fix. In a nutshell, login at the real Administrator, not an elevated user and go here:

HKEY_CLASSES_ROOT\Directory\Background\shell\cmd

1. Let local Administrator take ownership of the key, subkeys and values away from TrustedInstaller.
2. Give local Administrator Full Control of the key and subkeys.
3. Rename the DWORD value HideBasedOnVelocityId to ShowBasedOnVelocityId.

These are rather strange steps, but they seem to be the least worst or dangerous that work. I worry that step 1 might have side-effects, but I'll ignore that possibility for now because of the productivity benefit of getting the old menu back.

Orthogonal Programming

Set Windows Service Description

This is a quick follow-up to the previous post about creating MSI installers for Windows Services.

The previous post describes how to add rows to two tables in an MSI database to promote the installer to formally manage the stop, start and deletion of registered services. Unfortunately, it's not possible to set the description of the service by editing the MSI database.

To change a service description (not the name or display name), you have to call into native functions in advapi32.dll which contains service controller functionality. You can find some online articles and samples of how to do this from managed C# code, but many of them are erroneous or questionable. I have determined the minimum correct C# code needed on Windows 10 to change a service description. The source file is a part of a utility DOS command in this DevOps repository:

https://dev.azure.com/orthogonal/MsiUpdater

NOTE: A follow-up experiment a few days later confirms that you can add a CA class to your setup project which calls the code above in the Install override to set the service description. Luckily, Visual Studio set the CA to run in the install sequence after the service processing.

Orthogonal Programming

Visual Studio MSI Service Installer

This post is for anyone who uses Visual Studio Setup Projects to create MSI installer files for Windows Services.

Problem Overview

The "standard" Visual Studio way of creating a Windows Service when a product is installed is to create a Custom Action class derived from Installer which uses the ServiceInstaller and ServiceProcessInstaller classes to register the service. There are plenty of samples of this technique around.

The main problem with the "standard" way is that it only creates (registers) the service the first time you run the installer. The next install, say for an update, will crash because the service is already registered. I tried putting logic in the CA class to skip a duplicate install, but it there was a weird crash during install and it backed out.

Secondary problems with a "standard" service installer are related to the status of the service. There is no support to start a service after install or upgrade, or to stop it before uninstall or upgrade. You can write some OnAfterInstall event code to start the service, but any attempts to stop the service will be futile because the code runs at an inappropriate time in the Execute Install Sequence.

Once I realised that the Visual Studio and Framework class support for installing and creating Windows Service is seriously deficient, I had to find if there was a way of creating an MSI installer with full support for managing services. I have seen installers from Microsoft and other vendors that silently install or upgrade Windows Service with no manual intervention. So if other people can do it, then there must be a way I can do the same. The secret lies in two tables inside the installer.

Service Tables

An MSI installer file is a container for a large set of RDB-like tables which provide excruciatingly detailed control over install processing. Two of the tables named ServiceInstall and ServiceControl are specifically designed to control services. By carefully adding a row to each of these tables you can promote an installer to have the magical behaviour where a service is created, stopped, started and removed perfectly during install and uninstall processing.

The question becomes ... what's the easiest way of inserting rows into the service tables? If I could create some code to do this, then I planned to wrap the code in a library and a command line utility so that it can be called from MSBuild or Visual Studio project post-build events.

PowerShell

As an exercise in becoming more proficient in PowerShell coding I attempted to write a script that would insert rows into the service tables. If you run a search for "msi powershell opendatabase" you will many samples where people have used PowerShell to manipulate MSI files and their tables. There is no formal support for interaction with the installer COM object, so you have to use rather verbose and fragile reflection calls on COM objects.

After suffering terribly for many hours I concocted about 80 lines of script that were almost working. I could read the tables, but any attempt to open the database for update failed with cryptic and useless errors. At this point I threw in the towel, as I knew it would take hours of more suffering to perhaps solve the update failure. In any case, I felt like I was abusing PowerShell and using it for something it wasn't ideally designed for, as the script was becoming quite ugly.

On a whim, I opened up LINQPad to write some C# code to replace the script.

Managed Code

The C# dynamic type makes it really easy to work with COM objects. Within 15 minutes I had translated the PowerShell script into some C# code in LINQPad that successfully read and updated MSI installer service tables. The resulting code was a fraction of the original script size and it was much easier to read. To see how easy it is to read installer tables, here's some sample code the displays the contents of the [File] table.

string msifile = $"{testdata}\\MockService-1-0-4.msi";
Type t = Type.GetTypeFromProgID("WindowsInstaller.Installer");
dynamic installer = Activator.CreateInstance(t);
dynamic database = installer.OpenDatabase(msifile, 0);
dynamic view = database.OpenView("SELECT Component_, FileName, FileSize, Version FROM File");
view.Execute();
dynamic row = view.Fetch();
while (row != null)
{
  string component = row.StringData[1];
  string filename = row.StringData[2];
  string filesize = row.StringData[3];
  string version = row.StringData[4];
  Console.WriteLine($"{component} • {filename} • {filesize} • {version}");
  row = view.Fetch();
}
view.Close();
//database.Commit(); // Only needed if you update something
Marshal.FinalReleaseComObject(database);
Marshal.FinalReleaseComObject(installer);

The code that scans and updates the MSI service tables has been formalised and placed into a DOS utility command. Source is in this DevOps repository:

https://dev.azure.com/orthogonal/MsiUpdater

There is a method in the file that takes all of the parameters required to register a service in an MSI installer's service tables. It throws if conditions are unexpected. I have wrapped a small console command around the method and I call it in the post-build event of Visual Studio setup projects to silently promote the MSI file into a formal service installer.

Summary

It took me many hours of suffering spread out over many weeks to find the MSI service specific tables, learn how to update them correctly, and to test the resulting installers have the correct behaviour on install, upgrade and uninstall.

I hope this post will help others avoid suffering.

Orthogonal Programming

log4net string pattern

After a few years I had to define some log4net appenders in config files and I couldn't remember how to substitute runtime values into the appender parameters. A classic need was to change the address of the mail server for an SmtpAppender. I spent about 30 minutes searching for an answer and came across all sorts of stupid or outdated suggestions that required tedious coding. I knew there was an easy way, but it was so easy I couldn't remember it! Then I finally found it half way down the official FAQ page.

Set the required values at runtime:

log4net.GlobalContext.Properties["mailHost"] = /*something*/

Use a corresponding property in the config file:

<smtpHost type="log4net.Util.PatternString" value="%property{mailHost}"/>

Orthogonal Programming