LUKE PUPLETT

While I’d followed Mr Toub and his gang while they were making the TPL, I’ve since slept and forgotten it all. So I can always consider a concurrent approach to my classes, I thought I’d make a very simple overview ‘cheat sheet’ and stick it to the wall.

Get it here.

“The floor plan above shows the main blocks in Nehalem, and if you've followed previous Nehalem launches (most notably Bloomfield) then you may be able to spot what's missing: there is no QuickPath Interconnect (QPI) interface. Instead, in a significant twist that differentiates Intel's new PC system architecture from even AMD's offerings, there is now a PCIe interface that enables the GPU to attach directly to the processor socket. This latter move was made in anticipation of two things: 1) the GPU will migrate right into the processor socket at a later point when Intel releases a CPU with an on-die GPU integrated into it, and 2) for a discrete GPU, Intel hopes you'll use Larrabee.”

http://arstechnica.com/hardware/news/2009/09/intel-launches-all-new-pc-architecture-with-core-i5i7-cpus.ars

To get myself in the spirit of coding again after my holiday, I thought I’d implement an idea I’d had scribbled down for a few weeks. The concept goes something like this. An iterator usually dishes stuff up and then work is done upon that stuff before it is called again and more stuff is dished up. While the work is being done, would it not be prudent to prepare the next dish?

So I put my programming cape on and went to work. I created a simple class that takes and executes a delegate as the fetch or processing logic – this could either get something from a database, or process a bitmap or something – and then returns the result(s) to the caller in an iterator.

Using the class may look like this:

I’ll jump right to the results and put the code of the actual class at the bottom.

The following shows the average time in ticks it takes for a single test to run. The tests are ran many times to get an average time and minimise ‘interference’ from other processes on my PC. I tested the code above as well as an equivalent version that doesn’t use the PreprocessIterator.

Because the work being carried out above is not very time-consuming I also added a pretend cost, in the shape of a Sleep(ms) call, to the processor work delegate and to the code that consumes the data from the iterator.

10,000 tests without faux cost:

8,770      // parallel <—actually performs much worse.
4,496      // serial

10 tests with 100:100ms faux cost:

4,141,667   // parallel
6,062,455   // serial

100 tests with 20:20ms faux cost:

   899,104    // parallel
1,347,777    // serial

It’s clear from the first test how the overhead of queuing a task on the threadpool far outways any performance gain I stand to make from having that task performed at the same time as the work of consuming the last lot. This was as expected.

However when the consumption of data is more costly than this overhead, then speedups can be achieved.

Using this class may not be as beneficial as other methods, especially when with new CLR 4.0 ThreadPool. If the main thread (the one consuming the data) has to wait for the producer, then this is dead time.

Partitioning data and working concurrently on those partitions in serial may yield better results as any thread completing early will go on to steal from those that are still running.

Where the processing involves invoking work on another computer then it is surely always better to kick the process off sooner rather than later as the CPU cycles expended on performing that work do not effect any that may have been better put to use locally, of course.

Here’s the code for the whole class:

While getting my facts straight in an email conversation with a reader of this blog, I could not find where I had read about the new changes to the thread-pool threads in CLR 4.0 (why do I not know the code name for that, 10-4?) and the reason it proved so illusive is because its all in a Channel 9 interview with Erika Parsons and Eric Eilebrecht.

Here’s my para-bullet-pointing of the key information, from 12m0s in the video:

Previous Framework releases, years ago, started as a fixed amount and fairly low number then it was made per CPU and then in Orcas it was changed to "a frankly absurdly high limit to prevent a 'deadlock' like wait situation" – 250 x CPU count.

Potentially, this high amount could mean that on an 8-core machine you could use up all your virtual memory on your thread pool - potentially.

It’s changing again in 4.0: based on the amount of available virtual memory, it’ll only use half of it. Using this algo on 64-bit systems would end up with such a large number that the thread-tracking data structures used in the thread pool would overflow so its limited to 32,000ish.

This preparation for many-core is interesting given that the dispatcher and kernel locks in Windows 7 scale to (only) 256 cores.

Interesting link: http://blogs.technet.com/markrussinovich/archive/2008/07/21/3092070.aspx

A Technique for RIA UI Control Updates and MVVM

Its a fairly widely known fact of Windows programming that you can't update a UI control from the code executing in your painstakingly well-written worker object. There are a number of Framework constructs that are designed to help in these situations such as the Dispatcher as well as more low-level classes like the AsyncOperationManager which puts a slightly more friendly veneer over the SynchronisationContext. Whichever class you use, updating the UI usually involves writing dedicated methods in some place away from where the action is taking place.

I'm going to talk about using one of the simplest of the lot to re-unite things: BackgroundWorker.

You'll be able to go from writing asynchronous code straight to writing values to your view model with just a few lines in-between where the magic happens - all within the same method - so you can even pipe the variables declared and available within your concurrent method directly into the UI. Sounds to good to be true? Well I hope not because it seems to be working here.

About our Weapon of Choice

First up, a little bit of background about background, err worker. BackgroundWorker’s design centres on three events, DoWork, RunWorkerCompleted and ProgressChanged (I’d have called the first one RunWork but there you go).

Essentially, DoWork is triggered on a new thread-pool thread while the others are triggered on the UI thread.

Be careful though because ConsoleApplications seem not to jump back to Main Thread and I’m not totally sure if the design aim was to trigger on the UI thread specifically or just the same thread that the RunWorkerAsync method was called from.

I'd suggest reading more about it here (VB.NET examples).

The Delegate and its Wrapper

Jumping straight into code, the first thing needed is a vanilla delegate; parameterless and void. Next is a class which holds a delegate - we can't pass delegates in the way I'd like because they don't derive from Object so I’m sort of wrapping it up in a class which can be passed around.

Setting-up and Calling BackgroundWorker

To kick off the concurrent logic, we need to add some code somewhere in a method on the UI thread, either in a Window or some helper class. I’ve kept some extraneous code around to give it some context.

The VolatileState object is a hang-over from my test code so ignore it and pretend its says 'this' and points to a property on the Window1 class or something.

Interesting thing number 1: The DoWork method calls the StartSyncLoop method on the SyncManager passing in the BackgroundWorker that’s hosting it and a ViewModel which in turn has a constructor which points to the page we’re on. If we’re relying on data binding to do everything then we don’t need to pass a reference to our window, we just need a ViewModel instance with all the right bindings. My example uses an explicit UI control manipulation because its more obvious.

Interesting thing number 2: The ProgressChanged event handler method casts the UserState property of the event args back to a UIWorkWrapper and then invokes the delegate its wrapping!

The Synchroniser Thingy Class

If you haven't guessed by now, to get into the spirit of asynchronous activity I’m using a hypothetical example of a synchronisation loop which might keep an application in synch with a web service or something. The premise being that this method is called by the DoWork event handler code and so will begin its duties on another thread (note that thread-pool threads are actually supposed to be used for short bursts of work and not long-running background synch maintenance type tasks... but I won’t tell anyone if you won’t).

And that’s it. The asynchronously running loop can modify the ViewModel directly via an anonymous method delegate which will be run on the UI thread - note how the variables flow right through. The Status property could be data-bound and the PostStatusMessage could do something like this (this.Control having been set in the constructor to the Window1 instance):

Thanks and good day to you.

"Stay away from the nice doggy, children."

While reading the excellent (if somewhat late) Concurrent Programming on Windows: Architecture, Principles, and Patterns (Microsoft .Net Development) I began trying to recall the countless times that I have taken advantage of the framework's APM methods you get for free when you new up a delegate. Here's the noteworthy passage from page 418:

All delegate types, by convention, offer a BeginInvoke and EndInvoke method alongside the ordinary synchronous method. While this is a nice programming model feature, you should stay away from them wherever possible. The implementation uses remoting infrastructure that imposes a sizeable overhead...

Joe's choice of the word nice being a colloquial English word meaning not nice, shit, a poor effort. The scarcity of information from reliable sources and the arguments and proliferation of inaccuracies in the area of concurrency in .NET had me guessing that Joe's book would contain some nasty surprises which is why I'm somewhat peaved that such an important reference has only just come available, eight years after the Framework was released - and two years into my own person journey of asynchronous discovery! --grrrr.