Heavyweight Data Management...

...I am very concerned that I have previously missed an important requirement for data management solutions - a heavweight one judging by this great discussion on one of the Microsoft forums.

Microsoft CEP Surfaces as "Orinoco"

Seems like Microsoft have now gone public on the Microsoft TechEd site that they have a Complex Event Processing (CEP) engine that will be coming to market shortly (see MagmaSystems blog post ). One of my colleagues Mark Woodgate attended a briefing event at Microsoft for this technology back in February this year - here's an extract from some internal notes that Mark made back then:

"Microsoft CEP is very similar to StreamBase conceptually (and not unsurprisingly), in the sense that there are adapters and streams and how you merge and split them via some kind of query language is the same. However, StreamBase uses the StreamSQL which as we have seen is SQL-like in syntax but Microsoft CEP uses LINQ and .NET and although conceptually it is doing the same thing, it does not look the same. StreamBase’s argument was you can be an SQL programmer to use it and don’t need lower-level like .NET; however, it’s not SQL really as it has all these ‘extensions’ you have to learn so using .NET might look more tricky but in fact it makes sense. They don’t have a sexy GUI yet for designing CEP applications like StreamBase but it will be done in Visual Studio 2008.

 

Currently, you build various assemblies (I/O adapters, queries and functions) and then bolt them all together, called ‘binding’ by command line tool. You then deploy the application onto one or more machines using another tool so it’s a manual process right now. They are aware this needs to be made easier and more visual. They are allowing other libraries to be bolted in via the various SDKs so it’s pretty open and flexible. It works well with HPC and clusters/grids (or so they say) and of course can be used with SQL Server. The CEP engine also has a web interface based on SOAP so at least non-Windows based systems can talk to it"

 

The release of this technology will be an interesting addition to the CEP market and to the Microsoft technology stack in general. Assuming performance is at credible levels (i.e. not necessarily leading but not appalling either) it will certainly bring both technical and commercial pressure to bare on existing CEP vendors (see earlier post on Aleri/Coral8) and has the potential to broaden the usage of CEP. Obviously Linux-Lovers (sorry, I didn't mean to be personal...) will not agree with this, but Microsoft is putting together an interesting stack of technology when you see this CEP engine, Microsoft HPC and Microsoft Velocity coming together under .NET.

 

Merging in public is difficult...

Sounds like Aleri and Coral8 in the CEP (Complex Event Processing) market are not doing the best job they could of managing the publicity surrounding their recent merger, not helped by announcement of a CEP capability by Sybase, based on Coral8 source code. 

Explained more in a post on the Magmasystems Blog, and made more entertaining by the aggressive marketing tactics of Streambase in responding to the merger by offering a software trade-in facility for clients of Aleri and Coral8 (see press release).

Bankers playing computer games?

I attended a good set of presentations from one of our implementation partners, D-Fine, yesterday on high performance computing (HPC). They had one of their hedge fund clients present talking about some work they had done in valuing derivatives on a cluster, however the most interesting part of the event was talking about Graphics Processing Units (GPUs) being used within HPC grids and clusters.

Seems that GPUs such as those made by Nvidia offer performance levels that easily exceed that possible on traditional CPU-based HPC solutions. Key to it seems to be the GPU being more specifically designed for vector and floating point operations with several hundred processing cores being available on one chip, whereas CPUs are understandably designed for more general processing requirements. So if the problem is well suited to GPUs (not always the case), one of the technologists from QuantCatalyst said that with optimisation, the performance could be improved by up to 10,000 times relative to CPU based cluster solutions.

As with Chris's article on Solid State Drives then GPU usage is not exactly a free lunch, with specific tools and compilers needed to be used for the moment until a more generic, industry accepted abstraction comes along. At 10,000 times quicker it might be worth the effort though - maybe Hank Paulson should take a look given the CDOs he will need to be valuing any day soon...

Windows NoT - the new Microsoft OS?

Interesting article on plans from Microsoft for a completely new OS (i.e. not a new version of Windows), currently called Midori, motivated by the move to distributed computing and the commercial/technical need not be held back by the legacy of the Windows OS code base. Click here for more.