As someone actively covering directory services and virtual directories, several innovations have caught my attention. The players within the virtual directory space are (in alphabetical order) Optimal IDM, Oracle, SAP, Radiant Logic, Red Hat, and Symlabs. When it comes to product development and innovation within the last year, you can split those vendors right down the middle. – Optimal IDM, Radiant Logic and Symlabs have been actively developing their product and churning out new features in new versions. The others have not been adding any features, but instead spent time changing logos, product names, default file locations and otherwise integrating the virtual directory products into the respective Oracle/RedHat/SAP identity management ecosystems. In fact, in some of the latter cases I ask myself whether it is likely to expect any virtual directory product innovations anymore.

So what’s new? Where’s the innovation happening?

Optimal IDM: New connectors have been added for Microsoft SQL Server 2008, eDirectory. A special version for Microsoft Sharepoint integration has also been released, as well as “automated compliance features” that monitor for changes that violate definable rules and alert administrators.

Radiant Logic: Its flagship product, formerly Radiant VDS (Virtual Directory Server) has been split up into to new products: The VDS Proxy Edition and the VDS Context Edition. The former is a classical virtual directory product that falls into the same category than Oracle VDS and the Symlabs Virtual Directory products. The latter is a mix of meta-directory and virtual directory features. Radiant Logic has rewritten major parts of the virtual directory core to make it more efficient in order to overcome performance problems that used to be a weak point in the product.

Symlabs: A full virtual tree functionality has been added. This makes the product easier to configure. In the past, a virtual tree had to be constructed by manually configuring plugins to filter and route requests. This had made configuration more difficult compared to other virtual directory products. This used to be a weak point in their products, like the performance used to be a negative point in Radiant Logic’s virtual directory server. Symlabs has also added a complete web-based remote administration interface that can be used instead of, or side-by-side with the local Java configuration interface.

What else is new? The latest piece of news comes from Symlabs who have released a competitive benchmark paper that contains the results of a comprehensive benchmark of the virtual directory servers from Oracle, Radiant Logic and Symlabs. The numbers speak for themselves. Of course, comparative tests by vendors must always be taken with a grain of salt. In the report, Symlabs encourages companies to do their own benchmarks to verify the results in the Symlabs study. However, the numbers are credible and document what has already been known for some time. The Symlabs product comes out as the fastest virtual directory. This is unsurprising, due to a very efficient internal design and a small footprint that this translates to a level of efficiency that surpasses other virtual directory servers.

At second place in the competitive benchmark comes Radiant Logic’s VDS Proxy Edition server, which is also interesting. Until end of last year, Radiant Logic’s virtual directory product was at the tail end of all performance benchmarks, beaten by both Oracle and Symlabs by – at least – a scale of magnitude. Radiant Logic has done some hard work last year to catch up, and it shows by surpassing the Oracle product in the benchmarks and coming in second place.

The virtual directory segment continues to be innovative. This is good for customers that are increasingly adopting virtual directories as simple point solutions to solve integration issues between applications and directory servers. However, innovation does not happen everywhere. It has been very quiet around Red Hat’s, SAP’s and Oracle’s virtual directory products for a long time – up to now, little has happened with those products. Optimal IDM, Radiant Logic and Symlabs have done some serious enhancements to their products and compete head-on in the virtual directory arena. Remember the old stereotype that smaller companies tend to be much more innovative than the larger ones?

I recently received a press release from UnboundID announcing the availability of a new “synchronization server”. This software keeps two LDAP servers in sync (as the name suggests) – bidirectionally. In theory very useful, and it’s free too. But there’s a small trick: the synchronization server supports both Sun’s DSEE (Directory Server Enterprise Edition) and the new Unbound ID Directory Server. In the release, Unbound ID makes no secret of what this software should be used for: to migrate away from Sun’s directory toward Unbound ID’s competing solution.

UnboundID is a start-up based out of Austin, TX. It was founded by several ex-Sun employees, including Neil Wilson, author of the “slamd” load generation engine, and formerly one of the key people behind Sun’s OpenDS. I have already raved about their new LDAP SDK for Java, in my opinion the finest and most complete LDAP development kit for any language ever written.

The company is going after the very lucrative Telco and large service provider market, and launched a frontal attack on Sun Microsystems, who is the market leader in that space. UnboundID is offering a 30-40% reduction in yearly maintenance costs if customers switch from DSEE to their solution. Of course there is the usual fine print, and this offer is limited to medium-sized directories with less than two million entries. Why should Sun customers switch from DSEE to UnboundID Directory? According to UnboundID, their server is faster, has less footprint and is supported on a wider platform range.

It is not really obvious to me however why Telcos and large service providers would want to switch. For one, DSEE has been the de-facto market leader for massive-scale directory services, and customer satisfaction is high (not just if you believe the marketing – I’ve personally heard the same from Telcos using the product). A directory server running in a Telco is an absolutely super-critical component, and ripping it out and replacing it is akin to heart surgery. DSEE is very mature after having been around for many years and the kinks have been ironed out in many very large deployments a long time ago already (in fact, I was in one of those deployments in 2002 – that was fun). UnboundID would obviously need to make a very good case and give organisations a high level of assurance for them to switch over. The Telco sector is much more innovative than others, and tends to be on the bleeding edge of technology – but even so, there is a reluctance to switch from a very mature product that “just works” to a brand-new product.

That’s why UnboundID offers the “synchronization server” in order to try to entice organisations to run both directory servers next to each other for a longer period – to evaluate and eventually become comfortable enough with the UnboundID server to make the switch. It seems that the “synchronization server” has been written specifically for this purpose.

Which, personally speaking, I think is a bit of a pity, but hopefully UnboundID will realise the immense value that this synchronisation server could have once they’ve gotten over their frontal attack on Sun. A generic synchronization server that would keep multiple directories from multiple vendors synchronised is a fantastic value proposition, and I’m sure many organisations would jump at it. Especially when it comes from such brilliant minds like Neil Wilson’s who is known for his awesome LDAP stuff.

An interesting conversation is taking place within the blogsphere about meta-directories, with Dave Kearns and Kim Cameron on both sides of the argument. This was all inspired by a blog entry on the 4th of March from Jackson Shaw called “You won’t have to kick me around anymore!”. That musing was about HP’s retreat from the identity management market, but makes a statement about meta-directory technology:

Let’s be honest. The meta-directory is dead. Approaches that look like a meta-directory are dead. We talk about Identity 2.0 in the context of Web services and the evolution of digital identity but our infrastructure, enterprise identity “stuff” is decrepit and falling apart. I have visions of identity leprosy with this bit and that bit simply falling off because it was never built with Web services in mind.

I started in this area in 1993 and some of the same architectures are still out there.

The certainly struck a chord with me when I read it. Dave Kearns picked up the topic in his newsletter when he wrote about Optimal IDM, the new virtual directory kid on the block, and made the case that meta-directories have “finally given way to the virtual directory”. Kim Cameron picked up Dave’s entry and disagreed. Up to now, this has lead to an interesting ping-pong of opinions between Dave and Kim, which has not exactly been easy to follow, not just because new contributions are being made on a daily basis up to now, and also because Kim uses the term “meta-directory” to mean something different than what Dave (and myself included) understand. I am going to take this opportunity to jump into the commotion as well, knife not freshly sharpened, but armour freshly polished!

First of all, to clarify what “meta-directory” means (at least, to me!). I am thinking about “Via” (Kim’s baby, the product that Microsoft acquired in 1999 together with Kim’s company, Zoomit). I’m also thinking about Novell Dir-XML, Siemens DirXmetahub and the Critical Path Meta-Directory Server. Old products, created many years ago. You don’t really see much happening with this technology any more, because it has its share of problems, and unless assisted with other technologies, does not fit well into today’s much more dynamic identity and access models. The only exception to that is probably MIIS, but I’ll get to that in a minute.

The old traditional “meta-directory” technology works by creating one big “centralised directory” (or “metaverse” as it’s known in MS-speak), pulling data from everywhere into that centralised directory and then pushing data out into all directions either. This approach is usually not a good fit by itself, because it has several significant shortcomings. I would not go as far as call the technology “dead” (it’s impossible to ignore the many MIIS installations out there), but I’ll call it something else: “quaint”. Now that word has several meanings according to the dictionary, but I sure don’t mean “marked by skillful design, beauty or elegance”!!!

Microsoft has made an investment into that technology by rewriting MIIS pretty much from scratch. And Siemens to this date probably has the most comprehensive and advanced meta-directory implementation with its DirXmetahub component that is part of its Dir-X offering. Nevertheless, meta-directories are arguably still around mostly because Microsoft forces this technology onto its customers for what I think are political reasons: Several people working for Microsoft in the field have told me that it was in Microsoft’s interest to have Active Directory as a central component, and believe it against Microsoft’s interest to have a “filtered access”, such as a virtual directory in front of AD, abstracting information away from what should be the authoritative source. I never really understood this fear, but recently it seems that this brick wall may be slowly starting to crumble (see below).

Some experts in the field still obstinately (in my opinion) push meta-directory technology as the only way to integrate multiple sources of identity information. I think this is very short-sighted. This might have been true in the last century, which is not even that far ago. But in a truly dynamic environment, meta-directory technology and a “synchronisation-only” approach just tends to get into the way. Likewise, the idea that virtual directories by themselves could solve all integration issues is wrong. It’s never been only one or the other, unless you had a specific problem to solve. It’s not synchronisation or virtualisation. You need both, at least if you are in a dynamic identity environment, or have a vision to get there.

So what is the solution for the future? Some people believe that virtual directories will eventually fully supplant meta-directories. Coming from the virtual directory world myself (I worked for Symlabs before joining Kuppinger Cole), I never truly believed that – at least not the virtual directories that were around at that time. Virtual directories and meta-directories could co-exist, and the combination of both had in the past shown great benefits. Think of it as the screwdriver vs. the hammer. Sure, with some brute force you might argue that you can use a hammer to put a screw in, and with some agility you might use a screwdriver to hammer in a nail. But you’re likely to damage something in the way, or at best, not be very practical about it.

I think the future is definitely in the convergence of traditional directory servers, virtual directories and synchronisation solutions to provide rock-solid dynamic directory infrastructure. To a certain extent we can already see this. Maxware (before getting acquired by SAP) and Radiant Logic have already released early, basic versions of synchronisation solutions that harness the power of virtualisation and combine synchronisation with dynamic, abstracted multiple views of data, rather than the static meta-directory approach.

In the future I believe we will see “super-directories” that combine traditional data storage with LDAP access, virtual views and synchronisation features. Some of the players in this space are gearing up to do this already. As synchronisation is usually well-established technology by most of the large players in the identity management space, the missing part is currently still virtualisation, and especially the integration of virtualisation and synchronisation.

Sun and the OpenLDAP foundation, for example, have already added some basic virtualisation features to their directory servers. Oracle has acquired OctetString a while back, and has arguably the most complete, all-around implementation of directory services, synchronisation and virtualisation. Novell, IBM and Microsoft are still lagging behind in this space, with some of the “old guard” defiantly resisting directory virtualisation and hanging on to last century’s belief that synchronisation can solve everything. But there are signs that this resistance is crumbling. It better be. Recently, at DEC2008, Microsoft’s Stuart Kwan presented Microsoft’s vision of a truly dynamic identity infrastructure based on an “identity bus”, where applications could plug in, and “transformers allow us to fold, spindle and mutilate the data in any way we want” – changing internal claims into any other format required by applications. Surely virtualisation is not the only piece that is needed to fulfill such vision, but it is an important (and still missing!) piece. Kim Cameron has not been known to be a big fan of virtual directories – and he still shows some scepticism for the “virtual only” approach, but seems to be warming to virtualisation in combination with synchronisation in one of his recent postings:

So we are led to the conclusion that we need a spectrum of synchronization and remote access capabilities. We should be able to use policy to define what information is stored where, and how to get to information that is not stored locally – e.g., combine metadirectory and virtual directory functionality.

I pretty much agree with Dave and Jackson in that traditional meta-directory technology just doesn’t cut it anymore, at least by itself, and is at best “quaint”. I very much agree with Kim in what I think is his vision of a future “super directory service” that integrates synchronisation and virtualisation with traditional directory services. Where I completely have to disagree with Kim however, is his use of the term “meta-directory” for this new type of “super-directory” technology. OK, I agree that “super directory” sounds a bit tawdry. A better term should be found. But c’mon Kim, “meta-directory” is sooooo… 20th century

I am at Netpro’s Directory Expert Conference in Chicago this week, and very excited to be here! I’m keeping my eyes and ears wide open for the latest tech and trends around Microsoft AD and Identity Management, and also participating at an experts panel this afternoon. Knowing that DEC is an action-packed event, I came a day early, and it was well worth it. Sunday’s ramp-up to DEC 2008 was a pre-conference workshop on Microsoft Identity Lifecycle Manager (ILM) 2 beta, Certificate Lifecycle Manager (CLM), Active Directory Federation Services (ADFS) and Active Directory Rights Management Service (ADRMS). It was a hands-on lab experience given by David Lundell from the Oxford Computer Group, who did a brilliant job putting it together.

Microsoft’s vision is to have Directory Services in the centre of a comprehensive infrastructure that supports Identity Lifecycle Management, Strong Authentication, information protection and federation. Harnessing the tools presented in this workshop, one can see where this is going. Although some of the components (specifically ILM 2) are still in beta and not expected to be released until the “second half of 2008”, the picture may still be a bit rough and blurred, but one can see that it will be quite a beautiful one, once completed.

I was particularly impressed by Certificate Lifecycle Manager (CLM), an add-on to ILM that facilitates string authentication, specifically in the area of smart cards. It seems that Microsoft has managed to add significant value to an area that is often notoriously difficult for many enterprises to implement. Starting with an abstraction layer to the underlying card’s hardware stack to a comprehensive lifecycle implementation, CLM supports the full work-flow of the whole lifecycle of issuance, PIN reset, revocation and retirement. Self service is of course part of the offering and is streamlined for efficient and secure management from initial issuance to retirement and secure recycling. Just like the Dot Net Factory, Microsoft is harnessing the new Windows Workflow Foundation for all of its workflow management. For data flow, uses its MIIS meta-directory technology.

Just before the session closed, Microsoft’s Bobby Gill gave us a “sneak peak” of some additional features of ILM 2 beta 3 “hot off the disk” that he compiled a few hours ago. It is obvious that many significant enhancements are still being made, and Microsoft is very actively involved with its beta partners to collect their feedback and make improvements before the official ILM 2 is released.

Back to keeping my eyes and ears open, and I shall be back soon with some more news from DEC 2008!

Hello World! I am excited to have joined Kuppinger + Cole, and my responsibilities will be around the technologies of directory services and identity federation. I would like to kick off my blog by writing about an acquisition that actually happened a week ago, when Nokia Siemens Networks announced that it will acquire Apertio, a Bristol, UK based vendor of telecommunications software. Now what does this have to do with identity management, or even with directories? Simple. Apertio specialises in a very specific type of directory server software. They have come up with a in-memory based, highly efficient and super-scalable directory server that supports LDAP as well as access through protocols used in the Telco space (SS7, IMS).

So what role do directory servers play in the Telco world? Mobile carriers for example, use something called an HLR (Home Location Registry) as the data store for operational subscriber data. A HLR is effectively like a very large directory server, or user database if you wish, that must be highly available (otherwise you might lose service), and highly scalable (able to support many thousands of operations per second, otherwise you, again, might not get the service at the time that you need it). Traditionally, HLRs were sold as “big black boxes” at a juicy prices to mobile operators. What Apertio has done was to very elegantly merge traditional directory server standards and technology with the telco world by writing a specialised directory server that would be accessible via LDAP, and traditional SS7-base telco protocols. Granted – their directory server was so much geared towards that particular use case that it was not sold (nor made much sense) as an enterprise directory. But what fascinated me about Apertio was that they pioneered in successfully marketing the fusion between a LDAP directory and Telco HLRs. At the same time, they were in a great position to also sell the successor technology, called HSS (Home Subscriber Server, part of the IP Multimedia Subsystem, or IMS – effectively the “next generation” in communications).

Apertio has made great inroads with that successful combination. How will technology evolve in the conversion zone between LDAP and HLR/HSSes? I for one, firmly believe that many directory servers are ready for “prime time” when it comes to the stringent demands of the telco industry. Some of the LDAP servers available today can support the thousands of operations required, have the resilience features, and some of them even support transactions. Now that Apertio, who sold their products based on a software solution turns into “boxed HLRs” and “boxes HSSes” with a Nokia Siemens label on them, there might be competition arising from a new company brave enough to add the missing piece to today’s directory server in order to turn them into the next generation telco equipment. I doubt that the traditional vendors will go directly into this – at the end of the day, companies such as Sun and IBM might not want to encroach on the telco equipment manufacturers with whom they have built successful symbiosis by offering competing products – but a third party might well jump into that space.

Very large directory server projects fuel some important developments at the major directory vendors and add scalability and new features that can, to an extent also benefit enterprises, and large service providers that need to store millions of customer entries. Multi-master replication, partitioning and transactional features are all examples of this – who knows if this technology would have been developed if not for one or the other very large directory project. It will be interesting to see who, if anyone, jumps on that bandwagon to offer the next software based HLR/HSS systems based on LDAP technology, and how this may affect directory server vendors in making their software better and faster.