I’ve recently been to Sun’s directory labs in the the beautiful city of Grenoble, France to talk about what Sun has in store with their two directory servers: DSEE and OpenDS. I’ve used many predecessors of DSEE (starting with the good old Netscape Directory Server) on several projects over the last decade and used to know it inside out. I’ve grown quite fond of it, and so has everybody else I know who has used the product. I wasn’t exactly sure why Sun embarked on its OpenDS project. Why reinvent from scratch what is already a perfectly great product? This question was on my mind, and I was eager to find out why.

When it comes to directory servers, most analysts like to classify them according to the market segments they address. In no particular order, they are: operating system/network, telco and service provider, enterprise and embedded. When it comes to the operating system/network directory servers, Active Directory rules – not necessarily because it is the best for this purpose (and just to be clear: it’s not bad either!), but – well – it’s so intrinsically linked to Windows that you don’t really have a choice. When Novell Netware was around, NDS and e-Directory was another candidate in that area, but it’s pretty much down to AD at this point in time. It’s in the other segments where it gets really interesting because there is some very active development and strong competition.

The Telco/Service provider directory segment is particularly interesting because only the highest scalable directory servers can even attempt to survive in this area. Sun has been very strong in this area for many years, and for a good reason: experience and continuous improvement. I’ve been involved first hand in several very large deployments of Sun Directory Server 5.0 (I think it was during the time when Sun called it “iPlanet Directory Server”). At that time, in the early years of this millennium, we deployed the server for hosting several hundreds of millions of entries. Yes indeed, about 120 Million entries! This was 2002, and at the time the sheer scale was pushing the envelope quite a bit -  but it didn’t just work, it actually worked quite well! Performance, Multi-master replication, and resilience were absolutely key for these types of installations. And sure – in the early versions of 5.0 there were some kinks that had to be ironed out of the replication protocol, but even then it was quite amazing how scalable the directory was, and how well it could actually be managed with such an impressive number of entries. Over the last 7 years, the directory server evolved even further – multimaster replication is rock solid and Sun has tinkered continously with the software to increase scalability way beyond what was already impressive in 2002. Nowadays, there are quite a few reference customers who run Sun directory server with literally billions of entries (incidentally, many of them in China – why am I not surprised ), and this is considered perfectly normal.

When it comes to reliability, a key to deploying very large directories is redundancy, and the possibility to balance loads and fail over between multiple instances. In the early days, load balancing appliances were used to do this (Alteon was really good at this in its days), but unless those applicances had specialised proxy features to handle the instrinsics of the LDAP protocol, this by itself wasn’t a very good option for large deployments. Sun had acquired a company called Innosoft a decade ago, and with it came a product called “DAR” – Directory Access Router – a fully fledged LDAP proxy. Over the years, Sun has enhanced DAR and bundled its next generation into Directory Server (now known as “DSEE”, Directory Server Enterprise Edition”) at no additional cost. Being an important cornerstone of very large and complex directory deployments, it fits like a glove into the directory service and extends it by offering extensive request routing functionality, high availability and performance features and simple mapping features. Previously, only the CA eTrust directory had these features.

I can talk all day about deploying telco directory services, because I’ve used to do it for a living, and am still fascinated by the sheer volume and raw power involved But there’s another two very glorious aspects of directory services, and they can be found in the enterprise and in the still fairly recent embedded directory segment.

The enterprise directory segment is where most of the innovation is happening. Enterprises are typically not as focused on performance, and often more interested in integration, security and manageability. Integration is a very big topic, because the directory service is a crucial piece in any identity management infrastructure. And we’re usually not talking about “a” directory either – most enterprises have many different directory servers, containing either different user populations, or part of the same users but for different purposes. It is in the integration area where much innovation has happened in the directory area. Is doesn’t surprise me that most enterprise directories nowadays feature simple virtual directory functions. That was not the case five years ago, when I worked for a virtual directory vendor. At that time directory service vendors did not foresee virtualisation features as being an important part of their portfolio – perhaps because some of those vendors were also selling an “identity manager” type provisioning system and thought that any directory integration could be solved by deploying a full-blown provisioning system and brute force copying data around Well, this wasn’t really a feasible solution in all cases, so it is only natural that virtual directory companies such as OctetString and Maxware were acquired, and other vendors are “rolling their own” virtualisation features.

Some of the features that are not obvious, but extremely useful in the enterprise scenario are exactly those that allow a directory server to easily interoperate with provisioning, virtualisation and synchronisation products. Technically, the features in LDAP server that are relevant here are persistent queries, incremental updates and proxy auth. These are low-level features that are absolutely crucial when identity “managers” and provisioning services interface with directory servers.

Some other desired features within the enterprise directory segment are about password services and policies. In the vast list of featureds to be found in most modern directory servers are sophisticated access control lists that are expressive enough to configure a finely grained access control policy for deciding who gets access to what type of information. This used to be very important in the past but is getting less important as access control rules on the directory servers tend to be simpler nowadays, because changes typically ocurr through provisioning systems, and not that much any more directly to the LDAP server. Password policies are also a typical feature used in enterprise directory servers (you know – minimum length, character combination, auto-lockout,auto-expiry, and all those things). And of course, keeping track of when users last logged on – very helpful in order to identity dormant accounts.

Another important detail is also how passwords are stored, and how they can be migrated from one server to the other. As a general rule, it’s always good to offer administrators choice. Obviously passwords need to be well protected. But the approach of some directory vendors (specifically Microsoft and Novell) to “secure” their directories has backfired – the directory servers hoard the passwords and don’t even offer any possibility for administrators to export encrypted password hashes. You may wonder whether this “secure” feature is actually a hidden “lock-in trap”! That has created a secondary market around password “synchronisation solutions” in order to overcome the deficiency in the product itself, where the product’s designers thought they had to be smarter than the poor administrators who actually need to deploy, migrate and maintain them.

Last but not least, let’s not forget about one of the very important aspects of enterprise directory services. They need to be simple to deploy, administer and maintain! In the telco area it may be considered acceptable if the directory administrator team features several fully trained relational database administrators, but in enterprise environments that can be too much overhead. Directory servers that make use of relational databases for storing their directory data, such as Oracle’s OID and IBM’s Tivoli Directory Server can point to the advantages of running a directory services platform on a rock-solid database foundation (in these cases, Oracle and DB2 respectively). But the extra administration overhead can be considerable. CA has traditionally used the Ingres relational database for its eTrust Directory Server, but has now in the latest Version 12 switched to something called “DXgrid” – a revolutionary internal memory-mapped storage that not only offers incredible throughput, but also eliminated a significant portion of administration. Sun has since always used a simpler, but very fast and highly scalable data store for its directory server called BerkeleyDB – the same used also in most installations of OpenLDAP.

After mumbling on for quite a discourse I actually wanted to get to the point of Sun’s OpenDS, and the question that I wrote in the beginning of this entry. Why reinvent from scratch (OpenDS) what is already a perfectly great product (Sun DSEE)? As it turns out, there’s been a new segment for directory server that is steadily growing: the one of embedded directory services. For example, packaged solutions that require a directory server internally. Or “black box” appliances with a provisioning interface that contain – guess what – a directory server. A few years back, it was OpenLDAP that was typically shipped with those solutions, because it was free, open and could be embedded easier than other full-fledged directory server products. Now it is OpenDS that is continuously gaining ground, and for good reason. With its incredibly easy set-up, minimal administration, OpenDS epitomises what an embedded directory stands for. And on top of that, the scalability and performance are world-class. Development on OpenDS is, as the name implies, well – open. The development team features Sun employees and others outside Sun, just like OpenSSO. The release cycle is short and new features list is growing at an incredible rate.

So will OpenDS one day replace DSEE? Most likely. But this is still far in the future – for the next few years Sun is actively investing in DSEE as its flagship directory whilst continuing to nurture OpenDS and offering it as an embedded directory server, as well as to anyone interested in quickly deploying a directory server. Now, when I say “quickly” – I’ve managed to install it, extend the schema and load some data into it in less than fifteen minutes! Now that’s what I would call “quickly”. And once I had it up and running on my slow and overloaded laptop, I ran the “slamd” LDAP benchmark tool against it on the same laptop, and got back thousands of searches per second. Not bad at all! Now that’s what I call innovation in the world of LDAP

I’ll be speaking at TEC on Wednesday the 25th of March, on the topic “Cool LDAP Innovations”. OpenDS will definitely get a mention. On the presentation, I’ll also talk about some other real innovations that happened over the last few years in the directory services area. If you’re there, be sure to drop by!

Two weeks ago I was at Digital ID World in Anaheim, CA, followed by a briefing in Redmond. My mind is still returning to this action-packed event every once in a while, and I am still trying to make sense of it all. For me the most interesting aspect of DIDW has certainly been to meet face to face with lots of the usual suspects, some people I “know” virtually, but have never met face to face, and some new acquaintances. Over the next few week, as my busy research agenda allows, I will write up on some of the cool stuff, new technologies and new evolutions of products that I’ve learned about during those three days.

Just thought I’d just pay tribute to some of my experiences during those three days. For me as well as for many others, DIDW started off with a visit to the new “IDTBD” (ID To Be Determined) initiative that the Liberty Alliance sponsored. Bob Blakeley from the Burton group stood in the middle of a fully crowded room (including people standing outside). After a somewhat tedious roll call where everybody present stated why they actually went to this meeting, the discussion came into full swing. The idea behind the “IDTBD” was to provide an infrastructure framework for projects around identity. Instead of every project getting tied down with bureaucracies, legal agreements and organisational matters, IDTBD would provide support and let participants focus on what they can do best. I thought the idea was pretty good, but not everybody thought the same. As organisational matters like these were not my forte, I disappeared after the break, and when I walked past the open door an hour later, I could see that a very small crowd was still in very animated discussion.

I had my fun with Sun that afternoon, evening and night, and honestly, I had a blast. Sun brought me in twice for their Identity Buzz TV show. Daniel Raskin was my host, and we talked about open source within identity management – the specific nuances and what customers can expect from it. We also talked about one of my favourite topics, the identity bus (I did a round-table at our European Identity conference back in May), and in that one I managed to turn it around and have Daniel add his thoughts to the discussion (later on that week, I had the pleasure of meeting again with Stuart Kwan who explained me his vision, but more to that later). It was great to meet Daniel, I only had the virtual pleasure up to that point, and can attest that he is at least as cool and knowledgeable in real life as well. I also had some quality time with Pat Patterson, who I’ve met before, but only shortly between doors, and it was good to catch up. Saachin was there as well and turned on several light bulbs in my head when he talked to me about Sun’s 3 month roadmap for deploying Role Manager within an enterprise. My head was spinning a bit after so much information, and I was really grateful when Saachin’s colleague Neil Gandhi patiently spent a good two hours briefing me and walking me through the product in great detail a day later. As my colleague Sebastian Rohr and other noted, Sun certainly made a killing snapping up Vaau earlier this year, and now I can fully appreciate Sebastian’s enthusiasm.

John Barco very cunningly demonstrated a concept that is likely to pop up in the same basket as identity theft: identity exchange. By wearing Nicholas Crown’s badge around his neck the next day, he had me confused, because I just met both of them in person for the first time the day before. I had some great discussions with both of them later, especially with Nick, whom I talked after the Ping Identity party until the not-so-wee-anymore hours. Oh yes, the Ping party. Aren’t they legendary! As this event was held at the “Blues house”, the “house drink” was a blue liqueur. It did not glow in the dark, but turned out to be somewhat of an acquired taste. Andre Durand’s team were busy making sure that everyone held at least one cup in their hands at all times. I decided to be careful with it. At the party I made some great acquaintances, and ran into Doug Anter from Covisint. In a very forward-looking spirit that is common after successive libations in the later evening, we decided to set up a “breakfast briefing” for 9 AM the next morning. This turned out not to be painful at all (perhaps I can attribute this to my special care with the house drink), but to the contrary highly interesting, as I have an article in preparation on Covisint’s offerings on “Identity as a Service (IaaS)”.

In the same area, I was equally impressed with a briefing that I received earlier from Eric Olden who is the founder and CEO of Symplified. Having founded Securant in 1995 (which he later sold to RSA), he well understands the need, but also the entry barrier for small and medium enterprises when it comes to identity and access management. Symplified provides identity and access management as a service in both directions – incoming and outgoing. On the outgoing side, Symplified can connect an enterprise’s users to internal and external SaaS services (such as Salesforce, Workday, ADP, etc.) with single sign-on. On the incoming side, access to resources is controlled through a proxy layer that is either hosted by Symplified itself, or runs inside an organisation in several form factors: appliance or virtual machine. I think there is a photograph of myself wearing a Symplified T-Shirt towards the end of the Ping party.

Another very interesting briefing I received was from AEP Networks’ J. Alan Bird who is extending identity throughout the network with identity based access control. Their IDpoint solution tags every network packet (actually, the payload within IP packets) from an authenticated client PC with a special token. Specialised identity routers then act like firewalls by checking access against tokens and making access control decisions. A sophisticated auditing and reporting engine is included that can act as a feed to current GRC (Governance, Risk-Management and Compliance) solutions. As identity management has traditionally focused mainly on application security, I think that this pioneering approach offers a significant manageabilility gain and a previously not well-addressed need for extending GRC towards the network layer. I am convinced that this will become an important topic, especially with investments in strategic GRC projects increasing.

Oracle was a main sponsor at Digital ID World, and many of its brightest minds were roaming around. I was particularly happy to finally meet face to face with Nishant Kaushik whose blog I read regularly and recommend (it’s on my blogroll). Same with Clayton Donley, who I’ve seen already seen previously from far away, but have never had the opportunity to shake hands with. I had a great follow-up discussion with Eric Leach on Oracle’s new access management suite (he had briefed me on it a month before). And of course Phil Hunt, whose efforts around the Identity Governance Framework I wrote about previously. When I finally got to meet Dennis MacNeil in person, he gave me some good advise and helped me understand better how the individual pieces fit into Oracle’s strategy.

Understanding that it is impossible to mention everyone and everything that I met and discovered, it is perhaps worth mentioning what I wish I could have done. The time was limited, and unfortunately the exhibition floor closed very promptly, and I just plainly ran out of time. Matt Flynn was there and I shook his hand but had to run off and couldn’t catch up with him anymore. He will not escape me next time (or rather, I will not escape him) I also ran out of time and couldn’t properly catch up with the folks from Optimal IDM anymore, who briefly told me about the new features added to their virtual directory product. Equally with my old colleagues from Symlabs who would have loved to show me the upcoming full virtual tree feature in the next version of their virtual directory. Charles Andres who is now the head of the Information Card Foundation was all over the place but unfortunately so was I (and at the Information Card Foundation’s booth I ran into Axel Nennker, which was really cool). I did not have time for Sailpoint and Novell unfortunately – although I did have a brief chat with Dale Olds and some of the other “Bandits”, but would have loved to spend more time with his colleagues as well. Next time it will be!

Now who says that federated identity can’t be entertaining as well. On January 24th, Sun’s Daniel Raskin, who is involved in Sun’s OpenSSO project, poked a bit of fun at competitor Ping Identity by putting a short videoclip up on his blog which would help “explaining the differences” between Sun and Ping. It didn’t take long for Ping’s crew to respond in kind, promising an “epic battle” in their own video posted on Ping’s blog.

What I quite interesting about these little jabs carried out in good humour were the comments about “federation auto-connect” that Ping announced a few weeks ago in the latest version of PingFederate. The idea of this feature is to make federating between different entities easier by automating the exchange of meta-data. At Symlabs, the company I worked for previously, my then-colleague Sampo Kellomaki had developed the same feature about a year earlier, and had even mentioned it in his presentation at the first European Identity Congress last year. At that time, I must confess that I was unsure this feature’s value in most scenarios – apart from very specific low-risk “open” federations which were already being catered to by OpenID. Charts such as the one featured in this blog entry from Ping still raise a certain scepticism, but maybe that scepticism might prove to be wrong. I am certainly interested in exploring further the value proposition of auto-federation and will make sure to tickle some answers out of the participants in the federated identity track that I’m moderating at our next Conference in April.

As both Sun and Ping will be at the European Identity Conference 2008, we will try to set the stage for an epic battle to be carried out there! And to make it even more interesting, we’ll throw the other contenders into the foray as well!