I’ve recently been to Sun’s directory labs in the the beautiful city of Grenoble, France to talk about what Sun has in store with their two directory servers: DSEE and OpenDS. I’ve used many predecessors of DSEE (starting with the good old Netscape Directory Server) on several projects over the last decade and used to know it inside out. I’ve grown quite fond of it, and so has everybody else I know who has used the product. I wasn’t exactly sure why Sun embarked on its OpenDS project. Why reinvent from scratch what is already a perfectly great product? This question was on my mind, and I was eager to find out why.

When it comes to directory servers, most analysts like to classify them according to the market segments they address. In no particular order, they are: operating system/network, telco and service provider, enterprise and embedded. When it comes to the operating system/network directory servers, Active Directory rules – not necessarily because it is the best for this purpose (and just to be clear: it’s not bad either!), but – well – it’s so intrinsically linked to Windows that you don’t really have a choice. When Novell Netware was around, NDS and e-Directory was another candidate in that area, but it’s pretty much down to AD at this point in time. It’s in the other segments where it gets really interesting because there is some very active development and strong competition.

The Telco/Service provider directory segment is particularly interesting because only the highest scalable directory servers can even attempt to survive in this area. Sun has been very strong in this area for many years, and for a good reason: experience and continuous improvement. I’ve been involved first hand in several very large deployments of Sun Directory Server 5.0 (I think it was during the time when Sun called it “iPlanet Directory Server”). At that time, in the early years of this millennium, we deployed the server for hosting several hundreds of millions of entries. Yes indeed, about 120 Million entries! This was 2002, and at the time the sheer scale was pushing the envelope quite a bit -  but it didn’t just work, it actually worked quite well! Performance, Multi-master replication, and resilience were absolutely key for these types of installations. And sure – in the early versions of 5.0 there were some kinks that had to be ironed out of the replication protocol, but even then it was quite amazing how scalable the directory was, and how well it could actually be managed with such an impressive number of entries. Over the last 7 years, the directory server evolved even further – multimaster replication is rock solid and Sun has tinkered continously with the software to increase scalability way beyond what was already impressive in 2002. Nowadays, there are quite a few reference customers who run Sun directory server with literally billions of entries (incidentally, many of them in China – why am I not surprised ), and this is considered perfectly normal.

When it comes to reliability, a key to deploying very large directories is redundancy, and the possibility to balance loads and fail over between multiple instances. In the early days, load balancing appliances were used to do this (Alteon was really good at this in its days), but unless those applicances had specialised proxy features to handle the instrinsics of the LDAP protocol, this by itself wasn’t a very good option for large deployments. Sun had acquired a company called Innosoft a decade ago, and with it came a product called “DAR” – Directory Access Router – a fully fledged LDAP proxy. Over the years, Sun has enhanced DAR and bundled its next generation into Directory Server (now known as “DSEE”, Directory Server Enterprise Edition”) at no additional cost. Being an important cornerstone of very large and complex directory deployments, it fits like a glove into the directory service and extends it by offering extensive request routing functionality, high availability and performance features and simple mapping features. Previously, only the CA eTrust directory had these features.

I can talk all day about deploying telco directory services, because I’ve used to do it for a living, and am still fascinated by the sheer volume and raw power involved But there’s another two very glorious aspects of directory services, and they can be found in the enterprise and in the still fairly recent embedded directory segment.

The enterprise directory segment is where most of the innovation is happening. Enterprises are typically not as focused on performance, and often more interested in integration, security and manageability. Integration is a very big topic, because the directory service is a crucial piece in any identity management infrastructure. And we’re usually not talking about “a” directory either – most enterprises have many different directory servers, containing either different user populations, or part of the same users but for different purposes. It is in the integration area where much innovation has happened in the directory area. Is doesn’t surprise me that most enterprise directories nowadays feature simple virtual directory functions. That was not the case five years ago, when I worked for a virtual directory vendor. At that time directory service vendors did not foresee virtualisation features as being an important part of their portfolio – perhaps because some of those vendors were also selling an “identity manager” type provisioning system and thought that any directory integration could be solved by deploying a full-blown provisioning system and brute force copying data around Well, this wasn’t really a feasible solution in all cases, so it is only natural that virtual directory companies such as OctetString and Maxware were acquired, and other vendors are “rolling their own” virtualisation features.

Some of the features that are not obvious, but extremely useful in the enterprise scenario are exactly those that allow a directory server to easily interoperate with provisioning, virtualisation and synchronisation products. Technically, the features in LDAP server that are relevant here are persistent queries, incremental updates and proxy auth. These are low-level features that are absolutely crucial when identity “managers” and provisioning services interface with directory servers.

Some other desired features within the enterprise directory segment are about password services and policies. In the vast list of featureds to be found in most modern directory servers are sophisticated access control lists that are expressive enough to configure a finely grained access control policy for deciding who gets access to what type of information. This used to be very important in the past but is getting less important as access control rules on the directory servers tend to be simpler nowadays, because changes typically ocurr through provisioning systems, and not that much any more directly to the LDAP server. Password policies are also a typical feature used in enterprise directory servers (you know – minimum length, character combination, auto-lockout,auto-expiry, and all those things). And of course, keeping track of when users last logged on – very helpful in order to identity dormant accounts.

Another important detail is also how passwords are stored, and how they can be migrated from one server to the other. As a general rule, it’s always good to offer administrators choice. Obviously passwords need to be well protected. But the approach of some directory vendors (specifically Microsoft and Novell) to “secure” their directories has backfired – the directory servers hoard the passwords and don’t even offer any possibility for administrators to export encrypted password hashes. You may wonder whether this “secure” feature is actually a hidden “lock-in trap”! That has created a secondary market around password “synchronisation solutions” in order to overcome the deficiency in the product itself, where the product’s designers thought they had to be smarter than the poor administrators who actually need to deploy, migrate and maintain them.

Last but not least, let’s not forget about one of the very important aspects of enterprise directory services. They need to be simple to deploy, administer and maintain! In the telco area it may be considered acceptable if the directory administrator team features several fully trained relational database administrators, but in enterprise environments that can be too much overhead. Directory servers that make use of relational databases for storing their directory data, such as Oracle’s OID and IBM’s Tivoli Directory Server can point to the advantages of running a directory services platform on a rock-solid database foundation (in these cases, Oracle and DB2 respectively). But the extra administration overhead can be considerable. CA has traditionally used the Ingres relational database for its eTrust Directory Server, but has now in the latest Version 12 switched to something called “DXgrid” – a revolutionary internal memory-mapped storage that not only offers incredible throughput, but also eliminated a significant portion of administration. Sun has since always used a simpler, but very fast and highly scalable data store for its directory server called BerkeleyDB – the same used also in most installations of OpenLDAP.

After mumbling on for quite a discourse I actually wanted to get to the point of Sun’s OpenDS, and the question that I wrote in the beginning of this entry. Why reinvent from scratch (OpenDS) what is already a perfectly great product (Sun DSEE)? As it turns out, there’s been a new segment for directory server that is steadily growing: the one of embedded directory services. For example, packaged solutions that require a directory server internally. Or “black box” appliances with a provisioning interface that contain – guess what – a directory server. A few years back, it was OpenLDAP that was typically shipped with those solutions, because it was free, open and could be embedded easier than other full-fledged directory server products. Now it is OpenDS that is continuously gaining ground, and for good reason. With its incredibly easy set-up, minimal administration, OpenDS epitomises what an embedded directory stands for. And on top of that, the scalability and performance are world-class. Development on OpenDS is, as the name implies, well – open. The development team features Sun employees and others outside Sun, just like OpenSSO. The release cycle is short and new features list is growing at an incredible rate.

So will OpenDS one day replace DSEE? Most likely. But this is still far in the future – for the next few years Sun is actively investing in DSEE as its flagship directory whilst continuing to nurture OpenDS and offering it as an embedded directory server, as well as to anyone interested in quickly deploying a directory server. Now, when I say “quickly” – I’ve managed to install it, extend the schema and load some data into it in less than fifteen minutes! Now that’s what I would call “quickly”. And once I had it up and running on my slow and overloaded laptop, I ran the “slamd” LDAP benchmark tool against it on the same laptop, and got back thousands of searches per second. Not bad at all! Now that’s what I call innovation in the world of LDAP

I’ll be speaking at TEC on Wednesday the 25th of March, on the topic “Cool LDAP Innovations”. OpenDS will definitely get a mention. On the presentation, I’ll also talk about some other real innovations that happened over the last few years in the directory services area. If you’re there, be sure to drop by!

TUESDAY, March 4th. Chicago, back to freezing temperatures.

Microsoft’s Stuart Kwan kicked off the second day with his keynote address where he spoke about an “identity bus”, where off-the shelf applications can plug in for all their authentication and access control needs. It was exciting to hear this from Microsoft. The concept is actually not even that new – Phil Windley wrote about this in his book “Digital Identity. But it’s great to see that Microsoft seems fully committed to (several selected) open standards (most of them likely to start with the letters W and S) and interoperability, and assuming that this stays that way, I agree with Stuart that “it does not seem as far off as you might think”. His vision is that claims, or “assertions” in SAML-speak, are the core of identity data within the bus that can be transformed into different formats depending on application needs. Somehow I had a quick excursion down memory lane during the end of the presentation to a place in time a few years ago when I was briefly involved in a project for a large service provider who had deployed two web application environments. One of them was called the “Microsoft Environment” and the other one was called the “Open Environment”. I remember that they had quite a challenge getting identity management to work across both environments at that time. So in future, that unnamed company will just have one “Open environment” that includes all the Microsoft web applications as well.

Later on in the exhibition area I took a closer look at the latest virtual directory server product that had just been announced a day previously by a company called “Optimal IDM”. I was given a comprehensive tour by chief software architect Nada Dickerson who graciously let me monopolise her for an extended period (I couldn’t help it, coming from the virtual directory space, this really peaked my interest). Optimal IDM has deployed identity management solutions over several years and has run into the same types of issues over and over again, which led them to develop their virtual directory product. The product is specialised to handle three specific “modes”. The first mode is called “Union Mode” and aggregates unique entries from multiple containers into a virtual consolidated view. The other two modes are variants of this, and add join rules. Object Precedence Mode can be used to specify which back-end has the precedence when the same object exists in multiple back-ends. Attribute Precedence Mode can additionally join attributes from the same object in multiple backends so that the returned object contains data from all objects of the same name. This is essential the “shadow joiner” or “data augmentation” feature found in competing products. Optimal IDM has developed their virtual directory product entirely with .NET technology and believes this to be a competitive advantage over other virtual directories.

Even before the conference, I had already earmarked the slot at 11 AM for Pamela Dingle’s presentation entitled “Hanging Out in the CardSpace Kitchen”. I hadn’t met Pamela before, but read some of her excellent material on the subject. It turned out exactly as I had hoped – an excellent, educational and lively presentation from *the* authority on CardSpace (apart from Microsoft’s Kim Cameron, of course). She also made the connection clear between Microsoft’s implementation, Project Bandit and Project Higgins. Microsoft’s Brian Puhl chimed in at the end to give an excellent scenario on how CardSpace can be used to work around some tricky legal issues in federated environments.

I remembered Dave Kearns mentioning in one of his posts from way back when that when Pamela Dingle presented on CardSpace at one of the Digital ID World conferences, she had Microsoft employees clamoring to ask questions. Well they didn’t do this time, but I certainly did. I met Pamela again that night in Centrify’s hospitality suite at a virtual bowling tournament on a Wii. Both Pamela and I were new to the Wii and immediately noticed that it even offers its own variety of “InfoCards” in the way of avatars (pictures) of the participants that our host configured for us.

Another highlight was the BoF (birds of a feather session) on ADFS that I attended. Stuart Kwan and Brian Puhl were there as well, and shared some interesting details on why we actually need to use claims and can’t just instead stick additional identity information in a Kerberos token (some applications would just choke on that), as well as throw around some ideas on how home realm discovery could be used in an environment where multiple federation protocols are used. Then suddenly we were all deep into CardSpace again. Stuart also offered a very interesting opinion how internal security tokens might look like in a hypothetical, futuristic Windows version. Unfortunately I had to run out because I had an appointment, but I think the BoF ran well over its allocated time – definitely another DEC highlight.

Against good judgement and to the bewilderment of my bowling buddies at the Centrify hospitality suite, I decided to forego the invitation to Oxford Computer Group’s party featuring a band called “Hairbanger’s Ball” and instead head off to early retirement for the day. In hindsight this was not the smartest thing to do, because it must have been a real blast, and overall the particular session in the morning that I wanted to attend was cancelled. Well, I’ll know better next time!

MONDAY, March 3rd, Chicago (surprisingly warm).

I’ve already reported from the pre-conference workshop last Sunday that gave a very good introduction to Identity Lifecycle Manager 2 and Certificate Lifecycle Manager, and Microsoft’s Joe Long kicked off DEC 2008 with his keynote session presenting Microsoft’s vision on Identity Management, and how Active Directory will evolve to meet those needs in the future. Apart from being a good summary on what I had already heard a day before, it highlighted Active Directory being in the centre of Microsoft’s Identity Management ecosystem, surrounded by four cornerstones: Identity Lifecycle Management (ILM), Strong authentication (i.e. smart cards and CLM), data protection (Rights Management) and federation (ADFS). The idea is certainly going in the right direction. It is certain however, that Microsoft leaves many opportunities to be filled by partners that can harness the framework and complement it by filling in the gaps. No surprise of course, that some of these partners were present at DEC, showing off their latest wares.

As expected, many of the sponsors used the opportunity at DEC to announce availability of new products and releases. Netpro, the organisers of the event, went even further and “pre-announced” (announced that they will announce?) an upcoming new release to its ChangeAuditor product, a leader in the Windows auditing space. The upcoming 4.5 release is to feature a new SQL Server module and comprehensive Exchange auditing, including permission change and non-owner mailbox alerting. As companies in this space continuously strive to catch up with the ever-increasing demands of audits, this helps Netpro maintain a competitive edge in the space, as was explained to me by Brad Hibbert, who kindly took the time to brief me on Netpro’s plans. Brad also mentioned that Netpro will release a free tool built on top of Netpro’s SOA architecture that will integrate and extend the native Users and Computers interface. According to Brad, this AD Management tool will add business rules, workflow, and task automation to the ADUC interface. It help tighten security and instill better Idm practices into AD management, without requiring people to change how they manage AD today. The first release is planned to ship in May. Later this year a subsequent release will also provide a web console for AD management with these same capabilities.

Netpro is also planning a script management solution in Q3. This will allow organizations to integrate their custom scripts and batch jobs into an management console that will provide distribution, scheduling, security delegation, auditing and performance statistics. Over time NetPro will also publish its SDK such that other vendors and integrators will be able to extend the architecture to write and snap in additional management tools and utilities. This will make it much easier for many organisations to manage custom tools, scripts and batch jobs written for the purpose of administrating and automating the identity management infrastructure, and definitely tickled my interest. I shall be following up with Brad and share some further insight.

I’ve also taken a closer look at Centrify, after my colleague Martin told me to check them out, and he was right: I was positively impressed after talking to David McNeely, Centrify’s director of product management. He told me how Centrify’s DirectControl product seamlessly integrates Unix, Linux, Mac, Java and web platforms with Microsoft Active Directory. The product goes so far as to extend group policy objects onto those other platforms and allow for delegated administration. Another feature is zoning, which is used for two things: identity mapping from AD to the target system (so that my account “felix” on AD could, for example, be mapped to my Linux account “felixg” on the Linux development system, and to my accont “fga” on the production Solaris servers). Zones can also be used to manage granular access permission on specific sets of machines (like the “sudo” command on many UNIX machines). A second product, DirectAudit, can provide a complete log of everything that a user does on a system – up to the point of being able to replay an individual session like a VCR. Although I can understand the requirement for such a detailed audit on a highly sensitive system, I actually found it kind of scary from an old system administrator’s point of view.

Monday was also the day of the Directory & Identity Experts Panel Discussion, in which I had the privilege of joining Joe Long and Robert DeLuca from Microsoft, Kevin Kampman from the Burton Group, Gil Kirkpatrick from netpro, Laura DiDio from the Yankee Group and Christopher Voce from Forrester. Joshua Hoffman from TechNet magazine chaired the panel and opened up with a few questions before opening the flood gates to the audience. Joe was definitely in the front line of fire, being barraged with many questions with regards to when Microsoft would finally support SAML 2, SPML, virtual directories and other things that Microsoft doesn’t really seem to want to get its hands dirty with, at least at this time. I certainly felt sympathy, but he did a good job of defending Microsoft’s position. I got my share of questions as well. I have to admit that I was a bit nervous in the beginning, and in hindsight might have done a bit better with the first question about where I see OpenID in two years. But I think I did a pretty good job on the other questions of whether LDAP will be replaced by something else, and what needs to be done in order to enable applications for federation. After the expert panel, many lively discussions in the hospitality suites, and their aftermath! A perfect first conference day, and I collapsed happily into federated DreamSpace.

I am at Netpro’s Directory Expert Conference in Chicago this week, and very excited to be here! I’m keeping my eyes and ears wide open for the latest tech and trends around Microsoft AD and Identity Management, and also participating at an experts panel this afternoon. Knowing that DEC is an action-packed event, I came a day early, and it was well worth it. Sunday’s ramp-up to DEC 2008 was a pre-conference workshop on Microsoft Identity Lifecycle Manager (ILM) 2 beta, Certificate Lifecycle Manager (CLM), Active Directory Federation Services (ADFS) and Active Directory Rights Management Service (ADRMS). It was a hands-on lab experience given by David Lundell from the Oxford Computer Group, who did a brilliant job putting it together.

Microsoft’s vision is to have Directory Services in the centre of a comprehensive infrastructure that supports Identity Lifecycle Management, Strong Authentication, information protection and federation. Harnessing the tools presented in this workshop, one can see where this is going. Although some of the components (specifically ILM 2) are still in beta and not expected to be released until the “second half of 2008”, the picture may still be a bit rough and blurred, but one can see that it will be quite a beautiful one, once completed.

I was particularly impressed by Certificate Lifecycle Manager (CLM), an add-on to ILM that facilitates string authentication, specifically in the area of smart cards. It seems that Microsoft has managed to add significant value to an area that is often notoriously difficult for many enterprises to implement. Starting with an abstraction layer to the underlying card’s hardware stack to a comprehensive lifecycle implementation, CLM supports the full work-flow of the whole lifecycle of issuance, PIN reset, revocation and retirement. Self service is of course part of the offering and is streamlined for efficient and secure management from initial issuance to retirement and secure recycling. Just like the Dot Net Factory, Microsoft is harnessing the new Windows Workflow Foundation for all of its workflow management. For data flow, uses its MIIS meta-directory technology.

Just before the session closed, Microsoft’s Bobby Gill gave us a “sneak peak” of some additional features of ILM 2 beta 3 “hot off the disk” that he compiled a few hours ago. It is obvious that many significant enhancements are still being made, and Microsoft is very actively involved with its beta partners to collect their feedback and make improvements before the official ILM 2 is released.

Back to keeping my eyes and ears open, and I shall be back soon with some more news from DEC 2008!