I just came back from this year’s Expert conference, TEC 2009. Last year it was still called the “Directory Expert’s Conference” (DEC). This year the conference has been extended to include training on Microsoft Exchange as well, hence the name change. And of course not to forget that Quest has taken over Netpro – but has this really changed the scope or focus of TEC? Not at all, as was very immediately visible from the start, with a very funny introductory video. It started off just like a very glitzy marketing presentation that turned quickly into a hyperbole of fuzzy marketing buzzwords and photos of smiling executives. The initial bemusement turned into bewilderment, and quickly I could see some rolling eyes and frowns around me, just when the marketing fuzz stopped right in the middle of it, and into the video stepped the image of Gil Kirkpatrick, DEC’s founder and Quest’s Chief Architect who, looking annoyed, asked the marketing voice what all of this was about. Nothing at TEC was going to change from what DEC was – this was no marketing trade show, but rather a place for people to learn and exchange experience about Microsoft products – specifically Active Directory and Exchange. The video then stopped to make place for the real Gil Kirkpatrick coming on stage to a big applause and delivering the welcome speech.

As a sign of the times, the conference was somewhat smaller as last year – the organisers spoke about a difference of about 30% of attendees compared to last year’s DEC. When Gil asked the audience who had to jump through extra hoops to get to TEC, several hands flew up. Those who went however, had an excellent, varied and carefully balanced programme waiting for them. As with all conferences, it can sometimes be a challenge picking a presentation to go to from multiple presentations going on at the same time. I was ver pleasantly surprised to see that some key presentations were given more than once so that I could attend them even though I had missed them the day before. Also, presentations were recorded this time and will soon be made available to attendees which especially for me is an additional value.

The “day before” – i.e. Sunday, several pre-conference workshops had already been given. This was a tough decision for me, as I was torn between going to Laura Hunter’s workshop on ADFS and Bahram Rushenas’s workshop on codeless provisioning with ILM 2. I chose ILM and the workshop turned out to be very informative, as it gave me a very good glimpse into codeless provisioning with ILM. I still felt sad to have missed Laura’s ADFS workshop that has received high praises (which did not surprise me as Laura is an passioned expert on this topic, as well as a gifted speaker). But one can’t have everything!

The second workshop was again on ILM. Dave Lundell, a DEC veteran and one of the most knowledgeable sources on ILM that I have met to date, presented on the topic “Taming the Chaos – Building a Practical Lifecycle Mgt. Application in the ILM “2” Portal”. I knew it was going to be good because I already attended (and raved about) his ILM 2 workshop last year at DEC. This one turned out to be a truly wild ride! Dave and his colleague Brad Turner from Ensynch pushed the envelope by demonstrating what I’ve often heard but never really seen “in action”: that ILM 2 is more than just a provisioning tool, but in fact a whole platform that allows all kinds of lifecycle management for enterprise data. He took an excellent example out of the world of enterprise IT: the management of an OID (Object Identifier) management. Enterprises can receive an OID tree within the “private enterprise” branch by requesting it from IANA. This OID tree can then be used to number enterprise-specific schema extensions, SNMP objects and other things that need an OID and are used within an enterprise. The OID space should be properly managed in order to give it the correct structure and making sure that no OID is assigned twice. This unfortunately is very rarely done in any enterprise – perhaps because of its technical nature and because the negative effects are usually not visible immediately when the OID tree space not managed properly – and there are few who “do it right” and properly manage their OID space. Dave and Brad showed how to implement OID management with ILM 2. This was very interesting because it gave us participants a deep dive into the guts of ILM 2, its data structures and workflow possibilities. It also really pushed ILM 2 to its current limits. Ensynch has written several custom workflows and contributed them via the codeflow web site in order to get around some current limitations in ILM 2. Those guys continue to amaze me.

Of course, the news about Microsoft’s delaying ILM 2′s official release for a whole year put a bit of a damper on the party. Disappointment was tangible from customers and vendors alike. I can certainly understand that although ILM 2′s maturity has evolved since last year, Microsoft wants to play it safe and gain some more experience with deployments, and iron out some kinks that are still present in the current beta version. That however doesn’t help those partners of Microsoft who have made a significant investment for ILM 2′s supposed imminent release. Gemalto for example, was poised for a big launch and threw a big party that, well, was still a great party although with excitement rather muted because the cause for the celebration was gone. Attendees were also very disappointed, many of them having come to TEC specifically for the purpose of sharpening their skills in order to prepare for an imminent deployment of ILM 2.

But back to positive aspects of TEC 2009, which were many – an you obviously can’t blame Quest or TEC for Microsoft delaying ILM 2! The first presentation I went to was Brian Puhl presenting on his experience over the last few years rolling out federation agreements. As one can expect from Brian, it was interesting, funny and thoughtful. Of the lot of information provided I especially liked Brian’s experience with the entirely non-technical problem around creating trust agreements – and the multiple iterations of procedures that Microsoft went through until they had a model that actually works. In the beginning, there was the list of the “10 commandments” – you shall do this, you may not do that, and you must do it like this, and so on. The resulting list was probably bullet proof from the standpoint of mitigating every conceivable risk, but turned out to be so draconic that nobody, not even Microsoft’s departments could comply with it. The next iteration was an extensive questionnaire about the state of security and management of identities that a partner had to fill out. The problem there was that many partners certainly did not want to divulge all this information about their internal controls and security subsystems that they thought were confidential. The next iteration then was a definition of a lowest common denominator “bar” that a partner had to jump over in order to qualify for federation. Three “bars” were defined with diffierent classifications for non-critical, medium-value and high-value and confidentiality content. To qualify, a partner had to vouch that certain criteria were met. Each criteria then had a point score, and the resulting total score would determine which “bar” the customer had reached, and hence qualified for within the federation agreement. This turned out to be very workable.

Another TEC-veteran is Pamela Dingle, formerly of Calgary-based Nulli Secundus Identity Management consultancy. Pamela has just flown the coop and started a company called “Bonzai Identity” with the goal to help enterprises get to grips with identity management by carefully nurturing good practises, aligning business processes, making sure that data is correct, and helping organisations make the “right decisions” over time. She writes that “It is like gardening; you will have much better luck making small adjustments throughout the life of your garden than you will allowing a wilderness to grow and then wading in with a machete”. Her talk at TEC was entitled “A survivalist’s guide to identity management” and focused on the business process shortcomings and warnings signs that can really bog down identity management projects. A great overview and invaluable compilation of experience that can avoid very costly traps and maximise the value of those projects.

TEC is legendary for bringing out the best of Active Directory experts and get not just best practises from the real pros, but also hard-core technical info that you can’t find in other places. There is a gang of “usual suspects” whose presentation I always try to attend because it doesn’t get much better than that when you want to learn about Active Directory and dive deep into the technology. Apart from Brian Puhl, who is responsible for running AD in Microsoft’s IT department, there are Laura Hunter, Joe Kaplan and Dmitri Gavrilov. Interestingly enough, those AD Gurus have become quite turned on by ADFS and federation, and (except for Dmitri) presenting on that topic.

This has been the first time I’ve had the honour to speak at this TEC, and even twice! My first presentation was on the subject of authorisation: once you’ve authenticated the user, then what? How do, can and should applications decide how to allow (authorise) a user to do and see things? It is a subject that I’ve focused on quite a bit over the last months and something that I am dedicating a whole track to on May 6th at our European Identity Conference in Munich. I couldn’t help feeling that this particular presentation was a bit of an “odd one” at TEC, because I unfortunately could not just yet teach people how to use technology to do it: We are still early in the game because big vendors such as Microsoft and Sun have yet to commit to standards in this area, come up with frameworks and stipulate good practises. It’s not completely satisfying when at the end of the presentation you have illustrated the problems and pain, but can’t really point to a solution yet. However I see encouraging signs that vendors are taking this seriously and thinking about ways to tackle these problems. It is not just a lack of technology, but the fact that, well, there certainly is a lack of standardised technology and the current “best practises” that encourage application developers to just hardcode security into their applications just exacerbate the problem. I would obviously like to see more interaction between the vendors instead of everybody just thinking within their own box. At our European Identity Conference I am bringing some of the thought leaders, visionaries and experts together and will try to rally them into working together to find solutions together as an industry.

My second presentation was on the TEC’s equivalent of a “Friday afternoon” – on the last day of the conference shortly before lunch. I was very excited about the topic because I was presenting about “Cool LDAP Innovations”. As TEC is about Active Directory I thought it was important to share a different perspecitve on what is happening outside of AD with other directory servers. Since AD world is essentially closed (you can’t rip out AD from a windows network) there is no competition in this space, and in my opinion very little innovation. Compared to other directory servers, AD and ADAM has fallen behind in technology, so I felt a bit tongue-in-cheek, talking about some cool stuff that other vendors were doing. The evening before I managed to itercept Nathan Muggli and asked him if he’d attend, and he kindly did. I finished early and a lively discussion started. After a few minutes I was delighted to see the whole thing starting to look like a BoF session and I decided to sit down in the middle with the other participants and we continued disussing.

Kevin Kampman from the Burton Group (technically a competitor, but I prefer to see him and his co-workers as distant colleagues) gave a presentation entitled “the case for identity services”. Out of the pain points that he highlighted I could identity the same ones I talked about in the “authorisation” presentation the day before. It’s great when a smart experienced guy like Kevin arrive at the same conclusion – it means that we definitely have a case!

I’ve had to scramble after Kevin’s presentation, grab a quick lunch and then hop into the car to drive back to Los Angeles where I came from this time. I had thought that the drive through the desert would have been more exciting, but I’ve since been told that for things to get spectacular, Death Valley or Arizona would be the best option (both close, but I didn’t have time for the detour). Just having gotten back to Europe this morning, I am still thinking back about this intense and englightening experience and am definitely looking forward to the next one!

I’ve recently been to Sun’s directory labs in the the beautiful city of Grenoble, France to talk about what Sun has in store with their two directory servers: DSEE and OpenDS. I’ve used many predecessors of DSEE (starting with the good old Netscape Directory Server) on several projects over the last decade and used to know it inside out. I’ve grown quite fond of it, and so has everybody else I know who has used the product. I wasn’t exactly sure why Sun embarked on its OpenDS project. Why reinvent from scratch what is already a perfectly great product? This question was on my mind, and I was eager to find out why.

When it comes to directory servers, most analysts like to classify them according to the market segments they address. In no particular order, they are: operating system/network, telco and service provider, enterprise and embedded. When it comes to the operating system/network directory servers, Active Directory rules – not necessarily because it is the best for this purpose (and just to be clear: it’s not bad either!), but – well – it’s so intrinsically linked to Windows that you don’t really have a choice. When Novell Netware was around, NDS and e-Directory was another candidate in that area, but it’s pretty much down to AD at this point in time. It’s in the other segments where it gets really interesting because there is some very active development and strong competition.

The Telco/Service provider directory segment is particularly interesting because only the highest scalable directory servers can even attempt to survive in this area. Sun has been very strong in this area for many years, and for a good reason: experience and continuous improvement. I’ve been involved first hand in several very large deployments of Sun Directory Server 5.0 (I think it was during the time when Sun called it “iPlanet Directory Server”). At that time, in the early years of this millennium, we deployed the server for hosting several hundreds of millions of entries. Yes indeed, about 120 Million entries! This was 2002, and at the time the sheer scale was pushing the envelope quite a bit -  but it didn’t just work, it actually worked quite well! Performance, Multi-master replication, and resilience were absolutely key for these types of installations. And sure – in the early versions of 5.0 there were some kinks that had to be ironed out of the replication protocol, but even then it was quite amazing how scalable the directory was, and how well it could actually be managed with such an impressive number of entries. Over the last 7 years, the directory server evolved even further – multimaster replication is rock solid and Sun has tinkered continously with the software to increase scalability way beyond what was already impressive in 2002. Nowadays, there are quite a few reference customers who run Sun directory server with literally billions of entries (incidentally, many of them in China – why am I not surprised ), and this is considered perfectly normal.

When it comes to reliability, a key to deploying very large directories is redundancy, and the possibility to balance loads and fail over between multiple instances. In the early days, load balancing appliances were used to do this (Alteon was really good at this in its days), but unless those applicances had specialised proxy features to handle the instrinsics of the LDAP protocol, this by itself wasn’t a very good option for large deployments. Sun had acquired a company called Innosoft a decade ago, and with it came a product called “DAR” – Directory Access Router – a fully fledged LDAP proxy. Over the years, Sun has enhanced DAR and bundled its next generation into Directory Server (now known as “DSEE”, Directory Server Enterprise Edition”) at no additional cost. Being an important cornerstone of very large and complex directory deployments, it fits like a glove into the directory service and extends it by offering extensive request routing functionality, high availability and performance features and simple mapping features. Previously, only the CA eTrust directory had these features.

I can talk all day about deploying telco directory services, because I’ve used to do it for a living, and am still fascinated by the sheer volume and raw power involved But there’s another two very glorious aspects of directory services, and they can be found in the enterprise and in the still fairly recent embedded directory segment.

The enterprise directory segment is where most of the innovation is happening. Enterprises are typically not as focused on performance, and often more interested in integration, security and manageability. Integration is a very big topic, because the directory service is a crucial piece in any identity management infrastructure. And we’re usually not talking about “a” directory either – most enterprises have many different directory servers, containing either different user populations, or part of the same users but for different purposes. It is in the integration area where much innovation has happened in the directory area. Is doesn’t surprise me that most enterprise directories nowadays feature simple virtual directory functions. That was not the case five years ago, when I worked for a virtual directory vendor. At that time directory service vendors did not foresee virtualisation features as being an important part of their portfolio – perhaps because some of those vendors were also selling an “identity manager” type provisioning system and thought that any directory integration could be solved by deploying a full-blown provisioning system and brute force copying data around Well, this wasn’t really a feasible solution in all cases, so it is only natural that virtual directory companies such as OctetString and Maxware were acquired, and other vendors are “rolling their own” virtualisation features.

Some of the features that are not obvious, but extremely useful in the enterprise scenario are exactly those that allow a directory server to easily interoperate with provisioning, virtualisation and synchronisation products. Technically, the features in LDAP server that are relevant here are persistent queries, incremental updates and proxy auth. These are low-level features that are absolutely crucial when identity “managers” and provisioning services interface with directory servers.

Some other desired features within the enterprise directory segment are about password services and policies. In the vast list of featureds to be found in most modern directory servers are sophisticated access control lists that are expressive enough to configure a finely grained access control policy for deciding who gets access to what type of information. This used to be very important in the past but is getting less important as access control rules on the directory servers tend to be simpler nowadays, because changes typically ocurr through provisioning systems, and not that much any more directly to the LDAP server. Password policies are also a typical feature used in enterprise directory servers (you know – minimum length, character combination, auto-lockout,auto-expiry, and all those things). And of course, keeping track of when users last logged on – very helpful in order to identity dormant accounts.

Another important detail is also how passwords are stored, and how they can be migrated from one server to the other. As a general rule, it’s always good to offer administrators choice. Obviously passwords need to be well protected. But the approach of some directory vendors (specifically Microsoft and Novell) to “secure” their directories has backfired – the directory servers hoard the passwords and don’t even offer any possibility for administrators to export encrypted password hashes. You may wonder whether this “secure” feature is actually a hidden “lock-in trap”! That has created a secondary market around password “synchronisation solutions” in order to overcome the deficiency in the product itself, where the product’s designers thought they had to be smarter than the poor administrators who actually need to deploy, migrate and maintain them.

Last but not least, let’s not forget about one of the very important aspects of enterprise directory services. They need to be simple to deploy, administer and maintain! In the telco area it may be considered acceptable if the directory administrator team features several fully trained relational database administrators, but in enterprise environments that can be too much overhead. Directory servers that make use of relational databases for storing their directory data, such as Oracle’s OID and IBM’s Tivoli Directory Server can point to the advantages of running a directory services platform on a rock-solid database foundation (in these cases, Oracle and DB2 respectively). But the extra administration overhead can be considerable. CA has traditionally used the Ingres relational database for its eTrust Directory Server, but has now in the latest Version 12 switched to something called “DXgrid” – a revolutionary internal memory-mapped storage that not only offers incredible throughput, but also eliminated a significant portion of administration. Sun has since always used a simpler, but very fast and highly scalable data store for its directory server called BerkeleyDB – the same used also in most installations of OpenLDAP.

After mumbling on for quite a discourse I actually wanted to get to the point of Sun’s OpenDS, and the question that I wrote in the beginning of this entry. Why reinvent from scratch (OpenDS) what is already a perfectly great product (Sun DSEE)? As it turns out, there’s been a new segment for directory server that is steadily growing: the one of embedded directory services. For example, packaged solutions that require a directory server internally. Or “black box” appliances with a provisioning interface that contain – guess what – a directory server. A few years back, it was OpenLDAP that was typically shipped with those solutions, because it was free, open and could be embedded easier than other full-fledged directory server products. Now it is OpenDS that is continuously gaining ground, and for good reason. With its incredibly easy set-up, minimal administration, OpenDS epitomises what an embedded directory stands for. And on top of that, the scalability and performance are world-class. Development on OpenDS is, as the name implies, well – open. The development team features Sun employees and others outside Sun, just like OpenSSO. The release cycle is short and new features list is growing at an incredible rate.

So will OpenDS one day replace DSEE? Most likely. But this is still far in the future – for the next few years Sun is actively investing in DSEE as its flagship directory whilst continuing to nurture OpenDS and offering it as an embedded directory server, as well as to anyone interested in quickly deploying a directory server. Now, when I say “quickly” – I’ve managed to install it, extend the schema and load some data into it in less than fifteen minutes! Now that’s what I would call “quickly”. And once I had it up and running on my slow and overloaded laptop, I ran the “slamd” LDAP benchmark tool against it on the same laptop, and got back thousands of searches per second. Not bad at all! Now that’s what I call innovation in the world of LDAP

I’ll be speaking at TEC on Wednesday the 25th of March, on the topic “Cool LDAP Innovations”. OpenDS will definitely get a mention. On the presentation, I’ll also talk about some other real innovations that happened over the last few years in the directory services area. If you’re there, be sure to drop by!