Generic Connector and the Temple of Doom

by Deborah Volk on May 1st, 2009

The City of Atlantis. The Holy Grail. The Philosopher's Stone. The Perpetual Motion Machine. The Generic Identity Management Connector. This is the stuff of legend with folk tales reverberating through the ages. (The rise of scientific method during Renaissance with empirical evidence as a way of learning? An early example of reconciliation). Building connectors for customers and trying various frameworks as a way of decreasing the cost of creating a connector, I've been thinking about the notion of a generic connector
Connectors have always been the bane of any identity management solution. Without a way to understand or influence external (target) systems by, well, connecting to them, the business value of an identity management solution amounts to little. Therefore, connectors are a must-have...but the number of target systems found in the wild across all customers is large so it becomes impractical to build a connector for each individual target. This is where the analysis of what makes a connector tick becomes interesting.

Each target system is indeed different but there are degrees of difference. The difference between the two systems could be very small or very large. For example, two Oracle databases with the same structure are two different target systems but the only difference between them is the server they're hosted on. This is a small difference. Another example is having two databases, one a relational database such as Oracle and one a hierarchical database such as IMS. Being able to connect and retrieve data from a relational database will get you nowhere when you try the same approach on a hierarchical database. This is a large difference. How can we build a connector (with or without an underlying connector framework) that would allow us to quickly adapt it to a variety of systems while minimizing the effort having to do with differences between these systems? This would be our generic connector, the One-Size-Fits-All-And-Cooks-You-Breakfast Connector.
Thor Tech engineers working on Xellerate product (later acquired by Oracle and rebranded as Oracle Identity Manager) had a Eureka moment. Instead of tackling a problem of identifying all possible differences between target systems and building connectors that encompassed these differences, they partitioned the problem into two halves. One half would deal with procedural differences and the other half would deal with a technical specification having to do with a low-level integration details and protocol(s) for talking to the external system. A procedural difference can be illustrated when creating a user account in LDAP-compliant directory with one directory being Oracle Internet Directory (OID) and the other directory being Active Directory (AD). Even though in both cases we've got LDAP protocol as our "neutral" communication and data/command description substrate, the steps for creating an account in AD and OID are not the same since the container hierarchy in two directories and the placement of entries in the hierarchy are wildly different.

Having the problem consist of procedural and integration steps allowed Thor engineers to allocate these steps to two different components in Xellerate. The integration logic and all low-level technical specification details for talking to another system would live in Java code. The procedural logic having to do with needs of a particular workflow could live in the Java code or it could be be built in a visual composition environment proudly called the Adapter Factory. The idea behind Adapter Factory was that someone who knew the target system well but didn't know how to code (read: that elusive IT "business analyst") would be able to put together the steps for completing the transaction by clicking on various UI widgets. The Adapter Factory would then
generate the Java code underneath the hood. Here are the default (out-of-the-box) steps for creating account in OID as seen in the visual composition environment of the Adapter Factory:
The other Good Thing™ about the Adapter Factory was that each step in the above procedure could talk to another adapter or a pre-built piece of Java code (or a stored procedure or ...). This would allow organizations to use already written components for integrating with, say, Siebel or Peoplesoft and have the analyst configure the procedure (build the adapter in Adapter Factory) that used these components. Since adapters could talk to adapters, one could create reusable building blocks as small or as large as one would like. A small block could be creating an entry in LDAP, a large block could be creating a user account in Oracle database. This is an example of a Composite design pattern, taken to extreme by frameworks such as Qi4j and labeled as Composite-Oriented Programming.

On paper, the ideas and goals of Adapter Factory are fantastic. The implementation of these ideas is, unfortunately, a different story. It takes one some time to reach a comfort zone where clicking on this or that dialog in Adapter Factory becomes second nature. The visual composition UI is rooted in programming idioms closely resembling Java. While no programming is required, you have to understand the concepts of flow of control, loops, parameter passing and so on, few non-developer IT/business analysts would be able to handle this. The UI is slow and most people don't know how to achieve optimal results, creating poorly performing (slow or memory-hog or both) adapters. Since developers can't get the Adapter Factory to work they way they think, many OIM implementations end up with code, code and more code (unless you've hired Identigral, of course) . This drives up the implementation and maintenance cost.

If code generated by Adapter Factory coupled with pre-built or built-to-order code couldn't solve the Generic Connector Problem, maybe another approach would work. Oracle's solution was the Generic Technology Connector (GTC), a combination of architectural foundation/framework and (somewhat) generic connectors. The GTC framework allows developers to create reconciliation or provisioning providers for target systems. The GTC framework provides a standard roadmap and APIs for how providers should be developed; this fits well with the developer mentality. The GTC framework has a somewhat heavy footprint for small(er) tasks. If all you want to do is create a row in a database using a particular SQL expression that isn't supported by the out of the box database connector, you've got to do a fair amount of legwork.

Are there other approaches for genericizing a connector? Back in the olden days when identity management was still a nascent buzzword, provisioning was something out of a military jargon and reconciliation was strictly for accountants, Trulogica (acquired by HP in early 2004 for OpenView portfolio) tried to hang connectors on top of J2EE Connector Architecture (JCA) spec. For those not familiar with JCA, this was kind of like requiring a rocket engine for building a small model plane. Needless to say, this didn't work very well.

The most recent attempt at a generic connector solution is courtesy Sun. They've taken what I consider to be the most innovative approach with their Identity Connectors project. The project is an Open Source initiative whose goal is to produce connectors that can be easily decoupled from the applications using them, thus allowing easy replacement and substitution. Right now the connectors and the framework are very Sun Identity Manager-friendly, it would take some effort to make them work in a different identity manager product but the future is (potentially) bright.
Back to the original thought: Do you all think that it's possible to have a Generic Connector?

Posted in Identity Management, Oracle Identity Manager, Sun Identity Manager    Tagged with gtc, oim, sim, jca, adapters, connectors


alan lloyd - May 4th, 2009 at 5:29 AM
No is the short answer - simply because identity managed and engineered systems have to be governed by (for example) 3rd party service providers, self care, services managers and operations staff. Who gets at what via entitlements - which relate to the services of the system is critical as what happens under the systems business and liability rules is another.

Identity is relevant within a govenance context as is the entitlement.. So its not just a case of "connectors" and databases, its one of business and systems management.

hope this helps

regards alan
vinhant - May 5th, 2009 at 9:17 AM
The "generic connector" is a problem specific to OIM, not Sun IDM. Sun IDM already has the equivalent of a generic connector, which is a set of Java interfaces that all connectors must implement. It's a basic concept of object-oriented programming that the OIM architects forgot.

So now delivering a "generic connector" may be impossible because OIM did not abstract the connector from other entities (Resource Object, Process Definition, Form, etc.). These entities access the connector *directly*.

In contrast to Sun IDM, where entities never access the connector. It is completely abstracted via a "userview". Once the userview is saved Sun IDM takes care of calling the connector in the background.

Deborah Volk - May 5th, 2009 at 10:16 AM
Good point and I agree. GTC in OIM is a step in the right direction. (Note: the interface between the connector and OIM is not as wide as you write, only process tasks call adapters). My question is slightly different. Once you have a generic database connector (be it Sun or Oracle or whatever), will it work for all databases and all schemas out of the box?
vinhant - May 5th, 2009 at 11:43 AM
Resource Objects and Forms can call the connector via entities and prepop adapters. It's rare but sometimes needed to check something on the target system.

If we're talking about a simple database table then Sun has a connector that almost does what you are looking for. After configuring the database URL and which table to provision, Sun automatically discovers the table columns and creates an attributes schema for you. That's pretty much it, you are ready to go in terms of provisioning and reconciliation. So the answer is yes :)

I'm not familiar with the GTC so maybe it can do that too.

Deborah Volk - May 5th, 2009 at 11:55 AM
Even OIM has a simple database connector that will do a row. The idea is what if the row had to be filled in via a complex join, would the adapter work out of the box or would someone be writing some new java code? Rarely are adapters written with enough flexibility to account for everything, which is where a good framework is a fallback position. But a framework is NOT a generic connector.
vinhant - May 5th, 2009 at 12:31 PM
Of course if it's anything more than a simple table somebody has to tell the product how to create or reconcile the accounts. In a ideal world the adapter would magically discover how to do that. But then consultants would be out of work :)

Again the GTC/generic connector/framework solves a problem specific to OIM. Sun does not have this problem.

In the complex database example above one would use the Sun IDM "Scripted JDBC" connector to provision the database. As the name implies it uses scripted Java (BeanShell) to provision the database.
Deborah Volk - May 5th, 2009 at 2:28 PM
I don't see an architectural or principal difference between OIM's GTC and Sun's resource adapter API. In both cases you write Java code that implements interfaces. In Sun's approach the adapter is responsible for doing a lot more than just talking to the external system, it has XML (and methods) to deal with the entire metadata set for describing the resource and associated behavior. In OIM, the connector is narrowly defined, the resource, form (attributes) and other metadata types are not part of it. I think Sun's approach is heavy-handed even if it's oriented toward developers. It seems that Sun recognized this, Sun's Identity Connectors project has an SPI and a toolkit that is much cleaner than what I've previously seen.

Leave a Comment


follow on

2012 (1)
2011 (2)
2010 (2)
2009 (64)
March (11)
April (18)
May (18)
June (4)
July (1)
August (1)
September (5)
October (5)
December (1)