Traffic jams (part I)
by Deborah Volk on March 19th, 2009

As you may or may not know, many operations in Oracle Identity Manager (OIM) become messages that are processed by the Java Message Service (JMS) container. The container resides inside the application server and is usually referred to as messaging service. In particular, all reconciliation events, raised requests and audit events are turned into JMS messages.

Messages are processed by the JMS container using a queue. A queue works just like a pipe: if 5 messages are pushed onto the queue, the first message out of these five to enter the queue is also the first message to come out of the queue on the other end. Messages are pushed into a pipe by producers. OIM hides the producer/consumer machinery from you but there are many producers underneath the hood, some coming from the web UI, some from the Design Console, some are driven by scheduled tasks and so on. Messages are removed from the queue by consumers. Consumers are worker bees - once they grab a hold of the message, they process it by extracting the message content (headers and body) and acting on it. In versions prior to 9.1, OIM used a single queue for ALL message types which made it challenging to prioritize and tune the messaging service inside an application server for a specific category of messages. Starting with version 9.1, OIM comes with separate message queues for each class of messages , one queue for messages that get generated from reconciliation events, another queue for messages from attestation, another queue for audit, and a few other queues.

Consider what happens when producers generate messages faster than consumers can handle them. That is, producers work very fast and push a ton of messages into the pipe yet consumers are slow because each message on the consumer side is involved in an expensive transaction with a 2-phase commit. The pipe fills up and this creates a backlog. Messages are in a pending state - they're sent but not yet received, they're waiting in the pipe. A backlog in a queue is a natural state so by itself it's not a cause for an alarm. Only if the backlog grows exponentially, a server administrator should take action. It could be surprising to many how quickly this backlog could grow under certain scenarios.

For example, consider a company that runs a marketing campaign with a goal of getting the customers to register in the company's brand new customer service portal. The campaign sends out a large number of emails and the marketing message is very popular, it offers a coupon in exchange for registration. Lots of users click on a "Register" link in the email and the registration is implemented as a web page that talks to OIM and submits a reconciliation event to attach a "Portal" resource to an already existing customer identity. This scenario could result in an extreme degradation of performance that might eventually end up in a loss of data.

To be continued...

Posted in Oracle Identity Manager    Tagged with jms, messaging, performance, queues, oim


Leave a Comment

2012 (1)
2011 (2)
2010 (2)
2009 (64)
March (11)
April (18)
May (18)
June (4)
July (1)
August (1)
September (5)
October (5)
December (1)