A handy reference for Oracle Identity Manager 11g (R2 PS2)

Folks who are comfortable with the OIG stack (OIM to be specific) have most of the info, not-so-basic info, tricks etc up their sleeves – always ready to go! This is not always the case though.

I have created a simple set of curated links which can serve as a handy go-to reference for quick access.

oim-zeef

Sneak Peek

 

Applicability

  • When you’re looking for something specific – you know you saw it in one of the guides but just not able to dig it out !
  • There is a requirement which needs you to explore the product features/capabilities to ensure whether it is catered to OOTB
  • You’re new to the OIM ecosystem all-together
  • etc etc etc . . .

You might want to start with the new features or dive into product architecture etc. . . Feel free to suggest links to new information which you might find  :-)

Have fun !!

Posted in Oracle Identity Governance, Oracle Identity Manager | Tagged , , | Leave a comment

JVM PermGen – where art thou?

This post covers some basics of JVM memory structure and quickly peeks into PermGen to find out where it has disappeared since advent of Java SE 8

Bare Basics

The JVM is just another process running on your system and the magic begins with the java command. Like any OS process, it needs memory for its run time operations. Remember – the JVM itself is a software abstraction of a hardware on top of which Java programs run and boast of OS independence and WORA (write once run anywhere)

Quick coverage of the JVM memory structure

As per the spec, JVM is divided into 5 virtual memory segments.

  • Heap
  • Method (non heap)
  • JVM Stack
  • Native Stack
  • PC Registers

 

jvm-memory-segments

 

Heap

  • Every object allocated in your Java program requires to be stored in the memory. The heap is the area where all the instantiated objects get stored. Yes – blame the new operator for filling up your Java heap ;-)
  • Shared by all threads
  • The JVM throws java.lang.OutOfMemoryError when it’s exhausted
  • Use the -Xms and -Xmx JVM options to tune the Heap size

 

out-of-memory-error

 

Sub-divided into

  • Eden (Young) – New object or the ones with short life expectancy exist in this area and it is regulated using the -XX:NewSize and -XX:MaxNewSize parameters. GC (garbage collector) minor sweeps this space
  • Survivor – The objects which are still being referenced manage to survive garbage collection in the Eden space end up in this area. This is regulated via the -XX:SurvivorRatio JVM option
  • Old (Tenured) – This is for objects which survive long garbage collections in both the Eden and Survivor space (due to lingering references of course). A special garbage collector takes care of this space. Object de-alloaction in the tenured space is taken care of by GC major

Method Area

  • Also called the non heap area (in HotSpot JVM implementation)
  • It is divided into 2 major sub spaces

Permanent Generation – This area stores class related data from class definitions, structures, methods, field, method (data and code) and constants. Can be regulated using -XX:PermSize and -XX:MaxPermSize. IT can cause java.lang.OutOfMemoryError: PermGen space if it runs out if space

Code Cache – The cache area is used to store compiled code. The compiled code is nothing but native code (hardware specific) and is taken care of by the JIT (Just In Time) compiler which is specific to the Oracle HotSpot JVM

JVM Stack

  • Has a lot to do with methods in the Java classes
  • Stores local variables and regulates method invocation, partial result and return values
  • Each thread in Java has its own (private) copy of the stack and is not accessible to other threads.
  • Tuned using -Xss JVM option

Native Stack

  • Used for native methods (non Java code)
  • Per thread allocation

PC Registers

  • Program counter specific to a particular thread
  • Contains addresses for JVM instructions which are being exceuted (undefined in case of native methods)

So, that’s about it for the JVM memory segment basics. Coming to back to the Permanent Generation.

So where is PermGen ???

Essentially, the PermGen has been completely removed and replaced by another memory area known as the Metaspace

Metaspace – quick facts

  • It’s part of the native heap memory
  • Can be tuned using -XX:MetaspaceSize and -XX:MaxMetaspaceSize
  • Clean up initiation driven by XX:MetaspaceSize option i.e. when the MetaspaceSize is reached.
  • java.lang.OutOfMemoryError: Metadata space will be received if the native space is exhausted
  • The PermGen related JVM options i.e. -XX:PermSize and -XX:MaxPermSize will be ignored if present

This was obviously just the tip of the iceberg. For comprehensive coverage of the JVM, there is no reference better than the specification itself :-)

(Additional note – thanks to inputs by Jon Diamond)

Note: The JVM specification itself is a standard for vendors (who implement the spec to build their own JVM) to follow. It should not be used to figure out vendor specific features (e.g. SAP JVM might implement a spec feature differently than that of the Oracle HotSpot VM). The JVM specification provides scope for innovation by not enforcing each and every minor details w.r.t implementation. In fact, we are living in the age of Polyglot Languages – which compile to byte code supported by the JVM (Scala, Groovy, JPython, JRuby, Kotlin etc). But for most of us out there – it can be leveraged to gain a solid understanding of the Java fundamentals ranging from bytecode to class loading process

 

You can also explore

Cheers !!

Posted in Java, JVM | Tagged , , , , , | 2 Comments

Oracle IDM: Notifications

Notifications are an important functionality within Oracle Identity Manager

notification

A slide share is worth more than a huuuge blog post. . . some times ! ;-)

Posted in Oracle Identity & Access Management, Oracle Identity Manager | Tagged , , | Leave a comment

EJB 3.x : Lifecycle and Concurrency models (part 2)

This is the second post of the two part series. The first part covered the life cycle and the concurrency behavior of Stateful and Stateless EJBs. I’ll cover Singleton EJBs in this post

The Singleton pattern is arguably the most used (some times misused ;-)) pattern out there.

singleton1

Single-ton and loving it !

Java EE frees us from writing explicit code (like one on the above picture) to implement the Singleton pattern. Singleton EJBs were introduced in EJB 3.1 which itself was part of Java EE 6. All that’s required is a @javax.ejb.Singleton (class level) annotation (and a few more if you want to refine other aspects – read on) on a bean class to designate it as a Singleton session bean.

There is one and only one instance of a Singleton EJB in a JVM – no matter how many clients access it. Its not like Stateful SB –  one bean instance attached to a single client throughout its life cycle, neither like Stateless SB – a new instance for each client request

What are the distinct states in the life cycle of a Singleton Session Bean?

The life cycle for Singleton beans is the same as Stateless session beans – in fact it is one of the simpler aspects of this bean type

  • Does Not Exist
  • Ready

How do the states change? What triggers them?

Here is a quick tabular snap shot and a high level diagram . . .

 

singleton-lifecycle

Singleton Beans – State Transitions

 

State Transition Triggers Callbacks
DNE to R When the instance is first accessed via JNDI/DI or automatically instantiated by the container using the @Startup or @DependsOn @PostConstruct
R to DNE Container shuts down – destroys the bean instance or in case an Exception occurs in the @PostConstruct annotated method @PreDestroy

Note: DNE – Does Not Exist, R – Ready

As stated earlier, life cycle is one of simpler features of Singleton beans. It’s critical to understand their concurrency aspects.

Singleton Session Beans: Concurrency Management

As stated – a Singleton has just one instance in the JVM. In a Java EE environment, concurrent access is inevitable – that’s why we are using a technology like Java EE in the first place ! ;-) One needs to make sure that the concurrency (locking) strategies w.r.t Singleton beans are well thought through, depending upon the use case and requirements

singleton-12yr

Singleton – Consume With Care !

Singleton bean concurrency can be divided into 2 major categories

  • Container Managed (Default)
  • Bean Managed

Container Managed Concurrency

  • As the name suggests, the container applies sensible default configurations for the bean
  • Can be controlled using annotations as well as XML (deployment descriptors)
  • Explicitly declared using the @javax.ejb.ConcurrencyManagement annotation on the bean class itself
    • Default value is javax.ejb.ConcurrencyManagementType.CONTAINER
  • Two possible locking strategies provided by the container – applicable on both bean class or its individual methods
    • @javax.ejb.Lock with a value of javax.ejb.LockType.READ – allows concurrent access given no write locks
    • @javax.ejb.Lock with a value of javax.ejb.LockType.WRITE (Default) – guarantees exclusive access – only a single thread can execute a bean method at a given point
  • @javax.ejb.AccessTimeout can be specified on a bean class or method to ensure that a thread does not block or hold a lock for an indefinite time span

Bean Managed Concurrency

  • The name clearly indicates – the concurrency aspects of the bean are left to the developer. Makes sense when finer concurrency control is required as compared to what’s been offered by the container via aforementioned constructs
  • Usage of appropriate Java concurrency constructs required e.g. synchronized, volatile etc
  • Hard to get right !

Code example

Let’s look into a simple code snippet in order to better make sense of the above stated facts

Scenario one – Container managed concurrency (default, locking type not explicitly specified )


package com.abhirockzz.wordpress.ejb.lifecycle.singleton;

import com.abhirockzz.wordpress.ejb.lifecycle.stateful.MyStatefulBean;
import java.util.Date;
import java.util.logging.Level;
import java.util.logging.Logger;
import javax.ejb.Singleton;
import javax.ejb.Startup;

@Singleton
@Startup
public class MySingletonBean {

    public void act() {
        System.out.println("Entered MySingletonBean/act() on " + new Date().toString() + " . Singleton instance " + this.hashCode() + " Thread : " + Thread.currentThread().getName());
        try {
            Thread.sleep(2000);
        } catch (InterruptedException ex) {
            Logger.getLogger(MyStatefulBean.class.getName()).log(Level.SEVERE, null, ex);
        }

        System.out.println("Exit MySingletonBean/act() on " + new Date().toString() + " . Singleton instance " + this.hashCode() + " Thread : " + Thread.currentThread().getName());

    }
}

 


package com.abhirockzz.wordpress.ejb.lifecycle.singleton;

import java.io.IOException;
import java.util.Date;
import javax.inject.Inject;
import javax.servlet.ServletException;
import javax.servlet.annotation.WebServlet;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

@WebServlet(name = "SingletonTestServlet", urlPatterns = {"/SingletonTestServlet"})
public class SingletonTestServlet extends HttpServlet {

    public SingletonTestServlet() {
    }

    @Inject
    MySingletonBean mySingleton;

    @Override
    protected void doGet(HttpServletRequest request, HttpServletResponse response)
            throws ServletException, IOException {
        System.out.println("Entered SingletonTestServlet/doGet() on " + new Date().toString() + " . Servlet instance " + this.hashCode() + " Thread : " + Thread.currentThread().getName());
        mySingleton.act();
    }

}

 

Using Apache JMeter – I fired 2 concurrent threads at SingletonTestServlet (yes, just two.. this is more of a demonstration, not a load testing competition ;-) )

cmc-jmeter-1

 

cmc-jmeter-2

 

cmc-result

Observations

Looking at the logs, one can easily make out the following

  • The Servlet of course is not thread safe, hence two threads enter at the same time
  • One of the threads enters the method in the Singleton bean class (marked in red) and further access is forbidden due to the default WRITE lock type enforced by the container
  • As soon as the first thread finished execution, the second thread (marked in green) which was initially blocked, gets a chance to execute the Singleton bean method
  • Pretty simple!

Scenario two – Sticking with Container managed concurrency. Changing the explicit lock type from WRITE to READ

 


import com.abhirockzz.wordpress.ejb.lifecycle.stateful.MyStatefulBean;
import java.util.Date;
import java.util.logging.Level;
import java.util.logging.Logger;
import javax.ejb.ConcurrencyManagement;
import javax.ejb.ConcurrencyManagementType;
import javax.ejb.Lock;
import javax.ejb.LockType;
import javax.ejb.Singleton;
import javax.ejb.Startup;

@Singleton
@Startup
@ConcurrencyManagement(ConcurrencyManagementType.CONTAINER)
public class MySingletonBean {

    @Lock(LockType.READ)
    public void act() {
        System.out.println("Entered MySingletonBean/act() on " + new Date().toString() + " . Singleton instance " + this.hashCode() + " Thread : " + Thread.currentThread().getName());
        try {
            Thread.sleep(2000);
        } catch (InterruptedException ex) {
            Logger.getLogger(MyStatefulBean.class.getName()).log(Level.SEVERE, null, ex);
        }

        System.out.println("Exit MySingletonBean/act() on " + new Date().toString() + " . Singleton instance " + this.hashCode() + " Thread : " + Thread.currentThread().getName());

    }
}

 

What happens when the the application is bombarded (pun intended !) with 2 concurrent threads. . . ?

bmc-result

 

  • Two threads enter the Servlet at the same time – as expected
  • One of the threads enters the method in the Singleton bean class (marked in red)
  • The second thread (marked in green) also manages to enter the Singleton bean method at the same instant (check the time stamp)
  • Again – pretty simple! ;-)

Bean Managed concurrency is not something which I am depicting right now. As stated above, using BMC for a Singleton transfers the onus on to the developer and he is free to code concurrency features into the bean- this can be done simply using synchronized on each methods or other mechanisms e.g. from java.util.concurrent API

Suggested Reading

Cheers ! :-)

Posted in Java, Java EE | Tagged , , , , , | Leave a comment

EJB 3.x : Lifecycle and Concurrency models (part 1)

Java EE component life cycle and concurrency related details might not be new for seasoned professionals, but it is something which can take time to sink in for beginners.

concurrency

As far as EJBs are concerned, understanding their life cycle (and related concurrency scenarios) is extremely critical in order to ensure appropriate usage and solution design using EJBs. It’s easy to misuse them !

 

bean-lifecycle

Bean Life Cycle

 

I’ll quickly cover Stateless and Stateful beans in this post and skip Lima Beans for the time being ;-)

  • Stateful Session Beans – life cycle + concurrency handling
  • Stateless beans – concurrency model only, since I had briefly covered the lifecycle in one of my previous posts.

What are the distinct states in the life cycle of a Stateful Session Bean?

  • Does Not Exist
  • Ready
  • Passivated

How do the states change? What triggers them?

Here is a quick tabular snap shot and a high level diagram. For more details, read on . . .

stateless-ejb-lifecycle

Stateless Session Bean Life Cycle State diagram

Note: DNE – Does Not Exist, R – Ready, P – Passivated, SFSB – Statelful Session Bean

State Transition Triggers Callbacks
DNE to R When a SFSB instance is first accessed via JNDI or DI @PostConstruct
R to DNE Container shuts down, client invokes method annotated with @Remove, the bean reaches idle time out threshold designated by DD or @StatefulTimeout @PreDestroy
R to P The EJB container passivates idle beans and removes them from active memory based on specific algorithms @PrePassivate
P to DNE The bean reaches idle time out threshold designated by DD or @StatefulTimeout Note: @PreDestroy annotated method is NOT invoked
P to R When the client invokes a SFSB instance after it is passivated but has not timed out yet @PostActivate

Note: If a SFSB throws an exception during request processing, its instance is destroyed i.e. it goes to a DNE state. The @PreDestroy annotated method is not invoked in this case

Now that we have some idea about the life cycle of a SFSB, let’s try to take a look at how do these beans behave under load i.e. when the application is used by multiple users at a time which translates into concurrent access of SFSB instances

Stateful Session Beans: Concurrency Management

Thread safety is one of the core features of EJBs. The point to be noted is that this thread safety is free of cost and does not need any concurrency related constructs to be coded in by the bean developer himself (there are a few exceptions). As far as SFSB are concerned, the EJB container ensures that only one thread can access a bean instance at a particular time.

In this example, we are trying to simulate concurrent access to a single instance of a SFSB by invoking a test Servlet via JMeter. The Servlet injects the bean via DI and calls a method on it. The SFSB method just uses a Thread.sleep() to pretend as if it’s executing something.


package com.abhirockzz.wordpress.ejb.lifecycle.stateful;

import java.util.Date;
import java.util.logging.Level;
import java.util.logging.Logger;
import javax.ejb.Stateful;

@Stateful
public class MyStatefulBean {

    public MyStatefulBean() {
    }

    public void act() {
        System.out.println("Entered MyStatefulBean/act() on " + new Date().toString() + " . SFSB instance " + this.hashCode() + " Thread : " + Thread.currentThread().getName());
        try {
            Thread.sleep(2000);
        } catch (InterruptedException ex) {
            Logger.getLogger(MyStatefulBean.class.getName()).log(Level.SEVERE, null, ex);
        }

        System.out.println("Exit MyStatefulBean/act() on " + new Date().toString() + " . SFSB instance " + this.hashCode() + " Thread : " + Thread.currentThread().getName());

    }
}

 

 


package com.abhirockzz.wordpress.ejb.lifecycle.stateful;

import java.io.IOException;
import java.util.Date;
import javax.inject.Inject;
import javax.servlet.ServletException;
import javax.servlet.annotation.WebServlet;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

@WebServlet(name = "SFSBTestServlet", urlPatterns = {"/SFSBTestServlet"})
public class SFSBTestServlet extends HttpServlet {

    public SFSBTestServlet() {
    }

    @Inject
    MyStatefulBean mySFSB;

    @Override
    protected void doGet(HttpServletRequest request, HttpServletResponse response)
            throws ServletException, IOException {
        System.out.println("Entered SFSBTestServlet/doGet() on " + new Date().toString() + " . Servlet instance " + this.hashCode() + " Thread : " + Thread.currentThread().getName());
        mySFSB.act();
    }

}

 

sfsb-jmeter-1

HTTP GET request via JMeter

 

sfsb-jmeter-2

Concurrent request simulation via JMeter

 

Observations

  • Since a Servlet itself is not thread safe, multiple threads will in fact enter the doGet() method
  • A single instance of the SFSB (evident via the hashCode result) in being accessed concurrently (see the thread names in the logged statements)
  • Only one thread will be able to access the SFSB instance though – other threads wait for their turn while the SFSB method returns. This delay is noticeable via the log statements on the console

 

sfsb-result

Console logs

 

What about Stateless Beans?

These beans are inherently thread safe. Why? It is because by default, the container makes sure that each new request is served by a new instance of the bean. Remember, that a client can obtain a reference to stateless bean in 3 possible ways – DI, JNDI or via a remote interface (RMI). In all these cases, it is the container (proxy) which intercepts the call – thus, even if multiple threads are seemingly accessing the same bean instance, its actually not the same one ;-)


package com.abhirockzz.wordpress.ejb.lifecycle.stateless;

import java.util.Date;
import java.util.logging.Level;
import java.util.logging.Logger;
import javax.ejb.Stateless;

@Stateless
public class MyStatelesslBean {

    public void act() {

        System.out.println("Entered MyStatelesslBean/act() on " + new Date().toString() + " . SLSB instance " + this.hashCode() + " Thread : " + Thread.currentThread().getName());
        try {
            Thread.sleep(2000);
        } catch (InterruptedException ex) {
            Logger.getLogger(MyStatelesslBean.class.getName()).log(Level.SEVERE, null, ex);
        }

        System.out.println("Exit MyStatelesslBean/act() on " + new Date().toString() + " . SLSB instance " + this.hashCode() + " Thread : " + Thread.currentThread().getName());
    }
}

 


package com.abhirockzz.wordpress.ejb.lifecycle.stateless;

import java.io.IOException;
import java.util.Date;
import javax.inject.Inject;
import javax.servlet.ServletException;
import javax.servlet.annotation.WebServlet;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

@WebServlet(name = "SLSBTestServlet", urlPatterns = {"/SLSBTestServlet"})
public class SLSBTestServlet extends HttpServlet {

    @Inject
    MyStatelesslBean slsb;

    @Override
    protected void doGet(HttpServletRequest request, HttpServletResponse response)
            throws ServletException, IOException {

        System.out.println("Entered SLSBTestServlet/doGet() on " + new Date().toString() + " . Servlet instance " + this.hashCode() + " Thread : " + Thread.currentThread().getName());

        slsb.act();

    }

}

 

Observations

  • Since a Servlet itself is not thread safe, multiple threads will in fact enter the doGet() method
  • Different instances of the SLSB (evident via the hashCode result) are being picked by the container to manage concurrent requests (see the thread names in the logged statements).
  • In spite of concurrent requests, each request thread is getting serviced by a new instance
slsb-result

Console logs

 

That’s all for now! I am planning to cover Singleton Session beans in a future post. Stay tuned . . . .

Thanks for reading! :-)

 

Posted in Java, Java EE | Tagged , , , , , , , | 1 Comment

Book Review: Java EE 7 Performance Tuning and Optimization

This is a review of the book Java EE 7 Performance Tuning and Optimization

bookrevieew-javaeepertuning

What is this book about?

As the title clearly suggests, this book deals with Performance Tuning. It focuses on tuning, monitoring and optimizing the performance of Java EE applications in particular.

It starts off with basics of Performance Tuning and gives us an insight into the various kinds of performance related issues which one might face in the context of enterprise grade, distributed web (Java EE) applications. The book provides a quick fire tour of Java EE 7, familiarizes the reader with the basics of JVM (memory model, garbage collection. concurrency) and dives into the nitty-gritty of Performance Testing in Java.

The author guides us through some common performance issues and their symptoms along with a detailed insight of Java monitoring and profiling tools (in the JDK, IDEs etc). Topics like CPU, Memory and Thread Profiling have dedicated chapters and have been covered in great detail. The book delves into the topics of Tuning a Java EE application and its various facets (JVM, App Server, Web Server, the OS) along with a primer on Designing High performance Java EE applications. Last but not the least, the reader can dig into some Performance Tuning Tips (EJB, JPA, Servlets, JSF, Java SE) and apply the knowledge gained from the book in tuning a sample Java EE application hands-on!

Who has written this book?

The book has been authored by Osama Oransa. More about him here

What does the book cover?

The book is well organized and divided into twelve chapters. Let’s take a peek and see what each one has to offer.

Chapter 1 – Getting Started With Performance Tuning

This chapter introduces us to the world of Java EE performance tuning.

  • Classifies different types of performance related issues based on phases of discovery.
  • It goes on to differentiate typical application modes (standalone, thick and thin) along with a brief summary
  • Provides a quick peek into the all the components/layers of web applications and their properties which are critical from a performance standpoint – from the client end right down to the DB server

All in all, this lesson does a great job of getting things rolling and setting up the reader for what’s coming next. This is of great value for someone who is relatively new to the concept of Performance tuning and monitoring and allows her to get a decent head start.

Chapter 2 – Understanding Java Fundamentals

It lays a strong emphasis on strengthening the fundamentals associated with the Java Platform which are directly related to performance. Two primary topics which are covered are the JVM fundamentals and Java EE 7 (yes!)

  • Discussion of the JVM memory structure (as laid out by the JVM specification) and details around the Java HotSpot VM.
  • Exploration of the JVM Garbage Collector and its policies
  • Provides a brief overview of Java EE 7 – its specifications and features
  • Delves into some of the concurrency constructs available in Java EE – Async invocations, Singleton beans, non blocking I/O etc

Chapter 3- Getting Familiar with Performance Testing

Chapters 1 and 2 provided a good foundation and this chapter builds upon it. It deals with Performance Testing on a whole

  • It explores different facets of performance testing like types, components, tools, benchmarking etc along with an explanation of common testing terminologies and a discussion around performance testing in a Cloud environment.
  • The rest of the chapter deals with Apache JMeter and thoroughly covers web service, web application and database scripts with the help of succinct examples.

Chapter 4 – Monitoring Java Applications

This chapter deals with in depth coverage of the tools associated with performance monitoring and profiling in depth.

  • Starts off with an overview of the JDK monitoring and profiling tools landscape
  • It delves into details of these tools – memory map and heap analysis tools, JVisual VM and Oracle Java Mission Control
  • The NetBeans and Eclipse are covered as a part of the section which deals with IDE capabilities for JVM monitoring and profiling.
  • The chapter ends with a tour of JProfiler with a focus on the offline profiling mode.

Chapter 5 – Recognizing Common Performance Issues

This chapter builds on the performance issues overview provided in Chapter 1. It brings into focus, some of the frequently occurring performance issues and briefly looks at each of them

  • Explores Threading and deadlock related issues
  • Discusses performance issues which can be attributed to to memory (leakage), improper use of remote calls, database performance etc
  • The chapter also explores client side issues related to (JavaScript, CSS etc) and looks at some browser integrated dev tools (Chrome, IE, Firefox) which can help track such problems.

Chapter 6 – CPU Time Profiling

This chapter deals takes up one of the key aspects in Java profiling – CPU Time profiling

  • Discusses three profilers – NetBeans native profiler, the Java Mission Control and JProfiler
  • Explains how to interpret and make sense of the results obtained from the profiler tools
  • Delves into pin pointing and identification of performance issues (algorithm related, caching, threading etc) and provides a recipe/strategy for fixing them.

Chapter 7 – Thread Profiling

The deep dive into Java profiling continues. This time, the focus is one Thread profiling and its related nuances

  • As was the case with CPU Time profiling, the thread profiling options are explored with the help of NetBeans, Java Mission Control as well as JProfiler
  • Explains how to extract thread dumps with the help of JDK, application server and various profiler tools
  • Deals with interpretation of thread profiling results and analyzing thread dumps
  • Deep dive into exploration of threading related issues such as deadlock, blocked threads, unmanaged threads etc and builds on the recipe for fixing such issues (from the previous lesson)

Chapter 8 – Memory Profiling

This chapter marks the last section of the in-depth coverage of the types of profiling and deals with the all important area of memory profiling

  • How to use the NetBeans profiler and the JProfiler for memory profiling
  • Explains the process of heap dump extraction using a variety of tools such as jmap, JVisual VM, Eclipse MAT, JProfiler etc
  • Guides us through the process of heap dump analysis using visual tools as well as via he OQL (Object Query Language)
  • Analyses the potential issues emerging from memory related roots

Chapter 9 – Tuning an Application’s Environment

After all the discussion and exploration about potential performance related issues, related profiling tools and possible remediation, the focus now shifts to Tuning. This chapter deals with application environment tuning is specific which is divided into distinct categories

  • Detailed exploration of JVM tuning including both HotSpot and JRockit
  • Deep dive into application server tuning with GlassFish and Oracle Weblogic being the prime examples. Talks about tuning components such as EJB and Web containers, JDBC pools, thread pools etc
  • Exploration of HTTP server tuning with the help of Apache and Oracle HTTP server
  • Discusses tuning and optimization of operating system and hardware (capacity planning) related components

Chapter 10 – Designing High-performance Enterprise Applications

As the name clearly suggests, the chapter looks at the design aspects of enterprise applications. After having discussed different facets of performance tuning/monitoring/profiling, it makes sense to try and understand how to put certain best practices to practical use.

  • Explores different set of design decisions (app layer, security, framework etc) and their probable impact and introduces some common anti-patterns as well
  • Provides good insight/analysis of SOA and ROA based architectures and their performance aspects
  • Discusses the impact of data caching on performance in details and looks at topics like concurrency, data caching levels, how to assess caching performance etc
  • The chapter ends with an absorbing discussion on performance considerations from a Cloud deployment perspective

Chapter 11 – Performance Tuning Tips

This lesson provides general performance tuning tips, tricks and recipes for a variety of scenarios and use cases related to Java EE as well as SE

  • Deals with Java EE component specific performance secrets – EJB, JSF, JPA, web services etc
  • Explores general performance related topics from the Java SE space – I/O, Collections framework, synchronization etc and also peeks into the javap tool available in the JDK
  • The chapter ends with a discussion aobut database related performance tips ranging from choice of ORM library to executing bulk uploads in an optimized manner.

Chapter 12 – Tuning a Sample Application

This lesson is all about getting our hands dirty and applying the concepts learnt in the book in order to tune a real-world application

  • Introduces the sample application, and sets up an assessment plan
  • Guides us through the profiling process using previously described techniques such as CPU Time, memory profile etc
  • Deep dive into analysis of performance issues detected by the profiling process and resolution of each of those issues
  • Coverage of the testing process to assert performance improvements post issue resolutions

 

Highlights

Performance Tuning is not an easy topic to discuss or write a book about. Although it is based upon reasoning and science, more often than not, it is triggered due to unprecedented scenarios.

  • The author covers a vast landscape in general – from depths of the Java Platform basics, JVM to app server tuning, Databases and straight on to the Cloud !
  • In general, the book is applicable to wide variety of audience, but the day-to-day/ traditional Java developers stand to reap maximum benefit out of this book
  • The sample application tuning exercise is a great addition to the book. Again, performance tuning is not an easy thing to discuss and convey and setting up a application for hands on practice needs is indeed priceless
  • Leverages sample code, examples and pictorial representations in an effective manner.

 

Note: While most of the aspects of the book are great, but there are certain areas in the book where the grammar and overall English could have been better. Personally, it did not prove to be a hindrance for me, but can be improved.

 

Where can you grab the book?

A book this versatile and multi-faceted should not be missed – especially when it deals with Performance Tuning !

Visit the PacktPub web site to get your copy!!

Happy Reading :-)

Posted in Books, Java, Java EE | Tagged , , , , , | Leave a comment

A closer look at Oracle IDM Auditing

Reporting is a vital functionality in any product which deals with sensitive information. Same applies to Identity & Access Management tools. Oracle IDM’s Auditing module acts as a foundation for its OOTB Reporting capabilities. Let’s take a quick look at Auditing engine and how it facilitates the Reporting functionality within OIM

The use case presented here is simple – change to a user record in OIM.

What are the sequence of events which get triggered from an Audit perspective?

This is best explained by a diagram. I came up with the figure below in an attempt to better articulate the process.

 

oim-auditing

Although the diagram is self explanatory, a theoretical translation of the same is not going to harm us! :-)

  • The updated/created user record gets pushed into the USR table (stores the user information) – Its a normal process by which the information gets recorded in the OIM Database
  • The information is further propagated by the OIM Auditing engine (as a part of core back end server logic) and it initiates a transaction
  • The Audit Engine inserts a new entry in the AUD_JMS table as a part of the audit transaction completion. The AUD_JMS table is nothing but a staging table
  • The Issue Audit Messages scheduled job picks up the Audit messages in the AUD_JMS table and submits the key to the oimAuditQueue JMS queue.
  • The MDB corresponding to the queue initiates the Audit data processing – the data is seeded into the UPA table. This data is in the form of XML. These are snapshots of the user profile at the instant when the user record was actually modified/created. The UPA table also stores the delta (changes to the profile)
  • Finally, the Post processors of the Audit engine pick up the XML snapshots from the central UPA table and store them in specific audit tables (in a de-normalized format) like UPA_USR, UPA_USR_FIELDS, UPA_RESOURCE, UPA_UD_FORMS etc
  • These tables serve as the primary source of information for the Reporting module. If you have ever worked on the OIM Reporting module, I am sure you can relate to the Data Sources which you configure on your BI Publisher instance – these are for executing direct queries on the above mentioned Audit tables for its data.

That’s pretty much it ! This was not a coverage of the entire Audit module in OIM, but a preview of HOW the process is orchestrated on a high level.

Thanks for reading! :-)

Posted in Oracle Identity Governance, Oracle Identity Manager | Tagged , , , , , , , | Leave a comment

Java EE: Asynchronous constructs and capabilities

asynchronous

Introduction

Java EE has a number of APIs and constructs to support Asynchronous execution. This is vital from a scalability and performance stand point.

Let us assume 2 modules which are interacting with each other. When moduleA (the sender) sends a message to moduleB (the receiver) in a Synchronous fashion, the communication takes place in the context of a Single thread i.e. the thread which initiated the communication from moduleA is blocked until moduleB responds back.

This was a generic statement but can be extended to the context of a simple Java methods interacting with each other – in this case, a synchronous call from methodA to methodB would execute in the same thread which would be blocked until methodB returns or throws an exception

Why do we need Asynchronous behavior in Java EE based applications?

We can further extrapolate this point to the Java EE world – be it inter server communication e.g. between web tier and EJB tier (servlets and EJBs) or a typical client server interaction – a browser interacting with a RESTful end point, Servlets etc – the server thread which responds to a client request always blocks until the server component responds back.

Here is where Asynchronous execution comes into play – if the server thread processing the client request can be released/suspended and the actual business logic is executed in a separate thread (different than that of the original one), performance and scalability can be improved immensely ! E.g. if a HTTP listener thread allocated to listen to client requests is released immediately, then it is free to attend to requests from other clients and the business logic can be executed in a separate container thread which can then return the response via appropriate methods such as java.util.concurrent.Future object or via call back handlers registered by the client. Think from an end user perspective – responsiveness matters a lot!

Dive In: Async constructs and APIs in Java EE

Let’s take a look at few of the Async related features (APIs) in Java EE. This is not an exhaustive list – but should be a good starting point.

Different Java EE specs have their typical ways and APIs to facilitate Async capabilities. Let’s explore the below Java EE specifications

  • JAX-RS 2.0 (Java EE 7)
  • Websocket 1.0 (Java EE 7)
  • Concurrency Utilities 1.0 (Java EE 7)
  • EJB 3.1 (Java EE 6)
  • Servlet 3.0 (Java EE 6)

Note: The code presented below is in a snippet form (for obvious reasons). The complete samples can be accessed here 

JAX-RS 2.0

Async processing of requests is a new feature in 2.0 edition of JAX-RS (new in Java EE 7). In order to execute an aysnc request using JAX-RS APIs, one needs to inject a reference to a javax.ws.rs.container.AsyncResponse interface in the JAX-RS resource method itself. This parameter puts the request execution in async mode and the method proceeds with its execution. The resume method on the AsynResponse object needs to be called from within a separate thread after the business logic execution is complete. One can leverage the Java EE concurrency utility features (discussed later) such as the javax.enterprise.concurrent.ManagedExecutorService in order to encapsulate the business logic as a Runnable object and submitting it to the container’s executor service which takes care of the rest. No need to spawn un-managed, isolated threads on your own

@Path("/{id}")
    @GET
    @Produces("application/xml")
    public void asyncMethod(@Suspended AsyncResponse resp, @PathParam("id") String input) {

        System.out.println("Entered MyAsyncRESTResource/asyncMethod() executing in thread: "+ Thread.currentThread().getName());
        mes.execute(
                () -> {
                    System.out.println("Entered Async zone executing in thread: "+ Thread.currentThread().getName());
                    System.out.println("Simulating long running op via Thread sleep() started on "+ new Date().toString());
                    try {
                        Thread.sleep(5000);
                    } catch (InterruptedException ex) {
                        Logger.getLogger(MyAsyncRESTResource.class.getName()).log(Level.SEVERE, null, ex);
                    }
                    System.out.println("Completed Long running op on "+new Date().toString());
                    System.out.println("Exiting Async zone executing in thread: "+ Thread.currentThread().getName());

                    //creating a dummy instance of our model class (Student)

                    Student stud = new Student(input, "Abhishek", "Apr-08-1987");
                    resp.resume(Response.ok(stud).build());
                }
        );

        System.out.println("Exit MyAsyncRESTResource/asyncMethod() and returned thread "+Thread.currentThread().getName()+" back to thread pool");
    }

The JAX-RS Client API also has asynchronous capabilities but they have not been discussed in the post. They are definitely worth a look!

Websocket 1.0

The Websocket API is a brand new addition to the Java EE arsenal (introduced in Java EE 7). It facilitates bi-directional (both server and client initiated) communication which is also full duplex in nature (either the client or server can send messages to each other at any time).

In order to send async messages using the Websocket API, one needs to use the getAsyncRemote method available on the javax.websocket.Session interface. Internally, this is nothing but an instance of the nested interface of the javax.websocket.RemoteEnpoint – javax.websocket.RemoteEnpoint.Async. Calling the regular sendXXX methods on this would result in the sending process being executed in a separate thread. This is particularly useful when you consider exchange of large messages or handling large numbers of websocket clients to whom the messages need to be sent. The method wither returns a java.util.concurrent.Future object or one can register a callback in the form of a javax.websocket.SendHandler interface implementation

  public void sendMsg(@Observes Stock stock) {
        System.out.println("Message receieved by MessageObserver --> "+ stock);
        System.out.println("peers.size() --> "+ peers.size());
        peers.stream().forEach((aPeer) -> {
            //stock.setPrice();

                            aPeer.getAsyncRemote().sendText(stock.toString(), (result) -> {
                System.out.println("Message Sent? " + result.isOK());
                System.out.println("Thread : " + Thread.currentThread().getName());
            });

        });
    }

Concurrency Utilities 1.0

The Java EE Concurrency Utilities is another great addition to Java EE 7. It provides a standard way of spawning threads – the good part is that these are container managed and not just isolated/orphan threads about which the container has no contextual information.

In general, the Concurrency Utilities 1.0 provide a few standard constructs for execution of asynchronous tasks in separate threads. These are as follows – javax.enterprise.concurrent.ManagedExecutorService and javax.enterprise.concurrent.ManagedScheduledExecutorService.

In order to start a new task in a separate thread, one can use the ManagedExecutorService interface to submit a Runnable. In addition to implementing the Runnable interface, a class can also implement the javax.enterprise.concurrent.ManagedTask interface and provide a  javax.enterprise.concurrent.ManagedTaskListener implementation in order to listen to life  cycle changes to the task submitted via the ManagedExecutorService

@Override
    protected void doGet(HttpServletRequest request, HttpServletResponse response)
            throws ServletException, IOException {

        System.out.println("Enter AConcurrencyUtilsExample/doGet executing in thread "+ Thread.currentThread().getName());
        System.out.println("initiating task . . . ");
        mes.execute(new AManagedTask());
        System.out.println("Exit AConcurrencyUtilsExample/doGet and returning thread "+ Thread.currentThread().getName() +" back to pool");
    }

Servlet 3.0

Asynchronous HTTP was introduced in Servlet 3.0 (part of Java EE 6) which basically provided the capability to execute the request in separate thread and suspend the original thread which handled the client invocation.

The key player here is the javax.servlet.AsyncContext interface. In order to initiate asynchronous processing, the startAsync method of the java.servlet.ServletRequest interface is called. In order to execute the core logic, a java.lang.Runnable object needs to be submitted to the start method of the AsyncContext interface. One can choose to attach a listener by implementing the javax.servlet.AsyncListener in order to receive callback notifications during specific times of the Async task execution

 @Override

    public void doGet(HttpServletRequest req, HttpServletResponse resp) {

        PrintWriter writer = null;
        try {
            writer = resp.getWriter();
        } catch (IOException ex) {
            Logger.getLogger(MyAsyncServlet.class.getName()).log(Level.SEVERE, null, ex);
        }
        //System.out.println("entered doGet()");
        writer.println("ENTERING ... " + MyAsyncServlet.class.getSimpleName() + "/doGet()");
        writer.println("Executing in Thread: " + Thread.currentThread().getName());
        //step 1
        final AsyncContext asyncContext = req.startAsync();

        //step 2
        asyncContext.addListener(new CustomAsyncHandler(asyncContext));

        //step 3
        asyncContext.start(
                () -> {
                    PrintWriter logger = null;
                    try {
                        logger = asyncContext.getResponse().getWriter();
                    } catch (IOException ex) {
                        Logger.getLogger(MyAsyncServlet.class.getName()).log(Level.SEVERE, null, ex);
                    }

                    logger.println("Long running Aync task execution started : " + new Date().toString());

                    logger.println("Executing in Thread: " + Thread.currentThread().getName());
                    try {
                        Thread.sleep(5000);
                    } catch (InterruptedException e) {
                        Logger.getLogger(MyAsyncServlet.class.getName()).log(Level.SEVERE, null, e);
                    }

                    logger.println("Long task execution complete : " + new Date().toString());

                    logger.println("Calling complete() on AsyncContext");

                    //step 4
                    asyncContext.complete();
                }
        );

        writer.println("EXITING ... " + MyAsyncServlet.class.getSimpleName() + "/doGet() and returning initial thread back to the thread pool");

    }

EJB 3.1

Typically, (prior to EJB 3.1) EJB Message Driven Beans were used to fulfill async related requirements. A MDB bean listens to messages sent to a javax.jms.Destination (a Queue or Topic) and executes the required business logic – this might be anything from sending an email to initiating an order processing task. The important thing to understand is that the client which sends the message to the Queue in the first place is unaware of the MDB (decoupled) and does not have to wait/remain blocked until the end of the job (email receipt or order processing confirmation)

EJB 3.1 (part of Java EE 6) introduced the javax.ejb.Asynchronous annotation. This can be placed on a EJB Session bean (Stateless, Stateful or Singleton) class (makes all the methods Asynchronous) or at the method level itself – in case fine grained control is needed. A method with the @Asynchronous annotation can either return void (fire and forget) or an instance of java.util.concurrent.Future if the async method result needs to be tracked – this can be done by calling the Future.get() – the thing to note is that the get method itself is blocking in nature

  @Asynchronous
        public Future<String> asyncEJB2(){

        System.out.println("Entered MyAsyncEJB/asyncEJB2()");
       System.out.println("MyAsyncEJB/asyncEJB2() Executing in thread: "+ Thread.currentThread().getName());
        System.out.println("Pretending as if MyAsyncEJB/asyncEJB2() is doing something !");
        try {
            Thread.sleep(5000);
        } catch (InterruptedException ex) {
            java.util.logging.Logger.getLogger(MyAsyncEJB.class.getName()).log(Level.SEVERE, null, ex);
        }

        System.out.println("Exiting MyAsyncEJB/asyncEJB2()");
        return new AsyncResult("Finished Executing on "+ new Date().toString());

    }

This was a rather brief preview of Java EE capabilities. These APIs and specifications are functionally rich and it is hard cover all of them via a blog post :-) I hope this piques your interest and gives you a starting point to explore further.

Cheers! :-)

Posted in Java, Java EE | Tagged , , , , , , , , , | Leave a comment

New in JAX-RS 2.0 – @BeanParam annotation

JAX-RS is awesome to say the least and one of my favorites! Why?

  • Feature rich
  • Intuitive (hence the learning curve is not as steep)
  • Easy-to-use and develop with
  • Has great RIs – Jersey, RestEasy etc

There are enough JAX-RS fans out there who can add to this! ;-)

JAX-RS 2.0 is the latest version of the JSR 311 specification and it was released along with Java EE 7.

Life without @BeanParam

Before JAX-RS 2.0, in order to pass/inject information from an HTTP request into JAX-RS resource implementation methods, one could

1. Include multiple method arguments annotated with @FormParam, @PathParam, @QueryParam etc

1

 

1_1

 

2. Or, have a model class backed by JAXB/JSON or a custom MessageBodyReader implementation for JAX-RS Provider to be able to unmarshall the HTTP message body to a Java object – read more about this in one of my previous posts

 

2

 

3

 

This means that something like a HTML5 based client would need to extract the FORM input, convert it into JSON or XML payload and then POST it over the wire.

Simplification in JAX-RS 2.0

This process has been simplified by introduction of the @BeanParam annotation. It helps inject custom value/domain/model objects into fields or method parameters of JAX-RS resource classes.

In case you want to refer to the code (pretty simple) or download the example/run it yourself, here is the GitHub link

All we need to do is, annotate the fields of the model (POJO) class with the injection annotations that already exist i.e. @PathParam, @QueryParam, @HeaderParam, @MatrixParam etc – basically any of the @xxxParam metadata types and

4

 

Make sure that we include the @BeanParam annotation while injecting a reference variable of this POJO (only on METHOD, PARAMETER or FIELD).

5

JAX-RS provider automatically constructs and injects an instance of your domain object which you can now use within your methods.

Just fill in the form information and POST it !

6

 

7

 

That’s it. . . Short and Sweet :-)

Keep Coding !

Posted in Java EE | Tagged , , , , , | Leave a comment

Oracle IDM R2 PS2 : Rich Diagnostics

feature-img3

 

feature-img2 feature-img1

 

 

 

 

Oracle IDM R2 PS2 provides some really efficient troubleshooting, monitoring and diagnostic features. This post will provide a quick overview of the same and delve into using one such feature to resolve a practical issue.

New Diagnostic Features: Overview

1. Orchestration diagnostics – The EM console can be used to drill down into the Orchestration related details of events within OIM

  •      A Dashboard to view recent operations and events
orch-dashboard-1
orch-dashboard-2
  • Info regarding ALL the event handlers attached to a particular operation – including  CUSTOM developed. No more digging into XMLs form MDS !
orch-ops
  • Powerful search features for all Operations (search by Operation Type, Operation ID etc)
orch-instance

 

2. Dynamic Monitoring Service (DMS) is an Oracle FMW component which enables performance tracking of various OIM modules and services such as scheduled tasks, event handlers, platform resources etc. One can export DMS metrics for detailed introspection as well

3. Reconciliation Profile validation

  •      Validate Recon profiles via MBeans via EM console – great for debugging issues
  •      Also provides reconciliation tables related indices which can help compare Database tuning as per documented mandates
recon-pro-diag-feature

A detailed explanation of each of these features is difficult to cover in a single post – skipping this for a future blog post (series). For now, let us explore a real world use case where such diagnostic capability can be used

I happened to come across a particular issue and resolved it with the help one of the many OIM PS2 diagnostic features. This one happened to be related to OIM Reconciliation Profiles.

The problem statement

  • Reconciliation job execution was not resulting in event creation.
  • The server console logs pointed to a SAX parsing exception.

console-error

Note: Whenever I see an XML parsing (SAX or DOM) exception in context of reconciliation, the brain invariably gravitates towards the Reconciliation Profile XML – yes. It’s a boon in general, but a curse because it has a tendency to get corrupted (because of no apparent reason).

The investigation

Time to use the new Reconciliation Profile validation diagnostic feature in PS2 (mentioned above) which enables us to validate and detect errors in the OIM Reconciliation Profile. It does so by exposing this service as an MBean. Sweet !! :-)

So how to go about this?

1. Log into EM console and take up the route shown in the snapshot below (yes, a picture is worth a thousand words and 10 minutes of writing something which can be easily portrayed via a visual medium!)

em-console-profile-val-1
em-console-profile-val-2
em-console-profile-val-3

2. The next obvious step was to export the Recon profile XML itself in order to further drill down on matchingRule element (since there can be multiple such elements and the error indicates that one of them might be empty – hence the schema violation error resulting in a parsing error)

Exporting the recon profile XML via EM console

Note: This is not a diagnostic feature as such or a new functionality. This is present since OIM 11g R1 era

No need to execute manual processes – use the EM GUI to get the job done fast. Follow the snapshots

 

recon-prof-ex-1
recon-prof-ex-2

Finally – click on the Invoke button and you should have the profile exported to the location mentioned above

Note: If required, you can easily change/modify the XML and use the importMetadata operation of the same MBean to import with a single click (try it out)

The profile was exported in order to investigate it further. Let’s see what was the issue was

profile-xml-issue

 

So, finally it is evident as to which matchingRule tag was empty.

Conclusion

Well, it all boiled down to a missing configuration in the Connector artifact – missing KEY field configuration for the child field (in the Process Definition).

Take away point

OIM is a feature rich product. Hence, things can get complicated at times and errors can get hard to debug. We saw how easy it was to drill down to the cause of the error and ultimately fix it

Why/How did this happen? That was not really the point of this post. In case you are inquisitive, looks like this is a bug in the connector ! ;-) That’s for a later post – maybe after further introspection

Resolution

  • Include the KEY field config for child field in Process Definition
  • Recreate the Recon Profile
  • Validate the same via EM console (already explained)
  • Move on with your life ! ;-)

Remember – bugs/errors/issues are an opportunity for us to learn and doscover something new. Software can never be perfect in a single go. Be patient, try to diagnose, know your tools, know how to use them – if there are no tools, just build one and blog about it !! ;-)

Until then. . . Cheers ! ! !

Posted in Oracle Identity Governance, Oracle Identity Manager | Tagged , , , , , , , | Leave a comment