Application Performance : Part II

I have been dealing with many of the java applications for years and in the recent past i am finding all the areas that effect the performance. I have been reading lot of books and articles and i thought that a summary would help everyone. In the past 2 months, i have been reading the book Java Performance Platform. Thanks to the authors, they have really given valuable information in this book. As always, “Its not enough to read this book but to consciously write applications that takes care of the memory management and performance”.

Many of the developers and IT does complain about the huge foot print by applications and some times its difficult to know whats causes the foot print. Applications becomes memory hogs and instead of looking into the root cause, we tweak JVM settings, increase the virtual memory, increase the PermGen size, custom class loading. I am sure and any of you would agree that we do “all sorts of things” for the application performance. I have written some notes on the Garbage Collection  (which is Application Performance - Part I) in my previous blog. The current “notes” concentrates on the causes for the RAM footprint and different aspects of the things that we need to be aware of.

Memory foot print of the program is tricky to find out. Many of developers do look into the TaskManager to see RAM usage by the application. This definitely gives an idea of how much of memory your application requires. When this memory increases over a period of time, we suspect memory leaks and take corrective actions.

Programmatic Memory Usage : Certain in formation can be derived from the java.lang.Runtime class. This class can look for the heap size of the JVM. Two methods namely Runtime.totalMemory() and Runtime.freeMemory() gives the size (in bytes) information that many of us want. Heap memory can only give the size of the objects. But the actual size of the application is a combination of the Objects, Classes, Threads, Native Data Structures, Native Code.

App Runtime Size = funtion of (Objects + Classes + Threads + Native Data Structures + Native Code)

Depending on the OS, JVM and application the actual memory consumption changes. So any one of the above parameters could be the number one memory consumer for a particular application. In general the memory consumption depends on following items (but not limited to).

• usage the native code
• usage of java core libraries
• bulk of the frameworks used in the application
• number of classes loaded against the objects used

Most of the developers do have a great deal of control over the Objects and their sizes. It could help if the developer knows the approximate size contributed by the objects at the run-time to see what is the optimization area. i.e. if object memory is only a small % of the total size then perhaps we could concentrate on the classes or native libraries, etc.

When classes are loaded into the memory, there are few more dependant entities that contribute to the RAM footprint. Bytecodes is the intermediate format that a java class file gets compiled to. It is necessary that the bytecode gets loaded into the RAM. All the related contents are parsed and reflective data structures are created for methods and fields.  Constant pool is defined for all the classes. e.g. all the String literals are present in this constant pool along with all the class, method and fields names. Threads are another important item that could cause a large memory foot print. Its necessary to see what kind of computation is done in the Thread and what data-structures are used. Many of the UI level libraries/frameworks like SWT, AWT does depend on some sort of the native libraries. Its difficult to know which class of these frameworks directly access the native libraries.

Knowing some of the entities that increases the memory footprint would help many of developers. If not completely avoid the large memory footprints, we can at least be aware and work towards conscious usage of resources.

Is anyone doing Mashup IDEs on Eclipse ?

Gartner predicts that Mashups and Composite Apps is one of the top 10 technologies for 2008. There have been lot of companies embracing the so called bleeding-edge technologies like SOA, RIA, Web2.0, Virtualization, Social Networks and Enterprise Wikis. As these technologies and frameworks are supposed to increase organizations productivity, there is a need to look-into the developer’s productivity too. These days many are happy that you get to choose one in many of the development environments. 

Following are few of the very cool IDEs that i have seen in the area of AJAX, SOA, RIA …

1. Eclipse (WTP, ATF, RAP)
2. MyEclipse
3. Aptana
4. GWT Designer
5. Flex Builder
6. OpenLaszlo IDE
7. JackBe’s Presto
8. Nexaweb
9. BackBase
10. TIBCO General Interface

I am very curious to see anyone coming up with an IDE for Mashups and Composite applications in the Commercial and Open Source area. I am aware that the enterprise Mashups and composite applications is a budding concept. Of course Mashups from Google are there for a while (but these are not the real enterprise Mashups). Recently i have seen few projects like Damia from IBM, PopFly from Microsoft, Wires from JackBe, Kapow and Denodo. These projects do have some sorts for IDEs but none of them apparently have an IDE based on Eclipse.

“Is anyone out there who is doing Mashup IDEs on Eclipse ?”

Do you care about Garbage Collection ?

Recently there have been few instances our RCP application crashed with the OutOfMemory exception and it took lot of time to look reasons behind the crash. An easy solution is to increase the PermGen from 64mb (default) to 256mb (recommended). Well, this works for the application and i am happy. Since then, i explored few books and articles to look into the memory allocation, objects creation and garbage collection can effect the application and its memory needs.

Garbage Collection is the concept that its publicized as “JVM does automatic GC”. But in reality this does not seem to be as cool as it sounds. Developers should not completely relay on JVM to take care of your applications memory needs and garbage collection. A solid understanding on “How GC works” is essential to write robust and high-performance applications.

Its necessary to understand the complete life-cycle of an object and how it transforms its state from declaration till garbage collection. We will look into each of the state in detail.

• Created
• In Use
• Invisible
• Unreachable
• Collected
• Finalized
• Deallocated

Created: Creating the object makes many things to happen. Space is allocated for the Object. Object constructor is called. If at all there is a super class, it constructor is called. Instance variables are initialized. In the end its important to realize that the object construction does take time and this depends on the JVM implementation for sure.

In Use : Once an object has strong reference, its said to be In Use. It is normal for any object to be In Use state for relatively longer than any other states.

1. public class InUseClass {
2. static List someList = new ArrayList();
3. static void doSomething() {
4. Customer customer = new Customer();
5. somelist.add(customer);
6. }
7. public static void main(String[] args) {
8. doSomething();
9. }
10. }

In the above code you can see that the at the time doSometing() is executing, there are 2 instances of the Customer that’s holds on. One in the line 4 and one in the line 2. Once the doSomething() returns, there is only one instance that remain of Customer which is line from line #2. Which is a reference of the Customer object in the List. Thus you can imagine that there would be lots of references that remain strongly referenced even after they are used up.

Invisible : When an object is no longer strongly referenced but references still exists, its said to be in Invisible state. Its not necessary that all objects go via this state. In the following code snippet you can see that the strong object references are lost once the something() method is returned. In other words, they go out of scope once the method is returned.

1. public void something() {
2. for (int i =0; i<5;i++) {
3. Customer customer = new Customer();
4. customer.printName();
5. }
6. }

This scenario is dangerous and one of the main causes for the memory leaks. In this code block the references become invisible and its recommended to make the references null explicitly.

Unreachable: An object is in Unreachable state when there are no strong references and it will be marked for the collection. Of course “marked for collection” does not mean that JVM does the GC immediately. JVM has all its freedom to delay till memory is required by application.

Collected: When an object is unreachable the JVM readies it for the finalization. If an instance has the finalize method then it is marked and if the instance does not have any finalize method, it will move directly to Finalized state. Its very important to note that the when an instance has the finalize method, the deallocation process is delayed.

Finalized: Once the finalize method is run, if the object is still unreachable, its in the Finalized state. A finalized object is waiting for the deallocation. Its certain that the object’s life is prolonged when the finalize method is attached to the object. Also its not recommended to attach the finalize method for short-lived classes.

Deallocation: After all the above steps if the object is still unreachable, then the object is marked for the deallocation state. Its not sure when exactly the JVM deallocates, but this is dependent on the implementation of GC algorithms.

In the end, GC is not really what many of us perceive. An extra care must be taken to carefully write the applications with a special check on the object references and freeing the memory when ever possible.

“Nothing comes for free and so is GC”.

MyEclipse is Cool !!!

MyEclipse 6.0 GA has been released in August 2007 with a some cool features aiming at developers productivity. I just downloaded their 30 day trial version and played around little bit for two days. I was developing a AJAX and SOAEclipse IDE for my client; i was particularly interested to look into the ajax and database capabilities from MyEclipse.

Noticeable features from the 6.0 GA release includes (see myeclipse for complete list of features)…

• Europa eclipse distribution compatibility
• Integrated Libraries (for Java EE 5, JPA, Struts 1.1-1.3, Hibernate 2-3.2, Spring 1.2-2.0, etc…)
• Integrated Tomcat 6.0 Server
• Integrated Derby Server (10.2.2)
• Java Persistence Tools
• Visual JSF designer and Flow editor
• AJAX Tools
• Matisse4MyEclipse Swing Visual Designer

Ajax debugging capabilities are awesome from MyEclipse. They tried to provide most of the capabilities from the Firefox’s FireBug.

• It has the JavaScript Inspector to look-into the scripts loaded by the webpage.
• JavaScript Console to look into any kind of errors happening on the particular browser. One of the important issue that i was looking is to change the underlying browser that the page renders so that i can see the errors/warnings/info depending on the browser(IE/FF/SF). Some how i could not find this capability (perhaps i am missing something).
• Ajax Request Monitor is another great tool to see how many ajax requests are happening on the page. It also shows the Request/Response along with Headers and Body. This is very helpful for debugging any kind of ajax requests to indicate time, status and request and response.
• DOM Source view is very useful to see the code for a selected DOM node. The DOM Inspector view enables the developer to select any node on the dom and it highlights the specific area in the WebPage. This is great feature for the developer as he can see and inspect the inner elements from DOM.

I found the Java Persistence Perspective is very useful with DbBrowser, Table/Object Info, SQL Editor, Results, and Table Edit capabilities.

• How cool is it for a developer to be able to connect to different databases (via DB Browser), select the tables and view the table Info (via Table/Object Info) and the edit the tables if needed (via Edit Table).
• SQL Editor allows to select the different data base connections and then execute the query on a specific table or view. The query results are displayed in the SQL Results view. A developer oriented feature is to be able to copy the results on to the clipboard.
• I really liked the ease with which we can add a capability; some of the capabilities include Add Web Service Capabilities, Add Tapestry Capabilities, Add Struts Capabilities, Add Spring Capabilities, Add JPA Capabilities, Add JSTL Capabilities. All these capabilities can be added by a single click. Configuration files and property files are automatically generated for the developer.
• Java Persistence perspective enables to be able to select a table and create code supporting the persistence capabilities. Persistence mechanisms like JPA, Hibernate and EJB3 are supported via this feature (they call it reverse engineering… from table to code).

Last but not the least, Installing the MyEclipse examples dynamically is a great feature. User can decide what he wants and then he can get a full running application. I really like the bundling of the Tomcat and Derby so that all the minimal set of pre-requisites are satisfied.

Overall, working on MyEclipse was a great experience for me. All the mentioned features definitely improve developers productivity to a greater extent.

Demand and supply of webstartups

It was very expensive to start a start-up few years back. As years are passing by, software and hardware are getting cheaper, software products are also getting cheaper and the time to produce them is getting lesser. Initially when the technologies comes into the market, they are expensive. The same way now-a-days its becoming cheaper to start a start-up and there are more resources in-terms of ideas, VCs and academic resources.

I am really excited by some of the facts that Paul has cited in his recent article. I hope this article really helps to all those, who wants to start something and become success (me being one among them ). Thanks to Paul for writing all interesting articles.

Apple releasing SDK for iPhone

After few months of iPhone in the market, Apple announced the release of native-cocoa based SDK. This would enable all the enthusiasts to build the native applications on iPhone. I guess releasing the SDK is much better for Apple than letting people to hack and build applications for iPhone. Kudos to all the brains who have developed applications within few days after the iphone release.

Its a discussion issue that the introduction of the SDK for iphone is detrimental to the web applications that are built for safari and with a special emphasis on the iphone. Of course both have its advantages over other. A centralized web development requires no deployment and packaging mechanisms. It can be faster to develop with html and js. Developing applications with SDK can take full advantage of the application and can also overcome the entire restrictions of web-browser.

A detailed analysis is provided by Kevin Hoffman in an article from syscon.

Plugin dependency visualization plugin

Thanks to Bull for contributing this great plugin. He has created and hosted the plugin dependency visualization plugin on eclipse. The dependency visualization aims to provide a set of views to assist with plug-in dependency analysis tasks. In particular, the views will provide cognitive support to people as they attempt to understand the dependencies between their plug-ins.

It supports set of views with analytics and cognitive tools. This plugin gives a cool visualization of the dependencies between various developers plugins.

Earlier the only way to understand the dependencies is to keep different plugins on the whiteboard and run the plugin dependency tool in the pde. But with this plugin, it becomes much easier for the project leads and team members to manage the plugins.

Go online with RAP 1.0

Rich Ajax Platform(RAP) is a project from Eclipse aims to enable developers to build rich ajax and RIA by using eclipse development model, plugins and java-only apis. Innopract a Germany based company and an eclipse foundation member is the major contributor for this project.

The core of the RAP is a subset of SWT APIs and and some JFace. So over all the widgets available in the SWT and JFace are available to build the RIA. RAP is very similar to RCP, in the way that RCP runs on the desktop and RAP runs on the server. The RAP libraries intelligently renders the UI at the browser with the help of the JavaScript qooxdoo library. RAP lets many of the desktop based RCP applications to be run on the server with little or no changes in the code. Would it not be so cool to have a single code base and be able to have RCP and RIA? It’s very interesting to see the maturity of this project from incubator to 1.0 and all the new and note worthy features.

In the end it definitely competes with the GWT, AIR, Backbase, JackBe, Open Laszlo, etc… in the area of the AJAX Framework and Tools for building the applications. As APIs are based on Java, in this aspect it’s very close to GWT.

All the best to RAP !!!

Top 10 reasons to use eclipse…

1. Eclipse is Free: I have seen very few organizations that would support you with expensive IDEs. As a developer if i need to play with, i dont have too many choices. Eclipse is free. I can download without too many hassles. Of course there are some other (i dont want to mention the names) IDEs for free too. But they don’t score in some other aspects. I get so many features for free.
2. Eclipse Community & Industry Support: When i want to explain my boss and customers about eclipse, it important to know who is behind this project. Till now i had very few arguments about the credibility of the project and the people behind it. Especially the way it got spread through out the developer community from USA, Europe, and Asia is great. Developers celebrated eclipse’s birthday in Hyderabad, India in a huge way (doesn’t it say …).
3. API Documentation: As a developer and technical lead, every one is interested to have a good API documentation so that the learning cycle is less. You don’t spend too much time in digging into unclear documentation. However good is the software, i want to have a good API documentation and eclipse has it.
4. Free plugins: Once i have the base platform, i would want to use supporting and new features. And yes, many of VERY useful features are free. I have used so many plugins like findbugs, checkstyle, subclipse, etc… but in the end for IT departments its so nice to have something free and USEFUL.
5. Code Samples: Sometimes i better understand with the help of code rather than reading some documentation. Eclipse has great code samples for all top level projects. Whoever starts SWT, JFace would definitely get lots of samples form snippets and also from the articles. Thanks to the guys who have supported all these.
6. Design Philosophy: Many colleagues and subordinates have learned good design practices and programming patterns from eclipse code. Thanks to the book written by Gamma and Beck. Clean plugin architecture and clean interfaces.
7. Customization : With the base platform around, you can do whatever you want to do. Users have the every possibility to customize their plugins that way they want.
8. Extensibility : How else would we have seen the Java IDE, C++ IDE, Cobol IDE, PHP IDE, RCP Applications, etc…
9. Productization: Its so easy to do productization. Changing the icons, splash screens, custom messages etc… Seeing some applications, you would not even recognize that they are built out of eclipse.
10. Cross-Platform: A recent plugin i developed for a RIA/Web2.0 client, works on Windows, Linux and MacOSX; with the same code base and a single build. This is so much of a relief for many of the organizations who wants to develop the applications for multiple platforms.

XML vs JSON !!! Dont compare.

Java Script Object Notation (JSON) has become quite famous since 2006 and it has gone into the major companies application frameworks. JackBe, Google and Yahooare some of the companies that have adopted the JSON in their products. JSON is the response format for many kinds (rss, rest, wsdl, atom, dao, etc…) of the web service responses. Industry has already been talking “If JSON is an alternative to XML”.

I would like to keep the JSON as just a data interchange format. I would not mix up XML and JSON for the data representation and data exchange formats. Following are some differences that i have seen from my own experience…

• XML parsing is very generic, you don’t have to care about the Types. JSON parsing on the other hand is very tricky.
• XML has the data representation and semantics attached to it. JSON is very good for data representation.
• Perhaps JSON data over the HTTP is faster than the XML.
• As a developer the code to parse XML is much cleaner than the code to parse JSON.
• Evaluating the JSON on JS is much faster and efficient. Due to its inherent support on JS, the learning cycle and manipulation of JSON data is faster than manipulation of XML by JS.
• For WEB2.0/RIA applications, where there is frequent data transfers between the browser and server, it makes sense to get the response in JSON and manipulate it in the browser.
• I have used XML as domain models representation. It can contain the cross references, can contain types and attach semantics with the help of Schema.
• JSON does not have any namespaces. I am not sure if this is an added advantage for JSON.
• JSON Does not have validator. Developer is supposed to validate what ever he gets.
• I love the random access XML via XPath. Can i do the same thing in JSON.
• XLink, XQuery are some of the cool things i like on the XML.
• I can represent the entire design of a system as XMI. It has references, schema support, i can validate, etc…

In my view JSON is a great format for the internet/web applications over the browser. Its not the question of XML vs. JSON; Its the question of When XML and When JSON.