Sling - Who is closing my JCR Session?

Has this happened to you? - You got the JCR session, but at times, you find that "someone" closed the JCR session! And that too - its very inconsistent. Sometimes it works and other times it does not! Well then its that you have hit the Sling / CQ development anti-pattern. And possibly this is what you see in your log:

javax.jcr.RepositoryException: This session has been closed.

As suggested by Jorg Hoh on his blog on CQ development patterns, you should rely on Sling objects as much as possible, rather than relying directly on the JCR objects. But at those cases (e.g. writing to a repository) when you will have to directly rely on the JCR repository you will have to still use the JCR Session, its recommended that you directly get the JCR session from the Sling repository, and not adapt it from the ResourceResolver, IF you do not intend to use the ResourceResolver directly.

@Reference
private ResourceResolverFactory resolverFactory;

// Wrong way of obtaining JCR Session
private Session getSessionAntipattern() throws LoginException {
 ResourceResolver resolver = resolverFactory.getServiceResourceResolver(null);
 return resolver.adaptTo(Session.class);
}

Code Listing 1: JCR Session Creation Antipattern

@Reference
private SlingRepository slingRepository;

// Correct way of obtaining JCR session
private Session getJCRSession() throws RepositoryException {
 return slingRepository.loginService(null, null);
}

Code Listing 2: JCR Session Creation Pattern

You may wonder, what's wrong. Logically thinking, why would you retrieve the ResourceResolver (and  adapt it to JCR Session), if you do not intend to use the ResourceResolver?  This logic has made the Sling developer's to close the underlying JCR Session of the ResourceResolver when the ResourceResolver object is Garbage Collected. Although, how this is done is being changed across the releases, the basic idea of releasing the resources held up (JCR Session) is still valid.

If you observe in the code snippet 1 (antipattern), the scope of the ResourceResolver object ends as soon as the method returns, which makes it eligible for garbage collection. This means that whenever the garbage collector runs, the JCR Session got through the antipattern would be closed - oops! This explains your JCR Session getting closed inconsistently.

I would like to emphasize that adapting to JCR Session from ResourceResolver itself is not wrong, if you intend to use the ResourceResolver along, other than just adapting it to JCR Session.

Hope this post helps someone save some time debugging on the JCR Session getting closed inconsistently.

Insane sanity (testing)!

You are busy writing your code on a complex piece and suddenly a team wide announcement comes - "The new code has gone in for the build, please do 'sanity' for your components".

"My unit tests would save me!" is not something which will save you here if this happens to be something which has a lot of visual and "content driven" elements. So, you break out of writing code half-way and start doing the "sanity" because it is crucial to test your components before lunch, as this code will be going to production soon after. You do this half-heartedly because this is so common for you whenever a major release happens or a content restructuring happens - and anyway "I am not a tester!". So you lose at-least one hour of your otherwise productive time to doing this insanely mundane task. On top of this, you also have your "professional testers" doing the same testing -just few extra eyeballs! This is the typical scenario if you are developing content driven (CMS) applications. Bless you if you are not so!

Insane testing
Image Credit - Wikimedia Commons

At this age in Digital Marketing, where content is the king, faster development and delivery overpowers the more structured delivery methods often followed in typical enterprise applications, where you invest for a lot of integrated and automated testing. Reason - "If we invest time to build integration testing and automation, we lose time to market". This is a broken argument often upheld by people who most often do not even care about integration testing in its real sense, and automated testing which can test the UI elements and the content of the page.

If we can invest some time in bringing about continuous integration and UI automation, a lot - really lot of time could be saved on developers doing "sanity testing". This may be difficult to achieve for shorter duration projects of the span less than 6 months - all out of their own budget, but may be easily feasible for bigger projects. Better - if this could be planned by "practice leads" who could invest on this once, and reap the benefits multiple times across projects. I do not have metrics to prove this by maths, but just a little retrospection into this would be useful.

Personally, this topic is lower on my radar. First, there are higher priority tasks for learning and experimentation on my TODO list. Second, I work more in a consulting mode than delivery, so budgeting it on  my project is not possible. Additionally, I do not have metrics to convince my client to budget this either! However, I will try to squeeze in some time in the coming days and weeks to do some research into this topics and publish my findings.

Three great advice

Act on it. Don't think too hard. Be Agile. These are the three great advice that have been my favorite picks from the book - "The Passionate Programmer - Creating A REMARKABLE CAREER in software development (2009)".

"Act on it" means that a piece of good advice needs to have an action plan on it - however small it may be. Most of the chapters of the book had the "Act on it" section which forced us to review ourselves and try to find an action plan. Without an act, the advice just rusts in our minds.

Thinking hard - "to jump or not?"
Image Credits - stefaanroelofs

"Don't think too hard" means that it is not necessary to perfect the things in the first go. We should think on it, but not "too hard" to make it the best at the first attempt. This has been especially important to me because, often trying to do the "best" often resulted to the act not being done at the first place. The best example would be writing something for the blog. I would often have such a lot of topics but often either I choose a topic which requires itself to be expressed as an essay, or just make the writeup to be the best and cover all the aspects of the topic, leaving no open loops. Both of this essentially means a lot of time, which eventually just rusts in the backlog and never gets a chance to get published.

"Be Agile" when applied to career and life means that career and life are never done. Re-iterate over, and try to make them better every time, and correct the course of action if it needs so.

Putting this together would also mean that I should be blogging more often, even with not so well rounded topics still appearing in there. And of-course, this would also apply to other aspects of life as well.

Nice is just ordinary! You need to be Extra-ordinary - Amazing - Remarkable!

Today I happened to see a TV advertisement which had a beautiful story - "Nice is just ordinary!". This happened to  rephrase what Chad Fowler had to say in his amazing book - The Passionate Programmer - Creating A REMARKABLE CAREER in software development (2009), which I had finished reading one round just today.

Winning is not about not-losing. Winning is about being awesome. Winning is about being Amazing - Remarkable!

My career, as I look back, seemed to have stagnated. I never had a roller-coaster ride in my career. I had pigeon-holed myself into the mediocrity - especially for the last 4-5 years. No one else to be blamed - just me. This was why I forced myself to read this excellent book to be inspired, and bring out the best in me. After-all I got into this profession for pure passion.

This book was in my reading list ever since 2010, but it was two weeks back that I finally ordered a paperback edition from Amazon, which I would religiously read during my travel to and from office.

The world around us always demands extra-ordinary and amazing things. Everything else is just OK, and CAN just live without. But I would like my life to be something which would create a dent in the world! Something awesome; something remarkable!

Yes, I do need a lot of overhauling. I have to invest in myself - to be a remarkable product - to create remarkable products. And this should mark the beginning of the journey to a remarkable career.

Over the next course of time, I would post critique to the book, apart from my brain dumps of some interesting ideas brewing.

The secret sauce of wrapping a jar as OSGi bundle

OSGi has been on the mainstream technologies for some time now. Many of the Java application servers now are either natively OSGi based, or support OSGi. This has also led many to package their libraries as valid OSGi bundles. But still there are a lot of libraries, that have absolutely no knowledge of OSGi - and when we have to use one of such libraries - we have to depend on one of those many open source projects which gives an OSGi bundle for the library, or prepare our own recipe to wrap them into OSGi bundles.

Before you begin

Before you pack your toolbox to wrap vanilla jars into OSGi bundles, do search public OSGi repositories and projects which can save a lot of your effort. Here are some of the popular ones:

1. SpringSource Enterprise Bundle Repository 
2. ServiceMix Bundle Project  Github Repository - https://github.com/apache/servicemix-bundles
3. Pax Tipi Github Repository - https://github.com/ops4j/org.ops4j.pax.tipi

If you still can't find what you are looking for, you may have to consider building your own! You can even consider contributing your bundles to the ServiceMix Bundle project or the Pax Tipi project.

Wrapping of vanilla jars can sometimes become complicated, especially when it is related to dynamic class generation or that of XML parsing. And that is why I am writing this - to share my experience of wrapping OSGi bundles when the library is complicated.

Choosing your tools

No surprise, I am also going to suggest the BND tool here. However, the BND tool itself is available in various flavours depending on your development workflow and tool as described in the Download page of the BND tool:

1. Bndtools - Bundling tool for the Eclipse Plugin
2. Apache Maven Bundle Plugin
3. Bnd4sbt - Bundling tool for Scala application
4. Gradle OSGi Plugin - A Gradle based OSGi plugin. Gradle is a Groovy based builder. 
5. Ant - Available as an ant task
6. Osmorc - Plugin for IntelliJ IDEA
7. Command Line Utility 

All of them however use the bnd library at its heart. However here, the BND tool command line utility can be handy to quickly wrap vanilla jars into OSGi bundles and do some basic analysis of the vanilla jars.

The choice of tool is mostly a personal preference, and also driven on your development environment and workflow. I personally prefer the Maven Bundle Plugin. However, irrespective of the tool used, there are a few fundamental principles which will help us wrap the bundles successfully. This need not be the best way, but has always helped me to build out OSGi bundles.

For the Maven users

To begin with, you can clone one of the projects - ServiceMix Bundle Project or Pax Tipi project, and add your own module to start out wrapping your bundle. If you need to manipulate the vanilla jar other than just adding the Manifest, you could consider using the Maven Shade Plugin - as it is being used in the ServiceMix bundle project.

Setting up the lab

First, I would recommend setting up an environment to test your OSGified library. You should have the following in your testing environment:

1. An OSGi container - Felix or Equinox or whatever you are using. Ensure that the version matches your target environment.
2. The target OSGified jar - yeah we are not done yet, but this is the test target. We will perfect it over some iterations.
3. The client bundle - the bundle where your code uses the API which exists in the target library (#2 above). You can use an OSGi activator (or the SCR activate) to run your code which will test if you have your library OSGified correctly.
4. The supporting bundles in your target environment. This may range from logging bundles to that of XML parsers and so on. Just be as close as possible.

All this environment should be something that is easy to provision and setup. I personally use a vanilla Felix instance with necessary environment bundles provisioned using Pax Runner. However you are free to use the tool of your choice to do this provisioning. The objective is to be able to start, test and stop the OSGi container, make changes to your target OSGified bundle and client bundle and repeat. And you do need an environment which should be easy to provision. You can also consider using Pax Exam to setup this environment.

Bundle it up!

If the above setup gave you a success right away at your first try, then you need not read the rest of this writeup. But most often, we do not get it right at the first attempt. So then you will have to do some experimentation with the packaging of your bundle which will ultimately give you the success.

Realize what you need - single or family pack?

It is important to understand and realize that what you ultimately need, and how the final target OSGified bundle will look depends on your use case. If operating in a pure and strict OSGi world, every library should be in its own bundle. In other cases, where there is no strict requirements around your dependencies being strictly separate bundles, you can combine a few libraries together to get a single bundle (or may be still a few more). However, both these ways of bundling has their own pros and cons.

Single bundles are the citizens of perfect OSGi world. This bundle would only represent one vanilla jar. The advantage of this approach is that you fully leverage the OSGi modularity. If there are multiple bundles which wants to use this library, one instance of this library would do (as long as the version requirements also match).  Your artifacts would be light. However, you may have to deal with and manage a lot of these bundles if the transitive dependencies of your target library is just too big.

Family pack - as I love to call it - is usually made by combining a related set of libraries together to form a single bundle. In Maven world, this often can be related to as the transitive dependencies. This kind of packaging may sometimes be necessitated by the library itself, which often has code which has hard, non-OSGi compliant bad OSGi practices code for loading classes and resources, which requires some other dependent libraries to be on the same classpath (OK, you need to understand that in OSGi, every bundle has its own classpath). At other times this kind of packaging avoids you to manage a lot of bundles on your target environment. However, this would mean that tomorrow if another library needs to be added, which has one of the transitive dependencies that already exist somewhere in your existing family pack bundles, either you would duplicate the library over in your second family pack, or just expand your existing family pack. Family packs at times can also lead to cyclic dependencies being created between bundles, which really is just evil.

Things to Remember

Once you have decided the kind of packaging you require, you can move ahead and start creating and perfecting your bundle. The following are some of the things to be taken care of when you start your OSGification process:

1. Know what you are doing

Unless the vanilla jar you are trying to wrap is really simple, you would have to understand OSGi a little bit deeper - especially what each of the manifest headers mean, and how it affects the classloading and resource resolution. What each of the BND instructions mean and how it affects the bundle being generated. Read the BND documentation, and also the tool specific documentation - in my case the Maven Bundle Plugin's documentation.

2. Verify the vanilla jars versus your OSGified bundle

Verify that the contents of the vanilla jar and your OSGified bundle to ensure that you preserve the packages and the resources contained therein, unless you intend it otherwise. The resources iteself may contain some configuratoin files that are required by the library to function, or it may have some licenses. You should be particularly be aware that almost all the licenses and notices require that you keep them in the derived products. If you are merging more than one library, you may have to consider renaming the resources - Maven Shade Plugin can be useful here.

3. Perfect on the contract - Import and Export packages

The Export-Package and the Import-Package are the primary contract of your OSGi bundle. What goes into these should be absolutely clear. Try to understand the library clearly, and export those packages which are API, and hide the internal implementations (some libraries hardly distinguish this fact, so that forces you to have not-so-perfect OSGi practices). Also ensure that your exported packages has a logically valid version. For instance, if you are OSGifiying itext-2.1.7, you could start with an Export-Package of com.lowagie*;version="2.1.7".

What to keep for Import-Package is trickier sometimes. This is where you would have to experiment a bit. Start with all the import packages being optional (*;resolution:=optional), and then once your client bundle is all working perfect, come back here and understand what is absolutely necessary for the library to function, and which are optional. As an example, some libraries may give out dependencies to JUnit classes. Now, why would someone depend on JUnit classes, so quietly remove them from the import packages. On the other hand, dependencies to JAXB libraries could possibly be made as optional one. All this depends on the library under consideration.

4. Ensure no BND warnings

When you are perfecting the contract of Import and Export packages, ensure that you have no BND warnings. Especially around split packages. When you get the split packages, carefully study your vanilla jars (often split package warnings will come when you are making a family pack), and understand of how would you like to prioritize the packages across the libraries to apply the split-package instructions for BND.

5. Dealing with cyclic dependencies

As mentioned before, multiple family packs can lead to cyclic dependencies. When this happens, carefully study the bundles involved in cyclic dependencies and the packages on which these links are formed, and carefully remove the cyclic dependency by either separating out one of the transitive jar as a separate OSGi bundle, or simply keep all the packages of such a library as private. Take the decision based on your situation.

6. Adjusting your client bundle

At times, your bundle may be importing certain packages from the wrong bundle (instead of what you are intending). Example of such packages may be what is exported by the system bundle which is outdated, and you instead want your client bundle to import from a later version exported by a separate OSGi bundle. In these cases, use the version as your savior and explicitly restrict the package versions so that it would import from the bundle of your choice. Often the client bundle may have a relaxed import versioning, which makes it to import from the system bundle. When the version is stricter, you would often achieve what you intend to.

7. Classloader Hell

At times you may get into situations that even after following all the instructions above, your library is unable to load some classes or resources. Carefully try to debug through the library's source code and see where actually the code fails. Also compare this execution to one in a non-OSGi environment, and you would be able to identify that in OSGi, the library is unable to instantiate a class or load a library, because of the way that the code is written to do that. In these cases, the last option may be to patch the library. Download the source code of the original library and make modifications to the offending classes and put it back to the OSGi bundle. Follow the OSGi best practices when you do dynamic class loading or resource loading.

8. Test thoroughly

Test you client bundle thoroughly with all the possible use cases of using the library. If possible use the Pax Exam to write a test suite to be sure that you are covering all the use cases. Everytime you run the above environment, you will realize - most often with OSGi resolution error messages or error stack traces that something went wrong. This information is key to identifying what has to be tweaked to get it working. Repeat the all the steps above through this step, until you are done.

9. Be license compliant

Nothing technical, but respect the licenses of the libraries you are trying to OSGify.

For AEM / CQ developers

For sometime I have been working on the Adobe's marketing platform AEM /CQ which utilizes OSGi as the application container and so requires developers to package their code as OSGi bundles and many of them get panic when they have to OSGify an external dependency - especially when it is a complicated one. I have personally found that most of these AEM developers do not understand the OSGi beyond what is there in the product documentation. And this was my primary motivation for writing this article.

I have made some shortcuts for kick-starting your OSGification process for AEM environment, by creating Pax Runner profiles for few AEM versions at my Github repository https://github.com/ethomasjoseph/aem-pax-runner-profile. Feel free to tweak and use this and contribute to this if you find this useful.

Before you say - I am done (or not)!

Bundling a vanilla jar - especially the complicated ones require a lot of patience and experience. A lot of the above instructions are dependent on the context of which library are you trying to work with and what exactly is the need of the hour. Weigh the choices to choose your track. Feel free to reach out to me (blog comments are OK), in case you want some help in OSGification of your vanilla jar.

Happy Bundling!!!

References

1. BND's official site
2. Maven Bundle Plugin
3. Apache Commons OSGi tracker
4. Instruction of creating bundle using Maven and BND
• http://felix.apache.org/documentation/subprojects/apache-felix-commons/creating-bundles-using-bnd.html
• http://spring.io/blog/2008/02/18/creating-osgi-bundles/
• http://gnodet.blogspot.in/2008/09/id-like-to-talk-bit-about-third-party.html
• https://karaf.apache.org/manual/latest/developers-guide/creating-bundles.html