camunda BPM Platform OSGi 1.0.0 released

Today I am happy to announce the version 1.0.0 release of the camunda BPM Platform OSGi project.
Especially because I am the maintainer of the project ;-)

The project started on 17th of November when we moved the "old" OSGi module out of the core platform and made it a community project.
So let's start with a review.

What did we do?

 

First of all we now have a lot more test coverage. At the beginning there were zero tests and now we should have roughly 80% test coverage across all modules. Next to the tests there was a lot of refactoring to have smaller classes with clear responsibilities (that's what refactoring is all about, right? ;-))

Secondly, we have a Apache Karaf feature.xml and a Blueprint example project.

The first contribution from "outside" was the Apache Karaf commands module, which was developed by Elek from DCP Consulting. Thank you, again!

Finally, there is the new OSGiELResolver, which was included a few weeks ago. The new ELResolver gives us some independence from Blueprint.

As you can see, we did quite a few things, but there are still some tasks left.

What's left to do?

 

The ToDo-list states the following:

1. adapt Process Application API for OSGi
2. camunda webapp WAB (cockpit, tasklist, admin)
3. create example for configuring engine using PAX-CDI

Number one Daniel, Roman and I tried to solve in May. All the results are in the platfrom-api-hack branch. They still need some review.
Number two and three are still open.

That's what's left on the ToDo-list, but what else is there to do? 

The future

Of course it would be great to get some feedback from "real world" users and I hope more people will use the OSGi module in the future.
Then the open ToDos got to get done and I guess we'll find some ideas for the future (maybe Apache ServiceMix with camunda BPM).

After we've taken a look at the past, the present and the future there is only one thing left:

Finally

 

A big "thank you" to Daniel and Roman for guidance, support and having time for a hackathon with me! It is a pleasure working with you!

Read More

Consuming arbitrary remote services with the OSGiELResolver (camunda BPM OSGi)

In my last blog post I promised to give a slightly more advanced example about how to use the new OSGiELResolver. And as I promised, here it is ;-)

Prerequisites

The setup is quite simple. We have three bundles:

1. API
2. Service Provider
3. Service Consumer

You can find all the sources here. (feel free to suggest improvements, possible bugs, etc.)
As runtime I used two Apache Karaf instances on my computer (version 2.3.5; I had some problems with 3.0.1).
For remoting we'll use Apache CXF 1.4 (single bundle release).
And of course we'll need camunda BPM platform OSGi, which you'll have to build yourself.
Before I tell you more about the three bundles I'd like to point the book "Enterprise OSGi in Action" out. Without that great book I couldn't have provided this example. It's definitely worth reading.

So, enough advertisement, let's take a look at the bundles.

API bundle

 

The API bundle is really simple. It only contains one interface with a method. We'll need the bundle in both runtimes.

Provider bundle

 

Now we're getting a little bit more serious. The provider bundle contains the service implementation we want to use.
The context.xml contains the important parts for remoting:
<entry key="service.exported.interfaces" 
 value="de.blogspot.wrongtracks.osgielresolver.api.SomethingService"/><entry key="service.exported.configs"
       value="org.apache.cxf.ws"
<entry key="org.apache.cxf.ws.address"
       value="http://localhost:9001/somethingservice"/>

"service.exported.interfaces" should be obvious.
"service.exported.configs" tells Distributed OSGi to look for implementation specific properties.
Lastly "org.apache.cxf.ws.address" lets us define an alternative address. It is quite helpful if you don't want to type the fully qualified name of the class in your browser or other config files.

Consumer bundle

 

Let's take a look at the consumer. This bundle needs a little bit more information to work properly. To be able to consume remote services we need the OSGI-INF/remote-service/remote-services.xml. It doesn't have to be that name or that directory. You can specify the path inside the bundle with the "Remote-Service" header, which I set in the POM to:
      <Remote-Service>OSGI-INF/remote-service/*.xml</Remote-Service>
I won't walk you through the remote-services.xml. I'm sure you'll find better explanations somewhere else. (e.g. in Enterprise OSGi in Action ;-) )

After we configured this we can use the reference tag in the context.xml to find the service.
To make the service work with the OSGiELResolver we have to add two things. In the remote-services.xml the property "processExpression" has to be set and in the context.xml we have to use a filter.
As you may know the ELResolver uses the filter to search for classes. Searching only worked when both, attribute and filter, were set.

The provider Karaf

 

Like I said, I used Karaf as runtime. The "provider" Karaf needs three bundles:

1. API
2. Provider
3. Apache CXF

Just drop them into the deploy directory. It worked best for me when I started them in the order API, CXF and provider. Then everything should work as expected.

The consumer Karaf

 

The "consumer" Karaf needs a little bit more bundles (and if you run it on the same machine you'll have to change three ports). You have to add:

1. API
2. Consumer
3. Apache CXF
4. camunda BPM platform OSGi and dependencies

Drop API, consumer and CXF jars into deploy (again, starting API, CXF and then consumer works best). Adding camunda BPM platform OSGi isn't very difficult because there is a feature.xml (assumed it is installed in your local Maven repository).
To install it type:
features:addurl mvn:org.camunda.bpm.extension.osgi/camunda-bpm-karaf-feature/1.0.0-SNAPSHOT/xml/features

and then:
features:install camunda-bpm-karaf-feature-minimal

This should resolve all you bundles. Now, If you start the consumer bundle you should see the log saying "Started process". Strangely the logger of the service implementation was quiet. But if you uncomment the exception you can see that the service was called.

So, as you can see, the new OSGiELResolver makes it possible to consume arbitrary remote services, which is quite an improvement. I hope my example is understandable and helps to see the possibilities.

Hint

 

When you encounter this exception:
java.lang.IllegalStateException: Invalid BundleContext
just start the CXF bundle again, then it should work.

Read More

camunda BPM OSGi: the new OSGiELResolver

Introduction

Some of you may know that I am the maintainer of the camunda BPM OSGi project.
Several weeks ago I started to implement a new ELResolver (EL = expression language) and because it's finished now I want to do some shameless self-advertising for my work ;-)

The problem

The "old" ELResolver had some limitations: It could only work with one kind of classes (those who implement the JavaDelegate interface) and you had to register the ELResolver as service listener.
Also, the implementation depends on Blueprint because it used the registered component id to find the classes.

The new OSGiELResolver

The new OSGiELResolver doesn't have those limitations. You can use it theoretically with every class and it doesn't depend on Blueprint. If you want to know more, please have a look at the updated README. I would be happy if you could give me some feedback or ideas for improvement.

So far for now. I'll try to put together a more advanced example, soon.

Please note: this change breaks the API because I moved some classes, so this version would be a new major version number, if it weren't for the snapshot ;-)

Read More

First steps with Apache ACE

Introduction

"Apache ACE is a software distribution framework that allows you to centrally manage and distribute software components, configuration data and other artifacts to target systems." (from https://ace.apache.org/)
Well, that sounds good enough to try it out, at least for me.
I like the idea to centrally configure deployments with different version and have a way to automatically distribute those.

Starting ACE

Setting up Apache ACE was pretty easy. The Getting started guide contains all the necessary steps.
My MacBook was the ACE server and my Raspberry Pi a target.

Using the Web GUI is easy and straightforward (nice one guys ;-) ).
But that's for little children. I wanna find a way to automate everything with scripts.

The Client Shell API

Basically there are two ways to talk to the server remotely. One is the Client Shell API and the other way is via REST API. For now I'll stick with the Shell API

1st step: connecting to the server as shell client

Before we can write a script we have to connect to the server.
With some help from the iQSpot people (see here) I figured it out.
They suggest starting the client like this:

java -Dagent.discovery.serverurls="http://server:port"
     -Dorg.apache.ace.server="server:port"
     -Dorg.apache.ace.obr="server:port"
     -Dorg.osgi.service.http.port=-1
     -jar client.jar

Unfortunately, that didn't work for me (even after adding some missing backslashes).
The default is that you should be in the directory of client.jar. "-jar client.jar" wasn't the problem.
The startup searches for the client/conf directory, so when you see this exception:

java.lang.IllegalArgumentException: Bad arguments; either not an existing directory or an invalid interval.
    at org.apache.ace.configurator.Configurator.<init>(Configurator.java:89)
    at org.apache.ace.configurator.Activator.init(Activator.java:33)
    at org.apache.felix.dm.DependencyActivatorBase.start(DependencyActivatorBase.java:76)
    at org.apache.felix.framework.util.SecureAction.startActivator(SecureAction.java:645)
    at org.apache.felix.framework.Felix.activateBundle(Felix.java:2146)
    at org.apache.felix.framework.Felix.startBundle(Felix.java:2064)
    at org.apache.felix.framework.Felix.setActiveStartLevel(Felix.java:1291)
    at org.apache.felix.framework.FrameworkStartLevelImpl.run(FrameworkStartLevelImpl.java:304)
    at java.lang.Thread.run(Thread.java:722)

You're probably starting the client from a different directory.
To get rid of that exception we have to set a property:
-Dorg.apache.ace.configurator.CONFIG_DIR=
All in all the command to start the client looks like this:

java -Dagent.discovery.serverurls="http://server:port"\
     -Dorg.apache.ace.server="server:port"\
     -Dorg.apache.ace.obr="server:port"\
     -Dorg.osgi.service.http.port=-1\
     -Dorg.apache.ace.configurator.CONFIG_DIR="apache-ace-2.0.1-bin/client/conf"\
     -jar apache-ace-2.0.1-bin/client/client.jar

Now we can start the client.
But to pass a script to the shell we need two more arguments. Thanks again to the iQSpot people. They already pointed out those arguments:

• -Dgosh.args="–-args"
• -Dace.gogo.script.delay=delay
• -Dace.gogo.script=/path/to/script.gogo

What do those three do?
Everything you'll pass as "gosh.args" will be executed immidiately. If pass "--help" for example and start the client you'll see the help output.
The delay is helpful when you want to give your client some time to synchronize with the server.
"ace.gogo.script" should be obvious ;-)
We end up with the following command:

java -Dagent.discovery.serverurls="http://server:port"\
     -Dorg.apache.ace.server="server:port"\
     -Dorg.apache.ace.obr="server:port"\
     -Dorg.osgi.service.http.port=-1\
     -Dorg.apache.ace.configurator.CONFIG_DIR="apache-ace-2.0.1-bin/client/conf"\
     -Dace.gogo.script.delay="3000"\
     -Dace.gogo.script="script.foo"\
     -jar apache-ace-2.0.1-bin/client/client.jar

Now we have to find out, what we should put into "script.foo".
 

Shell commands

Every (basic) command is described here.
The steps are quite simple: cw, ca, cf, ca2f, cd, cf2d
If you don't like or get the abbreviations (it took me a while) there is also a nice picture in the REST API documentation:

What the picture is missing is cw or "create workspace". When using the Shell API you need a workspace, which you can commit later.

The script

 

So, what should skript.foo do? Let's assume we have to upload some generated artifacts from our CI server 
The steps are

1. create workspace
2. add the new Jars as artifacts from certain directory
3. create a new feature
4. add artifacts to feature
5. create a new distribution
6. add new feature and existing ones to distribution
7. add feature to existing target 

I have to admit that it took me quite a while to figure everything out because I'm not very experienced with Apache Felix GoGo.
Creating the workspace is easy: w = (cw)
Now we can call the workspace with $w. Adding the Jars was more difficult. Let's assume the directory is ./toAdd. Then the command looks like this: 

each ([(ls toAdd)]) {$w ca (($it toURL) toString) false}

You "toAdd" can be changed to any path and you could use some wildcards, like ls toAdd/*.jar
I guess if you're used to GoGo the command won't be a surprise. If you're not used to it, I would like to explain the different parts to you:
each takes a list and a function. ls toAdd returns a File array. That's why we need the brackets. They convert the array into a list. After that comes the function, indicated by the braces.
$w ca is the method to create an arfifact. $it is the iterator over the list that is provided by each.
Then we call the methods toURL and toString because reflection makes it possible ;-)

Third step: add all artifacts to feature

each ($w la "(Bundle-SymbolicName=org.camunda.*)") {symbolicName=($it getAttribute "Bundle-SymbolicName"); $w ca2f "(Bundle-SymbolicName="$symbolicName")" "(name=test-feature)"}

Again, we use a for-each-loop.
$w la lists all the bundles that match the passed pattern. (Here, I want to add all camunda bundles, no advertisement ;-))
Then I save the symbolic name in a variable, so it's easier for me later to reference it.
org.apache.ace.client.repository.RepositoryObject has a getAttribute method, which we use here.
Also, please note the semicolon.
We use the symbolic name as part of the first argument for ca2f (create artifact2feature).
The String contains three parts

• "(Bundle-SymbolicName="
•  $symbolicName
• ")"

I don't know why, but we don't ne a "+" for string concatenation. The second argument is the name of the feature. I just assume it to stay the same: "test-feature"
Creating a distribution and a feature2distribution are nothing special.

All in all we end up with the following:

w = (ace:cw)
$w cf test-feature
$w cd test-distro

each ([(ls toAdd)]) {$w ca (($it toURL) toString) false}

each ($w la "(Bundle-SymbolicName=org.camunda.*)") {symbolicName=($it getAttribute "Bundle-SymbolicName"); $w ca2f "(Bundle-SymbolicName="$symbolicName")" "(name=test-feature)"}

$w cf2d "(name=test-feature)" "(name=test-distro)"

$w commit

That should do the trick so far.
Stay tuned for my next steps with ACE ;-)

Read More

Glassfish 4, Commons Mail and "UnsupportedDataTypeException: no object DCH for MIME type multipart/mixed"

I know there are a bazillion posts/threads/etc. about the exception mentioned in the title and now there are a bazillion + one ;-)
Unfortunately I couldn't find the solution I want to present to you anywhere else.

First some context:
My class extends an Activiti class and uses Apache Commons Mail to send an email.
The email contains some text and has a file (txt/pdf/docs) attached.
Everything runs inside a Glassfish 4 and the Jars are deployed as OSGi bundles.

When calling email.send() the server threw the feared UnsupportedDataTypeException:

Caused by: javax.activation.UnsupportedDataTypeException: no object DCH for MIME type multipart/mixed;
    boundary="----=_Part_0_397989068.1398665205325"
    at javax.activation.ObjectDataContentHandler.writeTo(DataHandler.java:891)
    at javax.activation.DataHandler.writeTo(DataHandler.java:317)
    at javax.mail.internet.MimeBodyPart.writeTo(MimeBodyPart.java:1574)
    at javax.mail.internet.MimeMessage.writeTo(MimeMessage.java:1840)
    at com.sun.mail.smtp.SMTPTransport.sendMessage(SMTPTransport.java:1119)
    ... 120 more

Like I mentioned the Internet is full of solutions but none of them worked for me.
Because the Glassfish showed me that all of my bundles were correctly linked and resolved the problem had to be another place.

My colleague then told me I should try to change the TCCL. After some try-and-error it worked (I tried the one from commons.mail, the one from javax.activation and one I forgot ;-)).

The solution was to import javax.mail in my bundle and change the TCCL to the javax.mail classloader:

Thread.currentThread().setContextClassLoader(javax.mail.Message.class.getClassLoader());
email.send()

I am not sure why only the javax.mail classloader works. For me it is some arcane dependency/classloading/visibility problem.
Nevertheless, I hope this post helps some Glassfish/OSGi users.

Finally, I would like to thank my colleague @spost1970 for helping me find a solution.

Read More

Integrating Apache Aries blueprint into Glassfish 4

After experimenting with Glassfish 4 lately I would like to let you know what I am up to. It's nothing big (yet) ;-)

A little bit of background

Glassfish comes with integrated OSGi support (Apache Felix) but without Blueprint (as far as I know). So putting a Blueprint container into Glassfish became my task.

The nice thing about Glassfish is that it combines Java EE (especially EJBs) and OSGi and that, in contrast to JBoss, it has a nice OSGi web console.
If you want to get to know more about the OSGi-JEE combination the keyword for your favourite search engine is "fighterfish".

Most of the credit goes to Yong Tang and his blog entry. He describes the integration of Aries Application but also describes the basic parts necessary for my task.

The problem

So, before we start, what is the problem? If you're familiar with Glassfish you certainly know there is an autodeploy/bundles directory. Why don't I just drop the necessary bundles into the autodeploy/bundles directory?
When using the autodeploy directory Glassfish doesn't start the bundles so you'll have to do it by hand every time you empty the osgi-cache directory and, of course, initially.
But there's a more convenient way.

Let's get it on

What's the better way?
Just drop the jars

• Aries Blueprint Api(v1.0.0)
• Aries Blueprint(v1.1.0)
• Aries Proxy (v1.0.0) 
• Aries Util(v1.0.0)

into glassfish/modules/autostart.
To make sure all dependencies are there add slf4j api (v1.7.2), logback core(v1.0.13) and logback classic (v1.0.13) (or whichever logging framework you prefer). You don't need any additional configuration because we didn't create a subdirectory.

See, piece of cake. The trick is to find the right directory. Now the Blueprint extender will do its job right after start up.

glassfish/modules/autostart

glassfish/modules/autostart

glassfish/modules/autostart

glassfish/modules/autostar

glassfish/modules/autostar

Read More

Activiti/camunda BPM: custom behavior and BPMN extension elements using Blueprint

Introduction 

 

In the last few weeks I have been working on a problem regarding OSGi-Blueprint and Activiti. Because it wasn't as easy as I would have hoped I want to share my solution with you. I will start by explaining my environment, show you the problem and then I will explain my first attempt. After that I will present my solution. Finally I will show some ideas how to make it better and things that I did not test.

Just a short hint about the writing: when I reference a class or some XML it's written in italic, e.g. Object. When you see "process engine", I am talking about the whole thing, but when you see ProcessEngine it's the actual class.

My environment

 

I use the Activiti-framework in version 5.12.1, Apache Aries in version 1.0.0 with a little modification and my own ProSt bundles. ProSt can be found here. The README.md explains why and what I changed in Aries.

I haven't tried, yet, but I am pretty sure that camunda BPM suffers the same problem because
both share the same MailActivityBehavior and BlueprintELResolver classes and use <extension-elements> for injection. So if you prefer camunda and see "activiti" somewhere you just have to replace it with "camunda" in your head ;-)

The problem 

 

In general, I just wanted to send an e-mail during my process-execution. Sounds pretty easy, right?

My SendMailWithAttachmentBehaviour class extends the previously mentioned MailActivitiBehavior class. The process definition contains all the necessary information to send the e-mail, e.g. from, to and subject. Only the attachment is missing, which I get from the execution environment.

Because I use Blueprint I cannot use the activit:class or type="mail" attributes in the process definition. I have to declare the class this way:
activiti:deleExpression="${sendMailWithAttachmentBehavior}"

A little hint: the name in the braces has to match the one used as bean id in the blueprint.xml.

The other ways do not work with OSGi because of class visibility etc.

The easy part was to extend the BlueprintELResolver class (ProStBlueprintELResolver) and add a way to add custom behavior classes at the moment.

So, what happens when the process engine tries to resolve the expression?
When the bundle is loaded Blueprint creates a dynamic proxy and registers it at the ProStBlueprintELManager.
After the process reaches the ServiceTask which delegates to the ${sendMailWithAttachmentBehaviour} the process-engine asks its ExpressionLanguageResolvers if they know something with the name "sendMailWithAttachmentBehaviour". Logically the proxy is found.
After that the process engine tries to set the extension-elements at the class.
First it tries to find setter methods and if it cannot find setters it tries field injection. (see ClassDelegate.applyFieldDeclaration())
Both ways do not work.
But why?
Of course a proxy does not have any fields. But why is it not possible to just add the setters to the SendMailWithAttachmendBehaviour class?
The call is proxy.getClass().getMethods() and according to the documentation this will return all the methods of the interfaces that the proxy was created with. ActivityBehavior does not declare the setSubject() etc. methods because they are only needed for e-mails.

First attempt

 

At first I thought the solution was quite obvious. I would just export a second interface containing the setters like this:

<bean id="sendMailWithAttachment"
class="de.blogspot.wrongtracks.prost.example.behaviour.SendMailWithAttachmentBehaviour" />


  

<service ref="sendMailWithAttachment">

    <interfaces>

      <value>org.activiti.engine.impl.pvm.delegate.ActivityBehavior</value> 

      <value>de.blogspot.wrongtracks.prost.example.behavior.ExtensionElementsMailSetter</value> 

    </interfaces>

</services> 

But wait, if you take a look at the (old) context.xml you can see that my reference listener just listens for ActivityBehavior and not the other interface. That's why the created proxy won't contain the methods from the other interface. Too bad...

The solution

 

I found the solution accidentally while reading the Apache Aries Blueprint documentation. This chapter points out that you can also listen for service references.
I changed the methods to accept a ServiceReference instead of a ActivitiyBehavior and when the expression should be resolved I use the BundleContext to get the service. At that point it is not a proxy, it is the implementation.

Then everything works just fine. I don't even need the setter interface anymore.
You can see the solution when you look at the new context.xml and the previously mentioned ProStBlueprintELResolver. (the previously showed link pointed to an old version so nothing would be spoiled ;-) )

That's it, that is my solution to add custom behavior to the process engine and use extension elements in the BPMN XML.

How could we improve the whole thing?

 

Strangely, I have no idea how the whole thing could be improved. I would like to hear your ideas. Also, I would like to know if you think that the way presented here is good or bad or something in between.

What didn't I try?

 

You should note that I have not tried to find out how JavaDelegates behave in the same situation. I just did not have time and I wanted to show you my solution as soon as I finished it.

Read More