My experiences about a JMS interchange between HornetQ and ActiveMQ over Apache ServiceMix

I recently had a project where I had to deploy some Apache Camel routes, deployed over ServiceMix, that had to connect to a HornetQ messaging system, deployed on a JBoss Application Server.

My first intention was to wrap the HornetQ libraries as OSGi bundles and deploy them on Apache ServiceMix, but it was a real nightmare, with numerous class loading problems, terrible! There is an example of a similar use case of mine, written by Torsten Mielke, but I couldn’t establish the connection between Apache ServiceMix 6.0.0 and JBoss EAP 6.3. On the other hand, there is a HornetQ issue to make it OSGi compliant, but it also didn’t work out in my case.

My final approach to the problem was to set up HornetQ bridges between JBoss EAP 6.3 and the JMS installation of ServiceMix: Apache ActiveMQ. You can find the code of my proof of concept in one of my GitHub repositories. I honestly think that this solution desvirtues the concept of an ESB, because the routing is made outside, but it works!


Setting up a JMS bridge between Weblogic and ActiveMQ

Almost four years ago, I wrote about how to setup a JMS bridge between Weblogic and HornetQ. Lately, I’ve had to research how to do the same work with ActiveMQ. Here you have my findings. As it was for HornetQ, the first step was to copy the client libraries to a folder residing in a filesystem of my Weblogic server, in the case of ActiveMQ these files are:

  • activemq-client-5.10.0.jar
  • hawtbuf-1.10.jar
  • slf4j-api-1.7.5.jar

Later on, I set the PRE_CLASSPATH variable pointing it to these libraries into the script setDomainEnv. Creating messaging destinations is more or less the same job as it was for HornetQ, you just have to change the initial context factory and connection url parameters:

<jms-bridge-destination>
  <name>JMS Bridge Destination-Target</name>
  <adapter-jndi-name>eis.jms.WLSConnectionFactoryJNDIXA</adapter-jndi-name>
  <user-name>testUser</user-name>
  <user-password-encrypted>{AES}GZTa15osrQW6/5rbE2fzm+a9BX/YPY7Hm3xiIFi0oMQ=</user-password-encrypted>
  <classpath></classpath>
  <connection-factory-jndi-name>jms/TestXAQueueConnectionFactory</connection-factory-jndi-name>
  <initial-context-factory>org.apache.activemq.jndi.ActiveMQInitialContextFactory</initial-context-factory>
  <connection-url>tcp://localhost:61616</connection-url>
  <destination-jndi-name>jms/TestQueue</destination-jndi-name>
  <destination-type>Queue</destination-type>
</jms-bridge-destination>

But the former configuration is not enough, because ActiveMQ hasn’t his own JNDI provider (I had ActiveMQ as a JMS provider for a ServiceMix ESB) and requires a jndi.properties file with the mappings between physical destinations and tha jndi ones, but, how to configure the properties file in the context of a Weblogic messaging bridge? Here is the trick: create a JAR archive containing the jndi.properties file and put the JAR in the CLASSPATH (the same way at it was described before for activemq-client-5.10.0.jar). The steps to create the JAR archive are:

  • Create a file called jndi.properties with the following entries:

connectionFactoryNames=ConnectionFactory,XAConnectionFactory queue.TestQueue=TestQueue

  • Create the archive with the following command:

jar cvf jndi.jar jndi.properties

I found this trick in this post, after have tested many different configurations, including the setup of a Weblogic Foreign JMS Server, without success. Finally, I’d like to point out that I am not completely happy with this setup because it has an obvious drawback: you have modify and redeploy the jndi.jar archive every time you add a new queue or topic, so suggestions are welcome!!!


References


Oracle AQ: working with PL/SQL asynchronous notifications

I like Oracle Streams Advanced Queuing (AQ), it’s reliable and fast. I’ve been working with this technology for the last four years, with 10g and 11g database versions; most of the time, I’ve had to interact with Java EE systems, through Java Message Service (JMS), which it’s fully supported by Oracle AQ. JMS has Message Driven Beans (MDB) as the standard way to consume messages in Java EE, as its counter part on the Oracle database you can register asynchronous notifications to PL/SQL procedures. To be perfectly honest, I’ve always considered that the configuration of this functionality is a bit tricky, because sometimes you don’t get an error, it simply doesn’t work. That’s why I’m writing this post, to present a simple example of PLSQL asynchronous notifications that you can download from github.

The first point that I’d like to deal is the sign of the callback procedure, standalone or belonging to a package, that will consume the messages:

procedure receive_message_callback (
   context  raw,
   reginfo  sys.aq$_reg_info,
   descr    sys.aq$_descriptor,
   payload  raw,
   payloadl number
);

I think that the key here is the type of the argument payload: raw or varchar2, depending on the type of the message. I’ve prepared my sample with my personal Oracle Database Express Edition 11g Release 2, where I couldn’t work with Oracle AQ JMS types, so I’ve used a custom data type as the payload of the messages, which implies that the payload argument type has to be raw, but if you work, for example, with JMS Text Messages, SYS.AQ$_JMS_TEXT_MESSAGE in Oracle AQ, the payload argument type has to be varchar2.

The second issue is that the configuration needed varies depending if the destination is a queue or a topic.
In a queue each message is consumed by just one consumer, so you simply has to register the callback procedure. Here you have an excerpt of my code:

begin
   dbms_aqadm.create_queue_table (queue_table => 'queues_qt',
                                  queue_payload_type => 'TESTAQ.MESSAGES_T');

   dbms_aqadm.create_queue (queue_name => 'test_queue',
                            queue_table => 'queues_qt');

   dbms_aqadm.start_queue (queue_name => 'test_queue');

   dbms_aq.register(
      sys.aq$_reg_info_list(
         sys.aq$_reg_info(
            'TESTAQ.TEST_QUEUE',
            dbms_aq.namespace_aq,
            'plsql://TESTAQ.TEST_P.RECEIVE_MESSAGE_CALLBACK',
            hextoraw('FF')
         )
      ),
      1
   );
end;
/

The tip to remember here is not to forget the schema name prefix before queue and callback procedure names.

In a topic each message can be consumed by one or several subscriptors and each of them can process the message in a different way, for example sending an email, instead of processing by a PL/SQL procedure. So, you first have to register a subscriber, an agent in Oracle AQ terminology, and then register the PLSQL consumer. Here you have an excerpt of my code:

begin
   -- It's a topic, so multiple_consumers parameter is specified.
   dbms_aqadm.create_queue_table (queue_table => 'topics_qt',
                                  queue_payload_type => 'TESTAQ.MESSAGES_T',
                                  multiple_consumers => true);

   dbms_aqadm.create_queue (queue_name => 'test_topic',
                            queue_table => 'topics_qt');

   dbms_aqadm.start_queue (queue_name => 'test_topic');

   dbms_aqadm.add_subscriber (queue_name => 'test_topic',
                              subscriber => sys.aq$_agent(
                                               name => 'demo_subscriber',
                                               address => null,
                                               protocol => 0));

   dbms_aq.register(
      sys.aq$_reg_info_list(
         sys.aq$_reg_info(
            'TESTAQ.TEST_TOPIC:DEMO_SUBSCRIBER',
            dbms_aq.namespace_aq,
            'plsql://TESTAQ.TEST_P.RECEIVE_MESSAGE_CALLBACK',
            hextoraw('FF')
         )
     ),
     1
   );
end;
/

The tip to remember here is not to forget to put the name of the subscriber after the name of the topic, when you’re registering the callback procedure.


References



Weblogic: Interoperating with Oracle AQ JMS. A tip for sending messages

I usually work with Oracle Streams Advanced Queuing (AQ) integrated with Weblogic. In PL/SQL programs you can set the exception queue where you want to move the messages that couldn’t be delivered to their destinations, when you send a message, for example:


declare
  enq_msgid raw(16) ;
  eopt      dbms_aq.enqueue_options_t;
  mprop     dbms_aq.message_properties_t;
  message   sys.aq$_jms_text_message;
begin
  mprop.priority        := 1;
  mprop.exception_queue := 'test_exception_q';
  message               := sys.aq$_jms_text_message.construct() ;
  message.set_text('This is a Test') ;
  dbms_aq.enqueue(queue_name => 'test_q', 
                  enqueue_options => eopt,
                  message_properties => mprop, 
                  payload => message, 
                  msgid => enq_msgid) ;

  commit;
end;

But, how to set the exception queue when you’re programming in Java, in the context of an integration between Oracle AQ and Weblogic? The solution I’ve found is to set the string property called JMS_OracleExcpQ to the message with the name of the Oracle AQ exception queue, for example:


...
javax.jms.TextMessage msg = jmsSession.createTextMessage();
msg.setStringProperty("JMS_OracleExcpQ", "TEST.TEST_EXCEPTION_Q");
msg.setText("This is a Test");
jmsProducer.send(msg);
...

An important advice: put the name in uppercase and preceded by the name of the schema.

Finally, I’d like to point out that this is a good solution in the context of an integration between Oracle AQ and Weblogic, but makes the code non portable, if you want to change the messaging provider of your Weblogic Application Server.


References


Researching the HornetQ REST interface

I’ve been studying the HornetQ REST interface during the last weeks, because I consider it’s a very good solution if you have an application that uses JMS and you want to integrate messages from a third party that doesn’t use Java language. This is the first post of the ones I’ve decided to write about this issue and, in this case, the first task is to configure and install the interface.

As usual, it’s important to review the user’s manual of the technology, this time the HornetQ REST interface one.  One important restriction in my case was that I had to use the version 4.2.1 GA of the JBoss Application Server, with HornetQ 2.2.5 as its messaging provider. Therefore, the pom.xml file proposed by the manual had to be changed, in order to add the dependency on the RESTEasy libraries and the 1.5 Java version for both the source and target code:

<project xmlns="http://maven.apache.org/POM/4.0.0" 
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>

  <groupId>com.wordpress.fcosfc.hornetq</groupId>
  <artifactId>HornetQRESTInterface</artifactId>
  <packaging>war</packaging>
  <version>1.0</version>
  <name>HornetQRESTInterface</name>
  <repositories>
    <repository>
      <id>jboss</id>
      <url>http://repository.jboss.org/nexus/content/groups/public/</url>
    </repository>
  </repositories>

  <dependencies>
    <dependency>
      <groupId>org.hornetq.rest</groupId>
      <artifactId>hornetq-rest</artifactId>
      <version>2.2.2.Final</version>
    </dependency>
    <dependency>
      <groupId>org.jboss.resteasy</groupId>
      <artifactId>resteasy-jaxrs</artifactId>
      <version>2.2.2.GA</version>
      <exclusions>
        <exclusion>
          <artifactId>commons-logging</artifactId>
          <groupId>commons-logging</groupId>
        </exclusion>
      </exclusions>
    </dependency>
    <dependency>
      <groupId>org.jboss.resteasy</groupId>
      <artifactId>resteasy-jaxb-provider</artifactId>
      <version>2.2.2.GA</version>
    </dependency>
  </dependencies>

  <build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-compiler-plugin</artifactId>
        <version>2.0.2</version>
        <configuration>
          <source>1.5</source>
          <target>1.5</target>
        </configuration>
      </plugin>
    </plugins>
  </build>
</project>

Here you have a link to my Maven project, I hope it will be useful for you.


References


Trying to set up a JMS bridge between HornetQ and Oracle AQ

I’ve been interested in setting up a JMS bridge between HornetQ and Oracle Advanced Queuing (AQ) for a while, even I wrote about this issue in a response to a question of the HornetQ user’s forum. I resume the matter in this post with some improvements, although, to be perfectly honest, I am not happy with the configuration.

In theory, the bridge is possible because both systems are JMS 1.1. compliant. If you use a HornetQ 2.2.5 standalone installation, you simply have to add some managed beans to the hornetq-beans.xml file, as is described in the user manual. The key point here is to use the proper connection factory and set up the destination following the rules of the Weblogic manual: “Interoperating with Oracle AQ JMS”. Here you have an excerpt of the hornetq-beans.xml file:

<bean name="JMSBridge">
  <constructor>
  ...
    <!-- concatenate JMS messageID to the target's message header -->
    <parameter>true</parameter>
  ...
</bean>
...
<!-- TargetCFF describes the ConnectionFactory used to connect to the target destination -->
<bean name="TargetCFF">
  <constructor>
    <parameter>
      <inject bean="TargetJNDI" />
    </parameter>
    <parameter>QueueConnectionFactory</parameter>
  </constructor>
</bean>
...
<!-- TargetDestinationFactory describes the Destination used as the target -->
<bean name="TargetDestinationFactory">
  <constructor>
    <parameter>
      <inject bean="TargetJNDI" />
    </parameter>
    <parameter>Queues/testQueue</parameter>
  </constructor>
</bean>
...
<!-- JNDI is a Hashtable containing the JNDI properties required -->
<!-- to connect to the *target* JMS resources                    -->
<bean name="TargetJNDI">
  <constructor>
    <map keyClass="java.lang.String" valueClass="java.lang.String">
      <entry>
        <key>java.naming.factory.initial</key>
        <value>oracle.jms.AQjmsInitialContextFactory</value>
      </entry>
      <entry>
        <key>db_url</key>
        <value>jdbc:oracle:thin:@oraclehost:1521:test</value>
      </entry>
      <entry>
        <key>java.naming.security.principal</key>
        <value>test</value>
      </entry>
      <entry>
        <key>java.naming.security.credentials</key>
        <value>secret</value>
      </entry>
    </map>
  </constructor>
</bean>

The former configuration runs properly with the DUPLICATES_OK quality of service, but when you moves to a ONCE_AND_ONLY_ONCE one, changing the TargetCFF to XAQueueConnectionFactory, you get a NullPointerException:

  oracle.jms.AQjmsException: Error creating the db_connection
    at oracle.jms.AQjmsDBConnMgr.getConnection
    at oracle.jms.AQjmsDBConnMgr.
    at oracle.jms.AQjmsXAConnection.
    at oracle.jms.AQjmsXAQueueConnectionFactory.createAllXAConnection
    at oracle.jms.AQjmsXAQueueConnectionFactory.createXAQueueConnection

After a research on Oracle Support notes, I found that the Oracle API has a bug (10102373) that it will be fixed on the future release 12.1 There are workarounds for not receiving duplicate messages, for example: to save the JMS Message ID in a Oracle table and implement a unique index on that column, you have to take into account that the JMS Messege ID sent by HornetQ is a property that can be recovered with the method get_string_property of the Oracle type aq$_jms_message.

But, why Weblogic can interoperate with Oracle AQ in XA configurations? I suspect that the reason is because uses datasources with pre-created database connections. So, I’ve tried to deploy the bridge within JBoss AS 6.0.0, with HornetQ as its default JMS provider. First of all, I created an Oracle XA datasource. After that, I set up the configuration file jms-bridge-jboss-beans.xml in a similar way as before, the only change is to change the db_url parameter by the parameter datasource:

<bean name="TargetJNDI">
  <constructor>
    <map keyClass="java.lang.String" valueClass="java.lang.String">
      <entry>
        <key>java.naming.factory.initial</key>
        <value>oracle.jms.AQjmsInitialContextFactory</value>
      </entry>
      <entry>
        <key>datasource</key>
        <value>jdbc/testOracleXA</value>
      </entry>
    </map>
  </constructor>
</bean>

But, when I tried to run the former configuration, the bridge didn’t start. Another bug, a HornetQ one in this case, prevented me from reaching my goal. To be perfectly honest, I didn’t patch the installation, but I restart the managed bean through the JMX console and I got another exception, a ClassCastException, because JBoss pass a wrapped Oracle connection to the class AQjmsInitialContextFactory, which is waiting for an native internal connection. I really felt powerless!

Finally, I adopted the solution of setting up an Oracle Weblogic server as a mediator,  the guru Edwin Biemond describes this configuration in this article.


Security notes about Hornetq as a JBoss JMS provider

I’ve been recently involved in a project where my client had to share information with some partners, using HornetQ 2.2.5 queues deployed on a JBoss 4.2.1 GA application server, and I had to face up to a security problem. Here you have the case: the first step that a remote client has to perform when it tries to send a message to a queue is to lookup for a connection factory and the queue in a JNDI tree; at this point, you should not simply open your JBoss JNDI port (1099 by default), even having configured your firewall to open the port just to your partners’ servers and having an user authentication process for JNDI, because they might list all the names of your global JNDI namespace and exploit any security bug you may have. On the other hand, although I would have opened the port, our JBoss application server runs into a demilitarized zone (DMZ) with a NAT (Network Address Translation) standard configuration, in this situation you can’t use RMI.

The solution to my problem was to force the partners to access JNDI over HTTP an secure that access in a manner that they  just could perform lookup operations over a controlled read-only context, so they got their connection factories and queues from that context and, later on, opened their JMS sessions, through a HTTPS tunnel, after an user authentication process.

In order to run a basic test case,  the first step is to create a role for the remote users (named remoteRole in this example) and another for local ones (localRole) on JBoss. On the other hand, I created one key store for the SSL connection:

keytool -genkeypair -alias test -keyalg RSA -keysize 1024 -dname OU=TEST,CN=TEST,CN=LOCAL
        -keypass test1234 -keystore hornetq.test.keystore -storepass test1234

The following task is to deploy the netty servlet that provide the HTTP transport, please follow this link where you have the WAR file I setup.  The next step is to edit the file  hornetq-configuration.xml file  of the HornetQ installation and to add a servlet connector (please, notice that the path to the test keystore is referred to the client file system and you have to provide it with the keystore) and its corresponding acceptor :

<connectors>
...  
   <connector name="netty-servlet">
      <factory-class>org.hornetq.core.remoting.impl.netty.NettyConnectorFactory</factory-class>
      <param key="host" value="${jboss.bind.address:localhost}"/>
      <param key="port" value="8443"/>
      <param key="use-servlet" value="true"/>
      <param key="servlet-path" value="/messaging/HornetQServlet"/>
      <param key="ssl-enabled" value="true"/>
      <param key="key-store-path" value="C:\\Software\\KeyStores\\hornetq.test.keystore"/>
      <param key="key-store-password" value="test1234"/>
 </connector>
...
</connectors>

<acceptors>
..  
   <acceptor name="netty-invm">
      <factory-class>org.hornetq.core.remoting.impl.netty.NettyAcceptorFactory</factory-class>
      <param key="use-invm" value="true"/>
      <param key="host" value="org.hornetq"/>
   </acceptor>
...
</acceptors>

The following task is to be sure that the HTTPS connector of the JBoss 4.2.1 AS is running, if not you have to edit the file server.xml located in the folder \deploy\jboss-web.deployer

I also add a security constraint to this  hornetq-configuration.xml configuration file, so the remote users just can send messages to our queues:

...
<security-settings>
...
  <security-setting match="jms.queue.ReadOnly.#">
    <permission type="consume" roles="localRole"/>
    <permission type="send" roles="remoteRole"/>
  </security-setting>
...
</security-settings>
...

The following task is to edit the hornetq-jms.xml file and add a connection factory and a queue, deployed under the readonly context:

...
<connection-factory name="ReadOnly.NettyConnectionFactory">
  <xa>true</xa>
  <connectors>
    <connector-ref connector-name="netty-servlet"/>
  </connectors>
  <entries>
    <entry name="readonly/XAConnectionFactory"/>
  </entries>
</connection-factory>
...
<queue name="ReadOnly.TestQueue">
  <entry name="readonly/TestQueue"/>
</queue>
...

The next step is to code a basic messages producer, in order to be used by the remote part:

public class RemoteJMSProducer {

    public void sendMessages() {
        ConnectionFactory connectionFactory;
        Connection connection = null;
        Session session = null;
        Queue queue;
        MessageProducer producer;
        TextMessage message;
        Context initialContext;
        Properties jndiConfig;

        try {
            jndiConfig = new Properties();
            jndiConfig.put(Context.INITIAL_CONTEXT_FACTORY,
                           "org.jboss.naming.HttpNamingContextFactory");
            jndiConfig.put(Context.PROVIDER_URL,
                           "http://remote-jboss-server/invoker/ReadOnlyJNDIFactory");
            initialContext = new InitialContext(jndiConfig);

            connectionFactory = (ConnectionFactory)
                                 initialContext.lookup("readonly/XAConnectionFactory");
            connection = connectionFactory.createConnection("remoteUser", "test");
            queue = (Queue) initialContext.lookup("readonly/TestQueue");

            session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
            producer = session.createProducer(queue);
            message = session.createTextMessage();

            for (int i = 0; i < 10; i++) {
                message.setText("This is the test message number " + i);
                producer.send(message);
            }

            System.out.println("Messages sent!");
        } catch (NamingException ex) {
            System.err.println("JNDI exception: " + ex.getMessage());
        } catch (JMSException ex) {
            System.err.println("JMS exception: " + ex.getMessage());
        } finally {
            try {
                session.close();
            } catch (Exception ignore) {
            }
            try {
                connection.close();
            } catch (Exception ignore) {
            }
        }
    }

    /**
     * @param args the command line arguments
     */
    public static void main(String[] args) {
        RemoteJMSProducer remoteJMSProducer;

        remoteJMSProducer = new RemoteJMSProducer();
        remoteJMSProducer.sendMessages();
    }
}

The former class doesn’t use the typical org.jnp.interfaces.NamingContextFactory, but the org.jboss.naming.HttpNamingContextFactory as the initial context factory. On the other hand, I’d like to point out that the provider URL is http://remote-jboss-server/invoker/ReadOnlyJNDIFactory. In order to run the example, you need to put the following libraries on your classpath:

  • hornetq-core-client.jar
  • hornetq-jms-client.jar
  • netty.jar
  • concurrent.jar
  • jboss-client.jar
  • jboss-common-client.jar
  • jboss-j2ee.jar
  • jboss-remoting.jar
  • jboss-serialization.jar
  • jbosssx-client.jar

Please, realize that if you try to perform a list or a rename operation, you get an exception.


References