Setting up a JMS bridge between Weblogic and ActiveMQ

Almost four years ago, I wrote about how to setup a JMS bridge between Weblogic and HornetQ. Lately, I’ve had to research how to do the same work with ActiveMQ. Here you have my findings. As it was for HornetQ, the first step was to copy the client libraries to a folder residing in a filesystem of my Weblogic server, in the case of ActiveMQ these files are:

  • activemq-client-5.10.0.jar
  • hawtbuf-1.10.jar
  • slf4j-api-1.7.5.jar

Later on, I set the PRE_CLASSPATH variable pointing it to these libraries into the script setDomainEnv. Creating messaging destinations is more or less the same job as it was for HornetQ, you just have to change the initial context factory and connection url parameters:

<jms-bridge-destination>
  <name>JMS Bridge Destination-Target</name>
  <adapter-jndi-name>eis.jms.WLSConnectionFactoryJNDIXA</adapter-jndi-name>
  <user-name>testUser</user-name>
  <user-password-encrypted>{AES}GZTa15osrQW6/5rbE2fzm+a9BX/YPY7Hm3xiIFi0oMQ=</user-password-encrypted>
  <classpath></classpath>
  <connection-factory-jndi-name>jms/TestXAQueueConnectionFactory</connection-factory-jndi-name>
  <initial-context-factory>org.apache.activemq.jndi.ActiveMQInitialContextFactory</initial-context-factory>
  <connection-url>tcp://localhost:61616</connection-url>
  <destination-jndi-name>jms/TestQueue</destination-jndi-name>
  <destination-type>Queue</destination-type>
</jms-bridge-destination>

But the former configuration is not enough, because ActiveMQ hasn’t his own JNDI provider (I had ActiveMQ as a JMS provider for a ServiceMix ESB) and requires a jndi.properties file with the mappings between physical destinations and tha jndi ones, but, how to configure the properties file in the context of a Weblogic messaging bridge? Here is the trick: create a JAR archive containing the jndi.properties file and put the JAR in the CLASSPATH (the same way at it was described before for activemq-client-5.10.0.jar). The steps to create the JAR archive are:

  • Create a file called jndi.properties with the following entries:

connectionFactoryNames=ConnectionFactory,XAConnectionFactory queue.TestQueue=TestQueue

  • Create the archive with the following command:

jar cvf jndi.jar jndi.properties

I found this trick in this post, after have tested many different configurations, including the setup of a Weblogic Foreign JMS Server, without success. Finally, I’d like to point out that I am not completely happy with this setup because it has an obvious drawback: you have modify and redeploy the jndi.jar archive every time you add a new queue or topic, so suggestions are welcome!!!


References


Changing the datatype of a column from VARCHAR2 to CLOB

Some days ago, I realized I hadn’t estimated correctly the size of a column of a table that receives data from an external source. I defined a VARCHAR2(4000 CHAR) datatype but it came a longer message  (I got an ORA-01461 error), so I decided to move the column to a CLOB datatype.

At first sight, it seems a simple matter. For example, if you have the following table definition:

   create table test_messages
   (message_id number             primary key,
    content   varchar2(4000 char) not null);

You would think to issue the following statement:

   alter table test_messages
   modify (content clob not null);

But, if you do that, you get an ORA-22296 error.

I had several options to solve the problem (please review the reference of this post), but I finally selected the following one:

   alter table test_messages
   add (temp_content clob);

   update test_messages
   set temp_content = content;

   alter table test_messages
   drop column content;

   alter table test_messages
   rename column temp_content to content;

   alter table test_messages
   modify (content not null);

Anyway, it’s very important to review all the application code affected by the change, because, for example, if you use the substr function, you have to change it to dbms_lob.substr. On the other hand, if you use JPA and you have an entity like:

   @Entity
   @Table(name = "TEST_MESSAGES")
   public class TestMessages implements Serializable {

       @Id
       @Column(name = "MESSAGE_ID")
       private Long messageId;

       @Column(name = "CONTENT", length = 4000, nullable = false)
       private String content;

       public Long getMessageId() {
           return messageId;
       }

       public void setMessageId(Long messageId) {
           this.messageId = messageId;
       }

       public String getContent() {
           return content;
       }

       public void setContent(String content) {
           this.content = content;
       }

   }

You have to change the definition of content to:

   @Lob
   @Column(name = "CONTENT", nullable = false)
   private String content;

Finally, I’d like to point out two issues that could affect to performance. The first one is the possibility of defer the removing of the initial column, running the following statement:

   alter table test_messages
   set unused column content;

instead of:

   alter table test_messages
   drop column content;

Later, in a period of low use of the database, you can issue:

   alter table test_messages
   drop unused columns;

The other thing regarding to performance is that it’s convenient reorganizing the storage of the table by running the following command:

   alter table test_messages move;

Statements like alter index <index_name> rebuild could also be needed.


References


Weblogic: Interoperating with Oracle AQ JMS. A tip for sending messages

I usually work with Oracle Streams Advanced Queuing (AQ) integrated with Weblogic. In PL/SQL programs you can set the exception queue where you want to move the messages that couldn’t be delivered to their destinations, when you send a message, for example:


declare
  enq_msgid raw(16) ;
  eopt      dbms_aq.enqueue_options_t;
  mprop     dbms_aq.message_properties_t;
  message   sys.aq$_jms_text_message;
begin
  mprop.priority        := 1;
  mprop.exception_queue := 'test_exception_q';
  message               := sys.aq$_jms_text_message.construct() ;
  message.set_text('This is a Test') ;
  dbms_aq.enqueue(queue_name => 'test_q', 
                  enqueue_options => eopt,
                  message_properties => mprop, 
                  payload => message, 
                  msgid => enq_msgid) ;

  commit;
end;

But, how to set the exception queue when you’re programming in Java, in the context of an integration between Oracle AQ and Weblogic? The solution I’ve found is to set the string property called JMS_OracleExcpQ to the message with the name of the Oracle AQ exception queue, for example:


...
javax.jms.TextMessage msg = jmsSession.createTextMessage();
msg.setStringProperty("JMS_OracleExcpQ", "TEST.TEST_EXCEPTION_Q");
msg.setText("This is a Test");
jmsProducer.send(msg);
...

An important advice: put the name in uppercase and preceded by the name of the schema.

Finally, I’d like to point out that this is a good solution in the context of an integration between Oracle AQ and Weblogic, but makes the code non portable, if you want to change the messaging provider of your Weblogic Application Server.


References


Trying to set up a JMS bridge between HornetQ and Oracle AQ

I’ve been interested in setting up a JMS bridge between HornetQ and Oracle Advanced Queuing (AQ) for a while, even I wrote about this issue in a response to a question of the HornetQ user’s forum. I resume the matter in this post with some improvements, although, to be perfectly honest, I am not happy with the configuration.

In theory, the bridge is possible because both systems are JMS 1.1. compliant. If you use a HornetQ 2.2.5 standalone installation, you simply have to add some managed beans to the hornetq-beans.xml file, as is described in the user manual. The key point here is to use the proper connection factory and set up the destination following the rules of the Weblogic manual: “Interoperating with Oracle AQ JMS”. Here you have an excerpt of the hornetq-beans.xml file:

<bean name="JMSBridge">
  <constructor>
  ...
    <!-- concatenate JMS messageID to the target's message header -->
    <parameter>true</parameter>
  ...
</bean>
...
<!-- TargetCFF describes the ConnectionFactory used to connect to the target destination -->
<bean name="TargetCFF">
  <constructor>
    <parameter>
      <inject bean="TargetJNDI" />
    </parameter>
    <parameter>QueueConnectionFactory</parameter>
  </constructor>
</bean>
...
<!-- TargetDestinationFactory describes the Destination used as the target -->
<bean name="TargetDestinationFactory">
  <constructor>
    <parameter>
      <inject bean="TargetJNDI" />
    </parameter>
    <parameter>Queues/testQueue</parameter>
  </constructor>
</bean>
...
<!-- JNDI is a Hashtable containing the JNDI properties required -->
<!-- to connect to the *target* JMS resources                    -->
<bean name="TargetJNDI">
  <constructor>
    <map keyClass="java.lang.String" valueClass="java.lang.String">
      <entry>
        <key>java.naming.factory.initial</key>
        <value>oracle.jms.AQjmsInitialContextFactory</value>
      </entry>
      <entry>
        <key>db_url</key>
        <value>jdbc:oracle:thin:@oraclehost:1521:test</value>
      </entry>
      <entry>
        <key>java.naming.security.principal</key>
        <value>test</value>
      </entry>
      <entry>
        <key>java.naming.security.credentials</key>
        <value>secret</value>
      </entry>
    </map>
  </constructor>
</bean>

The former configuration runs properly with the DUPLICATES_OK quality of service, but when you moves to a ONCE_AND_ONLY_ONCE one, changing the TargetCFF to XAQueueConnectionFactory, you get a NullPointerException:

  oracle.jms.AQjmsException: Error creating the db_connection
    at oracle.jms.AQjmsDBConnMgr.getConnection
    at oracle.jms.AQjmsDBConnMgr.
    at oracle.jms.AQjmsXAConnection.
    at oracle.jms.AQjmsXAQueueConnectionFactory.createAllXAConnection
    at oracle.jms.AQjmsXAQueueConnectionFactory.createXAQueueConnection

After a research on Oracle Support notes, I found that the Oracle API has a bug (10102373) that it will be fixed on the future release 12.1 There are workarounds for not receiving duplicate messages, for example: to save the JMS Message ID in a Oracle table and implement a unique index on that column, you have to take into account that the JMS Messege ID sent by HornetQ is a property that can be recovered with the method get_string_property of the Oracle type aq$_jms_message.

But, why Weblogic can interoperate with Oracle AQ in XA configurations? I suspect that the reason is because uses datasources with pre-created database connections. So, I’ve tried to deploy the bridge within JBoss AS 6.0.0, with HornetQ as its default JMS provider. First of all, I created an Oracle XA datasource. After that, I set up the configuration file jms-bridge-jboss-beans.xml in a similar way as before, the only change is to change the db_url parameter by the parameter datasource:

<bean name="TargetJNDI">
  <constructor>
    <map keyClass="java.lang.String" valueClass="java.lang.String">
      <entry>
        <key>java.naming.factory.initial</key>
        <value>oracle.jms.AQjmsInitialContextFactory</value>
      </entry>
      <entry>
        <key>datasource</key>
        <value>jdbc/testOracleXA</value>
      </entry>
    </map>
  </constructor>
</bean>

But, when I tried to run the former configuration, the bridge didn’t start. Another bug, a HornetQ one in this case, prevented me from reaching my goal. To be perfectly honest, I didn’t patch the installation, but I restart the managed bean through the JMX console and I got another exception, a ClassCastException, because JBoss pass a wrapped Oracle connection to the class AQjmsInitialContextFactory, which is waiting for an native internal connection. I really felt powerless!

Finally, I adopted the solution of setting up an Oracle Weblogic server as a mediator,  the guru Edwin Biemond describes this configuration in this article.


SQL statement to get the weeks before the current date

I recently had to design a report to analyze the data of a week, so the user had to select that week using an input control. The data source of the combo box was the following SQL statement that runs in an Oracle Database:

select to_char(next_day(trunc(sysdate, 'DAY'), 'SATURDAY') - (level * 7) - 6, 
                    'mm/dd/yyyy')
  || ' - '
  || to_char(next_day(trunc(sysdate, 'DAY'), 'SATURDAY') - (level * 7), 
             'mm/dd/yyyy') week
from dual
  connect by level <= 52
order by level

This code employs the hierarchical query clause CONNECT BY to select the initial and the final day of the weeks before the current date, a period of a year more or less.


Tracking of a Weblogic JMS bridge

I’ve been talking about JMS bridges between Weblogic and HornetQ for a while, today I’d like to deal with the issue of tracking, which is especially useful when you need to know the time when a message was dispatched to the destination system.

The idea is to configure Weblogic to debug messaging bridge runtime events, add a custom logging handler that captures and saves these events into a table and to analyze them, in order to track the pass of the messages through the bridge.

Therefore, the first task is to enable the debugging of the messaging bridge events in Weblogic. One of the methods to achieve this goal is to add the following lines to the setDomainEnv.cmd script:

set JAVA_OPTIONS=%JAVA_OPTIONS% -Dweblogic.debug.DebugMessagingBridgeStartup=true
set JAVA_OPTIONS=%JAVA_OPTIONS% -Dweblogic.debug.DebugMessagingBridgeRuntime=true

The next step is to create a table to give persistence to the events:

CREATE TABLE wl_log
(log_id        NUMBER              CONSTRAINT wl_log_pk PRIMARY KEY,
 log_timestamp TIMESTAMP           NOT NULL,
 msg_id        VARCHAR2(25 CHAR)   NOT NULL,
 log_level     VARCHAR2(25 CHAR)   NOT NULL,
 subsystem     VARCHAR2(50 CHAR)   NOT NULL,
 message       VARCHAR2(4000 CHAR) NOT NULL);

CREATE SEQUENCE wl_log_s
INCREMENT BY 1
START WITH 1
MAXVALUE 999999999999
MINVALUE 1
CACHE 20;

The following task is to create a custom logging handler that will capture and save the events into the former table (this class has references to a properties file, where I put the parameters needed to connect to the Oracle database):

public class WLCustomLoggingHandler extends Handler {
    private Connection dbConn = null;
    private PreparedStatement psmt = null;

    public WLCustomLoggingHandler() throws IOException,
                                           ClassNotFoundException,
                                           SQLException {
        super();

        Properties dbProperties;

        dbProperties = new Properties();
        dbProperties.load(this.getClass().getResourceAsStream(
                                            "WLCustomLoggingHandlerDB.properties"));
        Class.forName(dbProperties.getProperty("DRIVER"));
        this.dbConn =
                DriverManager.getConnection(dbProperties.getProperty("URL"),
                                            dbProperties.getProperty("USER"),
                                            dbProperties.getProperty("PASS"));
        this.psmt =
                this.dbConn.prepareStatement("INSERT INTO wl_log " +
                                             "(log_id, log_timestamp, msg_id, " +
                                             "log_level, subsystem, message) " +
                                             "VALUES(wl_log_s.nextval, ?, ?, ?, ?, ?)");

        this.setErrorManager(new ErrorManager() {
            public void error(String msg, Exception ex, int code) {
                System.err.println("Error reported by TestWLLoggingHandler "
                                   + msg + ex.getMessage());
                LoggingHelper.getServerLogger().removeHandler(WLCustomLoggingHandler.this);
            }
        });
    }

    @Override
    public void publish(LogRecord record) {
        WLLogRecord wlLogRecord = (WLLogRecord)record;
        if (this.isLoggable(wlLogRecord)) {
            try {
                psmt.setEscapeProcessing(true);
                psmt.setTimestamp(1, new Timestamp(wlLogRecord.getMillis()));
                psmt.setString(2, wlLogRecord.getId());
                psmt.setString(3, wlLogRecord.getLevel().getName());
                psmt.setString(4, wlLogRecord.getLoggerName());
                psmt.setString(5, wlLogRecord.getMessage());
                psmt.executeUpdate();
                this.flush();
            } catch (SQLException ex) {
                this.reportError("WLCustomLoggingHandlerpublish: ",
                                 ex,
                                 ErrorManager.WRITE_FAILURE);
            }
        }
    }

    @Override
    public void flush() {
        try {
            this.dbConn.commit();
        } catch (SQLException ex) {
            this.reportError("WLCustomLoggingHandlerflush: ",
                             ex,
                             ErrorManager.FLUSH_FAILURE);
        }
    }

    @Override
    public void close() throws SecurityException {
        try {
            this.dbConn.close();
        } catch (SQLException ex) {
            this.reportError("WLCustomLoggingHandlerclose: ",
                             ex,
                             ErrorManager.CLOSE_FAILURE);
        }
    }
}

We just want to analyze the events regarding to the messaging bridge, so we have to create a logging filter:

public class WLCustomLoggingFilter implements Filter {
    public WLCustomLoggingFilter() {
        super();
    }

    @Override
    public boolean isLoggable(LogRecord record) {
        if (record instanceof WLLogRecord) {
            WLLogRecord wlLogRecord = (WLLogRecord)record;

            return wlLogRecord.getSubsystem().equals("MessagingBridge") ||
                wlLogRecord.getSubsystem().equals("MessagingBridgeRuntime");
        } else {
            return false;
        }
    }
}

The next step is to code a Weblogic startup class:

public class WLCustomLoggingHandlerStartupClass {
    public WLCustomLoggingHandlerStartupClass() {
        super();
    }

    public void configureLogger() {
        Logger logger = null;
        Handler handler = null;

        try {
            logger = LoggingHelper.getServerLogger();
            handler = new WLCustomLoggingHandler();
            handler.setFilter(new WLCustomLoggingFilter());
            logger.addHandler(handler);
        } catch (Exception ex) {
            System.err.println("WLCustomLoggingHandlerStartupClass.configureLogger: "
                               + ex.getMessage());
            ex.printStackTrace();
            try {
                logger.removeHandler(handler);
            } catch (Exception ignore) {
            }
        }
    }

    public static void main(String[] args) {
        WLCustomLoggingHandlerStartupClass testWLCustomLoggingHandlerStartupClass =
            new WLCustomLoggingHandlerStartupClass();
        testWLCustomLoggingHandlerStartupClass.configureLogger();
    }
}

This class has to be configured in Weblogic by going to the Enviroment->Startup and Shutdown Classes of the console (obviously, the corresponding jar file containing the former code has to be available on the weblogic classpath):

Weblogic Startup and Shutdown Classes
The last step is to analyze the log entries in order to know when each message is dispatched to its destination, when the bridge was up and running, etc. MessagingBridge Subsystem Messages are helpful to achieve this goal, especially:

  • BEA-200028: The bridge “arg01” has started transferring messages.
  • BEA-200024: Bridge “arg0” refused to obtain connections and transfer any messages, because it has been stopped or suspended, or is shutting down.
  • BEA-200015: An error occurred in bridge “arg0” during the transfer of messages (e).

But, the pass of a message through a bridge has to be discovered by analyzing the log entries with id BEA-000000, of course the sender program has to save the message and its ID on persistent storage at the time it sends it. The bridge dequeues each message from the source and enqueues it to the destination, all in the context of a XA batch transaction (10 messages by default):

03/03/12 10:12:11,859000000	"Bridge: TestBridge (processMessages()) received message:
                                 JMS Message Class: TextMessage
                                 JMSMessageID: ID:B3AD885BC54B4391976BEDA9EC00E8F4
                                 JMSCorrelationID: null
                                 JMSDeliveryMode: PERSISTENT
                                 JMSDestination: TestQueue
                                 JMSExpiration: 0
                                 JMSPriority: 8
                                 JMSRedelivered: false
                                 JMSReplyTo: null
                                 JMSTimestamp: 1330851452000 (Sun Mar 04 09:57:32 CET 2012)
                                 JMSType: null
                                 Transaction Id: BEA1-00018D19CC5B9E472B1E
                                 This is a test message: 010"
03/03/12 10:12:11,859000000	"Bridge: TestBridge (processMessages()) successfully sent message:
                                 JMS Message Class: TextMessage
                                 Old JMS MessageID: ID:B3AD885BC54B4391976BEDA9EC00E8F4
                                 New JMS MessageID: ID:2064fac7-65da-11e1-9824-005056c00008
                                 This is a test message: 010"
03/03/12 10:12:12,671000000	Bridge: TestBridge (processMessages()) committed the transaction

References


Weblogic: receiving messages from a remote HornetQ destination

I’ve talked in former posts about setting up a JMS bridge between Weblogic and HornetQ and how to secure that connection, but the bridge is just appropriate for sending messages from Weblogic to HornetQ, the proper way for receiving data send to HornetQ queues in Weblogic is to implement a Message Driven Bean (MDB), as you can read on the documentation of Weblogic. How to connect the MDB deployed in Weblogic to a HornetQ destination? After a research, I think that the best answer is a Foreign Server.

First of all, I copied the following HornetQ client libraries to a folder residing in a filesystem of my Weblogic server:

  • hornetq-core-client.jar
  • hornetq-jms-client.jar
  • netty.jar
  • jnp-client.jar
  • jboss-logging.jar

Later on, I set the PRE_CLASSPATH variable pointing it to these libraries into the script setDomainEnv and started Weblogic. The next step was to create a new JMS System Module on the console, accepting the defaults.  After that, I edited the module and setup a new Foreign Server with the defaults and I entered its configuration: Weblogic Foreign Server Main Configuration

The following step was to link the destinations (topics and queues) and the connection factories I previously setup on HornetQ. To do this I used the wizards provided by the Weblogic console, it’s easy, I just have to point out that I had to edit the configuration of my connection factory to add the security credentials:

Details of a Foreign Connection Factory

Finally, I mapped the queue and the connection factory in my MDB, here you have an example:

@MessageDriven(mappedName = "jms/Test/TestQueue")
@MessageDestinationConfiguration(
         connectionFactoryJNDIName = "jms/Test/TestXAConnectionFactory")
public class TestMDB implements MessageListener {
    static final Logger logger = LoggingHelper.getServerLogger();

    public void onMessage(Message message) {
        TextMessage msg = null;

        try {
            if (message instanceof TextMessage) {
                msg = (TextMessage) message;
                logger.info("MESSAGE BEAN: Message received: " + msg.getText());
            } else {
                logger.warning("Message of wrong type: " + 
                                    message.getClass().getName());
            }
        } catch (JMSException e) {
            logger.severe("TestMDB.onMessage: JMSException: " + e.toString());
            e.printStackTrace();
        } catch (Throwable te) {
            logger.severe("TestMDB.onMessage: Exception: " + te.toString());
            te.printStackTrace();
        }
    }
}