Saturday, February 20, 2016

Writing microservices with WSO2 MSF4J

WSO2 has concluded another WSO2Con conference in grand style. The WSO2Con Asia 2016 has had special focus on the new WSO2 Microservices Framework for Java (WSO2 MSF4J in short) product. I have done a tutorial on how to write microservices with WSO2 MSF4J on the AppDev track. You can see the Agenda here -

The WSO2 MSF4j is a lightweight, fast runtime with an annotation-based programming model that offers to create microservices in Java with container-based deployment in mind. Microservices developed using WSO2 MSF4J can boot in just a few milliseconds in a Docker container and can easily be added to a Docker image definition.

In the tutorial, we have covered the following topics -
  • Introduction to Microservices Architecture (MSA)
  • Introducing key features of WSO2 MSF4J
  • Developing microservices with WSO2 MSF4J
  • Monitoring and securing microservices with WSO2 MSF4J
  • Introducing new features of WSO2 Application Server

Friday, September 25, 2015

WSO2 Application Server (WSO2 AS) 5.3.0 Release

WSO2 Application Server 5.3.0 has been released on top WSO2 Carbon Kernel 4.4.1. See the release note below to find out the cool and exciting features of this latest release. 

The WSO2 Application Server is an enterprise-ready cloud-enabled application server, powered by Apache Tomcat, and Apache TomEE, it integrates Apache CXF framework. It provides first class support for standard Web applications, JAX-WS/JAX-RS applications and Jaggery scripting applications. Coupled with features of WSO2 Carbon, users can now manage their applications including JAX-WS and JAX-RS to web applications in a unified manner within the management console itself.

WSO2 Application Server also provides a comprehensive Web services server platform using CXF as its Web services runtime, and provides many value additions on top of this runtime. It can expose services using both SOAP and REST models and supports a comprehensive set of WS-* specifications such as WS-Security, WS-Trust, WS-SecureConversation, WS-Addressing, WS-SecurityPolicy, etc. WSO2 Application Server also has inbuilt support for Jaggery. WSO2 Application Server can be installed on on-premise or any public/private cloud infrastructure and provide unified management console and lifecycle management features which are independent from underlying deployment option.

An open source product, WSO2 Application Server is available under the Apache Software License (v2.0) . This includes all of the extra integration and management functionality as well. 
For more information on WSO2 Application Server and to download the product please visit

New Features
  • JavaEE 6 Web Profile support via TomEE 1.7.2 integration
  • WebSocket 1.1 API Support as defined by the JSR-356 specification
  • Tomcat Virtual Hosts support
  • SAML2 Single-Sign-On support for web applications
  • WS-Discovery support for CXF JAX-WS and JAX-RS services
  • OSGi ServiceLoader Mediator specification support via SPI-Fly
Key Features

  • Support for Servlet 3, JSP 2.2, EL 2.2, JSTL 1.2 specifications.
  • Full JAX-WS 2.2 and JAX-RS 2.0 Specification support
  • Integration of Jaggery - server side scripting framework
  • Unified Application listing and management UI for WebApps, JAX-WS/RS, Jaggery
  • Multi Tenant support for standalone deployment
  • 100% Apache Tomcat compliance runtime
  • Lazy loading for web applications and services
  • AppServer tooling - AppServer related artifacts can be easily generated using WSO2 Developer Studio
  • Clustering support for High Availability and High Scalability
  • Full support for WS-Security, WS-Trust, WS-Policy and WS-SecureConversation
  • JMX and Web interface based monitoring and management
  • WS-* and REST support
  • GUI, command line, and IDE based tools for Web service development
  • Equinox P2 based provisioning support
  • WSDL2Java/Java2WSDL/WSDL 1.1 and try it (invoke any remote Web service)

Wednesday, December 17, 2014

WSO2 BAM - APIM error - Failed to write data to database

When viewing statistics of APIs in WSO2 API Manager 1.7.0 via WSO2 BAM, you may have come across the following issue. [1]

As you might know, AM sends events about requests to APIs to WSO2 BAM. And, BAM stores these data in Cassandra storage which gets later by Hive analytics scripts. For AM, we have the am_stats_analyzer. After analyzing, the summarized information gets written to a RDBMS instance. The summarized db could be mysql, oracle or even a in-memory H2 DB.


This error [1] comes when a column in a summarized database table is too small to store a given value. In this, it's the resourcePath of an API. By default, the size for the resourcePath column is set to VARCHAR(100). If the resourcePath of an API is longer than 100 characters, then this error will be thrown.


The 'resourcePath' is defined in the API_Resource_USAGE_SUMMARY table in the summary db.

  • If the system is already up and running, the summarized mysql tables are already created. Therefore, we need to alter the tables to modify column lengths. You can use the following steps for that.

1. Since the issue is in a RDBMS such as mysql, you first need to log-in to console where you can execute SQL statements.
2. Then, execute the following statement.

alter table API_Resource_USAGE_SUMMARY modify resourcePath MEDIUMTEXT

  • If the summarized tables are not created yet, then you could go and and modify the table creation script in the am_stats_analyzer.
  1. Open the API_Manager_Analytics.tbox.
  2. In there, you will find am_stats_analyzer analytics script.
  3. Open that, and look for the hive.jdbc.table.create.query, which has the following summary table creation sql statement.
 CREATE TABLE API_Resource_USAGE_SUMMARY ( api VARCHAR(100), version VARCHAR(100),apiPublisher VARCHAR(100) , consumerKey VARCHAR(100),resourcePath VARCHAR(100) ,context VARCHAR(100),
        method VARCHAR(100), total_request_count INT, hostName VARCHAR(100), year SMALLINT, month SMALLINT, day SMALLINT, time VARCHAR(30), PRIMARY KEY(api,version,apiPublisher,consumerKey,context,method,time))

Change the type of resourcePath from VARCHAR(100) to MEDIUMTEXT. Save the script.

ERROR {} - Failed to write data to database {}
com.mysql.jdbc.MysqlDataTruncation: Data truncation: Data too long for column 'resourcePath' at row 1
        at com.mysql.jdbc.MysqlIO.checkErrorPacket(
        at com.mysql.jdbc.MysqlIO.checkErrorPacket(
        at com.mysql.jdbc.MysqlIO.sendCommand(
        at com.mysql.jdbc.MysqlIO.sqlQueryDirect(
        at com.mysql.jdbc.ConnectionImpl.execSQL(
        at com.mysql.jdbc.PreparedStatement.executeInternal(
        at com.mysql.jdbc.PreparedStatement.executeUpdate(
        at com.mysql.jdbc.PreparedStatement.executeUpdate(
        at com.mysql.jdbc.PreparedStatement.executeUpdate(
        at org.apache.hadoop.hive.ql.exec.FileSinkOperator.processOp(
        at org.apache.hadoop.hive.ql.exec.Operator.process(
        at org.apache.hadoop.hive.ql.exec.Operator.forward(
        at org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(
        at org.apache.hadoop.hive.ql.exec.Operator.process(
        at org.apache.hadoop.hive.ql.exec.Operator.forward(
        at org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(
        at org.apache.hadoop.hive.ql.exec.Operator.process(
        at org.apache.hadoop.hive.ql.exec.Operator.forward(
        at org.apache.hadoop.hive.ql.exec.GroupByOperator.forward(
        at org.apache.hadoop.hive.ql.exec.GroupByOperator.processAggr(
        at org.apache.hadoop.hive.ql.exec.GroupByOperator.processOp(
        at org.apache.hadoop.hive.ql.exec.Operator.process(
        at org.apache.hadoop.hive.ql.exec.ExecReducer.reduce(
        at org.apache.hadoop.mapred.ReduceTask.runOldReducer(

Tuesday, December 2, 2014

Fixing 101505 WSO2 Connection close issues


The error code 101505 means the HTTP connection initiated by the ESB has been terminated at the backend. According to the log you can determine whether message has been completely written to the backend prior to the connection close (See [Status SendingCompleted : true] in the log.) Users will usually get an error message such as the following.

fault: {
code: "101505"
type: "Status report"
message: "Runtime Error"
description: "Connection close For : x.x.x.x For Request : Axis2Request [Message ID : urn:uuid:84445830-ed4e-5c15-cd87-bb5b21869e3f] [Status Completed : false] [Status SendingCompleted : true]"

Possible fix:

Some Back-end servers do not support persistent connections. So, they tend close down the connections intermediately. If this is the case, then we should disable the Keep-Alive connections at ESB side for such back-ends.

By default Keep-Alive is enabled in WSO2 ESB/APIM. You can switch off this per proxy/api by adding the following property mediator.

<property name="NO_KEEPALIVE" value="true" scope="axis2"/> 

But If you need to disable Keep-Alive globally for all the connections initiated from ESB, then that configuration should be specified in transport configuration files.

So, if you are using NIO transport, then add the following property to ESB_HOME/repository/conf/ But, if you are using Passthru transport, then add the following property to ESB_HOME/repository/conf/


Saturday, November 29, 2014

Restrict WSO2 Carbon management console access

The WSO2 Carbon products provides users with a browser based management console to administer the WSO2 products. All the WSO2 products comes with a management console to make the administration easier. When hosting the products ready for production, it's imperative to restrict the access to the mgt console from the external network.

Following steps shows how to restrict the access to WSO2 Carbon management console by IP.

Carbon itself gets added to the Tomcat runtime as a webapp. The "/carbon" is a servlet under it. Hence, we can use the Tomcat servlet filter, org.apache.catalina.filters.RemoteAddrFilter to restrict access.

1. Open CARBON_HOME/repository/conf/tomcat/carbon/WEB-INF/web.xml
2. Then, add the filter, org.apache.catalina.filters.RemoteAddrFilter, as follows. This filter can be used to restrict access by IP. Following is an example valve configuration to restrict the access only to localhost (considering both IPv4 and IPv6).

    <filter-name>Remote Address Filter</filter-name>

    <filter-name>Remote Address Filter</filter-name>

For parameter value reference, see -

Tuesday, August 5, 2014

JSF webapps fronted with reverse proxy

JSF web applications can be fronted by a reverse proxy just like any other applications, except when the reverse proxy has a different context path compared to the application server that this JSF webapp is deployed to.

Look at the following HTML content of a JSF Forms application. The 'action' URL of the form begins with a forward-slash '/'. This URL is actually a absolute url without the hostname. So, this is not a relative URL. This is a problem if your reverse proxy has a different context. Since the JSF webapp is not aware of the context path of the reverse proxy, the JSF form action will fail.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "">
<html xmlns="">
  <h2>Welcome to Marks Calculator</h2>
  <form id="j_id_4" name="j_id_4" method="post" 
    <table cellspacing="10">
          <td>Subject 1 marks:</td>
            <input id="j_id_4:j_id_8" name="j_id_4:j_id_8" type="text" value="2" />
          <td>Subject 2 marks:</td>
            <input id="j_id_4:j_id_b" name="j_id_4:j_id_b" type="text" value="4" />
    <input id="j_id_4:j_id_c" name="j_id_4:j_id_c" type="submit" value="Calculate" />
    <input type="hidden" name="j_id_4_SUBMIT" value="1" />
    <input type="hidden" name="javax.faces.ViewState" id="javax.faces.ViewState" value="wpq2AMaIFASZtUJ+IghBD2X2mhXm5MAFTDfdI20attDvNPOF " />

Now, let's look at how the Apache HTTPD configuration might look like. Do not add forward-slash at the end of ProxyPass/ProxyPassReverse since the Location, '/as', do not have a forward-slash at the end either.

<Location /as>
    Order Deny,Allow
    Deny from none
    Allow from all
    ProxyPass http://localhost:9763 retry=0 timeout=5
    ProxyPassReverse http://localhost:9763

Apache is configured to forward requests that come to http://localhost/as/ to http://localhost:9763/.

Now, you can access the webapp by using the request URL - http://localhost/as/jsf-custom-viewhandler-marks/index.jsf. Now, because of the JSF form 'action' URL, the form submit will try to call http://localhost/jsf-custom-viewhandler-marks/index.jsf which is non-existent.

Now, I hope you understand what's the problem here. In my view, this is a limitation of JSF. So, let's look at how to fix this.


Implementing A ViewHandler

To fix this, you need to implement javax.faces.application.ViewHandler, and register it to your webapp. In this custom view handler, we will set the action url as a relative url rather than a absolute url without the host name.

Needed dependencies -
commons-lang 2.6
commons-logging 1.1


import org.apache.commons.lang.StringUtils;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;

import javax.faces.application.ViewHandler;
import javax.faces.application.ViewHandlerWrapper;
import javax.faces.context.FacesContext;
import javax.servlet.http.HttpServletRequest;
import java.util.List;
import java.util.Map;

public class CustomViewHandler extends ViewHandlerWrapper {
  private static final Log log = LogFactory.getLog(CustomViewHandler.class);

  private ViewHandler wrappped;

  public CustomViewHandler(ViewHandler wrappped) {
    this.wrappped = wrappped;


  public ViewHandler getWrapped() {
    return wrappped;

  public String getActionURL(FacesContext context, String viewId) {
    String url =  super.getActionURL(context, viewId);
    log.debug("The getActionURL: " + url);
    return addContextPath(context, url);

  public String getRedirectURL(FacesContext context, String viewId, Map<String,
          List<String>> parameters, boolean includeViewParams) {
    String url =  super.getRedirectURL(context, viewId, parameters, includeViewParams);
    log.debug("The getRedirectURL: " + url);
    return url;

  public String getResourceURL(FacesContext context, String path) {
    String url = super.getResourceURL(context, path);
    log.debug("The getResourceURL: = " + url);
    return addContextPath(context, url);

  private String addContextPath(FacesContext context, String url) {
      final HttpServletRequest request = ((HttpServletRequest) 
      String result = url;
      if (url.startsWith("/")) {
          int subpath = StringUtils.countMatches(getPath(request), "/") - 1;
          String pathPrefix = "";
          if (subpath > 0) {
              while (subpath > 0) {
                  pathPrefix += "/..";
              pathPrefix = StringUtils.removeStart(pathPrefix, "/");
          result = pathPrefix + result;
      return result;

    private String getPath(final HttpServletRequest request) {
        try {
            return StringUtils.replace(new URI(request.getRequestURI()).getPath(), "//", "/");
        } catch (final URISyntaxException e) {
            return StringUtils.EMPTY;


In the addContextPath private method, we modify the original url, and replace it with a relative url.

Then, build this class into a jar and pack it into WEB-INF/lib of your web application. You can find the source and a Apache Maven build file at the end of this article.

Now, you need to register this ViewHandler with your web application. For that, open your faces-config.xml and add your view handler as follows. The faces-config.xml is usually placed under WEB-INF/ folder in the webapp.


Now, the faces-config.xml might look as follows.

<?xml version='1.0' encoding='UTF-8'?>
<faces-config xmlns=""





Setting the sessionCookiePath

Some JSF webapps depends on sessions and JSESSIONID cookie. Usually, JSF sets this cookie to the webapp path. But if the reverse proxy has a different context, then the cookie fails to get stored in the browser properly. In my case, I use EJB with the JSF webapp, and I needed to fix this issue. Otherwise, I faced the below exception.

To fix this, you need to add context.xml to your webapp, and set the sessionCookiePath attribute. We can configure the JSESSIONID cookie path via this configuration file. In our case, we can set it as follows. See the tomcat documentation for more details.

<Context sessionCookiePath="/as/jsf-custom-viewhandler-marks">

There, the path has taken into account the reverse proxy configuration path "/as".

Now you are all set. You can use the given Custom View Handler sample code in your own JSF webapps. I have also posted a sample JSF webapp that uses EJB. The sample app can be deployed Apache TomEE.

You can find a sample JSF Custom View Handler code here -

You can find the sample webapp here -

Tuesday, May 27, 2014

Invoke STS Service in WSO2 IS using CXF STS Client

Following guide describes how to invoke the STS Service in WSO2 Identity Server using a CXF STS client. The STS Client is configured via Spring.

As I have noticed, CXF STS client fails to invoke the STS service if you use the wso2carbon-sts wsdl to generate the client. So, I have followed a different approach.


WSO2 Identity Server 4.6.0 or latter
Apache CXF 2.7.x


What I'm going to show you is using the CXF client via a simple stand-alone java class with Spring.


1. First, you need to define the spring bean. Let's say the file is named wssec-sts-bean.xml.

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns=""
       xsi:schemaLocation="    ">

<bean id="{}wso2carbon-stsHttpsSoap12Endpoint.sts-client"
    <constructor-arg ref="cxf"/>
    <!--<property name="wsdlLocation" value="https://localhost:9443/services/wso2carbon-sts?wsdl"/>-->
    <property name="Location"
    <property name="serviceName" value="{}wso2carbon-sts"/>
    <property name="endpointName" value="{}wso2carbon-stsHttpsSoap12Endpoint"/>

    <property name="properties">
            <entry key="ws-security.username" value="admin"/>
            <!--<entry key="ws-security.password" value="admin"/>-->
            <entry key="ws-security.callback-handler" value="com.cxf.sts.ClientCallbackHandler"/>
            <entry key=""
            <entry key="ws-security.encryption.username" value="wso2carbon"/>
            <entry key="ws-security.sts.applies-to"

NOTES: Please note that the wsdlLocation has NOT been set. Instead, we set the Location attribute which is the SOAP 1.2 STS https endpoint. Other attributes are the usual attributes that we normally set.

2. For the property in the spring bean config, you have to  point to a properties file that has the encryption properties. The contents of would look like follows.

3. We need one more configuration file to store the ws-policy of the STS service. First, you need to secure the STS service in IS as described in this documentation -
Now, if you browse to the wso2carbon-sts WSDL, you will see the policy that is applied to your service. Extract the policy element from the WSDL. It starts with <wsp:Policy ...>. If you secure the service with UsernameToken, then the policy would like below. Place it in a file called sts.policy.xml.

<wsp:Policy xmlns:wsp="" xmlns:wsu="" wsu:Id="UTOverTransport">
    <wsp:ExactlyOne  xmlns:wsp="" xmlns:wsu="" >
            <sp:TransportBinding xmlns:sp="">
                        <sp:HttpsToken RequireClientCertificate="false"></sp:HttpsToken>
            <sp:SignedSupportingTokens xmlns:sp="">
                    <sp:UsernameToken sp:IncludeToken=""></sp:UsernameToken>

Now all the configuration files are in place. Now, we can go ahead, and write java code to retrieve the sts tokens.

STS Service Invocation

1. First, we need to set the trust-store of wso2 is as a system property since we are invoking a https endpoint. And, then, we will read the spring beans file.

System.setProperty("", "wso2carbon");

2. Next, we should read the spring bean file, wssec-sts-bean.xml, and get the bean we created.

ApplicationContext ctx = new FileSystemXmlApplicationContext(
STSClient sts = (STSClient) ctx.

3. Now, we have the STSClient instance. We need to set some attributes to the client before we can request security tokens. One is the policy we just extracted from the WSDL.

//parse the ut policy xml, and get a DOM element
File f = new File("src/main/resources/sts.policy.xml");
Element stsPolicy = loadPolicy(f.getAbsolutePath());


4. Then, we can go ahead and request sts tokens.

SecurityToken samlToken = 
        "/Issue", null);

//convert the token dom element to string
String token = ((DOMImplementationLS) samlToken.getToken().getOwnerDocument().getImplementation()).

With these steps, you should be able to invoke STS service in WSO2 Identity Server using a CXF client. You can download the full source of this from here -



1. Start WSO2 Identity Server 4.6.0.
2. Secure STS Service with UsernameToken as described in this documentation -
3. Extract the client,
4. Run the client as follows.

mvn clean install -Pclient