I’ve just pubblished a post on the JBosWS blog regarding the JBossWS involvement in the Apache CXF project. In few words, the JBossWS team is increasing its collaboration with the CXF developers, the target being to improve both projects.

It’s not that simple to achieve an active bi-directional collaboration, with both parties’ needs being considered, but this is working quite well now. For instance, read what Daniel Kulp (CXF lead) writes about the collaboration. Needless to say I like this, that’s a nice example of what open source can make possible.

Today I have spent a lot of time with a very strange issue compiling JAX-WS generated classes. I have been using jbossws wsconsume to generate some classes from a .NET wsdl and I had a very strange behaviour:

1. Generate class calling wsconsume
2. compile them and its client using ant and run my test perfectly working
3. then calling my clean task to remove .class files and recompile them and run my tests doesn’t work!

IOW JAX-WS client have been working only the first time I compile them. I couldn’t figure out why it have been working in that manner, but after a lot of google search I got this commit. In practice what have been happening is better described in “Note on package-info.java” paragraph of javac target in ant manual. Starting from version 1.7.1 ant compile package-info.java only in these 3 case:

1. If a package-info.class file exists and is older than the package-info.java file.
2. If the directory for the package-info.class file does not exist.
3. If the directory for the package-info.class file exists, and has an older modification time than the the package-info.java file. In this case <javac> will touch the corresponding .class directory on successful compilation.

In practice if you havea ant task like mine:

<target name=”compile” depends=”init” description=”Compile the Java source code”>
<javac destdir=”${classes.dir}” classpathref=”build.classpath” debug=”${javac.debug}” deprecation=”${javac.deprecation}” target=”1.5″>
<src path=”${src.java.dir}” />
</javac>

Here the compilation target compile all files and so create directory where package-info.class will be contained during other generated files compilation. In this case the compilation target never re-generate package-info.class because no one of the 3 conditions is true. My workaround have been to change my build.xml file and have this compile target:

<target name=”compile” depends=”init” description=”Compile the Java source code”>
<touch>
<fileset dir=”${src.java.dir}” includes=”**/package-info.java”/>
</touch>
<javac destdir=”${classes.dir}” classpathref=”build.classpath” debug=”${javac.debug}” deprecation=”${javac.deprecation}” target=”1.5″>
<src path=”${src.java.dir}” />
</javac>

Hoping this post would be useful for someone, let me remark that is a problem of ant javac task, not of jbossws and you will get the same problem with any other jaxws stack.

Let me remark also that Wise perfectly work in this case regenerating it’s classes on the fly

Today I was importing a .NET generated wsdl and getting this error using JBossWS wsconsume.sh tool (yes I’m not using Wise for this particular case!! But the problem would be the same in Wise since it use wsconsume under the hood) :

[WARNING] src-resolve.4.2: Error resolving component 'sch:schema'. It was detected that 'sch:schema' is in namespace 'http://www.w3.org/2001/XMLSchema', but components from this namespace are not referenceable from schema document 'file:/home/oracle/Desktop/security.asmx#types?schema1'. If this is the incorrect namespace, perhaps the prefix of 'sch:schema' needs to be changed. If this is the correct namespace, then an appropriate 'import' tag should be added to 'file:/home/oracle/Desktop/security.asmx#types?schema1'.
  line 85 of file:/home/oracle/Desktop/security.asmx#types?schema1
[ERROR] undefined element declaration 'sch:schema'
  line 85 of file:/home/oracle/Desktop/security.asmx
[ERROR] undefined element declaration 'sch:schema'
  line 178 of file:/home/oracle/Desktop/security.asmx
[ERROR] undefined element declaration 'sch:schema'
  line 217 of file:/home/oracle/Desktop/security.asmx
[ERROR] undefined element declaration 'sch:schema'
  line 231 of file:/home/oracle/Desktop/security.asmx
[ERROR] undefined element declaration 'sch:schema'
  line 245 of file:/home/oracle/Desktop/security.asmx
[ERROR] undefined element declaration 'sch:schema'
  line 259 of file:/home/oracle/Desktop/security.asmx
[ERROR] undefined element declaration 'sch:schema'
  line 273 of file:/home/oracle/Desktop/security.asmx
 
Failed to invoke WsImport
java.lang.NullPointerException
        at com.sun.tools.xjc.reader.internalizer.SCDBasedBindingSet.apply(SCDBasedBindingSet.java:237)
        at com.sun.tools.xjc.ModelLoader.createXSOM(ModelLoader.java:518)
        at com.sun.tools.xjc.api.impl.s2j.SchemaCompilerImpl.bind(SchemaCompilerImpl.java:232)
        at com.sun.tools.xjc.api.impl.s2j.SchemaCompilerImpl.bind(SchemaCompilerImpl.java:85)
        at com.sun.tools.ws.processor.modeler.wsdl.JAXBModelBuilder.bind(JAXBModelBuilder.java:134)
        at com.sun.tools.ws.processor.modeler.wsdl.WSDLModeler.buildJAXBModel(WSDLModeler.java:2244)
        at com.sun.tools.ws.processor.modeler.wsdl.WSDLModeler.internalBuildModel(WSDLModeler.java:187)
        at com.sun.tools.ws.processor.modeler.wsdl.WSDLModeler.buildModel(WSDLModeler.java:133)
        at com.sun.tools.ws.wscompile.WsimportTool.run(WsimportTool.java:182)
        at org.jboss.ws.tools.jaxws.impl.SunRIConsumerImpl.consume(SunRIConsumerImpl.java:217)
        at org.jboss.wsf.spi.tools.cmd.WSConsume.importServices(WSConsume.java:223)
        at org.jboss.wsf.spi.tools.cmd.WSConsume.main(WSConsume.java:81)

After some search Alessio pointed me to this excellent post where the problem is well explained and solution provided. The “problem” is not specific to JBossWS and solution explained there is fine for JBossWS too.

Hoping this cross post could help someone looking for the error in relation to JBossWS in google

I’m participating in this poll as candidate for New features contribution.

As you probably remember I’ve contributed in DNA, JBossWS (sometime in the past), and JBossESB integrating Wise into ESB 4.4.  Moreover Wise is now a JBoss.org project: I’ve donated it some months ago and I’m leading the project there, and we have already released 0.9 and 1.0 versions.

If you like my efforts and would support me, and/or you are using Wise (within JBossESB or not) may I kindly ask  your vote there (you need a JBoss.org account, but it’s quite easy to register one):

http://www.jboss.org/community/poll.jspa?poll=1003

More infos from JBoss.og homepage:

Voting will end on January 30th 2009 and winners will be announced at the JBoss Virtual Experience, a web-based JBoss technology conference which will be held February 11th 2009. There is no charge for admission, but please advance register if you’d like to attend.
Terms and conditions are here. Please join us in giving these community members the recognition they deserve.

As said in this post, a lot of the strength spurring on an open source developer is narcissism. Help mine to grow up

As I wrote in some previous posts me and my fine team are working from a while to a project using JBossESB Wise action in a real world enterprise application. We are using it for the ETL (Extract Transfor Load) phase for a big DWH (Data Ware House) with an incremental loading of data.

In a nutshell we trace logical changes on an OLTP database (it’s a financial DB where all changes can be associated logically to a single company or at least to a network of company related for various reasons). Then we use JBossESB (and in particular SQLGateway) to periodically treat modified companies and extracting and enriching information to be loaded on the DWH instance. Where wise have its place? Well a lot of information and business rule to extract or enrich data have been implemented as webservices in last 3/4 years. So it’s pretty natural to reuse them to implement this last application.

Ok, it’s the bird eye view of the problem and the solution. On the rest of the post I’ll go in more formal details, starting with requirement and environment description

Requirement and environment description

The main requirement have been to collect a set of data regarding a large set of company (about 5 million) in a DWH for a marketing analysis. This data comes from different systems: 3 different OLTP relational database, and legacy host based system, an external provider. The good news is that both host system and external provider are accessible using webservices. Moreover OLTP databases have some webservices extracting data applying complex business rules; they doesn’t cover all requirements, but these DBs are completely under control of our development team, and dedicated jdbc and/or EJB3 access could be developed for new goals.

The final users would update it’s DWH with daily frequency. The large amount of data made impossible to extract transform and load the whole data every night. We have decided to keep track of changes on the main OLTP DB, and reload completely companies changed (some thousands a day).

Of course this approach isn’t totally new, incremental ETL are pretty common in DWH world, and all vendors have its own proprietary solution. While these proprietary system have its place and its plus, isn’t IMHO sufficient flexible to support an heterogeneous environment as one described. I thought it’s better to track with proprietary triggers logical significative changes (not a lot in fact) and adopt a SOA solution for ETL. It would be better in terms of flexibility and would permit us to reuse much more easily a lot of already written services containig complex business rules.

So the solution adopted have been based on JBossESB ant its composed by these macro steps:

1. A set of triggers on 2 of 3 named OLTP DB collect changes and write a unique identifier of the company in a dedicated table
2. A SQLGateway consume this table (the frequency of wake up and filters of the query are designed to avoid excessive and and not useful double treatment of companies due to double linked changes)
3. Any company is processed by a set of action chains. This actions could be locally defined actions reading relational database or Wise based web services invocations. A content based router policy route messages from an action chain to the next one.
4. Finally data extracted and transformed are written on the DWH.

Point 3 is of course the core of the system. The SQLGateway create a message containing a pojo object called Company and any successive action trasform or enrich this object with data collected and business rules applied. Wise’s based action calls webservices and use smooks to transform and enrich input object with ws returned values. Using CBR and continuous enrichment of the same object we get at last action (writeOnDWH) an object with all data needed t be written on the DWH.

Focus on Wise

A lot of actions are simply webservices calls implemented with a zero-code approach using Wise. We had just to write jboss-esb.xml fragment for webservice call and smooks config files to get a lot of business rules reused. It have been really GREAT!

I need to add some patch to current integration in ESB to obtain the max response from wise, but results have been really impressive: we had something like 90K company processed in an hour. What does it mean in finer details? Well from wise point of view about 300K web services calls in an hour!
Well also performance and numbers of ESB have been impressive: we are running on a single Linux64 machine (AMD64 double dual core) with 10 jms-listener processing 10 different chains  (200 concurrent 3ad for any jms-listener) for a total of 1.7M (wise and not) of actions called in an hour.

Isn’t it impressive numbers?

There is a list of patches I applied to wise/esb integration to support my requirement. All the code are committed on my workspace (maeste) in ESB svn:

	JBESB-2019	wise should pass to smooks response mapper also input data to permit continuos enrichement of message
	JBESB-2020	wise have a bug for which it may download too many wsdls and store them in a temporary dir
	JBESB-2021	add configurability for location where wise store smooks reports for its transformation
	JBESB-2022	wise doesn’t clean its internal smooks cache
	JBESB-2023	Wise is failed to consume a wsdl which contains two schema element with same name and different namespace .
	JBESB-2036	wise’s sample have problem because targetPackage not specified in properties files
	JBESB-2037	Avoid excessive reflective inspection of wise classes for better performance

I can’t go in more detail of the implementation or put here configs files because I cna’t reveal any business details of the application. I’ll try in next future to arrange an example totally equivalent in technology content, but without any link to real business content. If you are interested let me know, but be patients…it’s not a joke and I’m very very busy these days.

Thanks to my team (special thanks to Paolo and Luca)  and all contributors of Wise and ESB to make it possible

PS: what about huge split and route qs included in ESB 4.4. Well they cover different problems, even if not far each other. The main difference is that here we haven’t a huge message to split and route, but a lot of little message to enrich and then route (content based) to next enrichment phases.

As said in this post one of possible use of wise-core (the new core we get independent from Wise) is to integrate it in JBossESB to make a generic soap client invoking web service using Smooks transformation to hide final user the gap between their own object models and one generated by JAX-WS tools dynamically.

Well I contribute with some code to JBossESB providing an action which does what I described in a nutshell here. My efforts and possible improvements are described in this post on ESB developer forum.

Give your feed back there.

BTW I’m developing a real world application based on ESB and this wise-action: it takes some date from a db, enrich the message calling a set of webservices using wise, conditioning these calls with a content based routing approach, and then write the databack into db. I’m planning a post about this application as soon as me and my team will finish it….stay tuned!

It’s time to discuss with the community the future of Wise and Lms. First of all a clarification: Alessio and me have being very busy in last months, but we are not planning to stop development of Wise and Lms.

We have 2 different approach possible, not mutually exclusive each other:

1. integrate wise and Lms adding some functionality like saving request/response and code generation, making wise+lms a suite to test webservice, keeping QA people in mind and their relation with development people. We are thinking in this area about functionality to save, review and resend request and a common dashboard to manage wsdls, servers and so on.
2. Separate wise-core (dynamic generation of jaxws client and object exchange) from wise-gui (or wise-seam as you like to call it). This step would be important to use wise-core as generic API to call webservice

Ok, first point is well known for our blog’s readers, but why we are starting a discussion now about the second point? Well, it comes from the idea to use wise to make an action for jbossesb for call web service instead of the current implementation. I took a look to what jbossesb is doing at the moment in this area, and I thought wise can do a lot of these things, using standard JAX-WS wsconsume APIC generated classes.

I don’t want to discuss here the specif needs of jbossesb, but in general I believe it could be a good approach when you need a total and effective decoupling of the client and server using ws to inter operate. It’s matter of fact that wsconsume tools is great for java developer, generating needed stub class, but it introduce a new (or renewed ) level of coupling very similar to corba IDL. Generating statically webservice stub you are in fact coupling client and server.

So what is the alternative? Writing dynamic client using Dynamic dispatch JAX-WS API? Maybe, but I would prefer a solution using dynamic mapping on generated stub, exactly what wise 1.x does. In fact Wise did a mapping between a generuic user interface and generated stub objects. My idea is to extend this concept and permit to call a webservice mapping a generic Object Model to jaxws generated one. Are we reinventing the wheel? I don’t think so because IMHO JAX-WS dynamic dispatch isn’t a perfect turning wheel, and I hope Wise-core does the job better

How wise-core does perform this generic task? In a nutshell it does the same wise 1.x did, generating on the fly classes using wsconsume runtime API, loading them in current class loader and use them with Java Reflection API. What we add is generic mapping API to transform an arbitrary object model in wsconsume generated ones, make the call, and mapping the answer back again to custom model.

The implementation can be viewed in our svn (trunk), and it’s based on a refactoring of our original code and an extensive use of Smooks. Smooks is an amazing tool! Have a look! I will post about shortly, stay tuned!

Here is some ideas done or in plan in the wise-core

• Much more unit tests. The core was really poor in this area. [DONE]
• Calling WS using a custom object model, and delegating to wise-core the responsability of required mapping action [DONE]
• Adding jaxws client handler(custom one and some ones implemented in the core, at least one smooks’ based [DONE]
• smooks based javabeans mapping from user object to wsconsume generated object [DONE]
• write examples of use [partially DONE]
• write good javadoc and user manual [TODO]
• write integration and example of integration with jbossesb [DONE, not in repository]
• generating Smooks config skeleton for generated object. Without them who write the smooks config file for mappers have to know the classes hierarchy generated by wsconsume (IOW generate it statically first time to have a look ) [TODO]
• Use wise-core in wise-gui, using smooks mapper against our proprietary TreeElement Object to map to generated classes [TODO, it needs next point]
• Dynamic generation of smooks config knowing target Object hirearchy (our TreeElement) and wsconsume’s generated objects. [TODO]
• Evaluate different mapping tools (to be added to smooks, not to substitute it for sure!). Maybe Dozer, even if I suspect some problem with it for our goals. [TODO]

In our svn you can find a test demostrating the use in a stand alone application.

any opinion?

A couple of months ago one of the JBoss forum users suggested that notification messages sent by the jbossws ws-eventing engine should be validated against the underlying schema before the actual dispatch takes place (link). Heiko Braun opened a feature request issue on JIRA… and last week we submitted a patch for it.

If you’re interested in adding this feature and you’re running jbossws 1.0.4 GA, you can read about and download it here.

Still working on ws-eventing and JBossWS… We noticed that event notification failures are completely lost. If an event notification fails (for example because the event sink is down) the current jbossws engine (1.0.4.GA) just logs the error and does nothing else. Of course it could be useful at least to keep track of failed notification in order to send them again later or simply to warn the sink’s owner.

So we’ve just made a patch to jbossws, saving notification failures into a list stored in the SubscriptionManager mbean.

You can find the patch on jboss jira: http://jira.jboss.com/jira/browse/JBWS-1406

We got some troubles using ws-eventing jbossws implementation in our clustered/load balanced environment (for a deeper description read here). I’ll try to describe them in this post, with some proposed solutions:

Click here for a larger image.

The previous diagram describes the two main common problems we had with ws-eventing deployed in a clustered environment:

• Subscription on one of the clustered environment’s machine
  

  DESCRIPTION: when a client subscribes to our ws-eventing service the load balancer sends his requests to a real server that receives the subscription and store it locally (in current jbossws eventing implementation this is achieved using a Map owned by SubscriptionManagerMbean). An event source generates a notification and delegates its notification to jbossws; but in a cluster environment this event source could use a different real server knowing nothing about the subscriber. The notification would be lost. See messages from 1 to 5 in the above diagram. Of course a very similar situation may happen when renewing a subscription: a renew message cold be send to a server different from the one that received the first subscription, resulting in subscription renew failure. The client will receive an “Unable to renew” exception, and on the server side the subscription will expire for timeout! All notifications are lost until client re-subscribe to the right real server. See messages from 6 to 13 in the above diagram.

  PROPOSED SOLUTION: These two problems could be solved clustering the SubscriptionMap. The easiest solution would probably be arranged using HAJNDI to store subscriptions. Of course the use of a local Map instead of HAJNDI have to be configured at deploy time.

• Shutdown of one of the cluster’s machine
  

  DESCRIPTION: SubscriptionManagerMbean sends notifications to remove every subscriptions when it is stopped or the real server it runs on shutdowns. But in a cluster environment other servers continue to work and potentially to send notifications. Those notifications will be lost. See messages from 14 to 26 in the above diagram.

  PROPOSED SOLUTION: SubscriptionManagerMbeans have to be clustered and have to exchange their status before they can decide to send this kind of notifications. The solution could be implemented using jGroups (of course), but it needs further investigations and discussions, mainly to understand if SubscriptionManager is the only part of jbossws needing clustered solutions

I started a thread in JbossWS Design forum aboute this issue.