Today I was importing a .NET generated wsdl and getting this error using JBossWS wsconsume.sh tool (yes I’m not using Wise for this particular case!! But the problem would be the same in Wise since it use wsconsume under the hood) :

[WARNING] src-resolve.4.2: Error resolving component 'sch:schema'. It was detected that 'sch:schema' is in namespace 'http://www.w3.org/2001/XMLSchema', but components from this namespace are not referenceable from schema document 'file:/home/oracle/Desktop/security.asmx#types?schema1'. If this is the incorrect namespace, perhaps the prefix of 'sch:schema' needs to be changed. If this is the correct namespace, then an appropriate 'import' tag should be added to 'file:/home/oracle/Desktop/security.asmx#types?schema1'.
  line 85 of file:/home/oracle/Desktop/security.asmx#types?schema1
[ERROR] undefined element declaration 'sch:schema'
  line 85 of file:/home/oracle/Desktop/security.asmx
[ERROR] undefined element declaration 'sch:schema'
  line 178 of file:/home/oracle/Desktop/security.asmx
[ERROR] undefined element declaration 'sch:schema'
  line 217 of file:/home/oracle/Desktop/security.asmx
[ERROR] undefined element declaration 'sch:schema'
  line 231 of file:/home/oracle/Desktop/security.asmx
[ERROR] undefined element declaration 'sch:schema'
  line 245 of file:/home/oracle/Desktop/security.asmx
[ERROR] undefined element declaration 'sch:schema'
  line 259 of file:/home/oracle/Desktop/security.asmx
[ERROR] undefined element declaration 'sch:schema'
  line 273 of file:/home/oracle/Desktop/security.asmx
 
Failed to invoke WsImport
java.lang.NullPointerException
        at com.sun.tools.xjc.reader.internalizer.SCDBasedBindingSet.apply(SCDBasedBindingSet.java:237)
        at com.sun.tools.xjc.ModelLoader.createXSOM(ModelLoader.java:518)
        at com.sun.tools.xjc.api.impl.s2j.SchemaCompilerImpl.bind(SchemaCompilerImpl.java:232)
        at com.sun.tools.xjc.api.impl.s2j.SchemaCompilerImpl.bind(SchemaCompilerImpl.java:85)
        at com.sun.tools.ws.processor.modeler.wsdl.JAXBModelBuilder.bind(JAXBModelBuilder.java:134)
        at com.sun.tools.ws.processor.modeler.wsdl.WSDLModeler.buildJAXBModel(WSDLModeler.java:2244)
        at com.sun.tools.ws.processor.modeler.wsdl.WSDLModeler.internalBuildModel(WSDLModeler.java:187)
        at com.sun.tools.ws.processor.modeler.wsdl.WSDLModeler.buildModel(WSDLModeler.java:133)
        at com.sun.tools.ws.wscompile.WsimportTool.run(WsimportTool.java:182)
        at org.jboss.ws.tools.jaxws.impl.SunRIConsumerImpl.consume(SunRIConsumerImpl.java:217)
        at org.jboss.wsf.spi.tools.cmd.WSConsume.importServices(WSConsume.java:223)
        at org.jboss.wsf.spi.tools.cmd.WSConsume.main(WSConsume.java:81)

After some search Alessio pointed me to this excellent post where the problem is well explained and solution provided. The “problem” is not specific to JBossWS and solution explained there is fine for JBossWS too.

Hoping this cross post could help someone looking for the error in relation to JBossWS in google

As my reader have learn by my last post I (and Alessio) have just donated Wise (and LMS) to JBoss.org.

But what does it mean? And more important why?

What does it mean?

Wise has been since the beginning released on LGPL license. Of course donating our code to JBoss.org we haven’t changed nothing about license. Wise still be an open source project and neither is changed, you still have code, you still have a business friendly open source license.

Code donation just change the Intellectual Property owners, switching from us to JBoss, divison of Red Hat. So it seems we are leaving all what we have on this project, but we are doing that for some good reasons.

And what does it mean for the project? More or less nothing. We are not donating and leaving project, but we will be active members (more precisely I’m going to lead the project). The good news are two new contributors from JBoss world are lending a hand with the code.

Why?

The real question is why have I decided to donate? Well, I’ve used “we” until now because Alessio and me have generally agreed on this donation. I’m switching to first singular person because Alessio kindly give me last word on the donation both because his position could condition him and because all last code written to transform wise from a general web client to zero code web service client easily integrable in a lot of server side application comes by me.

Reasons are essentially 4:

• Wise is essentially an hacking against jbossws and jax-ws tools. Quite frankly I believe a pure hacking could be good for a personal use project, which more or less have been for years Wise. But when Wise have gotten a lot of interest integrating with the (cool) jbossesb project my point of view has changed. The question is: is possible to maintain wise and add features without having a strict join with jbossws and/or jbossesb with which is strictly dependent/integrated? And are there any better way to get it joined than make Wise a JBoss.org project? More users have been using Wise, and I have gotten some good feedback, but I think they need a serious approach of the project ensuring them Wise will not break (or at lest it will have great chance to don’t) it compatibility and interoperability which are the main reasons a lot of users is coming to us.
• Becoming a JBoss.org project Wise would probably have a larger and greater community. It should mean more and better feedbacks and contribution (already verified since we have gotten 2 new contributors). More feedbacks and contribution mean better software and it’s all what I care.
• I believe JBoss open source model is good. I think what they call professional open source is great. Maybe not all will agree with me, but it’s my own thoughts. It’s not planned at the moment, but I wouldn’t have any problem if one day Wise would be included in a JBoss product and someone will pay for support. Moreover I think it would be great, and for sure I wouldn’t have sufficient time to provide a professional support to them. I think open source is great, but home made product haven’t a real future, they need some kind of support, some company behind. I hope JBoss could be the company behind Wise.
• Last but not least because I’m honoured of JBoss’s interest on my project. A lot of the strength spurring on an open source developer is narcissism And I can ensure you sometimes it’s really needed with a little daughter and a lovely wife (with a great sense of humour) saying “ok, you don’t smoke, but you write code”

Have I taken a mistake? I don’t believe so. Anyway Wise is not going to an end, but to a bright new beginning: we have a lot of ideas, both increasing wise current capability and adding totally new features making it much more useful both in its traditional world (web services client/test) and in some new way I’m currently exploring and I’ll post about very soon. Stay tuned, and please join us and our building community contributing on our public forums, reporting bugs or writing some code. BTW Wise now have its own blog even if I’ll continue to cross post here for a while for wise’s older fans .

Moreover we are not discontinuing this blog where we will continue to post our experiences and point of views not necesseraly related to Wise (BTW the two most read posts aren’t about Wise, but subversion branching and a new approach to unit tests).

Stay tuned.

As I wrote in some previous posts me and my fine team are working from a while to a project using JBossESB Wise action in a real world enterprise application. We are using it for the ETL (Extract Transfor Load) phase for a big DWH (Data Ware House) with an incremental loading of data.

In a nutshell we trace logical changes on an OLTP database (it’s a financial DB where all changes can be associated logically to a single company or at least to a network of company related for various reasons). Then we use JBossESB (and in particular SQLGateway) to periodically treat modified companies and extracting and enriching information to be loaded on the DWH instance. Where wise have its place? Well a lot of information and business rule to extract or enrich data have been implemented as webservices in last 3/4 years. So it’s pretty natural to reuse them to implement this last application.

Ok, it’s the bird eye view of the problem and the solution. On the rest of the post I’ll go in more formal details, starting with requirement and environment description

Requirement and environment description

The main requirement have been to collect a set of data regarding a large set of company (about 5 million) in a DWH for a marketing analysis. This data comes from different systems: 3 different OLTP relational database, and legacy host based system, an external provider. The good news is that both host system and external provider are accessible using webservices. Moreover OLTP databases have some webservices extracting data applying complex business rules; they doesn’t cover all requirements, but these DBs are completely under control of our development team, and dedicated jdbc and/or EJB3 access could be developed for new goals.

The final users would update it’s DWH with daily frequency. The large amount of data made impossible to extract transform and load the whole data every night. We have decided to keep track of changes on the main OLTP DB, and reload completely companies changed (some thousands a day).

Of course this approach isn’t totally new, incremental ETL are pretty common in DWH world, and all vendors have its own proprietary solution. While these proprietary system have its place and its plus, isn’t IMHO sufficient flexible to support an heterogeneous environment as one described. I thought it’s better to track with proprietary triggers logical significative changes (not a lot in fact) and adopt a SOA solution for ETL. It would be better in terms of flexibility and would permit us to reuse much more easily a lot of already written services containig complex business rules.

So the solution adopted have been based on JBossESB ant its composed by these macro steps:

1. A set of triggers on 2 of 3 named OLTP DB collect changes and write a unique identifier of the company in a dedicated table
2. A SQLGateway consume this table (the frequency of wake up and filters of the query are designed to avoid excessive and and not useful double treatment of companies due to double linked changes)
3. Any company is processed by a set of action chains. This actions could be locally defined actions reading relational database or Wise based web services invocations. A content based router policy route messages from an action chain to the next one.
4. Finally data extracted and transformed are written on the DWH.

Point 3 is of course the core of the system. The SQLGateway create a message containing a pojo object called Company and any successive action trasform or enrich this object with data collected and business rules applied. Wise’s based action calls webservices and use smooks to transform and enrich input object with ws returned values. Using CBR and continuous enrichment of the same object we get at last action (writeOnDWH) an object with all data needed t be written on the DWH.

Focus on Wise

A lot of actions are simply webservices calls implemented with a zero-code approach using Wise. We had just to write jboss-esb.xml fragment for webservice call and smooks config files to get a lot of business rules reused. It have been really GREAT!

I need to add some patch to current integration in ESB to obtain the max response from wise, but results have been really impressive: we had something like 90K company processed in an hour. What does it mean in finer details? Well from wise point of view about 300K web services calls in an hour!
Well also performance and numbers of ESB have been impressive: we are running on a single Linux64 machine (AMD64 double dual core) with 10 jms-listener processing 10 different chains  (200 concurrent 3ad for any jms-listener) for a total of 1.7M (wise and not) of actions called in an hour.

Isn’t it impressive numbers?

There is a list of patches I applied to wise/esb integration to support my requirement. All the code are committed on my workspace (maeste) in ESB svn:

	JBESB-2019	wise should pass to smooks response mapper also input data to permit continuos enrichement of message
	JBESB-2020	wise have a bug for which it may download too many wsdls and store them in a temporary dir
	JBESB-2021	add configurability for location where wise store smooks reports for its transformation
	JBESB-2022	wise doesn’t clean its internal smooks cache
	JBESB-2023	Wise is failed to consume a wsdl which contains two schema element with same name and different namespace .
	JBESB-2036	wise’s sample have problem because targetPackage not specified in properties files
	JBESB-2037	Avoid excessive reflective inspection of wise classes for better performance

I can’t go in more detail of the implementation or put here configs files because I cna’t reveal any business details of the application. I’ll try in next future to arrange an example totally equivalent in technology content, but without any link to real business content. If you are interested let me know, but be patients…it’s not a joke and I’m very very busy these days.

Thanks to my team (special thanks to Paolo and Luca)  and all contributors of Wise and ESB to make it possible

PS: what about huge split and route qs included in ESB 4.4. Well they cover different problems, even if not far each other. The main difference is that here we haven’t a huge message to split and route, but a lot of little message to enrich and then route (content based) to next enrichment phases.

We are proud to announce that recently released JBossESB 4.4 contain a wise based implementation of webservice client invoker.

In a nutshell it is a zero-code webservice caller supporting smooks based mapping, and pluggable JAX-WS handler. Here is an abstract of the message with which I presented it to ESB community (here you find original message and related discussion):

It uses wsconsume API to dynamically generate client object and invoke web service, delagating to JBossWS JAX-WS implementation the dirty job.
It use smooks under the hood to transform user defined object into JAX-WS generated ones.

It support also standard JAX-WS handler and a generic smooks transformation handler to apply transformation to generated soap messages.

You can find it in my workspace under product/services/soap/src/main/java/org/jboss/soa/esb/actions/soap/wise/
I also wrote javadoc for the action class explaining how to use it and e example demonstrating 3 common use case:

* Direct call of a simple service without any mapping is needed
* Call of a service using a smooks mapper java-to-java
* Call a simple webservices without mapping, but with an handler
modifying header with smooks and an handler logging on System.out
request and response
In this 3 examples don’t forget to have a look to wise-core.properties for some important configs. Of course they could be integrated in action’s config in jboss-esb.xml in next future, but this first implementation leave them there.

On wise roadmap I have the implementation of webservices’ call receiving different resources (CSV, XML and so on) using smooks to map it on JAX-WS generated client objects, giving another interesting opportunity in ESB environment.

It is an initial implementation, and I need to integrate wise objects generation with new smooks configgenerator ( http://milyn.codehaus.org/Smooks+User+Guide#SmooksUserGuide-GeneratingtheSmooksBindingConfiguration ) to make user experience easier.

Moreover we are working on wise-core to improve it and make it more configurable an pluggable and support much more stuffs. I’ll post a roadmap soon.

Stay tuned!

Something happen at the office some days ago:

phone ringing

me: “Hello”
other: “Hi, I’m ***, I’m testing your webservices to implement X function of your B2B process”
me: “Ok, and…?”
other: “I’m getting timeout immediately”
me: “well timeout and immediately, seems two words in little contrast…” (some ritual questions about framework used, setting and so on) and finnally
me: “did you force a timeout client side?”
other: “Yes I set it at 300…but I get immediatly timeout”
me: “Immediately….. like 1/3 of second?”
other: “I don’t know…but I set it at 300 seconds, I’m not a newbie”
me: “No for sure, but may I ask if you sure they are seconds…..?”

ROTFL, sure he isn’t a newbie just a ……(put here what you want)……

Well, needless to say the answer about which framework he is using have been “.NET of course”. Yep of course

One of the use case for witch a SOA (ESB) solutions is recommended is when you have to manage a complex “technology heterogeneous” environment.

Well, I’m thinking about a good design for some new important feature to be added to our complex environment. Our environment is indeed complex, with wide impact, with heterogeneous needing, but it is quite homogeneous in technology. OK, it isn’t a monolithic system, it is build by a lot of part, but a lot of this part are java(2ee)/oracle based.

But the question is:do I like to keep my system so homogeneous? IOW if I invest a lot of money adding these new features to my system, which involve to use/review most of developed software, is it really the right choice to keep it all based on java?

I’m a java guru and fun using it as my main development language in last 10 years, but my answer is

NO

Why NO? Because if I take a look behind in the past I can see a lot of system architects answering “yes!” at same question 20 years ago substituting “Java” with “COBOL”. And a shudder come on my back…would I really sentence my system to be so strictly coupled with a single technology and loose flexibility and cool feature of newer technology? I’m not sure Java will become the next COBOL going to be static and legacy, but for sure, if I would answer yes I would be disown my ideas of “open system”.

There are so good languages and technologies kicking around, which probably solve better some kind of problem. Groovy, Scala and Ruby are the most famous, but we have also Erlang, Factor (with good ideas and a friend of mine behind), and even more legacy language like perl could have its place in some specific use cases. In general if something could be more productive or more flexible than java for some specific problem, I’d like to keep doors open. Randall did an interesting post saying java developers should learn other languages, I make a step over saying java developers should USE other languages

I’ve been always open to new technology and solution, would I miss my freedom of choice in favour of my beloved language? No, my freedom is much more important than java

Designing my new system I would use best technology and language for each part of the system. It’s always a good decision, the good news is integration of these parts could be seamless and painless, we haveSOA/ESB solution.

My conclusion is that isn’t necessary to have heterogeneous system to go for SOA, probably is the contrary: nowadays we need heterogeneous system to be time to market, to have easier maintenance, and so we need SOA to build and manage it.

SOA and heterogeneous technology environment seems to be the  eggs and chicken problem

Thoughts?

As said in this post one of possible use of wise-core (the new core we get independent from Wise) is to integrate it in JBossESB to make a generic soap client invoking web service using Smooks transformation to hide final user the gap between their own object models and one generated by JAX-WS tools dynamically.

Well I contribute with some code to JBossESB providing an action which does what I described in a nutshell here. My efforts and possible improvements are described in this post on ESB developer forum.

Give your feed back there.

BTW I’m developing a real world application based on ESB and this wise-action: it takes some date from a db, enrich the message calling a set of webservices using wise, conditioning these calls with a content based routing approach, and then write the databack into db. I’m planning a post about this application as soon as me and my team will finish it….stay tuned!

I had another interesting chat with Randall Hauch about JBossDNA next steps. Here is the log of our discussion. I put it here in agreement with Randall, because IRC chat sometime contain interesting ideas for all comunity memeber of JBossDNA, and it’s good it become public and with a longer time to leave than irc channel logs.

I haven’t time to extract a synthesis document, but in a nutshell we discussed about next steps of the project: federation engine and its connectors, graph SPI, new sequencers, views system.

Hoping it would be useful for someone, here you have it unabridged:

rhauch: I’ll get the graph API into SVN soon.
rhauch: Is there an area you want to work?
rhauch: Other sequencers? Federation?
maeste: Federation would be fun
maeste: and I’m thinking about sequencers useful for ESB/Overlord
rhauch: any experience with JBoss Cache?
maeste: in particulòar I’m looking at ESB code which write on JCR messages
maeste: just a little. Just ine project, not big
maeste: but I have an idea about it, not completely ignorant
maeste: concluding about ESB messages, I think a sequencer on it can be very useful, in particular if used togheter an UDDI sequencer
maeste: but as said Federation and connectors would be big fun for me
rhauch: i’m not sure i understand what would be sequenced
maeste: ESB track messages exchanged
maeste: and would be good to sequence message header
maeste: containing EPRs
maeste: of request and response
rhauch: those headers would be stored in JCR?
maeste: I think so. Am I wrong?
rhauch: oh, well they certainly could be stored in JCR.
rhauch: I just didn’t know how ESB would use JCR for that.
maeste: I think it would be used to query jcr to understand how many times messages pass on EPRs
rhauch: EPR = ?
maeste: and eventually we can sequence the route of message
maeste: EndPoint……I can’t recall the ‘R’
maeste: a moment…
rhauch: router?
maeste: no, it’s the ws-addressing EPR, the unique uidentifier of source/destination of a message
rhauch: ah.
maeste: R=Reference
rhauch: would it be useful to have a connector that publishes the message headers?
maeste: well ESB already have a JCR repository
rhauch: I guess I’m wondering how long those need to be stored? forever would mean a lot of data, right?
maeste: Yep
maeste: it’s the reason because ESB leave to the user the decision on how store on the repository
maeste: there is an action
maeste: that register a message on the repository
maeste: just body and attributes at the moment
maeste: not header
maeste: but also body (and mainly body) may be a lot of data
rhauch: would you rather I create JIRA tasks, and you assign them to yourself?
rhauch: or would you rather discuss what needs to be done and then you create the tasks for what you’d work on?
maeste: well it’s better to have a little discussion before
rhauch: where did you want to start, maeste?
maeste: well we can start from the roadmap
maeste: IOW what you have in mind for next release?
rhauch: Two areas: more sequencers and federation (at least read, probably not write)
maeste: oki
maeste: about sequencers i’d like to write some SOA related
rhauch: okay, that’d be great.
maeste: WSDLs, ESB Message….maybe UDDI
rhauch: johnny verhaeg works with me in STL, and he’ll be looking into a MetaMatrix sequencer (and maybe XSD)
maeste: I don’t think policy one yet, because we (in overlord) have to discuss about what’s kind of policy want to support
rhauch: Someone else on the discussion forums started on a Java sequencer, too. Not sure where he is.
rhauch: (where he is with the sequencer; I think he’s in germany)
rhauch: John is also looking into the view service, but really just some rough prototyping and investigation of some technologies.
rhauch: He won’t be spending too much time on that, tho.
maeste: about java seq…code or class?
rhauch: I believe source, although I’d love to have a bytecode sequencer!
maeste: about view service…another interesting area
rhauch: Both should really generate the same output structure.
maeste: I’d like to give my thought when it will be time
maeste: since I think some of these view could be used on overlord
rhauch: very cool.
rhauch: Oh, absolutely.
rhauch: I anticipate views for all kinds of data
maeste: and also because I’m used to works with web applications
maeste: already some technology in mind?
maeste: I suppose seam
rhauch: yes, seam
maeste: and….GWT? or JavaFX?
rhauch: not sure about view technology: jsf or Flex
maeste: ah…the two I not mentioned…;)
rhauch: well, GWT and JavaFX are options too.
maeste: Flex is indeed good technology too
rhauch: I’m concerned about the ability in GWT to dynamically generate views.
maeste: I have some more doubt about jsf
maeste: and I speak as leader of some big jsf projects
rhauch: We already tried that, and it worked, but it was a little painful.
rhauch: great feedback, then.
rhauch: i’ve never been a big fan of jsf.
maeste: yep right about dynamic generation of GWT
rhauch: i don’t know much about JavaFX
rhauch: Flex has an interesting XML-based view definition.
maeste: I’m studying it right now
rhauch: Hmm… data-driving views.
maeste: it looks very very promising
maeste: Yep I know about this Flex feature…I think something like that could be managed in JAvaFX, with a reflection like approach
rhauch: i need to read up on JavaFX
maeste: anyway it’s another story…we can discuss about views in next future on ML
rhauch: well, I’ll have to introduce you to John, maybe on the dna-dev@ list.
maeste: I need to understand better which kind of dynamic generation you have in mind
maeste: to give my thought
rhauch: well, think about this
rhauch: …
rhauch: in JCR there’s a structure representing a Java class
rhauch: so a node type for the class, with child nodes for the different members.
rhauch: and all these nodes have their own properties.
rhauch: The whole point of the view system is to build a domain-specific view using a subgraph.
rhauch: So the “Java class” view would know it needs to get the properties on the class node AND get the member child nodes and their properties.
rhauch: The resulting “message” (wrong term, really) would contain all the information necessary to build a view using a specific UI technology.
rhauch: The big thing is when navigating the repository
rhauch: and you come across a node that matches the Java class, that the “Java class” view is selected and used to generate the view.
maeste: ok, so I have to see in views a class as a class and not as row xml
maeste: and wsdls as wsdls and so on
maeste: great
rhauch: Right, and not a node with properties and links to child nodes.
rhauch: exactly
rhauch: Ideally, the views could be composed
maeste: of course
rhauch: so that if a node “looks like” a Java class AND a WSDL, then a single view would include both facets.
maeste: well it’s not a joke…doesn’t matter what technology we will choose…it still to don’t be a joke!
rhauch: But in the end, the user should understand what they’re looking at, because it’s expressed in their terms and in ways they understand.
maeste: ok, a grreat idea indeed
rhauch: Oh, and we should have views that show the views.
rhauch:
rhauch: Basically, we want to manage and configure DNA using DNA.
maeste: I can’t immagine an application implemented in jsf like this one which stay under control… jsf tend to don’t scale when you have an abstract approach
rhauch: Now can you see why I like the FLEX approach?
maeste: the easiest and natural way would be XML+XSL or XML driven Flex indeed…but let me investigate a little more about JAvaFX
rhauch: architecturally, it fits very nicely.
rhauch: that’d be great. again, I’ll introduce you to John over email.
maeste: oki
rhauch: so we talked about view system
rhauch: and a bit about sequencers.
maeste: coming back to Federation…
maeste: (BTW view system isn’t expected for next release, right?)
rhauch: no, views is not this next release.
rhauch: again, just investigating an approach so we know how to plan and how it might fit with anything we’re doing now.
maeste: yep of course…and it’s fun
rhauch: we talked a bit about federation a week or so ago
rhauch: and there’s some detail in the Getting Started doc.
maeste: yep I read and I recall our chat
rhauch: so any questions? areas that were vague?
maeste: the general idea is clear
maeste: my question is about
maeste: how do you think to tier this component
maeste: where stay cache
maeste: where the tree api
maeste: and so on
rhauch: ok
maeste: and another one
maeste: is
maeste: what do you think as container for DNA
maeste: JBoss Microcontainer?
rhauch: yes on MC
maeste: A standalone server or a deployable app?
rhauch: hmm…
maeste: And what about configuartion system?
maeste: And management system (MBean?)
rhauch: Ideally (from a user’s view) a standalone server, but that’s more work.
maeste: sure about that?
maeste: A deployable application is easier to understand
maeste: and easier to go live in user’s server farms
rhauch: well, easier for a developer. a bit more difficult from admins perspective.
rhauch: good points.
maeste: Not totally agree
maeste: I know some big farms with a lot of JBoss instance
rhauch: I’m still not convinced.
maeste: and for example some admins
maeste: prefers ESB as deployable
maeste: than standalone server
maeste: because JBossAS is already known by them
rhauch: no doubt. people have different preferences.
rhauch: I came from the MetaMatrix group
maeste: Anyway ideal approach
rhauch: and MetaMatrix is really like a database.
maeste: would be both
rhauch: and deploying a “database-like thing” into an app server really confused people.
maeste: as ESB group did for their product
rhauch: yes, ideal would be both.
maeste: yep u are right on this point
rhauch: in fact, the deployable would really be just a bundled JBoss server, right?
maeste: right
maeste: it’s just matter of packaging and let me say marketing
rhauch: sorry, I mean to say “stand alone server would really be just a deployable and a JBoss server”
maeste: I understood
rhauch: Right, and productization. (with my JBoss hat on)
maeste: LOL
rhauch: I think for the near term, we’ll actually be used inside other apps/projects/products.
maeste: or better with your RED HAT on
rhauch: my red fedora
maeste: lol
rhauch: we all have them, ya know
maeste: oki, so we start with deployable one..
maeste: my other questions:
maeste: config and management system
rhauch: oh yeah.
maeste: (not really coupled only with federation)
rhauch: and monitoring.
maeste: Let me say 0.1 is more an API than a server
maeste: I think next releases have to be PRODUCTIZED
maeste:
maeste: adding these features
rhauch: I think the best approach is to support MBeans, to allow for tools like JBoss ON to monitor and control the “system”
maeste: yes of course
rhauch: (although that’s strictly required for JBoss ON)
rhauch: I think there are two ways we can proceed with config/mgmt/monitoring
rhauch: One is the “traditional” way, with MBeans, Microcontainer config, and such.
rhauch: Another is to use the repository.
maeste: hmmm….I’m not sure to understand
rhauch: Let the repository define the configuration, and be the place we store monitoring results.
rhauch: To change the configuration, change the data in the “/dna:system” branch of the repository, and the system responds.
rhauch: For example, to configure a sequencer, add (or change) the data in “/dna:system/dna:sequencers/MySequencer”
maeste: The question here is….would the user be happy to have to learn a new approach?
rhauch: well, that’s where we’d have to adapt it to what they’re familiar with.
maeste: One more time the ideal approach is to use both and “gateway” the two
rhauch: Yes. Federation, anyone?
rhauch:
maeste: gotcha
rhauch: Seriously. I think that some of the “/dna:system” repository data could actually be federated from microcontainer config or from MBeans.
maeste: Federation read/write traditional config
rhauch: Yup.
maeste: And here we get back to views to admin DNA
rhauch: Exactly!
maeste: Great…I’m really impressed
rhauch: It’s just taking the “data-centric” approach to an extreme, I think.
maeste: And a lot of ideas crowd my mind about what we can do with this approach also for soa governance
rhauch: And, it’s possible that the projects like the Microcontainer could use DNA for managing their configuration.
rhauch: (once we advance enough)
maeste: yep
rhauch: I don’t want to push them this way, but if they see the benefits …
rhauch: like versioning of a configuration
maeste: I see the benefits for soa governance world for sure
maeste: and also for microcontainer
rhauch: Right now, I think we can basically use DI and IOC in our design
rhauch: and not have to pick a path.
maeste: yep
rhauch: I think we will want to use the Microcontainer inside the “DNA Service”
maeste: of course
rhauch: The MC is extensible enough that I think we can integrate the repository below it (sort of via a bootstrapping repository)
rhauch: in their configuration adapters.
maeste: my first question was about tiering of federation….MC is the base and then…?
rhauch: well, i think all the services and components will be managed by the MC.
rhauch: (and wired together)
maeste: (cool idea bootstrapping repository)
rhauch: I hope you could see that flavor in the example application code.
rhauch: I’ve been designing everything so that it can be configured in the MC.
rhauch: okay, so some of the other services/components.
maeste: yep I see
rhauch: One, we’ll have different connectors.
rhauch: the federation engine will delegate to the connectors to load a node,
rhauch: and the engine will merge the results for the same node from different connectors, placing all this in the cache
rhauch: Then, we’ll have a graph API that allows working with the stuff in the cache.
maeste: oki
rhauch: We can have a JCR implementation that uses this generic graph API.
maeste: merging done by JBossCache right?
rhauch: or a UDDI implementation.
rhauch: Merging the node data from different connectors is probably not something JBoss Cache can do, because we really need to keep the different “node fragments” separate
maeste: oki, JCR Implementation and UDDI implementation is better of course
rhauch: so we can manage them effectively.
maeste: oki understood
rhauch: But we’ll use JBoss Cache (should be pluggable, I think) to manage our cache of all the fragments.
maeste: any palns of certify these implementations (JCR and UDDI)?
rhauch: Basically, the graph implementation would work with the different fragments and do the merging.
rhauch: Yes, we’ll need to run the JCR TCK.
rhauch: Don’t know if there is a UDDI TCK. Do you?
maeste: No in fact
maeste: I think there some interoperability meeting
maeste: like ws-* ones
maeste: not totally sure
rhauch: the graph API also is much simpler than JCR
rhauch: basically, it includes versioning.
rhauch: All other JCR functionality (e.g., namespaces, node types, activities, etc.) would all be done by working with graph nodes.
rhauch: JCR 2.0 is closer to this model …
rhauch: moving closer to everything being accessible (and even editable) via JCR nodes.
maeste: oki… 2.0 is much better on these aspects
maeste: What about querying? Metamatrix engine?
rhauch: So I think if our graph API does versioning, then everything else can be done on top.
maeste: I agree
rhauch: (I think versioning will be important for any persistence layer, which is why I think it has to be in the graph API.)
rhauch: I’ve talked generically about the graph API being “above the cache”
rhauch: but really parts if it would also be used by the connectors.
rhauch: things like property values, names, etc. are the same at the connector level too.
rhauch: I have a strawman SPI that I’m about ready to check in.
rhauch: (We chose “SPI” because it fits for use at the bottom, but we also thought the “API”s would be JCR or UDDI or something else.)
rhauch: I was working on it late last night, and came up with something I think is pretty cool.
maeste: right
rhauch: basically, the connectors execute commands, where commands are things like “get node children” or “get node properties” or “create node” …
maeste: I’m anxious to have a look
rhauch: I’ll let you know when it’s in, because I’d love to get your feedback.
maeste: command…because command pattern is used?
rhauch: yes
maeste: so all command are rollbackable?
maeste: IOW transaction compensation?
rhauch: Well, that’s certainly my hope.
rhauch: I haven’t taken it that far yet, but theoretically using commands should make rollbacks (and compensated transactions) easier.
maeste: yep it was what I was thinking
rhauch: I’ve done that in other applications (rollbacks, not compensated txns)
rhauch: oh, regarding the JCR API
rhauch: depending upon how the JCR 2.0 spec shakes out
rhauch: (they’re trying to make it backward compatible)
rhauch: if it is not perfectly compatible, we should be able to have separate JCR 1.0 and JCR 2.0 implementations.
rhauch: I’d hate for us to have to do this, but it’s good to know it’s possible.
rhauch: i hope this all makes sense
maeste: all is very very interesting
maeste: so next steps?
maeste: you are checking the SPI in
rhauch: Well, the first is getting some SPI
rhauch: right.
rhauch: Then, we can proceed in a couple of directions.
maeste: and then you are going to create issues?
maeste: Or can we discuss now directions?
rhauch: 1) implementing the graph API on top of caching and on top of connectors.
rhauch: 2) implement some connectors
rhauch: 3) implement JCR API on top of graph SPI.
rhauch: (or at least part of the JCR API)
rhauch: I’d like to separate the caching from the federating logic.
maeste: I agree 100%
rhauch: We should be able to run without a cache (not good for performance, but caching shouldn’t really affect whether it works or not)
maeste: yes of course (not only for performance, also for cluster)
rhauch: so 1) above will probably break up into several bigger JIRA tasks.
rhauch: yup
maeste: yep
maeste: about 2) what you have in mind?
rhauch: well, probably a connector to another JCR repository. That should be straightforward.
maeste: yep
rhauch: Not sure we need others to get 1,2 & 3 working together.
rhauch: We’ll want others for 0.2 release, like file system and SCM (for ESB & Guvnor)
rhauch: file system being exposing the files as nt:files and nt:folder
maeste: yep
rhauch: That last one might be useful for testing.
maeste: well one also for UDDI would be fine
rhauch: I’d also like 0.2 to have a JBDC metadata connector, for exposing database schemas through the repository.
rhauch: That’s great demo stuff.
maeste: yep u r right
rhauch: (plus it would get us a long way in the MetaMatrix user community)
maeste: (I’m sure you are right…even if I don’t know anything about metamatrix )
rhauch: No problem.
rhauch: A few of the threads on the discussion forums had good use cases for federation,
rhauch: and interestingly they were all read-only.
maeste: yep very intersting
rhauch: So I think we can get buy with no solving the updates until 0.3.
maeste: yep, it simplify a lot of things
rhauch: Sure will.
rhauch: Anything I didn’t cover?
rhauch: Looking back at your questions from earlier …
maeste: I don’t think so. Point 3 is auto explaining
rhauch: Oh, we didn’t talk about caching policy and parameters.
maeste: right
rhauch: The doc covered this a bit, so not sure if you get the idea.
rhauch: I think the approach DNS uses will work really well.
rhauch: each fragment has a cache policy (with TTL, expiry time, etc.)
rhauch: and the federation engine respects those parameters.
rhauch: pretty simple.
rhauch: There was a dna-dev@ topic about that yesterday.
maeste: yep I saw
maeste: it makes sense for me
maeste: I know DNS protocol (not in deeper detail, but I have a general idea)
maeste: BTW I just have got an idea about a demo
rhauch: I think it will also work well if we federate multiple DNA federated repositories.
maeste: we can implement also a connector on apache access_log
rhauch: yeah? What’s the idea?
maeste: and sequence it to extract infos
rhauch: Oh, that’s good!
maeste: and do some statistic about
maeste: I’m sure it would be good
maeste: probably with a little work also for an application concurrent to aw-stat
rhauch: 0.2 should also have the event system, so the connector could publish events as the log is updated.
maeste: yep
maeste: and when we will have views………..
rhauch: ah, and great example of an analysis
maeste: aw-stats concurrent will be here
maeste: yep
rhauch: oh, that reminds me of one other thing
maeste: what about analyzer…03?
rhauch: about the graph API and sequencers/analysers
rhauch: I don’t know if you noticed this, but there is a StreamSequencer interface in the SPI.
rhauch: and a more generic Sequencer interface in the ‘dna-repository’ project.
maeste: yep I noticed
rhauch: Sequencer is much more powerful, but it uses the JCR node in its interface.
maeste: and adaptors to works together
rhauch: Right.
rhauch: So right now, StreamSequencer doesn’t depend on JCR, making it easier to implement and test.
maeste: yep I saw
rhauch: But other sequencers may want to do more advanced things.
rhauch: Okay, so here’s where the graph SPI comes in.
maeste: of course
rhauch: If the Sequencing framework is changed to work against the graph SPI
maeste: are yop thinking to Graèh API dependent sequencer?
rhauch: yes … your faster typist than me.
maeste: lol
rhauch: And analyses, too.
maeste: I made more typo
rhauch: LOL
rhauch: Anyway, they’d work regardless of what connectors are used and what API implementation is used.
maeste: they = StreamSequencer?
rhauch: Plus, since it’s more generic, analyzers and sequencers would work against nodes that represent built-in JCR things (like node types, namespaces, activities, etc.) without doing anything special.
rhauch: all sequencers.
maeste: oki
rhauch: StreamSequencers would still do what they do.
rhauch: but the StreamSequencerAdapter implementation would change.
maeste: yep it is what I have understood reading code
rhauch: At that point, the SPI for sequencers would become more powerful.
rhauch: It’s all about the graph.
maeste: and it is what I have in mind when I wrote you my complimets for very elegant code
rhauch: (ooh, some of my semantic days coming back!)
maeste: lol
rhauch:
rhauch: anything else to touch on, or should we call it for today?
maeste: oki, my last queston is about analysers: when do you plan to implement first?
rhauch: I dunno.
rhauch: I’ll be flexible
rhauch: They’re not too different than sequencers.
maeste: oki
rhauch: In my mind, federation is more important, so if we don’t have the bandwidth to do both, then federation should come first.
rhauch: however, if we have enough people to spread out, then we certainly can tackle them.
maeste: ok, that’s all…I’ll post the transcript of this chat on dna-dev
rhauch: My preference, tho, is to do shorter releases with more focused functionality.
maeste: and I will wait for spi checked in
rhauch: I’ll get that in today.
rhauch: Question:
maeste: and I’ll take a look to jira issues to take some one
maeste: shoot
rhauch: would you rather apply a patch, or do you think it is fine to check in?
maeste: Well I’m not sure to have understood you question about patch…
rhauch: Okay, this commit wouldn’t be invasive and wouldn’t break anything, so it may not matter in this case.
rhauch: But I’ve seen projects post a patch (to a JIRA) that the author wants feedback on
rhauch: people download and apply the patch, give their feedback, and the author changes then commits.
rhauch: Alternatively, I can just commit it and you can look at it that way.
maeste: oki… here comes my question about your svn policy….
maeste: all on trunk?
maeste: we can use branches for that
rhauch: well, that’s a good point.
maeste: I have a great experience of svn in quite large team….
maeste: svn could do a lot for us
rhauch: yeah, the larger the team, the more branches are the way to go.
maeste: hmmm..not totally agree
rhauch: so you’d prefer to do all major development on branches?
maeste: no absolutely not
maeste: not major development
maeste: but a specific development which need feedback yes
rhauch: ah, gotcha
maeste: well…may I post my ideas about svn on dna-dev
rhauch: That’d be a great discussion topic.
maeste: I can resume how I use svn in a lot of projects with plus and minus
rhauch: That’d be great.
maeste: I think with a good use of svn solve a lot of problems
rhauch: okay, so action items:
maeste: 2 post by me (svn and this chat)
rhauch: 1) I’ll check on commit privileges
maeste: yep
rhauch: 2) I’ll introduce you to John.
maeste: good
rhauch: 3) commit SPI (on trunk for now, or other?)
maeste: well if it isn’t invasive it’s good on trunk
maeste: or if you prefer create a branch with a good name
rhauch: 4) I’ll start putting in some JIRA tasks.
maeste: yep, and I’ll look at them
rhauch: I may do that, simply so that we can maybe try something else.
rhauch: something else if the SPI gets bad reviews.
maeste:

I had an interesting chat with Randall Hauch about JBossDNA and JBossOverlord. Here is the log of our discussion. I put it online (with Randall’s authorization of course) because I think it could be an interesting reading and a good introduction to JBossDNA as it have been for me. I have also summarized here some interesting point also for the report between JBossDNA and JBossOverlord. you will find in this chat.

Apr 25 16:08:27 rhauch: Well, 0.1 is really about sequencers. We’re using Maven 2 for builds, and we have a Maven 2 classloader that we’ve chosen to put on hold.

Apr 25 16:08:35 rhauch: Re the classloader (real quick)
Apr 25 16:08:45 maeste: oki
Apr 25 16:09:11 rhauch: Basically, since DNA is to be very extensible (i.e., sequencers, analyses, connectors, etc.)
Apr 25 16:09:49 rhauch: we thought it be good to have a runtime repository of JARs
Apr 25 16:09:59 rhauch: structured the same was as Maven2
Apr 25 16:10:15 maeste: ok I understand
Apr 25 16:10:57 rhauch: Works like a charm, but writing the tooling was lower priority.
Apr 25 16:11:10 rhauch: tooling for loading an “extension”, for example. Or upgrading one.
Apr 25 16:11:34 rhauch: So, the first release really has centered around sequencers.
Apr 25 16:12:14 rhauch: Basically, DNA monitors 1+ JCR repositories, and when a client uploads some content
Apr 25 16:12:37 rhauch: the system figures out whether any of the registered sequencers would be interested
Apr 25 16:12:46 rhauch: if there are any, it runs them.
Apr 25 16:13:03 maeste: oki
Apr 25 16:13:13 rhauch: A sequencer processes the content, extracts any relevant information, and then saves the information back into the repository.
Apr 25 16:13:41 rhauch: The idea is that you can just upload various kinds of files, and DNA automatically extracts meaningful structured information.
Apr 25 16:14:16 maeste: oki as metadata I suppose
Apr 25 16:14:23 rhauch: The simplest example of a sequencer now is an one that processes image files, extracting metadata (size, date, etc.)
Apr 25 16:14:28 rhauch: Right.
Apr 25 16:15:01 rhauch: There are a lot of really interesting ones, tho.
Apr 25 16:15:07 maeste: Great….the killer apps in a soa governace point of view
Apr 25 16:15:13 maeste: and not only IMHO
Apr 25 16:15:16 rhauch: WSDL, XSD, …
Apr 25 16:15:23 rhauch: Exactly.
Apr 25 16:15:25 maeste: exactly
Apr 25 16:15:27 maeste: lol
Apr 25 16:15:39 maeste: policy ws-security and so on
Apr 25 16:15:42 rhauch: ha … great minds …
Apr 25 16:16:09 rhauch: MetaMatrix is interested in this because they have a lot of models of data sources
Apr 25 16:16:37 rhauch: and they’re tooling converts those models into info they use at runtime to integrate data.
Apr 25 16:16:52 maeste: ok
Apr 25 16:16:52 rhauch: But people want to know what models they have, and what’s in them.
Apr 25 16:17:06 rhauch: Same story for SOA – what services do I have, and what are the details?
Apr 25 16:17:19 maeste: About SOA
Apr 25 16:17:21 rhauch: What are the policies? Where are the services used in an orchestration, etc.
Apr 25 16:17:32 maeste: that is my main interest
Apr 25 16:17:37 rhauch: This is your area of interest?
Apr 25 16:17:40 rhauch: Ha!
Apr 25 16:18:10 maeste: yep I arrived to dna starting from soa governance project overlord
Apr 25 16:18:29 maeste: and discussig these points with Mark Little
Apr 25 16:18:38 rhauch: So, there are a lot of possibilities for different sequencers.
Apr 25 16:18:55 maeste: about policies: in few words sla definitions
Apr 25 16:19:03 rhauch: Right. He’s one of our main “users”.
Apr 25 16:19:28 rhauch: Sure, but also security policies… whatever is needed.
Apr 25 16:19:41 maeste: well for sure there are a lot of possibilities for sequencers
Apr 25 16:19:54 rhauch: The idea is that we can automatically extract information into JCR, and then use analyses and other tools to tie all that information together.
Apr 25 16:20:33 rhauch: Okay … the 0.1 release.
Apr 25 16:20:44 maeste: oki and how do you extract infos?
Apr 25 16:20:50 rhauch: Most of the coding is complete, and we just need to put together some documentation.
Apr 25 16:20:51 maeste: jcr api?
Apr 25 16:21:04 maeste: about 0.1 ok
Apr 25 16:21:14 rhauch: Do you mean, how do the sequencers extract the information?
Apr 25 16:21:31 rhauch: (No, we can discuss whatever you’re interested in…)
Apr 25 16:22:05 maeste: no, how clients can extract infos? is them put back in original jcr?
Apr 25 16:22:44 maeste: and how dna helps in query them?
Apr 25 16:22:53 rhauch: Yeah, the information that sequencers extract is put back into the JCR repository (perhaps a different one).
Apr 25 16:23:13 rhauch: At that point, clients can just use JCR … search, navigation, editing, etc.
Apr 25 16:23:19 rhauch: That’s where we are now.
Apr 25 16:23:34 rhauch: “Analyses” are another component of DNA …
Apr 25 16:23:38 maeste: oki, any plan for a query language different than xpath?
Apr 25 16:24:01 rhauch: Well, I see JCR 2 is moving away from XQuery and XPath, and more towards SQL.
Apr 25 16:24:07 rhauch: I’m not sure how I feel about that.
Apr 25 16:24:19 maeste: me too
Apr 25 16:24:31 rhauch: They’re really working toward a more generic abstract query language,
Apr 25 16:24:44 rhauch: and I think they’d like to have multiple query languages on top.
Apr 25 16:24:57 rhauch: But that different JCR implementations (products) would offer these as features.
Apr 25 16:24:57 maeste: oki like sparql have been for rdf
Apr 25 16:25:04 rhauch: Sure.
Apr 25 16:25:20 rhauch: In fact, you could probably implement SPARQL.
Apr 25 16:25:36 maeste: I agree
Apr 25 16:25:37 rhauch: As long as you can generate a query in their abstract query language (AQL)
Apr 25 16:26:34 maeste: sorry for a lot of interrupt and question, please go on with your explaination of the future of the project…you are saying about analysis
Apr 25 16:26:49 rhauch: No, don’t apologize.
Apr 25 16:27:01 rhauch: One more thing …
Apr 25 16:27:08 rhauch: Someone else on the forums has started working on a Java sequencer
Apr 25 16:27:26 maeste: I dind’t see
Apr 25 16:27:30 maeste: user forum?
Apr 25 16:27:31 rhauch: and another (from MetaMatrix) has just started working on a MetaMatrix model sequencer.
Apr 25 16:28:23 rhauch: Yeah, its listed on our project page (http://www.jboss.org/dna/)
Apr 25 16:28:40 rhauch: Here’s the forum: http://www.jboss.com/index.html?module=bb&op=viewforum&f=272
Apr 25 16:29:06 rhauch: Nobody has uploaded any code yet.
Apr 25 16:29:31 rhauch: So, maybe this is a good segway back to the 0.1 release.
Apr 25 16:29:43 maeste: I known forum, just I didn’t see the message discussing these stuffs
Apr 25 16:30:31 rhauch: See this thread: http://www.jboss.com/index.html?module=bb&op=viewtopic&t=132494
Apr 25 16:31:24 rhauch: Yeah, i know. we don’t have much documentation.
Apr 25 16:31:31 rhauch:
Apr 25 16:31:55 rhauch: Anyway, I hope to release 0.1 next week.
Apr 25 16:32:10 rhauch: Most of the code is done, so we’re just working on some documentation.
Apr 25 16:32:30 rhauch: We’re going for frequent releases,
Apr 25 16:32:56 rhauch: and this initial release will get something stable that people can play with … primarily the sequencing framework.
Apr 25 16:33:36 rhauch: Then, we’ll start with the federation engine. I think that’ll be getting to some of the fun stuff.
Apr 25 16:34:14 maeste: As said in my mail to ML I’d like the idea to contribute in federation engine
Apr 25 16:37:30 rhauch: The federation engine is really going to be the interesting part of DNA.
Apr 25 16:37:41 rhauch: With it, we’ll be able to do a lot of very interesting things.
Apr 25 16:37:49 maeste: like?
Apr 25 16:38:00 rhauch: Well, remember the Maven 2 classloader?
Apr 25 16:38:04 maeste: yep
Apr 25 16:38:15 rhauch: The problem was getting stuff uploaded, right?
Apr 25 16:38:31 maeste: yep
Apr 25 16:38:42 rhauch: Well, we can have a connector that connects to an existing Maven 2 repository (via http), and presents it as if it’s already in a JCR repository.
Apr 25 16:39:07 rhauch: Then, the tooling can just copy from one JCR into another.
Apr 25 16:39:11 rhauch: That’s an example.
Apr 25 16:39:20 rhauch: Here’s another …
Apr 25 16:39:46 rhauch: A connector can talk to a relational database, and present the database schema information as if it’s already in JCR.
Apr 25 16:40:15 rhauch: Now I have a “federated JCR repository” that shows my database schemas live…. no copying, and it’s always up to date.
Apr 25 16:40:18 rhauch: Here’s another ….
Apr 25 16:40:35 rhauch: A connector can talk to a UDDI repository, and present it as if its in JCR.
Apr 25 16:40:58 rhauch: So now I can have a federated repository that lets me access my UDDI registry, but through JCR.
Apr 25 16:41:13 rhauch: And my policy information is also in the same federated JCR repository.
Apr 25 16:41:14 maeste: present a lot of information “as if it’s in JCR” this is the key
Apr 25 16:41:25 rhauch: That’s what the federation engine does …
Apr 25 16:41:31 maeste: exactly
Apr 25 16:41:51 rhauch: 1) connectors talk to some external system and build up subgraphs (as they’re queried/navigated)
Apr 25 16:42:06 rhauch: 2) integrate those different subgraphs into a unified graph, merging where required.
Apr 25 16:42:35 maeste: and then sequencer can do their job on this unified graph
Apr 25 16:42:41 rhauch: Bingo!
Apr 25 16:42:57 maeste: Great!
Apr 25 16:43:07 rhauch: Connector should be able to generate events to the federation engine (for example, new node at A/B/C/D)
Apr 25 16:43:27 rhauch: Now, here’s another connector that Mark’s team is interested in…
Apr 25 16:44:06 rhauch: A connector that sits on top of an SCM system (e.g, SVN, CVS, etc.), which is where the developers store their rules, policies and other artifacts.
Apr 25 16:44:36 rhauch: That connector should make those available to any tooling (e.g., Guvn’r/BRMS) that sits on top of JCR.
Apr 25 16:44:54 maeste: yes of course
Apr 25 16:45:14 rhauch: Data should be stored (as master data) only in one spot, and the federation engine lets this happen.
Apr 25 16:45:20 rhauch: Leave the data in SVN if that’s the master data.
Apr 25 16:45:28 rhauch: Or in the database (for the schema).
Apr 25 16:45:29 maeste: and maybe also a connector collecting runtime infos of soa environments
Apr 25 16:45:40 rhauch: Yes, exactly.
Apr 25 16:45:56 maeste: to collect status and how slas are respected or not
Apr 25 16:46:12 rhauch: Right. That’s a perfect example.
Apr 25 16:46:43 maeste: well you were telling me about analyzer
Apr 25 16:46:49 rhauch: Right ….
Apr 25 16:47:14 rhauch: analyzer are similar to sequencers in the sense that they’re automatically run based upon some criteria being met.
Apr 25 16:47:26 maeste: oki
Apr 25 16:47:35 rhauch: In the case of an analyzer, this criteria might be a scheduler,
Apr 25 16:47:45 rhauch: or it might watch for changes under certain paths.
Apr 25 16:48:07 rhauch: Unlike sequencers which really target a node’s content,
Apr 25 16:48:25 maeste: oki
Apr 25 16:48:37 rhauch: analyzers really process subgraphs as input, and produce output in the form of a subgraph
Apr 25 16:48:44 maeste: so it’s supposed to make some kind of massive job
Apr 25 16:48:46 rhauch: Essentially, they’re report generators.
Apr 25 16:49:17 maeste: oki and they don’t change jcr content, right?
Apr 25 16:49:26 rhauch: Yeah, the idea is that they could be set up to find statistics… about adhering to sla’s for example.
Apr 25 16:49:38 maeste: oki
Apr 25 16:49:46 rhauch: and then write those statistics to an area where they can be found.
Apr 25 16:50:13 maeste: I have a clear ides on how they can be useful in overlord
Apr 25 16:50:23 rhauch: So, a web tool (lets say) can look at that report in JCR. It’s always there, and the analyzers make sure it’s kept up to date.
Apr 25 16:50:25 maeste: exactly for sla reports
Apr 25 16:50:56 maeste: lol I’m thinking the same thing a web tools called overlord
Apr 25 16:51:10 rhauch: Basically, analyzers are big queries (or sets of queries) that take a while to run.
Apr 25 16:51:24 rhauch: So can you explain what overlord is?
Apr 25 16:51:39 rhauch: I’ve heard bits and pieces.
Apr 25 16:51:41 maeste: It’s not totally clear
Apr 25 16:51:59 maeste: it’s the project created to make a soa governace project
Apr 25 16:52:14 maeste: and it would include 2 kinds of tools
Apr 25 16:52:19 maeste: design time tools
Apr 25 16:52:39 rhauch: like the service modeler that Mark pointed to in the forums?
Apr 25 16:52:46 maeste: service modeler for instance based on Thomas Erl’s donation
Apr 25 16:52:53 maeste: lol
Apr 25 16:53:01 rhauch: … great minds …
Apr 25 16:53:14 maeste: and the other kinds it’s runtime governance tools
Apr 25 16:53:36 maeste: that would be some kind of (web) console
Apr 25 16:54:08 maeste: to collect infos and statistic of the soa environments
Apr 25 16:54:17 rhauch: ah!
Apr 25 16:54:23 maeste: slas, error, number of class and so on
Apr 25 16:54:40 maeste: and it would be able to change also
Apr 25 16:54:55 maeste: some paramiter runtime
Apr 25 16:55:06 maeste: to have policies enforcements and so on
Apr 25 16:55:30 maeste: in fact a lot of these things need also a developments under the hood
Apr 25 16:56:07 maeste: since for example there is no standard to define slas in ws-* or oasis panorama
Apr 25 16:56:15 maeste: but of course we need one
Apr 25 16:57:23 maeste: and it would contain also a BAM or as mark said better a SAM (service Activity Manager)
Apr 25 16:57:38 maeste: and of course a ws-cdl runtime tool too
Apr 25 16:57:58 maeste: and of course a lot of other stuffs we didn’t thought (yet)
Apr 25 16:58:28 rhauch: Right, and you’re thinking a repository for the artifacts, plus a place (repository?) to store the collected data?
Apr 25 16:58:32 maeste: As you can figure out there are a lot of point where we need a good repository
Apr 25 16:58:49 maeste: lol and rotfl
Apr 25 16:58:56 rhauch:
Apr 25 16:59:03 maeste: yep we are thinking to dna in fact
Apr 25 16:59:39 maeste: because a jcr repository isn’t sufficient, we need some sequencer and analyzer for sure
Apr 25 17:00:02 rhauch: Yeah, the JCR API is fairly low level — it’s basically just a graph API.
Apr 25 17:00:40 maeste: As said overlord is just started, and we (me and Mark) think the best starting point for my contribution is to be involved in dna
Apr 25 17:00:43 rhauch: The functionality is where the good stuff is.
Apr 25 17:00:52 maeste: to have a full understand of it
Apr 25 17:00:59 maeste: at least
Apr 25 17:01:10 maeste: BTW dna it’s big fun
Apr 25 17:01:23 rhauch: i hope that’s not sarcasm.
Apr 25 17:01:35 maeste: I had already a copy of trunk on my laptop before Mark told me about
Apr 25 17:01:44 maeste: Absolutely not
Apr 25 17:02:06 rhauch: Well, we’re glad your interested!
Apr 25 17:02:11 maeste: are there sarcasm in my phrase? If yes, I’m sorry for my english
Apr 25 17:02:52 rhauch: No, just my bad sense of humor not coming through…
Apr 25 17:02:57 rhauch: You’re english is great.
Apr 25 17:03:48 rhauch: There’s another area of DNA that is a bit further out on the roadmap … RESTful server.
Apr 25 17:04:14 rhauch: I thought I’d mention it in case that was much more up your alley.
Apr 25 17:04:46 rhauch: But if you’re interested in the federation engine, then there’s certainly a lot of work there!
Apr 25 17:05:40 maeste: Yep I’m interested in federation engine for sure
Apr 25 17:16:57 rhauch: The federation engine will have a couple of major components.
Apr 25 17:17:51 rhauch: At its core is a graph, stored in a cache and populated from the connectors.
Apr 25 17:18:05 rhauch: On top of that the fed engine will implement JCR API.
Apr 25 17:18:11 maeste: cache…JBoss cache? Pojo cache?
Apr 25 17:18:32 rhauch: Well, I’d like to not tie it to any particular cache, but I think we’d use JBoss cache.
Apr 25 17:18:48 rhauch: Pojo cache seems a bit heavy for this purpose.
Apr 25 17:19:12 maeste: yes of course not tie to a particular implementation is always good
Apr 25 17:19:14 rhauch: It is more efficient in a distributed sense, but it requires a lot of AOP weaving.
Apr 25 17:20:23 rhauch: anyway, so there are a couple of different areas.
Apr 25 17:20:57 maeste: yep
Apr 25 17:21:02 rhauch: As far as the JCR API implementation, we’ll want to set up the JCR TCK relatively early.
Apr 25 17:21:37 rhauch: I think we’ll have the mid-level graph API done pretty quickly, so we’ll definitely want your input there.
Apr 25 17:22:07 rhauch: BTW, my thought on that graph API is that the federation engine & cache should basically be a simple graph with versioning.
Apr 25 17:22:43 rhauch: If we do that, then we can put really almost any API on top of that, including UDDI (should we ever want to do that).
Apr 25 17:23:11 rhauch: If you look at the JCR API, there’s a lot of stuff that should be implemented on top of a more simple graph…
Apr 25 17:23:38 rhauch: node types, namespaces, version history, etc.
Apr 25 17:23:56 rhauch: They all need a simple graph that supports versioning.
Apr 25 17:24:12 rhauch: Anyway, that’s the middle of the federation engine.
Apr 25 17:24:16 rhauch: Then there are the connectors.
Apr 25 17:24:59 rhauch: I think we have JIRA feature requests for several different kinds of connectors, with a few initially slotted earlier than the others.
Apr 25 17:25:15 rhauch: I hope I’m making at least some sense.
Apr 25 17:25:21 maeste: ok, I understand