This article is the first in a long series of sessions about realising a working platform for Service Oriented Architecture - using Java and open source technologies. The motivation for this series came in three separate parts:

• Myths of SOA 
• Open source SOA 
• The next step in 'my own path'

I'll describe the above in detail first, and then get to the actual article.

Myths of SOA

I don't know if you have read Julian Browne's blog - if you haven't yet, believe me, you are missing something. Julian Browne is an Enterprise Architect with about 20+ years experience in designing applications. I came across his site about a year or so back when I was doing some analysis about Gigaspaces. His was one of those rare sites that I read through page-to-page almost in one sitting. He knows what he is talking about, he has a superb conversational style of writing, and most importantly, his articles speak to something deep within you. His articles filled me with a sense of melancholy though - mainly because his sense of pragmatism is [unfortunately] something really rare in a world filled with techno-babble and consultant-speak. 

He had recently written an article called 'Myths of SOA' - listing down some common mis-conceptions about service oriented architecture. I have been facing many of the myths he has listed down in real life - from 'everything is a service' to 'vendor-spin'. I couldn’t agree more with his conclusion - 'the more common sense that can be brought to bear on this right now the better'. As I read through the article, I started thinking about building a service oriented platform that was essentially free from these myths - this contributed to the first part of my motivation.

Open-Source SOA

The second part of my motivation came from the excellent book called 'Open Source SOA' from Manning Publications. This book describes how to implement an open source SOA platform exclusively with open source products like Tuscany, Synapse, jBoss jBPM, jBoss Drools and Esper. 

I have always been a fan of the open-source movement. Right from Linux to Spring and Hibernate, I personally believe that the open-source movement has benefited the developer community - and businesses using them - tremendously. I definitely don't buy in to the idea that you need to pay tremendously large sums of money to get an enterprise-grade platform for running your applications. But while the book focused on the major products of the stack and how to integrate them with the other selected products, there were certain aspects of the platform that were not addressed.

Having implemented a platform for service oriented architecture fairly recently, as I was going through the book, a lot of questions started cropping up in my mind. I started wondering that if I were to design and integrate a platform for service oriented architecture using pure open source technologies, what would it look like? I could immediately think of the decisions from the book that I wouldn't agree with, and those decisions I’d taken before which could have been done better.

The next step in 'my own path'

As mentioned above, I had recently put together a blueprint for a platform for service oriented architecture. Typically by the time you finish your work on a project, you would have come across a whole lot of scenarios that would have made you implement solutions along the way using already-selected products. There could be cases where you weren’t aware of a certain product or a certain product’s capabilities. There could also be cases where you took certain decisions due to a few constraints that were applicable for that context. These situations leave you wondering whether you would have built things differently if you’d been aware of these situations beforehand. Changing any of these decisions can lead to a different solution - probably something that is 'better integrated and more complete'.

In that sense, what I am doing here is the next step in my own path. I asked myself - given all of the above, if I was working on my own, and needed a SOA platform, how would I design it? How would I handle things like deployment, message routing, scalability, etc.? 

The article set you are seeing before you is the result.

I'm starting off on a journey to build a complete, working platform for service oriented enterprise application development. As of now, I don’t have a picture of the platform already in my mind - I only have a few goals for the platform. I intend to take up each of these goals [read 'aspects'] one by one, design a working solution for each using existing open source technologies, and integrate them together into a working 'stack'. I’ll be documenting the steps along the way - so that you will see the platform grow and take shape before you. As I complete each piece, I’ll be deploying a version of the platform to a section within this site so that you will be able to interact with the deployed instance directly. The complete source and binaries of the built stack will be available for download and use in your own projects as well.

Goals

Given below are my goals for the exercise:

1. Implementing services shouldn’t be painful [this is fairly self explanatory, so I’m not going to add more detail here]
2. Developers who are implementing services should be focussed on business logic, and not on infrastructure code
   
   As a principle, this was first explicitly stated as part of J2EE. Since then, this principle has also become a guiding rule for a lot of frameworks available in the open-source world. But if you take a deeper look at existing open source components for realising services, they do leave a few gaps which can be classified as 'infrastructure code'. One of the goals for this exercise is to fill these gaps wherever identified.
   
   
3. Address the disconnect between established 'standards' and situations faced in real-life application development where appropriate
   
   Don’t get me wrong, I do believe standards are good and should be followed where possible - what I do have an issue with are those standards that don’t adequately address real-life situations.
   
   Take for example, security and access control. The very same J2EE that established the second principle above falls amazingly short when it comes to security in real-life application scenarios. I’m not sure about the context you work in, but in all situations I have seen, an application wants the capability to define roles on-the-fly by specific users, without re-deploying the application or editing source/configuration files. The J2EE standard, on the other hand, defines roles and access control as being defined within the EAR file - which in-effect means a re-deployment for changing these definitions. Why?
   
   I can give you many more such examples - but I suppose you get the picture.
   
   
4. Evaluations and recommendations
   
   The first activity that happens during most of the architecture, definition phases of a project is to evaluate existing technology components and select the appropriate ones for use within that project. While the criteria used for the evaluation can vary between projects, there are many similarities between them as well. One of my goals is going to be to detail out evaluations for the technologies I end up selecting - in an attempt to shorten this cycle for your projects - obviously if the criteria I use matches your context.
   
   
5. Architects, administrators, and developers all need a higher level of abstraction when developing and/or deploying services
   
   This is a little ambitious since this can be achieved only after the above principles are put in place, platform components and processes are established and we have a working, integrated platform. Knowing fully well that this is ambitious, this is definitely going to stay in this list, we’ll look at how much of it was realised at the end of this exercise.
   
   

Aspects addressed

Listed below are the aspects of the solution that will be addressed - in order. I am not going to be describing these here, each of these aspects will be detailed out when I focus on that particular topic.

• Implementing services
• Service composition, dependencies and deployment flexibility
• Configuring services
• Testing services
• Transaction Management
• Business processes
• Deploying services
• Message routing, addressing, multiple environment support and the service registry
• Service request versioning
• Load balancing and failover

In some cases I have combined 2-3 sections together - these are intentional because I believe a solution can be implemented that addresses all the grouped aspects together. I may restructure the above grouping as I go forward, though.

Process

The way I intend to be going through this exercise is as follows:

1. Describe, detail out the reference implementation
   
   I’ve always felt that a framework/platform built with a specific project in mind is more complete, and addresses real-life scenarios better. We have seen this with the Java pet-store - an implementation of the pet-store more or less became a standard by itself for many open-source frameworks. With this goal in mind, I will be setting down the specifications for the reference implementation first and then work towards the solution. The reference implementation can then be used for testing the platform/integration code that we will be building as well.
   
   But for our reference implementation, I don’t intend to be using pet-store itself - I’m instead going to be describing something that we all can immediately relate to - a project/release/issue management system similar to JIRA, Bugzilla, Mantis, etc. We’ll be treating this both as a real-life project, and as a representation of any project that you might currently be working on.
   
   Though I’ll be detailing out what the user-interface for the system is going to look like as part of the specifications for the system, we will [intentionally] not be focussing on the user-interface during the implementation phase of our exercise. I’ll be coming to the user-interface implementation last, but rest assured I definitely have something in mind for the presentation tier.
   
   
2. Describe the development methodology, artefacts and tools that can be used for implementing the product(s)
   
   Once the specifications are detailed out, the next step would be to put in place some form of a development lifecycle. I’ll be describing this next - selecting tools/technologies, the development setup, etc - in effect what can be extrapolated to become the SDLC for the project. As said earlier, some of these selected tools/technologies will be deployed right here on the site so that you get to see the system working, live.
   
   
3. Develop and describe the layers of the architecture and the responsibilities of each layer in a completely product-agnostic manner
   
   This is where we’ll start describing the heart of the system. I’ll start with a few basic boxes, describe their responsibilities, and then begin increasing the levels of detail for each of the parts of the system. At the end of this phase, what we will have will be the final architecture of the solution, and the layers in the architecture, what we will not have will be the products we will use to realise each of these components.
   
   
4. For each of the layers, shortlist, evaluate and select a technology product that best fits the needs of that layer
   
   The obvious next step will be to evaluate and select a product for each of the layers described above with the reasons behind the decision, and in parallel, start implementing the reference implementation piece-by-piece. For example, once we have something for the OXM layer, we will have the object-XML mapping implemented in the reference implementation, once we select our persistence mechanism, we will get this layer implemented and so on.
   
   
5. Integrate the selected product into the existing stack, building the needed parts of the integration code
   
   This is probably part of the above step itself - as and when we finish selecting a new piece, we might need to build some ‘glue’ code that integrates the new piece with the existing parts. Again, as said earlier, as and when we finish this integration, you will have three things - 1. An article describing the design and implementation of the glue code, 2. A downloadable version that you can download and start ‘playing with’, and 3. A ‘live version’ that you can directly interact with here.
   

I originally had a section titled disclaimers that was supposed to be inserted here. But coincidentally, when going through some books on Zen today, I came across the following paragraph which I thought summarised what I wanted to convey in a much much concise manner. So here is it:

'There is a zen saying - 'Paths cannot be taught, they can only be taken'. So it is with this exercise. My function here will be to act as a guide. I do so in the spirit of of 'one who has been there before'. I do not claim to know all the answers. But what I have learned, I'll gladly share with those who wish to make a similar journey. If I can inspire you to follow your path, this series of articles would have served its purpose.' [Re-phrased from 'Zen Guitar']

So, without much further ado, let’s get started.