Service Oriented Architecture - SOA

In anticipation of a workshop/discussion meeting we are holding on September 24th, I have produced a short position paper, “Suggested Principles for Structuring Standards” (PDF 580kB), to outline the approach I speculate we should take in future in creating specifications and standards for learning education and training.

Responses, alternative views etc are most welcome before, during or after the meeting. I hope other attendees contribute papers and expect to synthesise conclusions of the meeting shortly afterwards.

#entarch @BBCRadio4 Just listening to a business story on the radio: the music retailer HMV is now going to be selling clothes from its flagship Oxford Street store. Of course this kind of thing is nothing new – retailers have always tried to diversify, and there are clothes shops in Oxford Street that are trying to sell CDs, while large retail chains that started as grocery stores now sell books, clothing, computers, mobile phones and furniture, as well as financial services.

For many enterprise architects, this kind of diversification strategy seems like a “no-brainer”. If you have a robust retail process, together with the assets and infrastructure to support the process, then surely it makes sense to make as much use of the process as you can.

But successful diversification is by no means a “no-brainer” (how I despise that term and its implications), but requires careful consideration and planning. This kind of planning should be meat and drink for enterprise architects, but not if they imagine that the decision follows automatically from some abstract process model.

So what models and techniques would be relevant to support an intelligent diversification strategy? And how comfortable are enterprise architects with using these models and techniques to support business decision-making?

@sbskmi gave an excellent presentation at the RESG AGM yesterday evening, offering a survey of sensemaking tools.

Simon’s research group at the Open University has a tool called Compendium, which belongs to a long and respectable line of issue and argument mapping tools, going back to IBIS (Horst Rittel, 1972). Simon showed a range of recent initiatives that demonstrate that issue and argument mapping has, as Simon puts it, “come of age”.

Compendium itself has been designed to allow logical arguments to be supported by rich media evidence, such as video. So we can manipulate conflicting knowledge claims, together with the ethnographic material that might be relevant to resolving these claims.

The particular interest of tools like Compendium (and its Web 2.0 cousin Cohere) to the Requirements Engineering community is the desire to document design rationale, and these tools can certainly be used for this purpose. But they can equally be used to support real-time control systems, and Simon showed an emergency response coordination scenario, with web service links to various information feeds, as well as the ability to produce mashups of various kinds.

However the aspect of Simon’s work that interested me most was the potential link to organizational intelligence. For example, he displayed a model of creative competences for complex challenges, based on the work of Palus and Horth, and showed how Compendium could use visual images to support the Palus and Horth methodology. (There are some parallels with the Repertory Grid technique, which I introduced into data modelling in the 1980s.) Simon also showed how Compendium could be used to compose machine intelligence with human intelligence. This helps to realise the original vision of Takehito Matsuda, who twenty years ago defined organizational intelligence as “the interactive-aggregative complex of human intelligence and artificial intelligence in an organization”.

As Brenda Dervin argues, sense-making is triggered by anomalies and exceptions. The point about an exception is that it forces us to review and revise our pet models and theories, or at least it should do so. As Lakatos pointed out however, in his brilliant essay Proofs and Refutations, people typically deploy various tactics for dismissing exceptions in order to preserve their favourite models and theories, including Monster-Barring and Monster-Adjustment. (See my piece on Models and Monsters.)

To understand an argument or rationale, we need to pay attention not just to the domain (subject matter) of the argument but also the discourse or discursive practice. Within a large organization, there are several competing discourses, and the intelligence of the whole organization depends critically on a healthy interchange between different discourses. For example, arguments based on conventional accounting practice may bias the organization towards certain ways of solving problems, and make some kinds of innovation impossible; so sometimes management needs to be able to step away from the accounting paradigm and look at other kinds of rationale for organizational change. It would be interesting to see if models and software tools could support a conversation that straddled multiple discourses.

But in any case, these tools for collective sense-making and decision-making should fit nicely into an overall architecture for organizational intelligence.

@sbskmi gave an excellent presentation at the RESG AGM yesterday evening, offering a survey of sensemaking tools.

Simon’s research group at the Open University has a tool called Compendium, which belongs to a long and respectable line of issue and argument mapping tools, going back to IBIS (Horst Rittel, 1972). Simon showed a range of recent initiatives that demonstrate that issue and argument mapping has, as Simon puts it, “come of age”.

Compendium itself has been designed to allow logical arguments to be supported by rich media evidence, such as video. So we can manipulate conflicting knowledge claims, together with the ethnographic material that might be relevant to resolving these claims.

The particular interest of tools like Compendium (and its Web 2.0 cousin Cohere) to the Requirements Engineering community is the desire to document design rationale, and these tools can certainly be used for this purpose. But they can equally be used to support real-time control systems, and Simon showed an emergency response coordination scenario, with web service links to various information feeds, as well as the ability to produce mashups of various kinds.

However the aspect of Simon’s work that interested me most was the potential link to organizational intelligence. For example, he displayed a model of creative competences for complex challenges, based on the work of Palus and Horth, and showed how Compendium could use visual images to support the Palus and Horth methodology. (There are some parallels with the Repertory Grid technique, which I introduced into data modelling in the 1980s.) Simon also showed how Compendium could be used to compose machine intelligence with human intelligence. This helps to realise the original vision of Takehito Matsuda, who twenty years ago defined organizational intelligence as “the interactive-aggregative complex of human intelligence and artificial intelligence in an organization”.

As Brenda Dervin argues, sense-making is triggered by anomalies and exceptions. The point about an exception is that it forces us to review and revise our pet models and theories, or at least it should do so. As Lakatos pointed out however, in his brilliant essay Proofs and Refutations, people typically deploy various tactics for dismissing exceptions in order to preserve their favourite models and theories, including Monster-Barring and Monster-Adjustment. (See my piece on Models and Monsters.)

To understand an argument or rationale, we need to pay attention not just to the domain (subject matter) of the argument but also the discourse or discursive practice. Within a large organization, there are several competing discourses, and the intelligence of the whole organization depends critically on a healthy interchange between different discourses. For example, arguments based on conventional accounting practice may bias the organization towards certain ways of solving problems, and make some kinds of innovation impossible; so sometimes management needs to be able to step away from the accounting paradigm and look at other kinds of rationale for organizational change. It would be interesting to see if models and software tools could support a conversation that straddled multiple discourses.

But in any case, these tools for collective sense-making and decision-making should fit nicely into an overall architecture for organizational intelligence.

Brown-Field Requirements

One view of requirements engineering is that its purpose is to produce a complete and coherent statement of what some system-of-systems is required to do – in other words, its behaviour in the broadest sense, including “functional”, “non-functional” and “commercial” requirements.

In a “Green-Field” scenario, we might imagine that this statement of requirements would result in the procurement and installation of a system of systems meeting the stated requirement. Requirements engineering is focused on understanding exactly what is required, and specifying it in an unambiguous and testable form.

But in almost all engineering projects, some system of systems – technical or sociotechnical – already exists, and the practical purpose is to make some planned changes to it. So people in the RE community are starting to talk more about “Brown-Field” requirements.

(RESG Event: Managing Brownfield Project Requirements, London, October 12th 2010)

Gap Analysis

An obvious starting point for brownfield requirements analysis would seem to be the identification of a gap between desire and reality. People often produce two models – an AS-IS model that describes the existing system, and a TO-BE model that describes its future replacement. The engineering requirements are then derived from the differences between the AS-IS model and the TO-BE model. This will typically result in a solution, possibly involving rebuilding some of the subsystems, replacing or upgrading some of the component parts, adding some new stuff or stripping out old stuff, rewiring the network, retraining the people, resetting system policies and parameters, and so on. In addition to a solution blueprint, showing how all these elements are to be configured, there will also be a transition strategy, indicating how (and in what sequence) all these changes will be installed. There are usually operational constraints – for example, a requirement to keep critical business processes running at an acceptable level during the transition period.

Imagine you want to rebuild your kitchen. You have to think about fitting new units into the existing space, or possibly moving a wall to give yourself more space. You have to decide whether you are going to keep the existing fridge (which you only bought last year) or buy a new one. And you have to think about how long you can manage without being able to cook. Moving the wall, or deciding to keep the old fridge, belong to the solution domain. But if you are going to do requirements analysis properly, there needs to be something in the statement of requirements that helps you determine these aspects of the solution.

In all but the smallest and most simple projects, there will be many solution variants. The decision to retain or replace a particular component may be based on a technical calculation of its likely performance and capacity within the new configuration, or may be based on a political calculation as to the most convenient budget from which to fund the replacement (in other words, preferably someone else’s budget, if we can get away with not replacing it now).

Ten years ago this month, I wrote a piece about this in relation to Component-Based Software Engineering. Supply and Fit (CBDI Journal, September 2000).

But there is a more fundamental reason why there are many possible solutions – because making sustainable changes to complex systems is a tough challenge. Large and complicated change programmes aren’t always the most effective; a small intelligent fix is often far better (and less risky) than any amount of optimistic meddle.

So before we can get to a solution blueprint and a transition strategy, we need an intervention strategy. This takes us out of the comfort zone of requirements engineering into general systems thinking.

Leverage Points

Donella Meadows identified twelve leverage points for making changes in complex systems, and suggested that these could be ranked according to their power. See original paper by Donella Meadows. A version is included in her posthumously published book Thinking in Systems (2008).

If the intervention strategy can be expressed as a combination of leverage points, then this raises the question for requirements engineering – how do we work through the requirements of changing a complex adaptive system in a way that could produce this kind of intervention strategy?

Zero Based Procurement

Finally, I wanted to make a comment about one of the (many) dysfunctional aspects of prevailing procurement practices. In his blogpost Was this NHS IT tender a stitch-up? (Computer World, September 2010), Tony Collins talks about the difficulties of referencing a specific product or system in a tender document. “If a user organisation has a system it’s happy with, and wants to keep and enhance, why would it want to go through the needless expense of an EC tendering, rather than simply renew the contract?”

Procurement rules may have been designed to prevent cosy and uncompetitive relationships between public sector organizations and their suppliers. They appear to have the effect of forcing each procurement to be treated as a separate exercise, starting each time from a blank sheet of paper, so that there is at least the theoretical possibility of giving new suppliers a chance. (This is similar to the principle of Zero-Based Budgeting.) Many people doubt that these mechanisms actually have any real effect on competition or value-for-money; but meanwhile, these mechanisms appear to have a strongly negative impact on through-life capability management. How can brownfield requirements engineering be done properly under these constraints?

Download OSBS11g here (14MB)..

Introduction
A few years ago, when Oracle had only the BPEL component, deployment was rather complex. Therefore I create my own build tool, replacing the obant tool and ignoring the existing build.xml files, supplied within your JDeveloper project. Now with Oracle 11g the SCA composite is introduced and we are running on the Oracle Weblogic Server.

It took me some time to upgrade the Oracle SOA Build Server to the 11g version. Many new features have been added:

• Supports Oracle SOA 11g (from 11.1.1.3, aka PS2)
• Deploy/Undeploy SCA composites
• Deploy/Undeploy MDS
• Support of partitions
• Deploy to diferent targets
• Restructured project directory
• Restructured environment properties
• Updated documentation (Use Guide)

Overview
The default tools of SOA 10g SOA 11g, is only focussing on deployment of a single component (BPEL / ESB / SCA). While in real life, more components are involved with a deployment. You can think on Java application; data-model changes, etc. To overcome this issue, and create a mechanism to deploy multiple type of components, a general “build.xml” that is able to compile and deploy components.

The solution described in the next paragraphs will take this into account.

Functionality
The build tool covers the following features and can be easily change and extended to customer requirements:

• Depends on SOA 11g environment, installed soa_domain
• Using full ANT tasks features, Ant 1.7 + Ant-Contrib tasks
• Support of Multiple environments; Dev / Test / Acceptance / Production / …
• Deploy SCA composites
• Undeploy SCA components
• UnDeploy MDS artifacts
• Deploy MDS artifacts
• Deploy SQL
• Integration with SubServersion; download latests or tags
• Stand-alone tool ; not depending on Oracle installation
• Compile and Deploy Java programs
• Supporting ‘MakeAll’

Configuration
To use the build tool some configuration must be made. Most of all settings are located in properties files. Configure the build tool configuration file to prepare the deployment for this release. Edit the following file:

$BUILD_TOOL/build.env

In build.env, edit ANT_HOME and ANT_JAVA_EXTENSIONS, change the first part of the path to where the package was copied to ($BUILD_TOOL). Also change LOG_DIR in build.env, log files will go to this directory.

An example of this file is here

...

LOG_DIR=/home/oracle/buildtool/logFMW_HOME=/app/oracle/products/11g/fmwSOA_HOME=/app/oracle/products/11g/admin/domains/soa_domain

... 

Manual Deployment

Download
Download the latest version of your project. Example for SystemTest environment:

obbuild.sh -e test Download

Note: it will use the file test/replace_.properties to execute the find and replace for the endpoints.

Example for tag 2.0.23 being the current version for acceptance:

obbuild.sh -e acc Download 2.0.22

Example for revision 1254 for development:

obbuild.sh Download rev 1254

Note: it will use the file test/replace_.properties to execute the find and replace for the endpoints.

The replace.properties file is used for a general replacement of tokens in your projects. The file describes how tokens can be replaced in one or
all BPEL processes, ESB services.

Syntax:

find.seq.name=[all|]
[all|].seq.find.value=
[all|].seq.replace.value=

Example:
Replace in all the files localhost:9700 into 127.0.0.1:8001

find.1.name=all
all.1.find.value=localhost:9700
all.1.replace.value=localhost:8001

Example:
Replace in HellWorld bpel process the default domain and the hostname

find.1.name=HelloWorld
HelloWorld.1.find.value=localhost:9700
HelloWorld.1.replace.value=192.168.0.1:8888

find.2.name=HelloWorld
HelloWorld.2.find.value=/orabpel/default
HelloWorld.2.replace.value=/orabpel/systest

Deploy generic SCA composite
Deploy each BPEL process separately. Example for Test environment:

For example: Compile and Deploy Helloworld 1.0 to the default domain of node1

obbuild.sh -e test DeploySCA HelloWorldSCA 1.0 default node1

Deploy Java WebService
Deploy each Java process separately. Example for Test environment:

obbuild.sh -e test DeployJava HelloJavaWorld

Note: default-application.xml must be included in source code.

Automatic Deployment
To deploy the whole system at once, use the task ‘MakeAll’.

obbuild.sh -e test MakeAll

Note: This target is using the ‘makeall_test.properties’ that describes the correct sequence of all the AIA components (Java, BPEL, and ESB) that will be built and deployed. The next table shows an example of the ‘makeall.properties’ file.

sql.make.10=cardeploy.version.10=1.0deploy.target.10=dbnode

mds.make.15=common_capesoadeploy.version.15=1.0deploy.target.15=node1

sca.make.100=HelloWorldSCArefxxdeploy.version.100=1.1deploy.target.100=node1

sca.make.200=HelloWorldSCArefxxdeploy.version.200=2.1deploy.target.200=node2

sca.make.300=HelloWorldSCArefxx

sca.make.20=HelloWorldSCArefbbsca.make.30=HelloWorldSCArefccsca.make.40=HelloWorldSCA

A target can be given to deploy to a particular node or database.

targets.propertiestarget.node1=node1.vijfhuizen.local:8010target.node1.username=weblogictarget.node1.password=welcome1

target.node2=node2.vijfhuizen.local:8020target.node2.username=weblogictarget.node2.password=welcome1

target.admin=t3://192.168.2.10:7001target.admin.username=weblogictarget.admin.password=welcome1

target.dbnode=jdbc:oracle:thin:@database.vijfhuizen.local:1521:XEtarget.dbnode.username=scotttarget.dbnode.password=tiger

Command line interface
The command line interface of the obbuild tool is as follows:

./obbuild.sh -?

obbuild.sh [-[hvtV] [-e environment] [-f build-file] [-m makefile] [-r replace-file] [task] [arg1] [arg2] .. [arg6]]

 -h      Shows this help -V      Show the version of this program -t      Show all the tasks in the build file -e      Select envirnonment property file, default 'dev', reading dev.properties -f      Read a specific build file, default is build.xml in current directory

-m      Read a specific makefile -r      Read a specific replacement file, use for promoting BPEL process to other environments. -v      Show verbose output task    The task to be executed, default is main arg1..6 Optional arguments passed to the build file as properties ANT.ARG1..ANT.ARG6

./obbuild.sh -t

obbuild.sh: Tasks in ./build.xml                                                   CreateDirs - Create directory structure                                    Download - Download latest source code                                              [arg=tagged-version | rev number]                                 main - Default task                                                   Promote - Find/Replace on one or more processes, see replace.properties         CompileJava - Compile Java files                       [arg1=name of the application]           CreateWar - Create WAR file from class files                       [arg1=name of application]           CreateEar - Create EAR file from WAR file                       [arg1=name of application]          DeployJava - Deploy an EAR file to                       [arg1=name of the application]        UndeployJava - Undeploy an application from target                       [arg1=name of the application]             MakeAll - Create a new set of the application                       [arg1=[version] [arg2=partition]            SendMail - Send a mail                       [arg1=to arg2=subject arg3=text]           DeployXSD - Deploy the XSD files: creating a ZIP file                       [arg1=Name]          CompileSCA - Compile SCA composite                       [arg=sca-composite-name]           DeploySCA - Deploy SCA composite                       [arg=composite-name] | [arg=version|1.0] [arg=partition] [arg=target]         UndeploySCA - Undeploy SCA composite                       [arg=sca-composite-name arg=revision | arg=target]           DeployMDS - Deploy MDS artifact                       [arg=oramds-artifact] arg=[target]         UndeployMDS - Undeploy MDS artifact                       [arg=oramds-artifact] arg=[target]           DeploySQL - Deploy SQL                       [arg=sql directory] | [arg=db-target]        DeployScript - Deploy Script                       [arg=ant|wlst] | [arg=script-name] [arg=target]

There is no support from Oracle or myself and I expect that you have the proper Oracle licences to use the Oracle software.

Druid Plant Oracle

On a Linked-In group discussion (CBDI Forum), Richard Gilyead asked some excellent questions about organizational intelligence, and I’m going to post an edited and expanded version of the discussion here.

Does size matter? Do organisations with “thin skin” have a better ability to sense and respond? Is it inevitable that organisations will develop “thicker skins” as they grow?

Of course it has long been a popular idea that large organizations should behave like small organizations, while small organizations have problems of their own. I believe we can use organizational intelligence as a “lens” to attain a more precise understanding of some of the things that get lost as an organization gets larger, and use this lens to try and preserve and restore these elements. Meanwhile, a small organization is more dependent than a larger one on the knowledge and intelligence of its business partners and other members of its ecosystem, and it is useful to pay explicit attention to these aspects of its external relationships.

Richard then asked a follow-up question

What aspects of cohesion should be emphasised to preserve small organisation benefits as they grow? 

I think this should be a vital question for enterprise architecture (even though enterprise architects mostly work in large organizations, it’s surely never to late to ask it), and I don’t think there is a one-size-fits-all answer.

An organization makes a strategic choice (whether consciously or unconsciously) about the kind of “structural coupling” that matters. For example, a high-tech start-up company may be physically located near a university campus, and the founders have good personal relationships with researchers at the university. This will help the organization remain close to technological trends, but possibly at the expense of developing a broader understanding of market opportunities. At some stage the venture capitalists may persuade them to move their head office closer to the customers and/or to bring in new executives whose personal network faces in a different direction. Meanwhile, a different company might be dominated by supply chain and distribution issues, and for that company it will be these relationships that are most strategically significant.

The point is that this “structural coupling” affects many aspects of organizational intelligence: the events and trends that the organization can respond to, knowledge flows, innovation and organizational change. Which brings us back to the question of stability – where are the fixed points around which organizational systems (including formal information systems and services) can be built, and where are the points where flexibility is required. This is not a new question, but the organizational intelligence “lens” provides a new way of thinking strategically about the implications here.

Many human beings have always been fascinated with the future, things mystical oracles, divination, and various forms of divination! People have always wanted to know at least a little “about the future, or at least a little spiritual guidance ‘or advice!

For many centuries, Catholic priests actually used tarot cards to recommend some of their congregations, who may have had doubts about certain things. soothsayers have always had an interesting and mystical to help people, and for a very long time, any reading or sessions have always been described, due to the laws today, and for entertainment purposes, which is good. . . We do not care!

We are all smart enough to know that there are no guarantees in life, and we are adults, and if we spend money on a psychic reading, tarot reading, or anything else, even a Kuan Yin is seen here reading Oracle , then we can do!

Kuan Yin is a Bodhisattva, and be light, and she is the goddess of compassion! And “well known and adored by the Taoists, Buddhists, Shinto e.

The Sutra of Saddarmapundarika also known as the Scripture of the Lotus Sutra and Sukhavativyuha also known as The Scriptures speak of the Pure Land of the two powers of the Bodhisattva to save lives and their ability to liberate man from any type of disaster, if we simply rely on your help! In fact, the whole of Scripture Chapter 25 Lotus is dedicated to Kuan Yin, and says his name is Avalokitesvara as Kannon, and, of course, Kuan Yin. The text describes what could save a person from any difficulty or disaster. It may also appear in one of at least 33 different ways.

There have been many books written about Kuan Yin in the last two years and thousands of history, mythology, rituals and stories about her and if you want to believe, there are some benefits seriously powerful to be derived from the Goddess Compassion.

My name is Father Time, and use the Oracle Kuan Yin, known as chien tung or 100 sticks of fate, every time I do a reading for the people through email. Many people loved their readings, and you too. Why not give it a try? You have nothing to lose!

In my fabulous website at www. FatherTimePublishing. com also have many other amazing things, including some powerful spells and charms! So if you need to find love, sex or money. . . come see!

Good luck and many blessings!

[Removed] / /

[Deleted]

@greblhad suggests building Lego models as a technique for enterprise architecture: “If your #entarch can not be illustrated in 3D using LEGO then you have a problem”.

The reason Lego cannot reliably express architecture is because just looking at the model doesn’t tell us which elements are architecturally significant. For example, if a particular Lego model has a row of small bricks over the window instead of larger ones, and uses green bricks for the base, how can we know whether the architect actually specified this detail, or whether the architect left these details unspecified and the person building the model simply used the bricks that were available? In other words, the same physical model could have been produced by two different architectures: one including the instruction “Use small bricks over the window and green bricks for the base”, and one including the instruction “Don’t buy more bricks if you don’t have to”. (The latter being derived from one of the popular architectural principles that supposedly drive architectural practice.) Note that in this example, the two possible instructions are at two different logical levels, and the illustration may mislead us about the architect’s real intentions. For a different example, see my post What if architects designed our communities?

At its best, LEGO (even with people figures, as suggested by @tetradian) shows a momentary instantiation of an architecture, and is therefore at the wrong level of abstraction.

I accept that we are not talking about using Lego models as a specification tool. But illustration only works if you know what is being illustrated, as Wittgenstein pointed out (Philosophical Investigations). Furthermore, the representational nature of the model may be problematic. If Lego only has a limited palette of colours, do we interpret green to mean any shade of green, or that particular shade? Does the use of green bricks in the model indicate the use of green bricks in the planned structure, or does it have some other meaning, for example indicating the need for specially treated bricks? And so on.

@greblhad was inspired by LEGO’s new “product”, called LEGO® Serious Play®. Perhaps enterprise architecture needs creative play as well as rigorous specification, but it would be surprising if the same tools and methods supported both.