Many captive software development groups operate at low efficiency.  There are reasons, but do they justify shunning all process improvement?

Captive shops produce software with only one customer, the business side of the same company.  From an outside point of view, this should be a great situation for efficient software development, however the opposite is often true.  Why? Well, FUD.

Risk Averse

If the business is successful, like making money, and assuming the products of the software group are viewed as important to that business success, many managers find themselves in a position of nervous comfort.  There is a de-facto proof that the software is good enough.  After all, the business relies on it and is making money.  So the methods and techniques, the governance, for maintaining and extending the software must be okay too.

Changing anything could potentially involve a risk of upsetting this happy situation.  Many managers are not willing to assume this type of risk because they fear that what is working may be broken.  These risks are not essential.  The software is viewed as supporting a larger product, and that product incurs essential risk.  The software is thought of like buying a delivery truck or copier where the risk is not essential to the business, therefore it should be minimized.

Couple this with the problem that many managers are not especially technical and it is understandable why “No.” becomes the only answer for any question of process refinement.

Contrast this with companies that produce software as a product.  They are usually inspired to refine their processes to gain an edge on competitors, improve responsiveness to customer issues, reduce cost, improve quality, and increase employee retention.  In other words, if the purpose of the company is to produce software, then the risks involved in process improvement are the essential risks that make the company interesting and viable.

From the developer’s perspective, when you know the answer will be “No.” before even asking, you stop asking.  The developers either look for greener grass or learn to live with low efficiency.  Worse, new graduates hired into this environment are trained that this is the way software development works in the world.

So there is a stalemate, a stasis that guarantees efficiency will not improve.  Management fears risk and developers are cowed to acquiescence.

Slippery Slope

However, there are several factors that may cause efficiency to decrease.

One problem is turnover of the development staff.  This is doubly harmful because the tacit knowledge may not have been captured in the low efficiency process and higher skilled people will get more frustrated by the situation and are more likely to leave.   Replacing a talented engineer has several significant costs; recruitment, the 3-6 months needed to become productive, and the opportunities lost during this period.  The net result is a staff that has a lower average skill set who must maintain and extend programs developed by brighter folks.  Entropy happens.

Another is a bias toward building newer features separately from existing features.  Since there is a desire to ‘leave what is working alone’, new features cannot be integrated with older features.  Rather than reworking parts of what exists to make it work well with new features, the new features are ‘glued on’ like a motorcycle side car.  So instead of morphing your Harley into a Land Cruiser, you wind up with a clown-car motorcycle with 5 sidecars.  Entropy accelerates.

Entropy compounds itself.  The software becomes unnecessarily complex.  Each bug fix, each feature addition takes longer.  New staff faces a longer learning curve to become productive.  Costs rise, feature additions slow.  Efficiency declines.

Moving Forward

As with any process modification, the first step is recognizing that you have a problem.  This can be a very difficult obstacle to overcome because ‘everything is working fine here’.  Often measurements of the development and maintenance process are ineffective or nonexistent.  You will need to know where you are before you can chart a new course.

Next try to get a bearing on how your processes compare to other companies’.  Draw on the experience of technologists and managers who have worked in other companies to learn about techniques that worked well.  “At XYZ Corp. we installed continuous builds and saw a 5-10% reduction in bugs.”  Also look for published research on areas that you suspect could be improved.  “TDD increased cost by 15-35%, but reduced defects by 60-90%.”

If you are a technologist, look for ways to incrementally improve the situation rather than lay out the big vision.  Broad statements like ‘we could double our productivity if…’ will prove to management that you’re a dreamer.  Try working with managers to understand their concerns and work to address them in your requests.  Initially, make proposals that you are very comfortable with, where the cost and benefit are measurable and unambiguous.  Advise your management of the cost, benefit, risk and your level of comfort about the correctness of all these.

If you’re a manager, realize that there are many way to improve, each with cost, risk and benefit.  Challenge the technologist to present opportunities with cost/risk/benefit assessment.  Try some small changes, measure the results, reap the benefit.  Remember also that the Hawthorne Effect can provide a measure of inspiration to the developers working with you, helping you be more successful; always be experimenting.

Most importantly, business and technology professionals need to work to build understanding and trust.

im having some troubles here..
i can save a log file from a web application using the Debug class from System.Diagnostics namespace.
when i debug the web service, it also saves the log file.
however, when the web service is called by the web application, only the application’s log file is saved. the web service’s log file is not even created.
i would appreciate any help, as i need this pretty urgently!
thanks in advance!!

qgl.-
hi fixedinseattle, thank you for answering.
im currently running the WS in debug, and user ASPNET is an administrator and has full control in local machine, where the WS is hosted.
the weird thing is that the log is saved when i run the WS by itself, but not when its called by the application.
the app is in VB.net and the WS in ASP.net
any clue of what could be the problem?
thanx!

Curious about services like Web Design, Telemarketing, Data Entry, etc. Please reference your answer if you can.

http://dragonswamp.com/caves/

I’d like to make something like this, what did they use?
can I have the link to please?

It is our pleasure to announce the publishing of another article in our “SOA Suite Essentials for WLI Users” series: Setting Web Service and JCA Adapter Endpoints Dynamically in Oracle SOA Suite.

This article describes how web services and JCA adapter endpoints in SOA Suite can be changed dynamically at run-time in a very similar way as the dynamic properties of WLI controls are used to change endpoint properties. The article uses the sample of a file adapter in a BPEL process and demonstrates how the output directory and the file name can be changed using header properties in the process or through the Enterprise Manager console.

A closely related article that describes the use of Domain Value Maps (DVM) in SOA Suite in comparison with the WLI XML Meta Data control will follow shortly and complete this use case.

Please let me know me know what you think about the series and this article

Navajo Integrator is the flagship product by Dexels BV to build enterprise solutions for large and medium scale organizations. With Navajo Integrator it becomes possible to actually set-up a Service Oriented Architecture and turn SOA and saas theory into practise. Navajo is real, based on open standards, and is proven technology. It has won a Duke Award by Sun in 2007 and has many real world appli…

Over the past decade, there’s been a push and pull between more technically inclined SOA proponents who worry about tools and platforms, and more business-oriented proponents who worry whether SOA is doing enough to advance profitability and market growth.

Both side are important, and often, SOA proponents need to maintain a balancing act to get projects through the pipeline. That’s the observation of Paul Brown, a co-author of the SOA Manifesto,
and
principle software architect at Tibco Software, and ebizQ Managing Editor Jessica Ann
Mola recently spoke with Paul about why business value is so important
to service oriented architecture. (Listen to the podcast podcast here; full transcript of the discussion available here.)

As Paul put it, “we do have to be good corporate citizens and realize that after all it
is the business value that is the reason for us being there.  It’s what
pays the rent and we have to deliver business value.”

However, he cautions, “we
can’t abandon the technical strategy because it is the technical
strategy that will give us the reduced cost, faster time to get results
out into the marketplace and all those good things that we expect out
of the technology.”

However, he states, “when
push comes to shove, business value has to win.”

The following is the full transcript of a discussion between Paul Brown, principle software architect at Tibco Software, and Jessica Ann Mola, managing editor of ebizQ, about some key takeaways from the SOA Manifesto.

(Listen to the podcast here.)

Jessica Ann Mola:  Hello, ebizQ listeners.  This is Jessica Ann Mola, Managing Editor of ebizQ.  I’m speaking with Paul Brown today who is Principle Software Architect at Tibco Software.  Thanks for joining me today Paul.  

Paul Brown:  Oh, my pleasure.  Thank you.

Jessica: So Paul I wanted to ask you today about the SOA Manifesto, which I know you’ve been involved in.  Can you give me first just a very brief background on the Manifesto and how you got involved?

Paul:  Well, those of us who are involved in making SOA a reality with customers often see that the concept has been somewhat twisted by sometimes software vendors who try and sell SOA as a product, or people who are in the companies that get so caught up with the technology that they lose focus of what’s SOA is all about and what they’re trying to accomplish.  Most importantly, what it takes to really get value out of this kind of initiative.  So we thought that we needed to sort of reset and redirect the thinking around SOA to dispel some of the myths, get rid of what’s jokingly referred to as the bad SOA and focus on the good SOA, the realities behind the concepts.  

Jessica: Okay.  How did you come to be involved with the SOA Manifesto?

Paul: Well, I was involved with both the first and the second SOA Symposium.  I’ve written a couple of books on Implementing and Succeeding with SOA.  And Thomas Erl invited me to participate in putting the SOA Manifesto together which I was more than happy to do.  So we got together kind of in parallel with the second SOA Symposium and spent several days kind of hammering out the wording of this manifesto.  

Jessica:  That’s great.  Now, the first principle of the Manifesto or I believe we call them value statements is “business value over technical strategy.” And you mentioned to me before that you feel that that’s the most important statement.  Can you explain why?

Paul: Yeah, it really is.  There is a tendency anyway in particularly large companies with large IT departments to get very focused on the technology and simply being successfully with the technology as opposed to providing the value to the business that after all is the justification for having IT to begin with.  So while it is important to pursue technical strategy that does give you the flexibility that you need to move forward, we can’t lose sight of the business value or the fact that we have to continually deliver real value to the business on a project-by-project basis as we pursue these technical strategies.  And sometimes we get in situations where the business value sort of comes into conflict with the technical strategy.  Not often, but it does happen.  

And in such cases, we do have to be good corporate citizens and realize that after all it is the business value that is the reason for us being there.  It’s what pays the rent and we have to deliver business value.  So sometimes, we need to compromise to give that business value.  On the other hand, we can’t abandon the technical strategy because it is the technical strategy that will give us the reduced cost, faster time to get results out into the marketplace and all those good things that we expect out of the technology.  So it’s a balancing act.  It’s not that we don’t value technical strategy but we have to keep them in balance and when push comes to shove, business value has to win.

Jessica: Well, thanks for clarifying that Paul.  And thank you very much for joining me today. 

Paul:  Oh, you’re quite welcome.

Wanted: advocates who can bridge the chasm between the SOA and data worlds.

Since it came about in its current form in the early-to-mid 2000s,
service-orientation (mainly focused on applications) has existed in a
separate world from data management.

Problem is, an SOA-enabled infrastructure with bad data flowing
through it can be useless, and even dangerous. One observer even compared SOA to a mosquito
that can deliver payloads of bad data (“viral data”) at lightning speed
all across the enterprise — pandemic style — before it can be stopped?

Now, we’re seeing the launch of the SOA Data Integration Architect Community (SDIAC),
an online community focused on the value of data integration and data
services in agile architectures such as SOA, promising to bring the
data and SOA worlds closer together. Such an organization will
definitely bring support to SOA proponents struggling to address data
quality issues, and data architects seeking to service-enable their
environments.

One of the moving forces behind the creation of SDIAC is Informatica’s Ash Parikh,
a colleague of mine who has been a long-time proponent of data services
within a SOA context. Ash has also appeared frequently as a guest speaker at ebizQ Webcasts and conferences. While Ash started the ball rolling, he prefers to
take a background role in the interest of the vendor-neutrality of the
group.  At least initially, Informatica is supporting and hosting the
SDIAC site. (Disclosure: I am a guest contributor to Informatica’s
Perspectives blogging community.)

So the SDIAC will be led by none other than Dave Linthicum, ebizQ’s resident authority on information management, and a respected expert on enterprise and data integration, SOA, and cloud computing. As Dave notes in a post
over at Perspectives, he views this community “as being sorely needed,
considering the lack of understanding of SOA data integration out there
today. Those charged with defining and building core IT
architectures are not likely to understand the best practices in
defining the architecture for the data up to the services, and up to
the processes. They are then left with architectures that are difficult to change, and don’t completely take advantage of the data assets.”

Dave also provides three reasons to consider involvement in SDIAC:

1) To become involved in the last mile of SOA – the need to deliver timely, trustworthy and relevant data using data services;

2) to network with peers and exchange knowledge; and

3) to create a nurture new ideas that will move the SOA-data services connection forward.

Another good reason for engaging with SDIAC is that it will help SOA
proponents better communicate their ideas and requirements to the data
folks, and likewise, helping data managers better understand the role
of SOA as a way to get the right information to decision makers and
applications. And, for both groups, it can help provide insights and
perspectives to approach and sell data services to the business.

And, as we move deeper into emerging areas such as complex event
processing and business activity monitoring, SOA can provide the
framework to help assure the relevance, accuracy and timeliness of data
moving through analytics systems.

There’s a difference between automation and orchestration, and knowing which one you’re really doing is half the battle in achieving a truly dynamic data center.

Randy Heffner on CIO.Com wrote an excellent article on SOA and its value, “SOA: Think Business Transformation, Not Code Reuse.” The problem I had with the article was not in any way related to its advice, conclusions, or suggestions. The problem I had was that I kept thinking about how perfectly much of his article could be applied to data center orchestration, operational transformation, and automation. Simply replace “SOA” with “orchestration”, “software reuse” with “automation”, and “business” with “operational” and you’ve pretty much got what needs to be said. Here, I’ll show you what I mean:

The worst CIO CTO misunderstanding about service-oriented architecture (SOA) orchestration is thinking of it as only another technical initiative for automation software reuse. Although SOA’s reuse orchestration’s automation potential is real and good, its business operational impact goes much further: In Forrester surveys, 38 percent of Global 2000 SOA orchestration users say they are using it for strategic business operational transformation. SOA’s Orchestration’s true source of power is in its business operational design models, not its technology — and this means that SOA orchestration provides a broad foundation for a much larger shift in business operational technology (BT) architecture that goes far beyond SOA orchestration itself. By correctly understanding orchestration SOA, CIOs CTOs can lead their organizations on a solid and well-managed path toward a strategic technology future and greater business value.

This is true for SOA, and it’s true for cloud computing, where it is the orchestration of the data center infrastructure that brings the value to the table through making more efficient the operational processes codified to automate and integrate systems.

This very short (perhaps too short) post on differentiating between automation and orchestration last year kicked off a very interesting (and also short) discussion on Twitter with the core theme appearing to be “why differentiate in the first place?” Why, indeed. I can answer that question with a question of my own: why differentiate between “software reuse” and “SOA”? After all, aren’t they the same thing?

They are not the same, as Randy very eloquently pointed out above. The biggest difference is that discrete components like services and automation are designed from the top-down; that is, they are not necessarily taking into consideration the business side of the processes.

Automation is akin to expert systems: it is the codification of a set of steps to perform some action. That action may be conditional upon variables extracted or shared by other systems within the environment, but they are environmental and focus on specific properties such as CPU utilization, number of connections, or even the user invoking the automation. Automation can answer the question “how do I do this?”

Orchestration, however, resides on the process level and more specifically on the business process level. Business process in this case relates to the operational IT processes which seek to achieve a specific business goal in its execution. Orchestration can answer the question “why do I do this?”

The subtle but significant difference between automation and orchestration is important if you’re looking to realize the maximum benefits from an implementation. Automation will reduce time spent on tedious tasks, yes, but orchestration can streamline processes by discovering – and one hopes ultimately eliminating – redundancy, and aligning the technical aspects of IT with the business.

Oh how you hate that “align IT with the business” line, don’t you? It’s fluff, you’re thinking; just another buzz phrase. In some cases when it’s used, yes, it’s just a phrase. But in the case of orchestration there isn’t a truer set of buzzwords that have been strung together in quite some time.

That’s because the goal of orchestration is to make operational decisions based on business goals and needs in real time. Instead of thinking about bandwidth in terms of how much an application might need, you start making decisions based on what it uses and what it costs and what the prioritization of the business function the application serves. Yes, you probably do factor those variables in today, but orchestration makes them an integral part of the decision making process and uses automation to make the adjustments required without human intervention. Orchestration is the art of tying together those integrations in such a way as to automatically execute against business logic based on operational and business requirements. In the case of orchestration, the whole is greater than the sum of the parts because the integration and collaboration that happens in an orchestration is enabled by automation, but not made from it.

Just as JBOWS (Just a Bunch of Web Services) don’t make a SOA, neither does a bunch of disconnected scripts automating IT tasks make an orchestration. It is the integration and flow of systems from the top down, with a view from the business layer of the stack, that makes an orchestration valuable to both business and IT folks alike. SOA is an architecture, not an implementation. So, too, is orchestration an architecture and not an implementation.

I think some folks from an EDS (HP) said it best in their vision paper: “Orchestration for the Business-Driven Data Center”

Orchestration technology is not the same as systems management, provisioning, workflow management or “run- book” automation (a type of workflow management). These disciplines seek to automate the tasks required to manage elements of the IT infrastructure, such as servers, network devices, storage or databases. The primary goals of automation are to improve response times, reduce errors, reduce IT staffing needs, avoid redundant manual procedures and improve consistency. These are operational goals specific to the mechanics of managing technology.

Orchestration, by contrast, focuses on business goals such as improving business up-time, improving customer satisfaction and retention levels, reducing order processing and delivery times, and reducing missed business opportunity costs.

Well, no. Not really, unless you’re implementing discrete automation points and calling it orchestration. The reason it matters then is that you may not know that you could be realizing more benefit out of the automation if you really were doing orchestration. If you’re doing orchestration and calling it automation, well, a rose by any other name, right?

What you call it isn’t nearly as important as recognizing that there are differences between the two and then applying that knowledge such that you can achieve the best benefits from what you’re trying to do.

Cause knowing is half the battle, right? The other half requires lasers…and for those, you’re on your own.

read more