ServiceTechMag.com > Archive > Issue XIV: January 2008 > A Strategic Approach to SOA: Using Pilot Projects and Effective Disciplines to Ensure Successful Adoption and Governance
Mark LaJeunesse

Mark LaJeunesse

Biography

Mark LaJeunesse is the HP Services SOA Worldwide Program Director and is responsible for the development of SOA services, sales and delivery training, and driving HP's EAS SOA visibility and differentiation in the global IT market.

Mark has 20 years of experience in the high tech industry. Prior to joining HP, he held a number of management positions at Digital Equipment Corporation, Compaq Computer, Cybertech Systems and most recently at BEA Systems.

Contributions

rss  subscribe to this author

Avrami Tzur

Avrami Tzur

Biography

Avrami Tzur serves as Vice President of SOA in the Technology Solutions Group at HP, responsible for the company's SOA products and strategy.

Previously, Tzur was Vice President of SOA at Mercury and General Manager for the Americas at Orsus Solutions, where he led the company's efforts in delivering a middleware platform for developing wireless software solutions for the enterprise.

Before joining Orsus, Tzur was a general partner with Wingspring Inc., where he was responsible for funding and providing operational guiding to a number of enterprise software startups. Prior to that, he was founder and Chief Executive officer of Prosight, a provider of top-down portfolio management solutions for the enterprise. Before Prosight, Tzur was an entrepreneur in resident with BRM and a researcher with the Interval Research Corp.

Contributions

rss  subscribe to this author

Bookmarks



A Strategic Approach to SOA: Using Pilot Projects and Effective Disciplines to Ensure Successful Adoption and Governance

Published: January 7th, 2008 • SOA Magazine Issue XV
 

Abstract: Organizations without a sound strategy for SOA adoption risk being outpaced and outperformed by competitors who are better equipped to serve customers, seize opportunities, and respond to change. While SOA brings new challenges that must be addressed and new complexities that must be managed, when done right, SOA can accelerate growth and mitigate risk, ultimately delivering significant competitive advantages. This article raises important considerations and describes key steps to ensuring the successful adoption and governance of SOA via strategic planning.


Introduction

With its promise for greater IT agility, flexibility, and speed, SOA has become the undisputed architectural model of the future. While companies may be at various stages of adoption, SOA is increasingly an IT initiative that ranks among the top IT priorities.

Centered on the concept of loosely coupled, reusable, standards-based and well-defined service design, the adoption of service-oriented computing and service-orientation ultimately becomes an approach for designing, building, and managing a distributed computing platform that is seen as the panacea to helping corporations align business and IT. To be successful however, requires a holistic and platform-neutral approach that can be harmoniously incorporated within existing IT investments.

To make all of this happen, the involvement of different organizational groups is required, including IT architects, analysts, developers, project managers, and even CIOs. Since SOA aims to enable cross-sectional functionality, IT professionals are often forced to step out of their individual silos in order to collaborate in new ways. Therefore, any adoption approach needs to take this "human factor" into account.


Phasing Into SOA

Choosing a starting point for your SOA journey is far from trivial. In fact, many organizations are overwhelmed by the potentially enormous impact of SOA adoption on their enterprise and their existing development, deployment, and integration approaches, as well as the changes required to organizational structure. The result has been inertia, where organizations effectively delay SOA-related initiatives based on a lack of clear guidance.

The best approach to overcoming this state is to proceed with a succession of small projects and solidly link each one to a combination of business and technology value metrics right from the start. Project success should then be measured against these metrics in a well documented plan that is widely distributed and understood by IT personnel and business sponsors.

Although this will result in a slower, more iterative adoption, it will help ensure that the phased incorporation of SOA is evolutionary and can be carried out with less risk. Ultimately, this leads to a service-oriented enterprise that is far more sustainable.


Lessons Learned So Far

Below are descriptions of two approaches that companies have historically taken at this early stage. Although they have worked in some cases, they are both considered to have a relatively high risk factor.


Approach 1: The SOA Crusade

"Collect SOA case studies and success stories and build a comprehensive, enterprise-wide business case (and bet your professional credibility on it)."

Depending on the history and culture of your IT environment, the odds of success with this approach will likely be slim. That is not to say that the motivations are wrong; SOA certainly has the potential to deliver benefits that can fundamentally transform the business, and the calculations of return on investment typically contained in these business cases are generally accurate.

However, the sad reality is that few board members or other senior business sponsors will be convinced by them, mainly because of the high level of upfront technological and organizational investment required and because of negative experiences with past, grandiose IT initiatives with similar objectives and overly optimistic promises.

There is a tiny minority of cases in which the push for SOA comes from senior business executives with a vision and firm grasp of how the conceptual benefits of service-oriented computing can be translated into strategic business gains. When that scenario exists, though, it is quite likely that the SOA initiative would start anyway, even without an overly comprehensive business case.


Approach 2: The Tactical Adopter

"Quickly select an aggressively marketed SOA platform, implement the product, and hope this results in an implementation that provides all the benefits associated with service-oriented computing."

For example, the technology might be an integration suite, an enterprise service bus (ESB) or a Web services framework. The chances of this approach working can actually be higher than with Approach #1 because:

  • the technology for SOA is getting better
  • architects are becoming more proficient in SOA and service-orientation
  • the project scope will be more effectively limited by the actual capabilities and functionality available within the product
  • the technology might address a short-term need to solve a pragmatic task, instead of being selected to build a company-wide infrastructure

The risk of this approach is that the SOA implementation will inevitably be tied to a particular project, solution, and/or vendor, rather than being properly understood and positioned as an architectural approach that cuts across individual projects and transcends particular products or technologies.

SOA does not equate to a simple set of products you can buy. Some technologies will definitely be required, but more important decisions exist around inculcating new processes and encouraging behaviors that maximize the chances of realizing long-term strategic goals. In addition, this approach risks SOA being categorized as yet another IT fashion wave, somewhat marginally linked to business value and always in competition with whichever other project is more urgent.

Committing investments to a tactical adopter approach is always a struggle, and small hiccups in the project will easily and incorrectly be linked to SOA as an architectural model instead of the inevitable learning curve. This sets up the situation whereby skeptical budget holders or other project managers competing for funding have ammunition to effectively (if mistakenly) attach SOA as "just another technology trend" or, worse, "a solution looking for a problem." Once this belief has become ingrained, counteracting and overcoming it can be difficult.


The Recommended Approach: Plan Strategically

To build credibility, we suggest pursuing a much more gradual, progressive route with a low initial investment and a strong link to business value. A strategic approach is typically much safer and can incrementally demonstrate value and generate support for the SOA project, ultimately helping to produce a more effective foundation for ongoing initiatives.

When planning strategically, the focus is on easier-to-justify business cases for smaller, leading-to-SOA projects that are tightly linked to a quick return of business value. This requires a longer period of time to transform your core IT infrastructure into one that is service-oriented. However, because every project - especially in the beginning - is strictly linked to business value and quick returns, the pressure and associated risk of extending the coverage of SOA over the whole IT infrastructure is somewhat diluted.


Step 1: Identify the Business Pain

"Business pain" is certainly something that today's companies and their IT enterprises are almost never short of. No IT department has unlimited time, money or resources and project prioritization is therefore an essential part of any transition toward SOA. The benefit of SOA ultimately derives from its usefulness in helping drive business value in such areas as process visibility and consistency, and improved business agility and responsiveness. Prioritization should therefore be centered on the areas of greatest business opportunity and/or threat.

opportunities (and business pain points) can manifest themselves in many ways, but almost invariably come associated with one or more IT constraints. For example, for an insurance provider the business pain might be a long time-to-market for new insurance products, which could seriously impair its ability to compete. The flexibility of automated systems to support a new offering is critical in addressing this type of business pain. An enterprise that has been properly service-oriented can help by enabling more cost-effective and standardized access to and modification and extension of these core IT systems.

In this first step, IT knowledge and technology priorities are secondary to process expertise and business priorities. For example, the IT department for the insurance provide would need to:

Below is a series of steps that outline a structured plan for phased SOA enablement via the strategic delivery of pilot projects:

  1. accurately identify the business pain points (such as the long time-to-market for insurance products)
  2. break them down into a series of factors (the business pain might be due to slow internal auditing procedures, over committing of resources to other projects, or a lack of flexibility in the IT systems)
  3. choose one factor (the lack of flexibility in the IT systems, for instance) that has an immediate IT counterpart that you know SOA can address

For a project to be a good candidate for a pilot you have to be able to measure the associated business pain via concrete metrics. In the insurance provider example, it might be the number of weeks it took to have IT support a new insurance product, averaged over the last five or so products. This type of historical data is frequently available and can be precisely calculated. The important thing is to have a simple, quantifiable metric that business people with no IT background can easily understand and relate to as a problem.


Step 2: Select the Pilot Project

SOA project candidates should have a narrow enough scope to ensure proper communication and minimize the risk of project failure. At the same time they should be significant enough to illustrate the full potential value of SOA.

In the previous example, improving the time of implementation for all products via one massive IT initiative is simply too ambitious for a pilot project. In this case, you will need to divide and conquer by partitioning the problem into smaller areas where the measure of business pain is still valid. For example, maybe there is a particular line of life insurance policy products where a streamlined, service-based product construction is possible to reduce the average time of implementation.

SOA pilot project candidates should not be highly mission-critical. As a general rule, good SOA pilot projects tend to have two characteristics: high visibility and low risk. By high visibility, we mean the project in question is important enough to the business to ensure initial funding as well as on-going attention from the sponsors (this also helps shift focus away from technology-led initiatives driven purely by IT).

Low risk refers to the fact that the project does not entail significant modification, extension or enablement of core, mission-critical business processes and minimizes the amount of new technology used (so don't introduce business process management and an ESB at the same time as a pilot project). In other words, a good pilot project is one in which delays during the project won't negatively impact core business operations.

Note that there is a "sweet spot" for SOA pilot project candidates, where the IT complexity is lower than average, but the business value is high. Hitting this spot is far from easy, but it has high payback.


Step 3: Measure the Business Pain Again and Get Credibility

Now it is time to measure the business pain once more. Thanks to the pilot project, our insurance provider can implement new life insurance products in a fraction of the time it used to take. Prove it by showing how easy it is now, and demonstrate the corresponding gains in business value.

Remind stakeholders how high the business pain was before the project, state how much it has decreased after the project, and tell everybody. It is the only way to get credibility and start raising the attention of the business stakeholders. Now go back to Step 1 and run another pilot project (or two) slightly bigger and/or more complex, on different but related IT areas (for example, new types of life insurance policies). This will leverage the skills and infrastructure established by your first pilot while also expanding the scope of SOA adoption.

Even though in the initial pilot project you may require specialized tools, be sure that (a) these products can scale to meet future demand as the number of services and users grows, and (b) that these tools will seamlessly integrate with your existing IT infrastructure. For example, when considering Web services management technologies, recognize that over time this type of functionality will become a transparent part of your overall operations infrastructure. Similarly, when considering ESB products and other messaging and brokering technologies, recognize that you may already have middleware that can support goals within your SOA adoption plan.


The Side Effects

While you are carrying out your initial projects, each of which addresses real business pain and are not likely to require excessive efforts for justification, a few important (and pleasant) things can happen:

  • The number of reusable services available for later projects will increase.
  • You will understand through experience how important it is to manage the proliferation of reusable services and to enforce design policies.
  • IT's credibility (and your own credibility as a professional) with the business will grow, as will the potential budget you'll be able to secure for future, more ambitious projects.
  • Starting from an integration competency center or from initial project-related resources, you will begin the long process of ramping up SOA-related competences. These individuals are likely to be part of your SOA center of excellence that will gradually form project after project (more details below).

Common Challenges

Once business services have been built and deployed, companies will undoubtedly experience new challenges in managing them. Traditional security, policy, and governance issues are compounded as service permutations grow and changes occur within the supporting architecture.

Furthermore, once a service is published, users who did not create that service may have a difficult time trusting it. Therefore, it's important to establish a controlled process for provisioning services, including checkpoints and approval mechanisms to ensure that services conform to both IT and business policies and best practices, and to guarantee an acceptable level of QoS.

Other challenges include:

  • Service Level Compliance - Business services can be composed of different services, which often depend on other services. All of these composition members may have inconsistent service level objectives. Service level compliance creates new requirements for performance, availability, policy definition, reporting and troubleshooting.
  • Service Security - Security requirements will become increasingly important as more services are published to a registry by a variety of constituents. Therefore, authentication and authorization of services not well-known within an environment will become critical to any enterprise-wide SOA.
  • Service Governance and Auditing - Defining overarching policies and auditing compliance on these complex and often dynamic relationships is very challenging. Messaging, orchestration, and governance issues (such as those imposed by Sarbanes-Oxley or Six Sigma) make it even more critical to get this right.

SOA Excellence Requires Essential Disciplines

A large financial information company wanted to fix the workflow around its existing technology for publishing enterprise Web-based services to create a foundation to automate application processes. It began by outlining its SOA vision and roadmap, but went beyond that by implementing a service registry to promote visibility and reuse. It also adopted a central system-of-record on all services that was easily accessible by IT and business management (and was, incidentally, also used for developer training). In due time, this company was able reduce the time-to-market for deploying services and products from a six-week cycle to just 15 minutes.

They enjoyed the fruits of SOA by applying several key disciplines that manage the entire lifecycle. From evaluating existing assets to building the infrastructure to operating and maintaining the underlying architecture, SOA has its own lifecycle. To ensure that your service-oriented environment is implemented and maintained properly throughout its lifecycle, you need to apply the disciplines of governance, quality, and management, as follows:


Governance

How do you gain visibility, trust, and control in an SOA? How can you maximize the controlled adoption of SOA? Without effective governance, both the adoption of a service-oriented architectural model and the ability for its underlying technology architecture implementation to scale as demand grows are impeded. And, without the ability to enforce enterprise policies, you place your business at risk. With proper governance, companies can guard against these risks by gaining:

  • Visibility - Create an environment where services at every phase of their lifecycle can be easily found and understood by consumers.
  • Trust - Establish consistency, interoperability, and formal service agreements between consumers and providers.
  • Control - Enable the management of the service lifecycle from inception to retirement.

This all begins by assessing your enterprise's current IT governance model and formulating a detailed specification of the functional model it must implement in order to transform itself into a service-oriented enterprise.

Even an iterative, project-based approach to SOA needs to be cognizant of the eventual end-goal of establishing an enterprise-wide SOA. This requires that individual projects adhere (where possible) to overall best practices and commonly accepted processes and conventions, thereby ensuring consistency, avoiding duplication, and creating a shared infrastructure.

The best way to do this in a "non-onerous" manner is to simply create a center of excellence (or a global program office) where best practices are agreed upon. Once the enterprise architecture program is established, you will need to execute a complete iteration of the service-oriented architecture definition process.


Quality

The increased complexity that a service-oriented architecture can introduce into an organization makes it harder to ensure that reusable services are adequately tested for functionality and performance before they are placed in production and shared across the enterprise.

You will need to consistently validate and optimize services as well as manage their testing in order to mitigate the risks of delivering ones that turn out to be inadequate. By rigorously subjecting services to pre-production test cycles, you can make better informed "go-live" decisions, reduce the time and cost of deployment, decrease software defects, and create consistent, high quality services and service compositions.

The goal of ensuring SOA quality is to:

  • optimize performance and deliver services that will scale in production
  • validate functional quality
  • manage the complexity of testing multiple services
  • provide traceability and impact analysis
  • create high quality individual services and composite applications comprised of multiple services

Management

In order to achieve the highest levels of operational integrity for SOA initiatives, IT needs to manage service level agreements (SLAs), enforce SOA policies, and obtain rich reporting, monitoring, and diagnostics.

Quality SOA management also delivers better business outcomes by proactively administering service levels from the business perspective. The company can rapidly isolate and resolve performance and availability issues and quickly detect changes to assess the impact on end-users by visualizing all service dependencies.

Key pieces of an effective SOA management program include:

  • Discovery - Use an up-to-date service dependency map to automatically find the relationships between services and to discover ungoverned "rogue" services (and bring them under management).
  • Monitoring - Gain visibility into performance or availability issues before they affect the business applications in order to meet SLAs.
  • Policy Enforcement - Create, manage, and enforce policies at runtime.
  • Availability Management - Ascertain service availability with real and synthetic end-user experiences and agent-less system and infrastructure monitoring.
  • Problem Isolation - Detect and isolate problems based on information from end-user and infrastructure monitoring systems and then drill down to find the root causes.

Figure 1: An overview of the three key SOA disciplines [REF-1].

As mentioned earlier, you should consider establishing a center of excellence to oversee the alignment of business and IT requirements. Because implementing SOA is a serious responsibility, you need an organizational body that administers proven methodologies, best-practice expertise, and provides experienced resources and mature tools. Such a center can develop a practical way to integrate a program for SOA governance, quality, and management with lifecycle phases for planning, assessing, implementing, and maintaining control of the SOA initiative.


Conclusion

It's important to remember that IT assets are also business assets, and that IT services are also business services. SOA better aligns business and IT and shifts the focus away from the nuances of underlying technologies toward services that deliver real value to the business in order to help mitigate risk and accelerate growth.

Since business services are meant to be reused, developers and project teams need to think and work in new ways. For SOA to succeed, individual teams and business units can no longer operate in secluded silos. Services need to harmonize with enterprise-wide mandates on quality and IT policies. The goal is for project teams to be able to easily find and reuse services to construct new applications and compose new business processes whenever required.

An SOA adoption is an opportunity for IT to make the transition from a bottleneck to a source of business agility and competitive advantage by enabling a number of business benefits. The key to taking advantage of this opportunity is in how the adoption itself is carried out.


References

[REF-1] Hewlett Packard SOA Technology Solutions Group