Mandy joined IBM in 1987. She is an IBM Distinguished Engineer, Master Inventor and member of the IBM Academy of Technology Leadership Team. Her current role is the Chief Architect for InfoSphere Solutions in the IBM Information Management CTO office. She leads the design of common information management patterns for different industries and solutions. This includes the Next Best Action solution and the strategy for Information Virtualization.
In earlier roles Mandy led the development of new features for the CICS, Encina, TxSeries and WebSphere products. She has over 50 issued patents worldwide in the fields of transaction processing, event management, business process management and model driven development.
In addition to her technical responsibilities, Mandy is involved in initiatives designed to enhance technical vitality with the IBM technical community. This includes mentoring, serving on technical career development and promotion boards along with Women in Technology initiatives.
Outside of IBM, Mandy is a Fellow of the Royal Academy of Engineering and a visiting professor at the University of Sheffield, UK. In 2001 she was the first woman to be awarded a Silver Medal by the Royal Academy of Engineering and in 2000 she was one of the "TR100" young innovators identified by MIT's Technology Review magazine. In 2006 she won a British Female Innovators and Inventors Network (BFIIN) "Building Capability" award for her work developing innovative people and the BlackBerry "2006 Best Woman in Technology - Corporate Sector" award. More recently she was granted an honorary fellowship of the Institution for Engineering Designers (IED), she won the "2012 Cisco everywoman Innovator of the Year" and in 2013, Plymouth University awarded Mandy a Honorary Doctor of Science.
Harald Smith is currently a Software Architect with IBM, with a diverse 30 year career specializing in information quality, integration, and governance products and solutions and a long-time interest in patterns and design practices. His work has spanned software product management, information technology, consulting services, technical support, system auditing, and business process re-engineering, and includes 4 issued patents worldwide. He writes extensively including his current blog "Journeys in the Information Landscape", particularly on methodology, best practices, and capabilities with IBM products and the topics of Big Data and Information Governance, and is an IBM developerWorks Contributing Author. Harald is also an IBM Certified Solution Developer in Information Management.
Design Patterns for Information in a Service-Oriented Architecture Published: October 24, 2013 • Service Technology Magazine Issue LXXVII PDF
Abstract: Organizations are driven by the information they manage. Organizations in this fast-paced world need to be more agile and more integrated, which is where service-oriented architecture adds value. Behind the services, there are systems that manage the information exchanged across the services.
This article is about design patterns for information management techniques that improve the information services used by your organization. It describes a pattern language called the Patterns of Information Management (PoIM). PoIM contains over 200 linked design patterns that use consistent terminology and concepts, resulting in a body of work that can be used at all levels, from the enterprise architect down to the individual project.
An example design pattern for an enterprise service bus (ESB) is used to illustrate the structure of a design pattern and how such a pattern links to other PoIM patterns to create a pattern language.
The success of service-oriented integration is determined by the quality of the information exchanged across the service interfaces. If the information is useful and reliable but difficult to get by any other means, then callers will flock to connect to a shared service.
Information that is core to the operation of an organization, such as information about customers, products, assets, and contracts, is often duplicated across many systems, since each system needs access to this information to perform its tasks. Where this type of information resides within any one system, it reflects the silo (typically the line of business) in which the system operates, leading to inconsistencies in this information across the different systems.
Consider creating a service that returns up-to-date customer details in a situation where a customer has multiple accounts and interacts through multiple channels, but where each account and channel are implemented by different systems. A centralized server that can consolidate, de-duplicate, and resolve inconsistencies is needed to integrate the details for each customer. This server (called a master data management hub) provides authoritative information through services and synchronizes this information with other systems, creating a consistent view of customer data across the organization.
Similar consolidation techniques are deployed with historical data for reporting and analytical services when we create data warehouses and data marts. And more recently, social and machine-generated data can be captured, synthesized, and made available through services using Big Data technology.
Information governance is the process wherein standards and policies relating to the management and use of information are regulated. Its supporting technology and practices assure the quality and protection of information throughout its lifetime.
All of these techniques come from the information management discipline. They are invaluable to enterprise architects and solution architects, but little understood. This article is an introduction to these design patterns.
As architects, we wish to use best practices and proven approaches to create robust solutions for our organizations. Those of us who have been in the industry for many years know there are fundamental principles of software engineering that do not change, even though the emergence of new technology seems endless.
When we document best practices, it is important to separate the fundamental principles from the technology's specifics. Then, when we encounter a new technology, its behavior can be mapped to the fundamental principles and special note is made of the technology's specifics that occur.
For example, Apache Hadoop provides a new type of repository and processing mechanism for processing information. When we compare it to existing database technology, such as a data warehouse, it is obvious that these are very different technologies.
However, there are similarities:
The major differences between the two are:
Design patterns document best practices, or architectural principles, in a consistent manner. They assemble a variety of information using a consistent terminology that helps us extract and understand the significant similarities and differences between technologies and approaches.
In this article we illustrate the value of design patterns using the Patterns of Information Management (PoIM) pattern language. PoIM covers the design patterns for managing information in distributed and interconnected systems.
Below is an example design pattern for an enterprise service bus (ESB). It was chosen as the example since it is most likely an architectural approach that you are familiar with and is a logical extension to the PoIM pattern language. This specific pattern is not included in the PoIM pattern language but is written in the same style, using the component patterns from the patterns of information management pattern language. As such, it is written in a technology-agnostic style.
The patterns in PoIM are grouped into pattern groups. The enterprise service bus pattern has been added to the Information Solution pattern group. The lead pattern in a given pattern group is named after the pattern group. It describes a generic design pattern while the rest of the patterns in the group are specializations. Information Solution patterns are composite patterns, built up from other underlying information management design patterns.
Table 1 shows the pattern summaries (also known as patlets) for Information Solution and Enterprise Service Bus:
Table 1 – A summary of the Information Solutions Pattern Group is displayed [REF-1].
Notice that each pattern has a name and an icon. The name should be descriptive and easy to remember. The icon is a visual reminder of the pattern's content. We also use the pattern icons in the pattern descriptions when a pattern is being used as a component in the pattern's solution. The pattern summary includes a summary of the problem that the pattern solves and the recommended solution.
The full pattern description for the enterprise service bus is documented next, with the following subsections:
Each section adds another dimension to the understanding of the design pattern's intent and potential value.
The Enterprise Service Bus Pattern
An organization has adopted a service-oriented architecture as their system integration strategy, and is experienced with using service-oriented integration for point-to-point exchange of messages between two systems.
The point-to-point integration between systems is complex whenever the needs of the service requesters are inconsistent with the capabilities of the service providers, both in terms of the type of connection that is used and the scope of the functions available.
A fictitious retailer called MCHS Trading has three systems accepting orders from customers. These are called E-Shop, Mail-Shop, and Stores. The orders flow to a Shipping system that communicates with the appropriate warehouse to request that the ordered goods be sent to the customer's address. The order is also sent to the Invoicing system to ensure that the order is paid for.
Figure 1 illustrates how these order-processing systems are connected. No single system sees the end-to-end order processing process:
Figure 1 – MCHS Trading's order-processing systems [REF-2].
MCHS Trading has recently introduced two Master Data Management (MDM) hubs. Customer Hub is a system for synchronizing customer data between E-Shop, Mail-Shop, and Stores. Product Hub is the system where products are chosen for the catalogues, descriptions and images of each product are perfected, and suppliers are selected. Subsets of this product information are distributed read-only to the other systems.
MCHS Trading wishes to introduce a new Customer-Care system to help employees answer questions on the status of orders, record issues with products, and update customer details. This is the first stage for providing a social media interface for customer service, enabling customers to check on their orders and chat with employees electronically.
How should the Customer-Care system connect with the other systems, especially given the fact that each system was introduced at a different point in time and supports different communication mechanisms?
Create an intermediary system to act as a broker between the service requesters and the service providers.
This intermediary system is called an enterprise service bus. The implementation of each service within the enterprise service bus is responsible for issuing the remote calls to the appropriate systems and combining the results. It must handle the different communication mechanisms, security credentials, and data formats and standards.
This is illustrated in Figure 2, which depicts the following steps:
Figure 2 – An enterprise service bus is being used as an intermediary.
Some of the more important benefits of using this solution are:
Conversely, some of the liabilities associated with implementing an intermediary system are:
The Customer-Care system is given access to the services it needs through an enterprise service bus. These services offer clean interfaces to information about customers, their loyalty card status, products offered, the orders made, and how far the orders are through processing.
The enterprise service bus handles the implementation of these services. It is responsible for making the appropriate calls to the different systems involved in the order processing and extracting of the appropriate information. It also makes calls to the Customer Hub system for customer data. This system ensures customer information is kept consistent amongst the order taking systems: E-Shop, Mail-Shop, and Stores. For product information, the enterprise service bus calls the Product Hub since it is the most appropriate place to retrieve details of products and report problems that customers are having with a specific product.
The enterprise service bus hides the system complexity from the Customer-Care system. The services it offers are also adopted by other applications that need access to the core operational data of the organization (Figure 3).
Figure 3 – An enterprise service bus is being used to support the Customer-Care system.
The enterprise service bus is used by organizations that are experienced in using service-oriented integration between different parts of the business and wish to create further decoupling between the systems that are service providers and those that are service requesters.
The Enterprise Service Bus pattern relies heavily on the PoIM Information Service pattern group that includes Local Information Service, Remote Information Service, and Triggering Information Service. Together, these patterns describe the substitutability of the remote information service for a local one, and how an information service triggers new work when called from a remote system.
The processing that is performed by the enterprise service bus runs on a specialized server. In the PoIM patterns, this server is described by the Information Broker pattern in the Information Node pattern group.
The processing that occurs inside the service implementations in the enterprise service bus follows the Information Federation Process pattern from the Information Process pattern group.
Moving beyond the Enterprise Service Bus
The enterprise service bus improves access to the organization's information and is able to hide the complexity of how function and information is distributed between the systems. What it cannot do is resolve inconsistencies in information stored in different systems, or errors or gaps in the information stored within a single system. These issues must be resolved using information management techniques and technology. Otherwise, service-oriented integration becomes an effective technique for distributing bad data further.
A Service-Oriented Architecture needs a companion Information Architecture to cover how information is managed within systems and ensure that what is exchanged across the services is valid.
The information architecture defines standards for information, such as a common information model comprised of:
It also covers the additional systems and integration mechanisms that are used to consolidate, validate, enrich, and synchronize information between systems so that their information is kept consistent. This integration capability is in addition to the services. We call this the information supply chain that defines how information flows between the systems and ensures that information synchronization remains efficient, with minimal latency. When an error in a data item is detected, it can be traced back through every affected system to the source and corrected.
Each type of information is typically synchronized using its own information supply chain. For example, Figure 4 shows MCHS Trading's information supply chain for product information that emits from Product Hub. Product Hub is the single source of product information and so is called a cascading information supply chain.
Figure 4 – The product information supply chain is shown in a flowchart [REF-3].
Figure 5 shows the information supply chain for customer information. In this case, customer information may originate from E-Shop, Mail-Shop, or the Stores system. It is sent to the Customer Hub, which is then responsible for matching it to existing information about the customer. Each system uses a different approach for identifying people and contains duplicate records for individuals, with different subsets of attributes filled in in each record. Once the Customer Hub has the new information about the customer matched up with existing information, it synchronizes this new data with each of the relevant records in the E-Shop, Mail-Shop, or the Stores system. This type of information supply chain is called the Hub Interchange Information Supply Chain.
Figure 5 – The customer information supply chain is illustrated in a flowchart [REF-4].
The information integration mechanisms must operate continuously in the background so that the information is in the right place when services are called on the participating systems.
Figure 6 summaries the pattern groups in PoIM:
Figure 6 – A summary of the Patterns of Information Management pattern group is shown [REF-5].
In this article we have provided an example of a design pattern for an enterprise service bus (ESB). We chose a familiar concept to illustrate how effective the design pattern approach is for documenting design guidance, and to whet your appetite for more detail on how information should be managed in a service-oriented environment.
[REF-1] Chessell, Mandy and Harald Smith, Patterns of Information Management, IBM Press, 2013, p.578
[REF-2] Chessell, Mandy and Harald Smith, Patterns of Information Management, IBM Press, 2013, p.33
[REF-3] Chessell, Mandy and Harald Smith, Patterns of Information Management, IBM Press, 2013, p.35
[REF-4] Chessell, Mandy and Harald Smith, Patterns of Information Management, IBM Press, 2013, p.40
[REF-5] Chessell, Mandy and Harald Smith, Patterns of Information Management, IBM Press, 2013, p.67
Mandy Chessell and Harald Smith are authors of the book, "Patterns of Information Management", published by Pearson/IBM Press, ISBN 0-13-315550-1, May 2013, © Copyright 2013 by International Business Machines Corporation; for more info please visit the publisher page: http://www.ibmpressbooks.com/store/patterns-of-information-management-9780133155501