Vijay Alagarasan

Vijay Alagarasan

Biography

Vijay Alagarasan is a Principal Architect for the Enterprise Architecture and Strategy department at Asurion, a leading global technology support and protection company (www.asurion.com).

As a principal architect, his focus areas are services, API management and integration architecture across different business units in Asurion. His contribution includes defining the vision, strategy, reference architectures, and reference implementations, patterns, principles, and best practices for services, API management and integration. He tailored some of the industry reference architecture and principles to the organization needs, through which it realizes full benefits. He closely works with other architects and development teams to drive services and integration in alignment with the vision.

Vijay is a hands-on architect. Before he joined enterprise architecture for Asurion he worked as a Principal Software Engineer, Senior Developer, Technical Lead, Solution Designer and Developer for Asurion, Cognizant Technology Solutions, and Clerysys Inc. He started his career as a J2EE developer and transitioned to a hard code TIBCO developer. He graduated from the University of Madras in Chennai, India with Bachelor of Engineering degree in Computer Science and Engineering.

His special interests include Enterprise Integration Architecture, Service Oriented Architecture, Microservices, API Management, RESTful WebAPIs, Containerization and Service/API Governance.

Contributions

rss  subscribe to this author

Bookmarks

API Management Introduction and Principles Published: April 3, 2015 • Service Technology Magazine Issue LXXXIX PDF

Abstract: This article is focused on providing solutions for common problems faced by large enterprises implementing SOA when they try to transform digitally as the market demands. It covers pain points commonly faced by such enterprises, and how API management product capabilities can be used to eliminate issues. In addition, the article also provides a set of principles to keep in mind when implementing API management for an enterprise.

Challenges faced by the Enterprises as they Transform Digitally

Today, enterprises that have adopted SOA in the past want their information assets and capabilities exposed without any boundaries as they transform digitally. Change at the speed of innovation drives enterprises to be flexible, adaptable, and accessible, with varying levels of security. Product development teams are aimed to get the new products into the market within a few weeks, and are prepared to fail fast and go back. On the other hand, service providers are trying to meet those new demands and timelines.

Large enterprises are desired to transform digitally and improve their agility in responding to market feedback. They want to get new features into the market within a week or so, just like startups. Enterprises want their minimal viable product (MVP) out before their competitors, and deliver the changes/features incrementally. This style of working demands small independent releases and favors A/B experiments in order to keep up with customer satisfaction.

While experimenting with new techniques, they must somehow preserve their established market share and revenue streams that still run on legacy applications where the core business rules and processes are bundled.

Most enterprises spent their dollars building SOA platforms in the last decade or so, and some of them are still in the migration process. Most of them bought expensive SOA suites, adopted one technology and built large SOAP web services. As their market expectations shift towards MVP, fail fast and faster response to customer feedback, first generation SOA principles are getting compromised at the service layer and the room for innovation is pushed out by a myopic focus on availability and stability.

It's not just application-to-application or business-to-business communication anymore. Today, it can be between thing-to-thing, i.e. machine-to-machine. To meet such demands, the services have to be mobile friendly, for example RESTful Web APIs. Large SOAP Web services are not mobile or machine friendly.

Some of the common challenges and question asked are:

  • Contract inflexibility by forcing consumers to adapt to single contract, i.e. can you expose the same service as REST?
  • Security mechanism inflexibility,
i.e. do we have multiple authentication choices? The same service is exposed to more than one type of consumer.
  • Data formatting inflexibility, i.e. can we expose the same service to the public and suppress PII?
  • Documentation gaps due to ungoverned service profiles, i.e. do you have documentation for your service? Consumers don't have time to talk to your expert.
  • Unable to allocate system capacity based on the consumer's value proposition, i.e. can services allocate more of their resources to agent applications than mobile applications? We don't to want to break existing customer satisfaction while we are trying something new.
  • Unable to govern consumers,
 i.e. do you know who is consuming what? Can we handle unknown consumers? We are planning to drive innovation from third party developers.
  • Trend analysis on consumer and provider behavior,
i.e. is there a way to block a consumer?
  • Forcing the consumer to upgrade due to service provider upgrade, i.e. why should the consumer change if they are not really impacted? They don't have time.

Now product teams are all of a sudden expecting service operations and development teams to do this, and of course, they don't express in technical terms. If they have not yet asked for your enterprise, they will soon ask for it.

So, how do we address these? It's not the best idea to solve these concerns for every service individually, or by the technology by which they were developed, or by the environment in which they were hosted. None of these options will drive consistency. It will become more expensive and will end up in more overheads to service providers and operations. Solving the problem may add more weight than the problem itself.

API Management is the Remedy

So, how do we enable the enterprises to reach their goals? An API management solution, which resolves these pain points in a centralized fashion, represents the Big SOA services as a slim-down API that has real, meaningful agility results. A range of enterprise-level API management solutions are now available in the market, like CA Layer 7, Apigee Edge, SOA Software, Intel Mashrey, TIBCO API Exchange, Microsoft API Management, etc. The enabling technology should at least include capabilities around:

  • Appropriate user authorization access in securing API and service endpoints
  • Dynamic routing and transformation for legacy solutions, including sensitive information protection in motion and at rest

  • Mobile platform access to legacy enterprise queuing mechanisms

  • Consumer quota management of resource consumption so shared services can guarantee a quality of service to all consumers
  • Consumer/provider usage monitoring to aid in incident resolution and operational readiness
  • Consumer self-service for service integration and consumption to accelerate new consumption models
  • Service cataloging for API and service capability discovery to encourage service reuse

API Management: Conceptual View

Each API management solution available in the market achieves these capabilities differently. Each of them has different logical and physical product architecture.

In general, there are four components:

  • API Gateway

  • API Developer Portal

  • API Analytics Portal

  • API Service Development UI

The diagram below purely represents the conceptual view of API management.

img

API Gateway:

It's a logical and physical entity that sits between consumer and provider (backend services). Service providers are not visible to the consumers anymore; therefore all transactions must pass through the gateway services or proxy. In general, gateway exposes the backend services, such as Web APIs. However, it's not limited to Web APIs; they can also mediate SOAP, JMS, XMPP, AMQP, and WebSocket services. These capabilities will vary based on the API product, so the right product should be picked based on the enterprise needs.

The key capabilities that it provides include, but are not limited to, the security, transformation, throttling, caching and routing. It separates concerns from service providers and allows the enterprise to manage them centrally. It also reduces cost for backend service providers and improves consistency across organizations.

API Developer Portal:

It's a self-servicing portal through which developers can register themselves to gain access to the Web APIs. Every API must be published to the portal. The primary goal of such a portal is to eliminate human dependency quickly onboard the known and unknown API consumers/developers. Of course, APIs should meet the consumer requirements for whatever product that they are aiming to build.

Some of the key capabilities that are accomplished by the portal are API Key Management, Consumer/User Management, Subscription Management, API Discovery, API Lifecycle and API Documentation.

API Analytics:

Every gateway proxy or service must be hooked up with at least basic analytics. Most API products collect inbound and outbound metrics as soon as the API is published or enabled, which is essential to measure the API consumption trend. Analytics provide transparency around consumer behavior and the backend service behavior, and allows the API owner to proactively identify and allocate necessary capacity and resources based on the consumption trend.

API Development UI:

It's a user interface where the API services are developed, deployed, un-deployed and migrated. The API development UI, in general, comes with several built-in policies that can be configured to build services. Policies are building blocks for a gateway service. However, when there are no built-in policies available for a specific need, developers can also build custom policies.

API Types

APIs are required to be classified based on their visibility and scope. This allows for layout of a strategy, security, SLA, and to measure their performance based on their objective.

  • Private APIs: These APIs are not visible outside of an enterprise. They are available only to the internal applications. Consumers are known. In general, the objective of these APIs is to drive down the cost.
  • Public APIs: Public APIs are visible outside of an enterprise, i.e. they are publicly available for anyone to use. There are known and unknown consumers. In general, the objective for these APIs is monetizing the assets exposed to mobile, Telemetry collection, and drive innovation.
  • Partner APIs: These APIs are visible only to partners. Consumers are known. In general, the objective for these APIs is collaborative business process and operation. These APIs sometimes fall under the realm of Private API when they are shared.

Applied Principles for API Management

For any new components that are introduced in software architecture, it's good to define the boundary conditions so that the solutions are implemented only for the intended purposes. This provides a scale to measure the solution quality and measure the criticality of the violation in terms of impact to the enterprise goals and benefits. Further, it enables technical debts to be logged and resolved as part of subsequent iterations. Likewise, there are 8 applied principles, which should be kept in mind when developing and implementing an API management solution for the enterprise.

Name Description Rationale Implication
API Gateway is lightweight API Gateway service includes the policies that are only related to: security, routing, caching, transformation and throttling. Having complex policy logics in API Gateway services degrades overall solution performance and will increase maintenance cost, leads to scalability concerns.

API Gateway services should not include complex policy logics (e.g.: XPATH transformation rules, parsing the message body for routing), transformation rules, and business process orchestration.

API Gateway layer may validate the consumers, apply data representation transformation, route the requests between backend services, and secure the backend services.

Transport Headers and URL parameters are the only ones that should be used for routing, throttling, and security policies.

API Gateway is Performant API Gateway services are designed and configured so that the API layer will perform with exceptionally low latency.

API management should not add latency in order to keep up with the customer satisfaction and existing QoS agreement.

Eliminates the need for every backend service to implement caching and throttling to improve performance.

Operations should monitor API usage and consumption and proactively add more nodes as the business grows.

Data from backend systems should be considered to deliver in lightweight formats for better performance.

Consider reducing the data transformation complexity to improve performance.

Measure and monitor the API gateway service latency and backend service latency to continuously improve.

Include API Gateway as part of the performance testing, establish the API Gateway service benchmark, and validate the performance requirements.

API Gateway availability API gateway is adequately available to handle traffic spikes, works around temporary failures in the enterprise backend, and fails gracefully in the event of a backend outage. The goal is to keep up with the availability requirements as required by the solution.

API Gateway should be highly available since it brokers all the transactions between the consumer and provider.

Mobile apps that leverage backend systems can lead to sudden growth in IT traffic that can result in crashes and consequent unavailability.

Availability and stability is key to establish consumer trust and adoption.

API proxies should include policies to throttle incoming requests.

Employ monitoring and alerts for the API Gateway and API Portal to ensure the availability of the gateway endpoints, backend service endpoints, gateway nodes, portal services, and portal nodes.

API Gateway services are built to handle the backend failures gracefully as needed by the solution.

API Life cycle must be Managed API lifecycle has to be carefully governed to provide guidance for creation, assure the quality, and evolve for the needs.

There is a need to manage the API changes gracefully for consumers to operate without any downtime.

Assures that APIs are rationalized at definition and development, and managed as an asset that has a lifecycle.

Ensuring the APIs for its adherence towards architecture/design principles and QoS that it is required to meet.

Define API maturity model and lifecycle stages with clear metrics that need to be obtained in each stage.

Run notification and upgrade campaigns to migrate developers to new API versions in order to keep API maintenance cost low.

Manage API catalog in API developer portal so that consumers can browse and understand the API availability along with its quality of services.

Track and highlight significant technical debts to executive leadership.

API Methods are accessed by authorized consumers API services include policies to verify consumer authorization. The goal is to protect APIs and from unauthorized access appropriately as defined by security requirements.

Managing the security in a centralized API management system reduces cost and improves security coverage.

Protecting the informational assets from unauthorized users improves enterprise credibility with partners.

Consider the API scope (public or private API) and employ the required security needs as needed by enterprise security.

Ability to block, blacklist and whitelist the API consumers.

API gateway should provide multiple options for consumers to authenticate and authorize, such as IAM systems, API key validation, mutual authentication, and service account-based access.

API Resources are Uniquely Addressable Principle Every API published under Asurion domain should have its own unique URL so that it can be referred to unambiguously.

Eliminates confusion and ambiguity about the API for the API consumers.

Having meaningful names for resources in alignment with business entity models gives visibility towards API supported business capabilities.

Every API has a unique URL and is categorized by business capabilities.

Every resource name in a URL should be a noun and meaningful, i.e. customer, address, order, shipment, incident, interaction, endpoints.

APIs are monitored for utilization and behavior Collect metrics and to monitor APIs for usage, QoS compliance, exceptions, performance, latency, and much more, as needed. The goal is to be transparent with the utilization and behavior of the APIs and their associated backend services.

Gain realtime visibility into API performance and trend analysis by tracking metrics overtime.

Align the API consumption with the consumer's value proposition.

Transparency around API consumption and how they are consumed.

Every API gateway service must be hooked up with basic built-in analytics such as usage, exceptions, performance, and latency.

Capture the exception information to measure the trend and drive those to resolution.

Determining the consumer usage patterns so that the required resources can be allocated accordingly.

APIs are Published with Documentation APIs are easy to understand and utilize. API developer portal offer interactive documentation and communication tools to make it easy to create and manage the applications built on APIs.

Improves agility, as the developers build and test APIs without human dependency.

Favors the atmosphere for building new solutions or applications by composing existing APIs faster and more cost effectively.

Create and manage technical and interactive documentation for all the APIs and their methods.

Self-servicing tools and processes are defined to perform user management, consumer management, key generation, and API provisioning.

Manage subscription plans for the consuming applications and users.

API roadmap is managed and published to all API consumers.

Conclusion

API management found its space in larger enterprises to keep them moving towards their digital transformation strategy. It adds a buffer to the transformation process so that the organization can migrate gracefully.

In addition, an enterprise should also consider investing in a thoughtful API strategy. This would enable the enterprise to monetize their capabilities and information assets, driving innovation from the third party developer community via Open APIs.