Business Modeling and Outside-In SOA

Document ID: ZAPFLASH-20061012 | Document Type: ZapFlash
By Jason Bloomberg

Anybody who has ever attempted to teach computer programming to a room full of tech novices knows that some people easily get it, while others simply don't. There's something about the basic "do this, make a decision, then do that" execution flow of any program that is comes naturally to some, while other people simply don't seem to be wired in such a way as to make sense of such algorithmic thinking. As it happens, the algorithm-resistant population is every bit as clever as the programming-friendly group; it's just that they think differently about the behavior of systems and thus choose to represent them in different ways than their more technical peers.

The underlying observation here is that human beings are marvelously adept at representing arbitrarily complex concepts with a wide range of different kinds of models. In fact, our innate ability to model reality is so pervasive in our day-to-day lives that the modeling process tends to fade from our awareness. Nevertheless, whether in business or in any other aspect of our lives, we are actively dealing with the complexity that surrounds us by building internal models that represent various aspects of reality. Furthermore, the types of models we use are amazingly diverse, as each of us leverages different types of models for representing the full spectrum of experiences we encounter in our lives.

As organizations look to Service Orientation to provide best practices for leveraging information technology (IT) to provide agile resources for the business through the power of the Services abstraction, the ubiquitous, varied, and subtle nature of the human ability to model complex situations becomes a central concern. After all, the abstractions that Services represent are a type of model themselves, as are all the ways that people think about using Services in their business processes. Furthermore, the dichotomy between people who are comfortable thinking algorithmically and those that aren't resonates well beyond the classroom, with the algorithmically inclined leaning toward careers in IT, while people who prefer other types of models gravitating toward business roles.

This natural division of aptitude presents a challenge for any organization implementing Service-Oriented Architecture (SOA), because Service Orientation brings together these disparate ways of thinking about modeling, and requires IT to support the variety of models that business people prefer. Indeed, technical approaches to business modeling have fallen short of addressing the needs of business people who tend to think in non-technical terms. As a result, learning how to accommodate the different ways that people in the organization think about both business and technology is a critical best practice in the Service-oriented architect's toolbelt. Without this understanding, the business is unlikely to extract the value and flexibility from the SOA implementation that they desire.

Understanding Models in the Business World
Ask an average cubicle dweller who's not in IT to break down what they do every day into its most basic elements, and you're likely to solicit a list of human interaction-based activities that move information around: conversations, meetings, sending and receiving communications, reading and writing, and the like. Knowledge workers might throw in activities like analyzing or evaluating. Few business people, however, will reply that they spend their days executing business processes.

From the IT perspective, however, if you define a business process as "what a business does," or even as "a set of activities intended to achieve a business goal," then all the various activities that our cubicle crowd undertake fall into the category of executing business processes. In fact, ZapThink frequently espouses the perspective that the Service-oriented approach to business process calls for flexible business processes that respond to the way humans work, rather than processes that constrain humans to work the way the systems want them to work. The problem is, the various models IT uses to represent business processes -- flowchart-based orchestrations, flexible choreographies, defined flows -- are only a small subset of all the useful models that businesspeople might use to represent the way that they understand how the business actually works.

In fact, businesspeople have a rich, varied set of models for representing how they move information around: conversations, messages, documents, spreadsheets, Web interfaces, mind maps, flowcharts, agendas, schedules, calendars, and other loosely structured forms of information representation. To be truly successful with SOA, therefore, IT must support these multiple approaches for representing and exchanging information, instead of shoehorning business operations into an IT-centric representation of how techies think the business is supposed to work.

Taking Business Models to the Next Level with SOA
Models, of course, are core to architecture, and SOA is no exception. ZapThink's SOA Metamodel consists of a number of different models, including the business model, which represents the requirements of the business, the Service model, which represents both the Services in production as well as the requirements for new or changed Services, as well as implementation models that represent the underlying technology that supports the Services abstraction. Of these, the Service model is what makes SOA unique, and implementing the Service model properly is the essence of effective SOA.

The best way to implement the Service model is through metadata: Service contracts, policies, Service composition logic, and other metadata that represent all the salient aspects of existing or required Services and their use in the organization. Metadata are the appropriate vehicle for such models, because metadata support declarative representations of dynamic configurations. This declarative nature of metadata, in fact, is an essential enabler of SOA, and one of the primary reasons that the prevalence of XML has provided the raw material that characterizes today's SOA initiatives.

You might assert that the opposite of "declarative" is "programmatic," where metadata implement declarative configuration while executable code supports programmatic logic. The context of the variety and subtlety of the human reliance on models, however, blurs the distinction between declarative and programmatic approaches. Clearly, if you're writing Java or C# code manually, you're behaving programmatically, while if you're picking options off of pull-down menus on a Web page, you're behaving declaratively. But what if you're using an XML-based programming language like Water? The Web Services Business Process Execution Language (WS-BPEL, or BPEL) is similarly an XML-based execution language, as its name suggests.

Furthermore, a business user may be interacting with a fully declarative interface, but might type in some simple JavaScript or Visual Basic in a field to express some business logic. Indeed, while it's possible for a declarative tool to support pull-down menus that can duplicate, say, a line of JavaScript, just how important is it to avoid typing in a simple script? Furthermore, it's possible to argue that text-based programming languages are themselves a form of metadata that represent the underlying bytecode that the appropriate platform can execute.

Lest we allow this discussion to devolve into an unproductive battle over declarative vs. programmatic approaches, it's important to realize that both techniques are models -- and furthermore, represent only two among many different models people use for modeling the flow of information in the organization. Some people prefer to think programmatically, others favor declarative thinking, while still others would rather use a spreadsheet, rules engine, calendar, email client, or some other representation. Is typing a formula into a spreadsheet cell programmatic or declarative? Regardless of your answer, the point is that it doesn't matter, since spreadsheets effectively represent certain information-related tasks consistent with popular, well-understood models for such tasks that the business uses every day.

The ZapThink Take
There is an essential lesson for the architect here: design is an outward-in process, not inward-out. All good architects begin with humans and what they're looking for from the systems the architects are designing. Since Service Orientation requires an enterprise perspective on the interactions between business and IT, it's essential for the Service-oriented architect to be able to wear the hat of a business architect, and consider all the various models that the business is apt to use to represent the capabilities both of the business and the IT that serves it.

Furthermore, there is also an important lesson here for technology vendors as well: SOA tooling must be as diverse as the business users who wish to use it. People may utilize any number of different tools that leverage numerous models to take advantage of the Services their SOA provides. Gone are the days where tools vendors can manufacture only hammers, and expect their customers to recast all their problems as nails. By abstracting the interaction between business and IT, the business now expects IT to take care of business however the business sees fit, and not vice-versa.

High Performance SOA

Document ID: ZAPFLASH-2006920 | Document Type: ZapFlash
By Jason Bloomberg

In the world of information technology, the concept of abstractions are particularly handy. Take, for example, the Services abstraction at the heart of SOA, which masks the complexity of the underlying technology implementation while presenting composable business Services to internal and external users. But every abstraction comes at a price, and the Services abstraction is no exception. Loose coupling, composability, agility, and the other benefits of SOA all introduce performance overhead. For limited sets of Services with small numbers of users, this performance hit may be minimal. For SOA implementations with large numbers of users, Services, or traffic, however, maintaining the necessary performance levels presents a substantial challenge, both to the architects who design the infrastructure as well as IT operations personnel who are responsible for keeping the lights on.

In fact, in SOA environments with the highest performance requirements, maintaining the Services abstraction in the face of high traffic is a paramount concern. Fail to maintain the abstraction, and the Services no longer meet the agile needs of the business, and the quality of the SOA implementation comes crashing down like a house of cards.

Performance Beneath the Services Abstraction
The SOA performance problem falls into two broad areas: ensuring sufficient performance of atomic Services as well as of composite Services. Atomic Services provide Service interfaces that abstract existing systems, so ensuring their performance necessitates managing the performance of the components, applications, and systems that lie beneath the Services abstraction. As you might expect, dealing with the performance of atomic Services leverages well-established capacity planning and performance quality assurance (PQA) techniques, including clustering, virtualization, and load testing. Today's architects are adept at making infrastructural decisions that ensure, for example, sufficient database performance, distribution of traffic onto a cluster of application servers, and the like.

Furthermore, traditional PQA also serves atomic Services well. Simulating loads on Service interfaces is quite similar to simulating traditional Web page performance, after all -- and many Web Service PQA tool vendors have predictably based their products on Web page PQA tools that performance test traditional Web applications via their Web interfaces. But while Web Services share some similarities with Web pages, there are some fundamental differences. In particular, Web page interactions are usually request/reply, but Web Services support a wide variety of interaction styles, including asynchronous, synchronous, event-driven, publish/subscribe, and one-way. Load testing a Service that has such a wide range of interaction styles requires more sophisticated tooling than traditional Web page-centric PQA tools.

Performance Above the Services Abstraction
While SOA manifestly relies upon Services, there is far more to properly architecting SOA than simply building a bunch of Services. Architects must consider the consumption of those Services as well, including the dynamic, business-driven composition of Services into Service-Oriented Business Applications (SOBAs). Unfortunately, the very nature of SOBAs as flexible, continually changing, potentially ad hoc compositions presents complex performance challenges to architects and operations personnel alike.

In fact, there are several dimensions of SOBA performance that architects must consider as they plan their SOA:

• Balance between use and reuse -- Some services expect high use, meaning large volumes of traffic from specific consuming applications, while others expect high reuse, implying a range of different consumers that may use Services in different ways, for example, in different SOBAs. High usage is often (but not always) more predictable than high levels of reuse, but the architect must plan for both, as well as the combination of the two.

• Very Large Messages and Granularity -- Some Service interactions involve the exchange of very large messages (VLMs). Addressing the performance issue of VLMs requires different infrastructure and different planning from Service interactions that exchange large volumes of messages. Sometimes the VLM problem directly relates to the granularity of the Services, while other times it might concern SOAP attachments, encryption, or other features that increase the size of messages. But in any case, the architect must take the sizes of messages into account as part of the SOA performance plan.

• Dynamic Performance policies -- In some cases, the service level requirements of individual Services is part of the contract for each Service, but in other cases, the organization requires performance policies that they can apply to Services as part of their governance framework. In fact, being able to reconfigure performance policies may be a business requirement the architect must take into account when planning the SOA.

• Service dependencies -- Service compositions come in many flavors: orchestrated flows, flexible choreographies, data virtualizations, and various combinations thereof. String several Services together where the output of one contributes to the input of the next, for example, and if one Service in the chain is too slow, the entire composition suffers. Multiply this bottleneck issue by the numerous ways that people can create SOBAs, and the architect has a complex task ahead.

Tackling the SOA Performance Problem
Dealing with performance bottlenecks is like playing whack-a-mole: defeat one and another immediately pops up. Even worse, implementing SOA just increases the number of moles you have to whack. It's essential, therefore, for the architect to plan for performance bottlenecks at different levels, both above and beneath the Services abstraction. In other words, the architect must craft a performance plan that might take advantage of some combination of the following approaches:

• Service and infrastructure virtualization -- Various virtualization techniques can provide cost-effective approaches to dealing with variable performance issues, essentially by abstracting a specific part of the infrastructure. Virtualization is especially useful for dealing with unexpected spikes in demand, but the complexity of virtualizing heterogeneous resources can often limit such approaches' effectiveness.

• Combining judicious loose coupling with strategic tight coupling -- While a simplistic view of SOA might suggest that loose coupling is always better than tight coupling, the fact of the matter is that loose coupling introduces overhead, while tight coupling can smooth out bottlenecks. The architect's challenge, therefore, is in identifying those situations where the business requires some level of loose coupling, and then providing only as much as it needs, for example, by implementing transactionality in compiled code and distributing that code for high concurrency parallel processing underneath the Services abstraction.

• Registry-based dynamic routing -- A Service-oriented approach to load balancing leverages registry-based location independence to route Service requests to the appropriate Service instances in order to satisfy performance requirements. This dynamic routing approach to scalability and fault tolerance is unlikely to be as performant as tightly coupled clustering, but works well in heterogeneous environments, and can leverage the declarative policy management capabilities of the registry.

• Performance as part of the SOA governance framework -- Leveraging the registry's policy management features should be more than an afterthought, however. In fact, the broadest, most agile approach to SOA performance is to plan for it as part of the governance framework for the SOA implementation. Based on the performance constraints of the technology, craft policies that will maintain the required performance levels while empowering users as much as is practical. For example, if the architecture team can distinguish particular types of SOBAs that the infrastructure can support from others that it cannot, then the team should be able to craft policies that will appropriately limit SOBA creation, and thus create predictable limits for overall performance.

• PQA throughout the Service lifecycle -- Traditional PQA typically applies simulated loads before deployment to a QA environment that closely mimics the production environment. After all, such traffic loads can bring systems to their knees, so nobody would want to run them in a live environment. Maintaining parallel QA environments for SOA implementations, however, is difficult to impossible, because of the wide range of possible configurations of SOBAs and the Services that feed them. PQA for SOA must therefore take a more subtle approach than the "pound on it until it fails" technique. Instead, SOA PQA relies upon the use of configurable policies that indicate whether a Service is in live or test mode, combined with active SOA management that proactively monitors live performance and takes preventative steps should Services cross pre-set warning thresholds.

The ZapThink Take
Analyzing SOA performance highlights the fact that SOA is more evolutionary than revolutionary. Architects must still know how to use every capacity planning and performance enhancement tool in their toolbelt, only now they're able to add a few new tools to the mix. In fact, there's no way we'd be able to figure out how to scale Web Services if we hadn't already worked out how to scale traditional Web applications.

It's also important to note that SOA performance is about more than ensuring that Services perform as required, just as SOA is about more than building Services. SOA best practices also cover the consumption of Services -- within SOBAs as well as at the user interface. As a result, the comparatively mundane world of SOA performance has direct relevance to the sexy world of Enterprise Web 2.0. After all, no enterprise would depend upon rich, collaborative applications if there were no way to ensure their performance.

Finally, dealing with SOA performance requires an Enterprise Architecture approach to SOA. Those bottleneck moles in the whack-a-mole game can appear anywhere in the enterprise, at any level of abstraction. The fact that SOA hides the complexity of the infrastructure from the user only exacerbates the need for an enterprise perspective, because high quality, high performance SOA requires high performance from every part of the enterprise.