How ESBs Fill Web Services Management Gaps

While XML and WS-I standards continue to evolve to support enterprise-caliber web services, another piece is growing in importance. The Enterprise Service Bus is poised to provide needed manageability and visibility to service-driven integrations, says one seasoned Java and .NET technologist. Further, ESB architectures may provide interop options, as they begin to play big roles in Microsoft's .NET framework.

Tags: ESB, Manager, Business, Web Services, Technologies, Architecture, Enterprise,

Principal Technical Product Manager,

Increased adoption of web services and the virtualization of the underlying complexity of the systems, the Enterprise Service Bus (ESB) will drive the need for a way to manage those connected services in today's and tomorrow's enterprise IT environment. The ESB has become increasingly important as a method to tie all the pieces together in a manageable fashion.

The ESB Enterprise Portal provides the dashboard of the relevant data. The ESB as a COT package, or as a set of services offered through the OS, will continue to drive the SOA architecture and what it promises. Microsoft continues to incrementally add pieces to their .NET architecture resulting in an ESB equivalent suite of services that we find today. Historically, it had been a term predominately heard in the Java world.

As we look at the current business climate, stiff competition and the demand for near real-time access to the state of the business is now driving the need to map software solutions across a complex maze of enterprise systems. The challenge, however, is to link these islands of data, in a mission critical manner that avoids breakage through customization or changes.

As we all know, a large-scale swap of legacy systems for new systems is not a viable option. So, the real choice is to extend what is in place by applying new technologies that can provide the benefits of a modular system.

As we look at some of the emerging technologies, there are several likely candidates that will help ease this transition. It will not be one technology, but the application of several new technologies—taken together—to create an environment that is more flexible. Among the core ones are web services, XML and a growing list of interop standards from IBM, Microsoft, BEA and others (such as WS-Security, WS-Reliability and/or WS-Reliable Messaging, etc.)

But, there is another piece of the puzzle, the ESB, which looks to provide needed manageability and visibility to these service-driven integrations.

In simple terms, the ESB provides the management layer to tie all of these services together in a standard way. The resulting marriage of ESB, web services, XML and WS standards now provides the ability to leverage the various web services, talking to our legacy systems and former islands of data, and expose them in a way that allows aggregation and roll-up of information that can then be delivered inside a management console.

Tacking Strategic Integration
Within the Enterprise Web Services, ESBs

Why is this marriage of technology so strategic or important to the business? It is the aggregation of this information from those web services that make it strategic.

For example, poor sales in North America could be exposed through a management dashboard where the sales manager could then drill down and see that the volume of PCs is low due to price competition. With this new dashboard, the manager can now run a model on what it would mean to lower the price slightly below the competition to see what the impact would be to the bottom line. From the ad hoc query he sees that he can maintain her margin while still pricing below his competitors. He commits the change and the information is updated across their CRM, ERP systems all exposed as web services talking to each other through the bus.

So, in this example we have multiple systems that are feeding up this information, multiple web services, including services from other value chain partners. If we assume this was the 15th of the month, we have a manager that was now able to proactively manage her business that will have a net positive effect on her month.

This kind of scenario supports the concept of "real time" access to the business information. It also points out those independent web services, without a common backbone and messaging service, would not be able to provide the aggregation of the necessary information.

While the ESB is still a fairly recent concept, it continues to evolve and gain acceptance. The combination of both web services as connected services into an ESB now provides the ability to manage those systems and services in a consistent and integrated fashion through the bus.

With the ability to create reusable adapters, and consume those adapters, built on open standards, we now have the option for both the new systems, as well as the legacy systems, to be added in an incremental way that provides both continuity as well as extensibility as the business needs change.

Some of the big players in this space are IBM WebSphere, Sonic, Fiorano, and WRQ, all of which are Java-based and have their roots in the back office side of the Enterprise IT infrastructure. Through several products and services, Microsoft provides most of what you find in today's ESB; BizTalk, MSMQ, RPC and other technologies. Microsoft is also working on future technologies such as Indigo, Avalon and WinFS which will further enrich the services exposed in the SOA architecture.

ESB Provides Virtualization, Masking Complexity
A resulting benefit that this set of technologies provide, is the virtualization of these services. This new view of these formerly disparate services, can now be provide to the appropriate stakeholders through portals and feature-rich clients as key performance indicators with drill-downs in to the detail if needed.

The ability to wade through all of the available information to get to what is most important and relevant is even more valuable today as most people have too much information to wade through on a daily basis.

The resulting virtualization that the ESB provides then leads to the next requirement of today's enterprise IT environment. That is the monitoring and reporting on the state of those services. As the business intelligence is built into the adapter and bus framework, we can now leverage existing systems to track and monitor the health of these systems, with those services looking more and more like utility type of services that departments can subscribe to base on their needs. This virtualization also maximizes the physical hardware in a way that was not previously possible through load balancing, fail over and redundancy.

The "Developer View" of ESBs
From a developer's perspective, we see this involving the developer more through the use of design patterns and application blocks being provided. As you look at the SOA as a set of services which the ESB now connects, an OO type of approach to the infrastructure building blocks, where developers can now provide additional benefit and work inside those blocks, which the architect then assembles against the existing environment.

The result is you end up with greater alignment between how the developer builds and how the Architect looks at the problem. What we are seeing is a more "Agile" approach to the standard SDLC where the Architect and the Business Analyst pass down the design; we see it being much more iterative and collaborative as those blocks get built.

We also see the developer doing more architectural type of things as they build reusable components that match or create new patterns. These patterns allow any new project to benefit from lessons learned and commonality around projects that get distilled into best practices. This results in more time for the developer to focus on the "new" problems, not the same old problem over and over again. This also, over time, creates higher quality in the deployments and ultimately a more stable IT infrastructure.

The resulting ESB now provides the ability to add and manage both your new services, be they .NET or J2EE, or your in-house, custom built systems into an architecture that will provide a roll-up of those systems as services. We can now monitor and track the consumption and changes to those services as they relate to the business. This now comes much closer to matching, from a conceptual view, the business, and how work truly flows within the business.

As the year progresses Microsoft will play a much larger role in this space. With the next version of SQL server, .NET and Server, Microsoft will have a suite of compelling pieces that are necessary to meet the business needs that we described, agility, real time access to information and the ability to act on that information through the middle-tier and a SOA architecture.

In summary, the combination of each of these emerging technologies, taken together, provides the building blocks necessary to create virtual utilities of existing legacy systems. This roll-up and management provides the necessary architecture via the ESB for doing true cross-departmental, event-driven workflow, which in today's accelerated and highly service orientated environments becomes a clear strategic differentiator for organizations that are trying to control cost and leverage existing investments while at the same time provide true business agility.


Don Laursen is the Principal Technical Product Manager of Captaris and works closely with engineering, product management and key partners and customers to drive the requirements and plans to build the company's products into a complete, integrated, and flexible Captaris Business Information Delivery platform and product suite. Mr. Laursen has broad and deep technical knowledge about Microsoft technologies and the technology trends in the industry.