Mainframe Integration Tips for Java, .NET

Java and .NET devs should look to a "runtime execution model" for using non-programmatic web services to leverage legacy apps, say experts at SD 2004's Best Practices event. See why these approaches, which offer quick and easy integration with mainframes, are boosting Java and .NET enterprise dev prospects.

Tags: Mainframe, Integration, Web Services, Guruge, Screen Scraping, Applications, Legacy,

Web services could give new life to old-fashioned "screen scraping" styles of client-to-mainframe integration, according to an enterprise apps consultant speaking at last week's SD 2004 Best Practices event.

Anura Guruge, an independent web services and legacy apps consultant from Gilford, N.H., told attendees at SD Best Practices that after massive spending in the mainframe software sector in the years leading up to Y2K, the sector has slumped in budget-constrained "apathy." But, he added, a new wave of web services-based approaches can be used to extend mainframe assets throughout the enterprise, sparking new interest in IT pros who can deliver "mainframe modernization."

The current climate, in fact, is presenting a unique opportunity for traditional web, Java and .NET developers and architects to add value to mainframe assets, without the need to learn COBOL, CORBA or other mainframe-centric technologies.

The Mainframe Integration Opps for Java, .NET
Guruge suggested that a "non-programmatic" technique like screen scraping can be an advantage, not a handicap, to Java and .NET devs: one they can wrap their arms -- and applications -- around. The trick for devs and architects who want to tap into valuable legacy data, applications and business rules, Guruge said, is to understand just how to construct a web service that can call to a legacy (proprietary) destination.

"The point is that you are developing new applications, but you aren't re-creating transactions that have been around for 20-plus years," said Guruge.

To succeed at this style of mainframe web services, integration developers must:

  • Identify the target mainframe transactions
  • Set up a SOAP connection
  • Call and retrieve the mainframe-side data, and
  • Merge the results with other elements of a composite application

"Screen Scraping" Grows Up
"Screen scraping" has become a somewhat derogatory term for many enterprise IT professionals, mainly because it's been associated with more than a few integration debacles over the past decade or so. Moreover, many programmers looked down at the idea of interacting with mainframes using calls that effectively emulated a terminal.
But Guruge insists that today's web services environment presents a whole new range of capabilities for using such calls,

But, rather than call the approach "screen scraping," Guruge prefers to describe the approach as a "runtime execution model." The key is that the approach is non-programmatic.

Inside the "Runtime Execution Model" Integration Approach
With REM, the dev isolates the applications he wants to invoke on the mainframe side, and then calls these applications into a "composite of applications" running in the environment the dev knows best; .NET or Java, for instance.

The integration is accomplished by the call to the mainframe app, which in effect becomes simply just another object in the dev's web service.

The approach is enabled by use of a host integration server such as those offered by IBM, NetManage, WRQ and others. When needed, the integration server steps through a mainframe access process -- "scraping" a succession of screens -- and returns results. The legacy application continues to run unchanged on its native platform.

The runtime execution model (which has also been described as a "host integrator" model) was well-established before web services and XML came about. Now that a call to the mainframe app is a web services call, the approach may acquire respectability, especially in an era when IT managers are looking for more standards-based ways to integrate with the mainframe, rather than replace it.

The types of legacy "applications" Guruge recommends for "runtime execution" are not those with clean separation of data and business logic -- these can be handled by data integrators and app servers. Instead, the recommended apps are those full of CICS-like transactions intermingling data and logic. This code has gone through many iterations in a long life, and is in turn difficult to identify.

"Y2K proved conclusively that finding source code is not as easy as you think," he said.

At the SD Best Practices session, Guruge said a key to REM's quick deployment arises from the introduction of what he calls "Host Integration Studio tools" that help developers capture the necessary mainframe log-in and authentication access procedures, as well as the general screen navigation [screen scraping] to access needed transactions. The tools, which may increasingly hook into framework standards such as Eclipse, also help the dev describe the system I/O in terms of WSDL.

Capturing of I/O screens occurs almost as with Microsoft Office application development, running and recording macros. WSDL descriptions are also built with help from wizards.

With this plethora of tools running in this execution model, what about performance overhead? The mainframe is not greatly affected, according to Guruge. But the web services engines cannot be underpowered. "You may not want low-end Pentiums running your host integration," Guruge said dryly. "Platforms in the middle must deliver [according to] mission-critical criteria," he said.

"This is a mature paradigm with advantages," he continued, "what was driven in the past from the terminal is now run by an exterior application."

One common mistake made by IT professionals doing legacy integration is not looking at the wide picture, Guruge said. "People get blindsided [by] not looking at everything that's out there. In fact, they don't have to restrict themselves to a single scheme," he said. "Start with something small, and manageable, but representative."

Bad integration design can derail any project, but better designs may be more widely obtainable with defined, standard web services implementations, Guruge suggested. "With things like CORBA -- which is still a very powerful technology -- there was complexity. With web services, you're getting standards infrastructure -- XML, SOAP, WSDL. And you simplify the invocation."

"Web services are simpler, at least in terms of a usage model; it's easier to understand and realize a model," he said.

Screen scraping should not be derided, said Guruge. noting that "it works, it's a proven technique, it tends to be simple, and it also allows you to combine transactions from multiple applications."

"In terms of drawbacks, well, here we are talking about continuing with legacy applications. With this technique, legacy applications never go away," said Guruge.

It's been said before: Big Iron platforms are not dead. "For the big volumes and workload, the mainframes still tend to be the ideal platform," Guruge said.