Global technology suppliers take their seats on the enterprise service bus

When Phoenix-based Avnet, a global technology distributor, began looking for ways to build greater flexibility into an IT environment dominated by legacy applications, it had to come up with its own description of the architecture it was envisioning: enterprise integration optimization.

Eight years later, the type of infrastructure Avnet developed is now called a service-oriented architecture (SOA). And at the heart of Avnet's SOA is what the company once referred to as a message hub, but which is now gaining prominence as an enabler of SOAs under another name: the enterprise service bus (ESB).

According to Larry Fulton, senior analyst at Cambridge, Mass.-based Forrester Research, ESB is an important component in many SOA implementations. “ESB brings a lot of little things to the table that make it much easier to move an existing IT infrastructure into an SOA world,” he says.

An SOA is an infrastructure in which software applications are broken into modular components, called services, and placed in a repository where they can be easily accessed by users or other services. When this is done properly, a company can implement new business processes almost at will by writing procedures that call for specific sets of services to interact with one another. An ESB is the messaging layer of an SOA. An ESB typically does the job of translating information created in disparate applications to a neutral format, and then routing those messages to their intended destinations.

Fulton describes an ESB as “middleware used in its purest sense to connect services in an SOA.” While it enables the inherent flexibility of the SOA by providing a communications infrastructure for connecting disparate applications so that services and resources can be shared, an ESB isn't always a necessary component of an SOA.

Indeed, some would suggest that an ESB—or even an SOA—could be overkill in certain situations.

“We've seen a lot of failures when customers just go about creating SOA because they want SOA,” says Archie Roboostoff, director of product management for NetManage, which has solutions for integrating, Web-enabling, and accessing enterprise information systems.

Roboostoff sees problems arise when companies undergo large ESB implementations before they have an actual business need.

“We're not opponents of ESB,” Roboostoff says. “We have many partners in the ESB layer. The problem we have is when a customer tries to do what we call top-down service delivery—meaning they just implement everything for SOA and they hope their users one day adopt those services.

“What we're saying is to start from a business unit, solving a specific problem using technologies [for encapsulating bits of application logic into services],” he continues. “Once you deliver on that one business unit and you want to leapfrog to other units, then bring in the ESB, because now you're driving an actual business need as opposed to some theoretical need.”

Among the ESB's virtues is its ability to hide what Fulton refers to as the IT plumbing that enables the often complex task of integrating applications that were never designed to work together in the first place.

It's a scenario familiar to Avnet, which started its SOA journey while contemplating how to extend customizable e-commerce services to its diverse array of customers and trading partners. Avnet wanted to do this without replacing its legacy applications, such as the homegrown, COBOL-based order-transaction system it had been using since the 1980s.

Yet connecting with each customer's and partner's ERP system using Avnet's existing point-to-point, static infrastructure was out of the question.

“We would be stifled trying to make all these changes in a hard-coded environment,” explains Avnet CTO Bill Chapman. “When we used the service-oriented concept, we could augment applications and not always be in the rewriting or revising mode with those applications.”

A large part of this flexibility was fueled by the company's in-house-developed ESB. “It was a message hub architecture that performed all of the functions of an ESB,” says Sean Valcamp, director of solutions integration for Avnet. “It did the translation; it did the transformation; it did the routing.”

Eventually the company decided to move from its homegrown ESB-type software to a commercial offering. “We used our existing architectural strategy to create criteria that would allow us to see which [off-the-shelf] products would fit best in an ESB for Avnet,” Valcamp says. “In doing so, webMethods ended up at the top of that list.”

Chapman says Avnet's ESB-based SOA architecture has made a big difference in how the company does business. He points to quicker ramp-up of trading partners, and the ability to develop new business opportunities as two examples.

“The architecture is a significant business driver,” Chapman says. “Avnet looks at IT and the services that IT provides as a differentiating value in the supply chain.”

Being able to leverage internal expertise and existing assets also are benefits of the architecture, adds Valcamp. “Our focus is on wrapping what [developers] have already built, and what Avnet has already invested in. It hasn't been to rip and replace the technologies that have existed for years. It's to leverage those technologies, and the people who are familiar with them.”

Avnet's story illustrates the rising popularity of the ESB/SOA model, which is catching on with large enterprises. According to a recently released report from Forrester Research, 67 percent of organizations with 40,000 or more employees implemented SOAs in 2006.

Fulton says this interest in SOA also drives growth of the ESB market, and is attracting new vendors to the space. “The number of ESB vendors is unquestionably growing,” says Fulton. “I have a list of 22. That doesn't include the [roughly half-dozen] open-source ESBs that are available today.”

With so many options, how does a company choose the right ESB for its business?

“People see differences between ESBs on a lot of fronts,” Fulton explains. “Some users will eliminate an ESB because they don't like its tool set, or they like how another ESB integrates its services.”

One way to understand the ESB market is to categorize the available offerings.

“You could bucket them in a couple ways,” Fulton explains. “Some have a broker-based technology where essentially everything goes through a central hub. Others have a differentiated topology where there's software on all the end points and they communicate directly as needed.”

Progress Software, however, would argue with that assessment. The application infrastructure software vendor, which claims to have invented the ESB category, says that a true ESB is defined by a distributed execution architecture.

Hub Vandervoort, CTO of Progress Software's Enterprise Infrastructure Division, argues strenuously against calling any platform with a hub-and-spoke configuration an ESB. He believes distributed execution should be included as a strict criterion when defining ESBs.

“That certainly was the definition we proffered early on,” Vandervoort says. And it's the architecture on which the company has based its own product, Progress Sonic ESB.

“There's this philosophy that's guided us from day one that basically says: Distribute wherever possible and centralize only where absolutely necessary,” says Vandervoort. “That's important for two reasons, and that's distribution and federation and all the underlying qualities that go along with that—reliability, scalability, and the ability to span security domains and large geographic boundaries, and even enterprise organizational boundaries.”

According to Vandervoort, enabling federation—the ability for individual business units or functional areas to maintain their own databases while still communicating with the rest of the enterprise—is particularly important for Progress Software's customers, which are primarily large, multinational organizations with a strong need to maintain localized sovereignty.

Oftentimes in an SOA-based infrastructure, companies try to implement a single database that all individual organizational subsets must access to enable their business processes—something Vandervoort describes as a “brute-force technique.”

“That may work on the theoretical level or even on a technical level, but what that completely ignores is the fundamental nature of organizations,” he says.

Individual groups within an organization—whether they are departments, business units, wholly owned companies, or loosely linked agencies—tend to fiercely guard their independence.

“They're not about to cede control of their data to one central database owner,” Vandervoort contends. “The back end that needs to be available in a single face to the [user] is going to be driven by data that is owned on a federated landscape where the owners want to remain sovereign.”

While Progress Software differentiates itself in part with its ability to support a federated organizational model, webMethods says its ESB-plus strategy is what appeals to its customers.

webMethods describes its flagship product—webMethods Fabric—as a business process integration suite. It offers a lightweight, ESB-only product, but Lance Hill, webMethods general manager for SOA solutions, contends most companies need more than just an ESB.

As the definition of an ESB evolved, says Hill, “what also happened is the realization that ESB is an infrastructure. It's plumbing, but in almost all cases in which you're implementing that plumbing, there are additional things you are doing in a project context to meet some business needs—and that requires additional technology to work with the ESB.”

For instance, while an ESB can transmit messages between services in an SOA, the ESB cannot create services. “For most large companies, a lot of the business logic they use to create services exists in applications they already own, but which aren't SOA-enabled. The technology to create services using those back-end applications typically has to be added to an ESB,” says Hill.

Despite potential confusion arising from the proliferation of ESB offerings on the market, companies such as Avnet believe in their value.

Avnet's Chapman even thinks they can play a role in changing the face of IT. He points to his own company as an example of how using the ESB/SOA model has enabled Avnet to reuse rather than rework assets.

“Everyone is talking about the cost of IT outsourcing and offshoring, and trying to drive those costs down,” says Chapman. “But if IT doesn't have to revisit and reengineer code all the time—and can spend more time developing new features and essentially augmenting modular components—that's a great win. That's a great win for American industry, and a great win forIT professionals.”