IT Architecture Transformation Articles -- Flashline Whitepaper: Integration, Transformation, Reuse

The key to unleashing the power of information technology lies at the intersection of three essential IT disciplines; integration, transformation and reuse. Integration projects connect business processes, applications, data and business partners. Transformation efforts analyze, upgrade, migrate, consolidate and retool application and data architectures to realign existing IT environments with critical business requirements. Reuse provides the philosophical foundation and management framework for integration and transformation initiatives.

Integration and transformation are essential to leveraging the enormous installed base of existing application systems to deliver quantifiable benefits to the business bottom line. A reuse strategy drives integration and transformation initiatives while ensuring that future application environments remain agile and responsive to ongoing business demands. This white paper introduces integration and transformation benefits and options and discusses how reuse plays an essential role in harnessing the power of existing application systems in major IT environments.

Recent CIO surveys[1] have consistently placed integration at the top of the list of goals for their IT organization. The reasons behind these findings are no mystery. Legacy application and data structures evolved from highly segregated departmental designs. These stovepipe infrastructures have become entrenched in organizations over the past thirty years and significantly hinder information flow across applications, business units and organizational boundaries.

Consider the U.S. Federal Government's efforts to tie together a large number of distinct agencies and functions under the Homeland Security Act. Each agency has their own computers, databases, files, users and application systems. The information used by individual agencies is stored in an endless variety of disparate formats. Semantic meanings across agencies are likely to be difficult to interpret - let alone consolidate. The bottom line is that integration is one of the primary responsibilities of this new agency.[2]

Other examples of fragmented, redundant information architectures abound. Consider, for example, a long distance company that has customer information scattered across multiple applications, data bases, business units and geographic locations. It takes multiple customer inquiries to reflect a change in service or provider. This not only annoys customers, but results in a corporate infrastructure that maintains multiple, redundant business units on a scale that costs the company millions of dollars annually.

Fragmentation, duplication and inconsistency are the hallmark of most information infrastructures and integration is key to dealing with these issues. IT can aim an integration effort at business processes, applications, data and / or business-to-business intercommunication requirements. Various approaches typically involve a Web-driven front-end coupled with some type of middleware technology to facilitate front-end / back-end integration.

A typical "non-invasiveī" integration response to the above challenge could be deployed on multiple fronts. Business process integration technology could be used to create a process-driven front-end that triggers multiple back-end applications to update customer data in a more timely fashion. This approach can involve application integration middleware that invokes back-end application transactions. An alternative approach would apply back-end data integration where data bases are synchronized through intelligent data reconciliation software.

Integration benefits include creating a more cohesive and fluid business processing environment while streamlining business responsiveness. Enterprise data becomes more accessible through Web-based interfaces, which in turn provides business environments with more timely and consistent information with which to perform their jobs. In the earlier example, customer service calls would be reduced dramatically and customers would stop being pestered by phone calls based on outdated information.

Unfortunately, non-invasive integration, where back-end application and data architectures are invoked but not modified or even understood to any degree, is limited in its ability to fulfill a variety of business requirements. In the earlier example, it is highly probable that the hardware, data definition discrepancies, application processing rules, business nomenclature and other factors are sufficiently diverse as to limit the effectiveness of a non-invasive integration approach. Application and data transformation takes over where integration stops.

Transformation can be applied in limited or far reaching ways based on the business and technological objectives of an enterprise or project team. The difference between integration, as previously discussed, and transformation is that transformation examines and unravels data and application architectures while integration does not. The transformation approach allows IT to realign application and data structures in situations where non-invasive integration cannot fulfill the requirements driving such a project.

For example, integration middleware technology would not be able to decipher and interpret the myriad data representations across the agencies being pulled together under the Homeland Security Act. Project teams will need to cross-reference, analyze, rationalize, redesign, consolidate and reorganize data in order for that data to be useful. In doing so, those same teams will need to interrogate the applications and the people that access that data. This is no small feat, but feasible based on various transformation tools and techniques.

The end result of any transformation project is driven by the business objectives for that project. For example, in the aforementioned long distance carrier's scenario, the business goals might be to consolidate and eliminate redundant business units and related support structures while streamlining customer service across the enterprise. Such a project could utilize a phased transformation approach as follows.

This is just one example of a phased transformation project. Again, the tasks and goals of such a project must be business-driven and technologically feasible. Transformation opponents argue that such an approach carries a great degree of risk. In reality, the phased approach inherent in a transformation project, which strings together a series of short-term, highly achievable projects into a larger initiative, is actually a lower risk proposition than many large-scale replacement efforts.

Transformation can be coupled with integration to deliver the best results for a given project. Front-end / back-end integration can, for example, be more readily applied to the newly consolidated application architecture than to the highly redundant and fragmented legacy architecture initially found in the long distance provider scenario.

Inherent in any integration or transformation initiative is the need for reusing portions of data or application functionality that already exists. Based on this premise, reuse and a reuse repository (see Figure 1) becomes the intellectual foundation as well as the management discipline behind integration and transformation. After all, if an IT organization did not see any value in integrating or transforming existing application and data structures, management would pursue "from scratch" replacement efforts as an alternative. Yet integration remains a top priority for IT executives. Reuse is, therefore, the de facto goal for IT and businesses because it is inherent in integration and transformation projects.

With this in mind, management will want to carefully define the role of reuse within these types of projects. In an integration initiative, reuse involves identifying the business processes, legacy transactions or data that need to be integrated and building the front-end and middleware architecture needed to deliver results. For example, if a project team creates an order processing front-end that triggers a series of back-end, host-based applications, reuse of the source code that contains the order processing capabilities is a de facto reuse requirement. Without delving into the various approaches for accomplishing this type of front-end / back-end integration, suffice it to say that this is a common integration model for companies today.

Managing the growing number of integration interfaces is a major challenge. Integration projects require the use of pre-built or custom adapters that define the formats of the information and transactions being accessed. Pre-built adapters exist for application packages, such as SAP, but in-house applications, which account for more than 80% of the applications currently running in production environments, are more commonly required. Keeping track of the growing number of interfaces, adapters and tools being used by integration teams requires a reuse repository. In the absence of such a capability, an integration environment will quickly become unmanageable from a maintenance perspective.

Adapter proliferation is the dirty little secret of the integration industry. No one wants to talk about it because it is messy work and will likely result in future maintenance nightmares for companies over the long-term. One company had back-end adapters fail because their payroll provider changed interfaces. This shut down processing capabilities for days until it could be repaired. A reuse repository could be used to track the development, proliferation and reuse of adapters. In this way, reuse becomes the strategic management tool for an integration environment.

Transformational reuse involves front-end analysis that is similar to an integration project. The differentiating factor involves determining if non-invasive integration can meet the demands of a particular business initiative. If it cannot, as was the case with the long distance provider scenario, then a transformational approach may be required. Under this approach, reuse plays several roles.

Rationalization and consolidation of redundant data and functionality is an underlying principal of the transformation approach. In other words, transformation embodies the antithesis of the ī"eplicate and modify" mentality found in many development projects. The folly of the replication model commonly found in development scenarios is evident when viewing the issue from a future maintenance perspective. If an IT organization needs to apply an interest rate modification, it takes much longer and is significantly more risk prone to apply such a change to hundreds of instances of that business rule than to a handful. Because transformation facilitates of process, data and business rule consolidation, it also facilitates reuse.

For example, transformation can be an important stepping stone for a company that needs to leverage legacy applications to create a functionally rich Web services environment. Under such an environment, dynamic reuse is triggered via a series of legacy business rules and data that has been captured, consolidated, componentized and made into a service. This would create a scenario in which essential business logic can be triggered from within (or eventually beyond the bounds of) the enterprise.

In other words, transformation views reuse as a deliberate goal that reverses the "replicate and modify" trend. The resulting IT environment is eventually streamlined into a subset of itself and becomes more readily understood, modifiable and agile. The reuse repository plays a key role in tracking these redundancies as they are discovered, consolidated and componentized. Here again, reuse is both the motivation for transformational consolidation as well as the management tool for achieving and maintaining a highly streamlined IT environment over the long-term.

In summary, viewing reuse within the context of integration demands and transformation requirements provides much greater value than simply viewing reuse within a "from scratch" development perspective. Similarly, reuse must become a central principal for integration and transformation planning and implementation teams. This will allow these teams to focus on the management, maintenance and consolidation implications as key components of each project. Only through this holistic approach can IT deliver essential agility to the business community while unleashing the true power of IT within their enterprise.

[1] CIO Magazine, Fall 2001, "State of the CIO"

[2] Homeland Security Act, Title II: Information Analysis And Infrastructure Protection, Section 201

System Transformation Portal