by
Kaleem Aziz
Two
perspectives to each: Technology and Business
ERP,
CRM, Mainframes, Connectors
Business
to Consumer commerce through 24x7 .com websites
Three
technologies to solve integration: CORBA, DCOM & J2EE
One
technology to integrate the three technologies.
.NET
vs. Web Services vs. REST
Future
predictions and warnings
Business organizations are preparing for yet another wave of innovation and change with the advent of Web Services and related technologies. This report provides analysis, recommendations, warnings and predictions related to these technologies. Two domain perspectives (1) technology and (2) business are explored alongside two time-line perspectives, namely, (1) past and (2) present. The predictions are the best estimates of the author, derived directly from analyzing the factors involved and are provided with warnings as to which key factors might cause it to proceed as predicted.
The only difficulties with innovation and change seem to be in getting the right information. And the only difficulty in staying ahead of the competition, without knowing what the competition is up to, is in differentiating the strategy. This strategy often depends upon the source of the information and what an organization plans on doing with that information. One of the most difficult aspects of adopting a new technology is the trade off between waiting for the technology to mature versus jumping in to stay ahead of competition by driving the innovation. Innovation in technology has been a classic reminder of how bad we are in understanding new technology, where the competitive advantage lies, the timing of when you adopt the new technology, and the architectural framework you build your solutions in.
Unless you are informed of the underlying factors (i.e., why of it) it will be a gamble to get the solutions (i.e., how of it) right. What differentiates a technology ending up being hype or becoming practical implementations is the understanding and appreciation in the industry about which business aspects it must use for solving rather than following the trend.
The greatest appreciation and impact of Web Services to business organizations is in Integration projects, also known as Enterprise Application Integration (or EAI). EAI has existed before Web Services, and therefore, our best bet to appreciate the essence and impact of Web Services happens to be through finding out how these integration projects were done in the past, and Web Services answers which specific needs of integration.
Two perspectives to each: Technology and Business
The challenge with innovation, especially in Information Technology, is the multi-dimensional complexity created by people perspectives. The technology may always stand the prototype tests, but it’s the people that skew the understanding or implementation. EAI has been no different in this matter. So, we will dwell into each of the following aspects with two major perspectives: technology and business. Managers understand technology partially and developers understand business partially, and we leverage these qualities in the report to provide mutual appreciation of the two perspectives. Finally, for each section, we provide a punch line that merges both perspectives in the recommended best practices and the warnings. We urge you to appreciate the factors rather than absorb the recommendations and risks.
The primary reason business depends upon technology is the need for speed. This need may be realized in summarizing, communicating or number crunching faster, but the basic factor is that businesses have been summarizing, communicating and number crunching even before technology took up these challenges to do them faster. Understanding business with and without a technology, shows the real advantages of the technology. The same analogy applies to EAI with and without Web Services.
In the most primitive sense, enterprises can be connected by hand-written letters mailed and delivered by a runner or messenger between two enterprises. What EAI brings is means to make this runner or messenger faster, while Web Services makes this runner reach more enterprises that agree to follow a particular type of agreed upon address and lookup convention.
Technology came to understand that it was economic to have server (a powerful computer) could do the heavy handling, requiring users to buy smaller computers and yet be able to do more. It had an added advantage that updating any changes required them to deploy the changes only on the server machine(s). Technologically, two kinds of systems were created: Fat-client and Thin-client. The fat client would require a more powerful desktop on the users’ end, but would have the advantage of more features, speed and seamless integration with other desktop utilities. The thin client could be accessed even from low bandwidth lines (like dial-up networking), but would have the advantages of more proliferation, scalability and extensibility in that only installation or upgrade would be required at the server side.
This mode of operation helped businesses provide their business online by hosting a server. The only prerogative of the business was then to keep the server up and available as much time as they could, by hiring an expert administrator of the server machine(s).
Not long later the advent of Client-Server computing paradigm, greater flexibility was achieved by having another abstraction in the middle layer. The need drove this technology, in that client and server were diverse systems. To make them speak, in the Client-Server era, required tight coupling of logic to handle as many types of systems and protocols as the market had accepted. Middleware, in the most simplistic sense, essentially provided a separate program to do this interface. This spawned the 3-tier architectures consisting of Client, Middleware and Server.
As the amount of configuration and maintenance of middleware was lesser than the amount businesses were forced to spend connecting the diversity to their servers, middleware era saw a great swing in seeing middleware abstraction as one way to drive costs down as well as drive revenues (through online services) up. Infact, it helped businesses to be flexible and scalable, which was a continuously growing need of businesses by this time. Businesses that started off their technology implementations at this time bought directly and heavily into middleware based solutions, thereby circumventing the problems of being tied into older Client-Server configurations and then upgrading them.
The problems middleware solved created a different set of problems, but at a higher level of abstraction. Middleware enabled enterprises to create gigantic technology solutions that would enable greater speed and efficiency. However, an enterprise that wanted to connect several of their 3-tier architectures at much larger levels of software saw a progressive need to integrate the problems created by diversity in such 3-tier systems. A software concept called Connectors was employed to connect these diverse intra-enterprise systems. Connectors were essentially program modules written primarily to enable cross communication between two diverse systems. Since each enterprise software company wanted to improve its sales and customer base, they provided connectors within their software to as many other popular enterprise softwares that market had accepted. As one can imagine, the number of connectors required to be developed kept growing with the diversity; so much so that most companies strategized to provide connectors either of major players, or provided only customized connectors for their most valuable customers.
Businesses saw a progress and the new problems posed by diversity in middleware were merely a higher level of implementation of building blocks that were much larger. The returns businesses would have by integrating several of their 2-tier and 3-tier architectures was based on competitive needs like turning out a product to the market faster; mostly by communication between the enterprise softwares. This stage was the true advent of Enterprise Application Integration (EAI), where means were sought to connect large business softwares residing in ERP, CRM, Mainframes, etc. Developing connectors proved to be a very costly affair, and the market soon started looking for better means to integrate, and parallelly tried to avoid these problems by developing even more flexible and pervasive software to startwith.
The next stage of growth was from customer thin-client forms to usage of network to communicate inter-enterprise. A form of trusted network (VPN) was created among large organizations that were interdependent on each other for supply chain tasks. Electronic Data Interchange (EDI) was a standard that businesses had agreed to communicate in, and some of the large organizations already had softwares that connected them with their suppliers and dealerships.
Automation of communication with suppliers and dealerships drove down the time and cost of operating. The improvement in cost was due to two reasons, (1) reduction of work force to deal with the numerous suppliers and dealerships, by creation of fewer and trusted suppliers and dealers
Simplicity of HTTP and internet, brought out customers in droves on to using websites; and technology responded to it by providing sale and advertising online through Business to Consumer (B2C) commerce. Diverse HTTP and FTP servers were developed by technology companies. The advantage of websites was that (1) they would be available 24 hours, 7 days a week, and (2) reached as far as the internet could (or a phone like could). Parallel development in infrastructure for internet and support for websites made it easier to develop thin-clients with HTML and JavaScript on the front-end client, and with numerous server side technologies on the back-end. Servers machines and application server programs were used to render communications, primarily with a live user, through HTML forms.
Almost at the same time as the enterprise wide computing engines, internet e-commerce technologies were maturing fast. To a large extent the affordable ISP charges, free website domains, simplicity of HTML, web e-mail, etc., increased the popularity of internet, and businesses looked towards their technology departments to increase their online presence.
Much of the server-side and subsequent logistic enterprise integration (with ERP, CRM, etc) was implemented by OMG standards called CORBA. However, Microsoft broke off the universal integration effort by providing proprietary technology, called DCOM, for its operating systems. This split in the market created a fissure in integration efforts, providing another set of diversity in the core technology platforms to implement standard integration. Java, a new language, revolutionized integration by creating a virtual processor on top of the hardware microprocessor. It standardized a set of byte codes for any system, and took up the challenge of implementing the standard byte code to behave the same on diverse systems through the ‘Write Once, Run Anywhere’ challenge. However, each of these technologies CORBA, DCOM & J2EE while great at integrating diverse systems failed in the problem of integrating any further due to lack of interfaces between themselves. Again, as they integrated at a certain level, they failed to integrate amongst themselves at a higher level of abstraction!
Businesses learnt a lot from technology and IT experience developing server side back-end and integrating their diverse systems. The development costs to integrate systems rose higher due to the specialized staff required for it. But majority of enterprises were content with the fact that such integrations were keeping them competitive with the rest of the market. However, in this arms race to get more and more technologically advanced; the technologies warped and complex. Also fissures in the standard integration methods proved the fact of the market forces: People, not tools, decide what’s best for them. That solutions, not technology, actually sells. Businesses that were locked into decisions made solely by technology pundits set themselves up to being sucked into the hype, whereas those that made decisions based on business drivers (rather than technology religions) performed better. Both Microsoft’s stand for proprietary solutions as well as the stand by Java and Open Source community to not co-operate with Microsoft segmented the browser market, ISP market, OS market, database market, development tools market and integration market. These divisions proved to be a failing point for technology, where competition was taking the toll over the industry’s need for collaboration. From business perspective the challenge for integration could never be solved until compliance could be achieved through mutual agreement. It was evident that further integration depended upon people factors (i.e., politics of the software vendors, through agreement in technology industry), while technologically proof of concept had already been provided by CORBA, DCOM and J2EE each, independently.
A meta-language called XML, which was a superset of HTML, was a developed as a simplified form of text and structure. Unlike Java, it had a shorter learning curve because it was being introduced in the developer base that was well versed with HTML, didn’t require a byte-code generator, didn’t require learning a complex language, and saw promise from almost every quarter of the now mature IT community. Subsequent development of XSLT (a powerful language to transform XML text, almost in ways Perl could on standard text) and XML Schema (rules engine to XML data, that itself was in XML) opened ways in which XML could be crunched without human intervention by means of simple written scripts. The advantages of XSLT and XML Schema over CORBA’s IDL, for example, were that much of the development would drive itself, without the requirement of hiring large number of IT developers to do the same. Technologically, XML gave the proof of concept of simplicity and flexibility, that the industry had learned succeeds in adoption and proliferation through historical experiences. However, the rise of XML also drove the need for far-fetched thinking architects who could foresee the profound design, automation of development and integrations that were larger than anything seen so far. Major breakthroughs were made in this respect, so much so that EDI like facilities could now be provided via untrusted internet-based network.
The rise of XML proved to be a universal unifier for businesses. (Although Microsoft initially broke off into devising its own XML standard ahead of the standard committee’s, it complied its browsers later to comply with the finished standard.) XML was applicable not only to rendering UI as HTML was, but also to integration tasks, primarily because it provided highly structured mechanisms. Simultaneous development of tools to operate on XML simplified the only complexity with XML, the bulk of data XML would create due to it being text. But since internet operated on a non-binary (i.e. text) base, XML was suited to take advantage of the existing internet for several business functions. A projection of what could be integrated in EDI style where large enterprises could automate several of their functions, given they trusted each other and succeeded in securing their communications, held the promise of huge savings (through the speed and automation of such technology). The only two factors that businesses found in their way to successful benefits was (1) people factors of trust and compliance, and (2) security from malicious attempts (hacking) that internet was known for.
With the growth of networking activity to the span of internet, the race was on technologically to proliferate and benefit from Business to Business (B2B) commerce. In this era, pioneering companies in these technologies identified a vertical market that could be connected by a set of common standards. Major stake-holders in the integration arena on the scale of B2B businesses started implementing solutions in a brute-force fashion, i.e. worked in each domain to identify the structure that the industry could agree upon and provided one solution for each industry. As can quickly be understood from history, the diversity of such industries would be a stumbling block in the future in integrating different industries. A few pioneers started implementing solutions horizontally, again in a brute-force fashion of finding the ways in which enterprises cross-communicated. Now that vertical and horizontal factors were taken care of, the only hurdle to complete integration at internet scale was dependent on huge investments to IT – due to the amount of CORBA, DCOM, J2EE and Mainframe programmers companies would need to employ inevitably. The extrapolation of this financial need showed a picture of companies spending trillions of dollars in becoming more efficient and competitive. Few companies wanted to be left behind, and the pioneering technology companies had enough work for a decade or so. But a few systems and companies still could not be connected because of the price-tag these integrations came at.
B2B commerce promised a new era of competition – with technology companies scrambling for a market share, and the other sectors paying exorbitant amounts to get B2B functionality. To many businesses B2B seemed like the industrial revolution done all over again with computing taking up almost every part of automation. However, the gold rush to be B2B complaint caused the number of providers to increase as well, so much so that the presence of numerous competitors forced companies to reduce the cost of their offerings. A diversity of standards and B2B pioneers created proprietary solutions, which would mean that at some stage enterprises would find themselves locked with certain IT offerings and locked out of some other offerings. But integration at internet scale meant enough benefits in the short term to worry of the proprietary solution issues right then.
The diversity of proprietary solutions would’ve created another need some day to cross-integrate them. Besides the amount of development effort and price-tag seemed too high for it to be simple and extensible enough. Microsoft came out with what it called .NET, which provided a service oriented architecture much like CORBA, DCOM or J2EE had, but that employed XML for all its functionality. The simplicity of this architecture proved that it could solve both the challenge of intra-enterprise integrations (EAI) as well as inter-enterprise integrations (B2B).
This simplification in integrations at EAI and B2B level was highly lucrative to businesses. So much so that the segment of enterprises that wouldn’t jump into B2B integration by brute-force now found it economical to integrate via a service oriented Web Service. Service oriented CORBA, DCOM or J2EE could not only not integrate inter-enterprise as seamlessly, but also it was complex and uneconomical to be undertaken at such a large abstraction. Web Services promised a means to hide the proprietary solutions underneath the enterprise, as long as organizations agreed to communicate in the industry standard XML (a little of which was developed during the brute-force B2B implementation attempts in the previous years).
.NET was Microsoft way of complying to the standards as well as providing a superior architecture that solved two problems of integration with one technology means. Web Services, a poorly coined term for alternative to .NET, by rest of the non-Microsoft community did not pose another rift in technology standards, because Web Services would only use XML as messaging – irrespective of which server side software the XML was generated from. This was a huge win in integrating the Microsoft and non-Microsoft technologies. Also, Web Services came with concepts of integrating business processes, namely, workflows, orchestration and choreography -- unlike how CORBA, DCOM and J2EE only integrated software systems while businesses were left to manage business processes at a higher level of abstraction. Technology companies realize now that network-based computing can be faster than the desktop computer’s computing power in near future: starting numerous drives like ‘Network is the computer’ by Sun, ‘On-Demand Computing’ by IBM, ‘Internet changes everything’ by Oracle, etc. Web Services, it was found, could be implemented also over the internet through URLs and hyperlinks (core philosophy on which internet is based), rather than messaging. This architectural style is called REST, but not many enterprises can use this style because their previous systems (that they look to integrate) are not URLs.
Within the industry, similar to the rise of Java, the hardware vendors saw this opportunity as a means to increase their sales – because irrespective of the hardware an organization purchased, it had the capability of being integrated with rest of their existing systems. On the integration front, businesses became bullish on EDI style integrations over the internet – the push from large players like IBM, Microsoft, etc., convinced them to want to have their systems integrated, if they could be guaranteed security. The attitude in the industry was that of wanting to solve the people factors of trust and security, for mutual benefits. The inherent characteristics of workflows, orchestration and choreography that Web Services provided further empowered business, rather than IT, to control the integration efforts. Microsoft, after losing the legal battle over Java with Sun, developed its own byte code engine for a language called C#, which is very similar to Java yet provides a few other development goodies like that of unifying several languages into single byte code (called MSIL).
Technologically, some innovations bring forth radically new concepts; for instance, the new concepts OO programming brought over structured programming, or the Java programming language brought over C++. In each of these cases, the biggest challenge was unlearning, i.e. forgetting the old concepts to develop the new systems in the newly recommended way. There have been OO programmers who have programmed in structured way with C++. As an industry, transitioning to XML will see similar problems as we saw with HTML – lots of software that will be developed will misunderstand the actual need for the technology, and instead follow the crowd. Understanding the core factors why these technologies were spawned in the first place often help developers visualize the business and market problems for which they are providing solutions.
Alongside the regular integration operations, one should expect to see development of tools for testing, monitoring and micro-payments. What these bring forth will dictate or guide other things in technology and business processes.
Internet sockets will be similar to water taps or electricity sockets. Failure of internet will almost be as catastrophic as failure of water supply or electricity. Not much is being planned for failure backup, and technologically this sets us up to a major catastrophe the first time that such an event happens.
Hacking, and protection against it, will carry higher stakes, as the losses will not be similar to that of identity theft of gullible individuals or loss of investment in B2C commerce. Mainframes are still known to manage financial traffic worth trillions of dollars, and similar traffic will be over the internet. There are bound to be messages that are forgotten to be encrypted either by the programmer, or decrypted by the hacker by sophisticated tools, traveling over the internet medium. Resolution of these may be required at a higher level of business abstraction.
Speed of XML operations is much slower than binary. BEEP, alternative to HTTP, offers integrations over internet that can be close to binary. However, the lack of adoption of BEEP forces technology to use the accepted text medium over HTTP. As has always been, technology is first expected (by business) to get things right, and then expected to perform more efficiently in the next stage. Expect to see a second wave of performance improvement projects.
As understood by where Web Services will ultimately lead us, technologists often predict the creation of Internet ver 2.0 or Semantic Web, where machines will browse the web instead of the current internet which is suited only to humans. Whether they will or they will not depends entirely upon our people factors. Technology, mostly hardware, is available for such capabilities but since it is a future priority for most businesses, such a Semantic Web, where your refrigerator orders milk automatically when it runs below a certain mark, offers thin economic advantage as of now. Especially, since the software infrastructure isn’t in place for Web Services to integrate enterprises, these consumer level integrations will have to wait until such time.
Businesses, needless to mention, have learnt that some of the technologies can be hyped up – creativity that is IT’s latest toy. What it holds for business had been arcane upto now. Web Services implementations can change that by helping organizations view their business processes while technology implements them. Besides no longer being as arcane as to how technology automates a business process and to what extent; this control will define a new meaning, in that the economic needs of business will drive the technology more directly than it has been before. The organizations that integrate business processes, rather than computer systems, stand to win. Standardization and security cannot happen without some level of trust in exposing business operations; and this trust is key people factor for success in enterprise integration through Web Services. At the same scale of integration, without the standards, Web Services offers little more than other alternatives.
While Web Services workflows connect business processes over internet, with the power of Choreography and Orchestration, one can expect technology to convert businesses into delegating numerous small business functions to subsidiary enterprises in real-time. This could mean that expertise in micro-solutions to be delegated to several internet services that are looked up and executed in real-time. Unlike today, large IT development teams may no longer be required on-site on enterprises that feature non-IT products, as the same solutions would be carried out through calls to services over internet.
Some form of insurance mechanisms will be required to safe-guard companies operating in the B2B space over the internet.