One of the more inefficient aspects of IT provisioning is that enterprises must often overbuild the compute and storage capacity of their systems to accommodate periods of peak demand, with a lot of bandwidth going to waste during off-peak times. Such overcapacity and underutilization can become an expensive proposition. On the heels of a recession that has devastated the IT business and inspired more disciplined technology buying, most vendors have realized that they risk losing business unless they can deliver on a utility-like model, in which customers pay only for what they use.

It has been almost a year since Hewlett-Packard director of utility computing Nick van der Zweep explained to me HP's notion of Instant Capacity on Demand (ICOD). Back then, van der Zweep said, "Our vision is that some day, all computer resources -- the CPU, storage, networking -- will be connected to a fabric or grid and people will be billed for it on a usage basis. You might have your own data center, but if you don't have enough resources, you could get them from next door or somewhere on the other side of the world. The more you use, the more you pay."

How much closer is HP to realizing this dream today?

According to van der Zweep, HP is now demonstrating how the aggregate processor resources on an HP-UX-based SuperDome server can be re-allocated dynamically not only between applications running within a virtual server on a single system, but also between virtual machines on the same system. The next obvious step is to go outside the system. In a perfect world, instead of capacity planning on an application-by-application basis, or even a machine-by-machine basis, the dynamic allocation principal behind utility computing allows enterprises to plan for the maximum amount of capacity that might be needed at any one time across an entire enterprise. Currently, an HP customer whose requirements are exceeding the "utility's" maximum capacity can activate more resources on a pay-as-they-go basis, and then deactivate the surplus resources when they are no longer needed.

HP isn't the only company addressing the overcapacity problem through aggregate capacity planning, dynamic re-allocation of resources, and the publication of idle compute and storage resources to the network. During a recent briefing in Boston, Jonathan Schwartz, Sun Microsystems president and chief operating officer, explained how Sun is contemplating, through its N1 grid technology, how enterprises or departments could make their spare compute cycles available to other companies or departments for a fee through the Internet or local area network, much the way that electricity-generating windmill owners can sell their surplus energy back to the local utility company.

But many challenges remain before the dream can become a reality.

van der Zweep said that the goal of HP's ICOD technology is to deliver both the execution technology that allows for dynamic re-allocation of resources as well as the management technologies that handle security, provisioning, and -- most important of all to budding on-demand aficionados -- the billing infrastructure that facilitates the entire pay-per-use model.

Earlier this month, at Gartner's Symposium/ITxpo, HP demonstrated how the dynamic re-allocation of compute and storage resources within an HP SuperDome server is triggered by the execution of business rules rather than by systems management applications.

Through HP's Workload Manager, business and system managers can describe the priority of applications relative to other ones running in the same operating system instance, or the priority of virtual machines (VMs) relative to other VMs running on the same physical server. For example, in an eight-way HP-UX box, two VMs can be splitting the processors evenly until the higher priority VM (based on what applications are running inside of it) crosses a utilization threshold that automatically triggers re-allocation of two of the lower priority VM's processors to the higher priority VM. The result is that, with six processors, the application in the higher priority VM isn't starved for resources and, with two processors, the lower priority VM just takes longer to complete its task. Within VMs, HP's Process Resource Manager works almost identically, but instead of dynamically reallocating physical CPUs, its compute resources are divided based on a percentage of the total compute resource available to that VM.

Although an HP SuperDome server's ability to dynamically allocate resources isn't trivial, it's only a baby step towards standardized grid-based delivery and chargeback for compute resources the way electrical power is delivered over a grid using a standard chargeback unit (the kilowatt). For starters, such grid-like DNA cannot be found in most applications. It's one thing for a system to recognize that its compute resources, as allocated to a particular VM or application, are crossing a certain threshold, and to scrounge around the system for more horsepower to assign to those processes. Within a system, the application that needs the resources is fairly guaranteed of finding the same type of compute resources that it's already depending on. To the extent that such re-allocations are tied to a billable model, systems generally have the necessary accounting models within themselves to track such usage. For an application to recognize that it is starved, and for it to seek out additional resources from somewhere out on the network, is a completely different story.

At issue are the basic fundamentals for resource allocation to any application, even in a single processor, non-grid scenario. Perhaps the easiest example for understanding how applications are tuned to the loads placed on them is to look at an application like a Web server.

Web server administrators know that responding to requests from Web users in a timeframe acceptable to those users is tied to the server's ability to sustain a certain load. Beyond the hardware configuration, that ability is tied largely to the maximum number of http processes (known as daemons in Unix-speak) that can listen for inbound requests, and how deep of a backlog the Web server will allow. In a Unix or Linux environment, these daemons are listed as distinctly separate processes, each of which consumes some amount of compute bandwidth. As the load moves up and down, a Web server will spawn and kill daemons, usually making sure there are spare ones to quickly service the next new connection from a client. When a Web server spawns more daemons, more compute resources are consumed and, as Web requests come in, those daemons take turns servicing those requests. At some point, if the number and frequency of Web requests exceeds the maximum number of requests that the server is configured to service simultaneously, the backlog starts to fill up and performance from an end user's perspective will degrade. In a worst-case scenario, browsers will time out.

Beyond spawning and killing daemons as necessary, one autonomic response is to look elsewhere for additional daemons that can service the inbound requests. In the same way that the original daemons were engaged in a round-robin of servicing Web requests within the context of a single system's bus, perhaps the network can be looked upon as nothing more than an extension of that bus. If one system is maxed out, perhaps another daemon on another system can service the request. And herein lies the rub. Sure enough, there could be another daemon on the network, or even out on the Internet, that's ready, willing, and able to service the Web request that finally broke the camel's back. But, in the simplest of such load-balancing scenarios, that spare daemon in waiting must have something in common with the others that have reached their limit. For example, can the daemon physically reach the content and, if so, will it have the required security privileges? What if the Web application involves dynamically served content via a scripting framework like VBScript, JSP, Perl, Python or PHP? Can the daemon's host even support execution of those scripts? The key here -- and a gating factor to an electricity-like, on-demand utility of compute resources -- is that the pool of unused compute capacity to which the stressed-out system looks for reinforcement must have some commonalities with the stressed-out system.

In the world of Web serving, such load-balancing is already commonplace within the confines of a local area network or by using high-performance server interconnects specifically for clustering systems. But try extrapolating that idea to an ERP application like SAP or PeopleSoft. Whereas the Web is almost like middleware, and HTML standards from the World Wide Web consortium make it technically possible for a Windows and Linux system to combine forces in a way that the two can divide the workload of a very basic HTML application, there are no such cross-platform standards with proprietary commercial software. Commercial software providers generally certify their wares to work on (and in conjunction with) extremely specific versions and revisions of underlying software, operating systems, and hardware.

According to HP's van der Zweep, if the commonalities needed to successfully defer a Web request to a daemon on another system are complicated, the deferral of an enterprise application's workload to another system is exponentially more difficult. Not only must the software be capable of a utility-like response where it can seek additional computing resources on the network or the Internet (which most applications are incapable of), but it's very likely that the computing environment behind the resources it's looking for will have to be a direct match to the computing environment that it's running on. According to van der Zweep, "it's not just 'go out and find me a copy of SAP that can take this request.' It's 'go out and find me a copy of SAP running on this specific operating system with those specific patches with that version of Java on top of a certain processor architecture.' "

Demonstrating how configuration diversity challenges the concept of scalable utility computing, a recent attempt by the University of San Francisco to produce one of the world's 500 fastest supercomputers by aggregating the computing power of 700 laptops fell short of the experiment's goal partly because of system diversity. The San Jose Mercury News reported that Patrick Miller, a computer scientist at Lawrence Livermore National Laboratory and a part-time USF computer science professor, said, "You'll never find a thousand identical laptops when you're begging and borrowing." The story goes on to compare the project to one where such diversity wasn't an issue. "Virginia Tech built the world's third-fastest supercomputer last year by purchasing 1,100 PowerMac G5 computers from Apple and wiring them together. That was relatively easy compared with [the USF attempt], because the G5s were identical to each other -- so no hardware problem had to be solved more than once."

The dilemma is akin to a household where the refrigerators, televisions, VCRs, stereos and other appliances might each require a different form of electricity. Not only would the utility grid need access an untold number of electricity types, but it would have to come up with some sort of protocol so that your appliances could find a matching source of electricity. Now put that idea against the backdrop of van der Zweep's or Schwartz's dream where other users with excess VCR electricity can publish the availability of that excess, and then your VCR, which needs electricity, must find all of them, select from among them (perhaps via auction), meter the delivery, and then work out the subsequent billing and electronic payment.

Provided enterprise applications could perform in the dual role of user and provider (giving and taking compute resources as needed), engaging in that sort of give-and-take arbitration (which applications can't do now) still requires a mutual understanding (which also doesn't exist) over such basic elements as what the billable unit of measure is and how it's measured. Imagine coming up with a unit of measure for some SAP-based compute time, let alone a single unit of measure that applies to SAP, PeopleSoft, and other enterprise software.

In my previous discussions with van der Zweep, it was clear that, whether HP was going to provide compute resources on a utility-like basis or users were going to buy and sell their excess resources to each other using the Internet, HP was struggling with what the universal unit of measure might be and how it would be applied in a standard fashion. After all, if vendors are moving to a utility model that can support the most common enterprise applications, the last thing a CIO wants to reconcile into his or her budget or payables is a bunch of vendor-specific billable units of measure.

With proprietary units of measure, IT shops might end up locked into certain vendors just to keep their cost accounting simple. In the name of such simplification, "number of transactions" was one guess that van der Zweep made when discussing what the compute equivalent of a kilowatt in the electricity world might be. But how does one decide exactly what constitutes a transaction, especially if the world heads to a more Web services-like situation where a transaction consists of many sub-transactions distributed among multiple systems? It's all very, very complex.

van der Zweep indicated that HP is looking to move beyond such guesswork, and is angling towards a billable unit known as a "computon," which is universally applicable to both compute and storage resources.

Setting aside for a moment whether companies like Sun and IBM might buy into HP's notion of a computon, perhaps the real answer to simplification (both in terms of billing and the types of electricity) lies not in coming up with a universal unit a measure that can be applied to an almost infinite number of stack configurations, but rather, in greatly simplifying the stack side of the equation first. Use middleware to take most of the elements of the computing environment (such as hardware and operating system) out of the equation as much as possible and base the units of measure on the remaining components.

Indeed, Sun CTO Greg Papadopoulos already sees a future without too much stack-related complexity. Suggesting that there might come a time when Java compute time is a commodity, Papadopoulos said, "We've got to build systems that are far more regular at the network layer so that when I walk into a data center, I can say, 'Oh, this looks very familiar, it looks just like the one I was in yesterday because it's built out of the same architecture and the same sort of organizing principles.' "

Currently, however, Papadopoulos is bearish on utility computing. "Right now, utility computing is fool's gold. If you don't go and work on the systemic issues, you actually end up in this perverse thing where if someone asks you to double the size of your utility, you have a failure," said Papadopoulos. "If you don't properly get the architecture right underneath, you'll in fact diverge on your operating costs. It should be the other way. You should be able to say 'Hey, I'm twice as big and I am just motoring along in terms of being able to deliver incremental cycles.' Incremental cycles should be practically free."

Papadopoulos insists that some form of middleware is necessary for compute time to be delivered like electricity. But before that can happen, he added, the installed base of enterprise applications must catch up to developers who, for the most part, are already developing at the middleware layer.

Not surprisingly, after several rounds of secret face-to-face discussions with Microsoft chairman and chief software architect Bill Gates that resulted in the recently-inked Microsoft-Sun détente, Papadopoulos came across as unbiased about which middleware technology--Java or .Net--the installed base has to follow. He estimates that while 80- to 90 percent of developers are working at the middleware layer , "80- to 90-percent of your installed base is not. It's running SAP and it's running the old client server applications. SAP and other enterprise ISVs have moved their developers over to Java pretty seriously. But that's the new stuff. There still has to be a phase shift in the installed base [before middleware-driven utility computing is real]."

Sun's Papadopoulos was also unwavering on the long-term cost of not planning for a move to a middleware-oriented architecture. "If you aren't working to navigate across [the gap], you will fail and be cursed as a company," said Papadopoulos. "At some point, the people who aren't working on making that transition will be caught in the back commoditization of the old stuff. Since there's no opportunity to add new value to it, you're just going to drive it down."

In some ways, the SETI@home screensaver sets a precedent for a middleware-driven utility with accountability baked in. Systems that run the screensaver, thereby offering their spare compute resources to the SETI@home grid, don't have to be running on a specific operating system on a specific type of processor. Rather, the SETI@home servers divide the compute workload into 0.25 megabyte work units that are then parceled out to available systems. Although billing isn't a function, SETI@home has an accounting system for how much work those computers are doing.

"In a certain sense, we have a billing problem in SETI. One of the main motivators [to participation] is credit," David Anderson, director of the SETI@home project, told me. "Using a unit of credit we call a 'cobblestone,' we can account for how much work each computer is doing and we show where people are on the list in terms of how much work they've done. So, we have to do accounting, too. A cobblestone reflects computations, storage and network traffic."

Whereas Sun, HP and other commercial IT providers are looking to commercialize the utility model, Anderson was quick to remind me that even if the technological barriers are overcome, other issues remain before spare capacity can be bought and sold on the open market.

According to Anderson, the initial idea behind United Devices, a company where he was once CTO, was to build just such a billable compute utility based on the SETI@home technology. But, once United Devices' customers learned of the shortcomings of the SETI-like approach, the company had to shoot for a slightly less ambitious goal. "Our attempt to do a billable model didn't work," said Anderson. "One reason is the sort of companies that need that kind of computing power. For example, pharmaceutical companies will use computers to predict what chemicals will be effective against diseases. The problem is that if you run one of these programs that are assessing chemicals, then you reveal your intellectual property [the list of chemicals] and the results are impossible to hide on the computer doing the computations. The pharmaceutical companies backed out because of security concerns. Now, United Devices is concentrating on enterprises interested in the same principle, but within a company [instead of turning to the public]."

Another security problem Anderson pointed out is cheating. Even though no money is involved, Anderson said that SETI@home users will go to great lengths to overstate their contribution in hopes of getting a higher ranking in the statistics. After observing this behavior in SETI@home's user base, Anderson and his team had to employ countermeasures to ensure the accuracy of the accounting system. "For example, if someone claims to have stored a gigabyte of data on their hard disk, we must have some way to verify this."

Further evidence that a middleware-like software infrastructure is where utility computing will have to go before it's a reality is the successor to the current SETI@home software design. Anderson is confident that the Berkeley Open Infrastructure for Network Computing ("BOINC" for short) will represent an architectural advance. "[While] SETI@home combines infrastructure with the application... BOINC is developing middleware that separates those two things," said Anderson. "People would download the core client and then that client downloads and runs applications. This way, it's possible for a project to add, change, or update the applications that are running [on the distributed systems]."

Anderson noted that there are some fundamental differences between BOINC and traditional middleware like Java. "With Java, you theoretically write the code once and it can be downloaded and run anywhere; with BOINC, the code you download is compiled and is specific to your platform. The commonality is that it's dynamic. Code is getting moved around the network."

If SETI@home is the best real-world example of the marriage of a billable model to a middleware-like, platform-agnostic compute utility, how far from a commercialized version of the compute utility dream can we really be?

"The whole idea of completely and fluidly sharing computing power carries with it a set of technical problems that aren't resolved right now," said Anderson. "But this is precisely the problem that the Global Grid Forum (GGF) is addressing." When I mentioned the idea of HP's computon to Anderson, he chuckled, "The GGF has subcommittees devoted to this idea of the economics of computer sharing. There's a tendency for people to come up with old ideas and give them new names."

Will vendors like HP solve the utility problem before the GGF does? "The GGF is doing more talking than anything else because of competing interests of the vendors involved," said Anderson. "I suspect that companies like HP and Sun will get there first."

You can write to me at david.berlind@cnet.com. If you're looking for my commentaries on other IT topics, check the archives.