The amount of power consumed by IT is also becoming a huge problem, HP's Fink said.
"If we actually took the cloud and called it a country, on a power consumption basis, the cloud would be the fifth-largest country in the world. ... And, we are barely scratching the surface of the amount of data that's going to come at us," he said.
Managing the ever-increasing data center power consumption could be tackled by having a lot of smart people looking at how to better manage energy, but that would be the wrong approach, he said.
"We can continue down the path of how to become the best scientists on Planet Earth in terms of managing energy," he said. "Why don't we start thinking about the problem in terms of how we stop using energy in the first place."
HP, Fink said, is doing its part with the development of its Project Moonshot server, which takes advantage of mobile device technology to use 89 percent less energy than other servers.
Hp.com is one of the busiest websites on the planet, and HP moved a significant part of that site to Moonshot servers, which Fink called "HP on HP."
"We can basically power hp.com on the equivalent power of 12 60-watt bulbs," he said.
Nearly everyone in IT departments has been in a situation where the focus is on IT, with the problem of power consumption being "someone else's problem."
"As an industry, we have ... to look beyond the 'I'm measured on opex' or 'I'm measured on capex' or 'somebody else owns building my data center and I just put the stuff in it,'" Fink said. "And you really need to start forcing the conversation around looking at this holistically."
Another area where computer architecture is changing is in network connectivity, where new photonics cables can be used to replace the traditional copper cables currently in use.
Fink called thick copper cables the 'energy-sucking devil,' and said that the more copper one uses, the more energy it consumes.
"And our math right now suggests that we can get to a point where we will use 16,000 times less energy per bit to process [data] by moving from pure copper-based systems to photonics-enabled systems," he said.
NEXT: Rethinking Computer Storage, Memory