Here is a summary quote from an article by the Pythian Group worth reading to help people remember that bad design can hobble even the most insane hardware rigs.
"There is a time and place for hardware upgrades. The problem is that most people find hardware easier and just aren’t aware of how expensive their app logic really is. I find tuning easier and more cost effective."
The article explains an example of a database query and data model that brings a 10 node Oracle RAC cluster on a SAN to it's knees. After properly tuning and indexing the need for the 10 node cluster disappeared and it was replaced with a mere two nodes. I'd say that saved someone a few hundred thousand dollars a year.
Resources:
Good Database Design is Mightier than Hardware