I was recently asked to make some general recommendations to a client for monitoring a MySQL database, memcached, and machines in general. So, off the top of my head, here is what I came up with. I'm sure with more thought it could be duly refined but, it's a decent list I think to get the ball rolling.
For monitoring your memcached instances Cacti is likely a good choice.
Cacti memcached Monitoring Sample:
General Systems and Traffic Monitoring you might like to use Nagios and Cacti. Installing these tools is not trivial. But, they are extremely useful one you get them online and configured. This will give you an excellent view into the inner workings of your machines, network devices, VM's, and services.
There are some virtual appliances out there at at rbuilder and vmware that we could use to get up and running faster w/o building from scratch. Just hit their sites and search for the applicance by name.
The other part of this is getting some sort of objective measure of traffic on the site. You can use, for free, Google Analytics. You can buy a copy of Mint and install it. You can pick up an account with FiveRuns also. That’s one part, the other part is server log analysis. The logs need to be copied somewhere central and processed using something like awstats or any of a variety of others out there.
Third party monitoring like WebMetrics or Gomez is nice to keep an eye in the sky on things and help track down some types of problems when they come up. I personally vastly prefer Gomez as a service but for some it does come off a bit pricey compared to WebMetrics. Gomez, are you listening?
Finally, for log data monitoring and searching you may want to investigate and invest in the tool Splunk after you do all that other stuff to be able to make better use of various types of log data throughout your network.
Finally, if you are indeed running a highly virtualized infrastructure then the extra money to go w/ the Enterprise level management tools by either Xen or VMWare could be worth it to you. Those are XenSource (web site is down right now so I can't get the link) and VMWare Virtual Center.