This note isn't so much about overall scale but what is here can allow a site to do more with less in some ways. It can also save you money. But, recently I was faced with a server farm that saw a pretty good increase in traffic. But, I'm not going to go into that so much. All I really wanted to point out with this post is that a little apache module add-on called "mod_deflate" can save you 66% on your bandwidth over time. Take a look at this screen shot from a monitoring tool.
There are a couple of down sides to mod_deflate you might not think of right away. It can cause a pretty dramatic increase in CPU usage on a busy site. So, be mindful that to use it effectively, you'll need a bigger CPU in some cases. Additionally, not all web browsers like mod_deflate. So, you'll have to content w/ the usual suspects. Finally, it's not a magic cure-all. You still have to write good code on the back end please. But, using mod_deflate with apache can certainly make things more efficient with regards to outbound traffic. Also, I do believe the lighttpd 1.5 has built in support for deflate and 1.4 can be configured with it as well.
Here is the most basic type of mod-deflate setup, for example only, that will compress a few number different files
AddOutputFilterByType DEFLATE text/html text/plain text/xml