Network performance of moving data in and out of S3 storage has come up from time to time for me. I've seen some data on the matter published here and there as well. Most recently I posted a short note to this article.
From a recent email from AWS, "The Amazon S3 team is now beta-testing support for an important low-level networking feature which has the potential to significantly increase the performance of large data transfers to and from S3, particularly (but not limited to) for long distance data transfers." This is being enabled by supporting TCP window scaling.
This all has to play itself out through testing and this is all rough data. But, the important bits from the note are that reported transfer speed improvements of 4x to 18x. It's not exactly slow already and bumping it up several times will be quite nice for sites that move a LOT of data and need to move it fast. Additionally, this is not a trivial matter to set up so let your systems administrator out of the basement to work on this one.
In summary, you can move more data faster in an out of S3 storage. It won't be terrible easy to set up but looks like it'll be worth the work.
My final thought is what will this do for performance on one of Amazon's own AMI's on EC2? I'll look forward to those test results!
Resources:
Note in the forum:
We're pleased to announce beta availability of TCP window scaling support ( http://tools.ietf.org/html/rfc1323) for Amazon S3. Using our beta endpoints, you should now be able to negotiate window sizes larger than 64k. In many cases, this will improve throughput for large transfers over long-latency links. To try this out:1. Enable large window support in your client/OS. This varies by client/OS, so please refer to local documentation for instructions.2. Use the s3-ws-beta.amazonaws.com endpoint. Note that this beta endpoint will not work with buckets located in the EU. This is a temporary location and this endpoint will be removed when the beta is completed.
We've set up the beta in the US, but will roll out the functionality in both Europe and the US when we make the feature Generally Available. We expect the beta phase for this feature to be a couple of weeks, so give it a try today and let us know what you think.
Thanks,The Amazon S3 Team
AWS Forum Thread: http://developer.amazonwebservices.com/connect/thread.jspa?threadID=18863&tstart=0