When you need to transfer very large amounts of data over the internet, sooner or later you will hit a limit where it will actually be faster to send that data on disks over regular mail (often called sneakernet). Internet transfer rates are simply not enough for large data sets.
Imagine a company with two offices in different cities, perhaps even in different countries. Each office has a 100 megabit internet connection. If the company needs to send a large amount of data from one office to the other, theoretically a 100 megabit connection can muster about 45 gigabyte in one hour if there are no bottlenecks on the way. This ends up being just over one terabyte of data in 24 hours.
In other words, for anything larger than one terabyte, it would be faster for this company to just send the data on disks for over-night delivery.
What does Google do?
Google’s initiative to transfer all the Hubble space telescope data is a good case study. The Hubble data takes up 120 terabyte (120,000,000,000,000 byte). How does Google transfer it? Not over the internet. Instead they send actual physical disk arrays via regular mail, something they have dubbed, for fun, FedExNet. This allows them to get the data within 24 hours.
To transfer the same amount over the internet in 24 hours, Google would have to be able to achieve transfer rates of more than 11 gigabit/s running constantly maxed out. On a regular 100 megabit connection, transferring 120 terabyte of data would take almost four months (111 days).
Is Internet 2 the answer?
The basic point here is that the internet still needs a lot more capacity. How soon can we reach such massive transfer rates? There is hope at the horizon, but FedEx probably hopes the answer is “never.” 🙂
