In a Few Years Petabytes Will Be Quaint

March 23, 2011

I got my first computer for Christmas, my freshman year of college. In addition to being far less powerful than a years-old smartphone, it came with a 420-megabyte hard drive. At the time this was pretty good-sized; MS-DOS didn't take up much room, and there was no such thing as online video. Even MP3s had just been invented the year before, and besides that my computer wasn't fast enough to decode them on the fly. So just under half a gigabyte of storage seemed like enough to last quite a while.

Technology advanced, of course, and a couple years later I had a Pentium with a 4-gigabyte drive to store all those MP3s I was downloading off the campus network. I still had the 420 MB drive as a boot drive, which Windows 95 took up a hefty portion of. That 4-gig drive? Cost me $400 back in 1997.

The computer went through a few more iterations and mutations, ending up with a 120-gig drive and 2 gigs of RAM -- meaning I could load that original computer's entire hard disk into memory four times with room to spare. I was starting to look into external drives for video, because really, how much stuff can you fit into just 120 GB?

Then the PC's power supply died and I moved to Mac for about three and a half years. When its own hard drive got full I rsync'ed everything onto a new 320 GB drive and left myself plenty of room to collect things.

After Firefox killed my Mac, I went back to the PC platform and found myself a nice, big 2-terabyte drive. Lacking a useful purpose for all of that space for the time being, I decided to help these nutjobs rescue the second-largest video site on the internet before the fuckweasel owners pull the plug and delete everything.

When the 1 TB virtual disk started getting full, I realized I'd need: (1) more space and (2) a backup of what I'd already downloaded, since none of my files have been uploaded to safekeeping yet. Whip out the external drive I'd been meaning to do my Windows backups on, and fire up rsync.

Wanna know how long it takes to rsync 864 gigs over USB2? About 48 hours.

So anyway, that's where I am now. I'm continuing my share of downloads because Yahoo hasn't pulled the plug just yet. As of right now I have this:


Filesystem Size Used Avail Use% Mounted on
/dev/sda1 1006G 864G 91G 91% /
none 493M 228K 493M 1% /dev
none 500M 260K 500M 1% /dev/shm
none 500M 96K 500M 1% /var/run
none 500M 0 500M 0% /var/lock
/dev/sdb1 1.8T 1.1T 657G 63% /media/ArchiveTeam

Meaning that in a little shy of four weeks, I've managed to fill two-thirds full a hard drive. A hard drive that's more than 4,700 times the size of my original disk. The one that I couldn't imagine filling up any time soon.

March 21, 2011April 8, 2011