Storage For Photographers

Your Hard Drive Will Fail;

It’s Just A Question Of When

A yawn is the typical reaction from my colleagues when the topic of digital storage, archiving and backing up is talked about. That is, until one of them loses their work, because they weren’t paying attention to the need for a reliable back-up strategy. The yawns are then replaced by much shouting of obscenities and tears.

In this day and age of digital photography, the obsession of ever increasing megapixels, the need to shoot video and record audio, one thing is for certain; we need somewhere to store it all. The lazy and stupid will leave it all on their laptop until the disk’s full and then drag it off in a hurry, onto an external drive, usually losing stuff. Sometimes they’ll even be stuck on a job with no choice but to delete older stuff just so they can download the latest job and process it. I’ve even seen colleagues work straight of a CF card when in a hurry send a low res jpeg, forget to download the card and then format it. Disaster.

Although it’s a pain, at the end of a long day, it still pays to have a system. My personal way is to never format my CF cards until I have the same work on at least two hard drives. If it’s just on my MacBook Pro, then the CFs remain in my belt pouch until this has been backed up at home. I have a ThinkTank Photo Pixel Pocket Rocket which is always full of CF cards. I make it a point to have more than enough cards with me. I always have one small external hard drive with me too, so I might back up onto this. When on foreign assignments, I carry two external drives and back up onto these (these portable drives are never stored with my laptop. I’ll leave one in my luggage and have the other with me if leaving the laptop in a hotel or car). Once home, after a typical day in town or a trip, I’ll back up the assignment onto my Mac Pro’s Aperture Library.

I’m a big fan of Aperture and have it on all my machines. Apart from RAW processing, Aperture also acts as a fully searchable image database. At the moment I have more than 385,000 images in my Aperture library, and this number is constantly growing. RAW images are stored on an internal drive, in separate Project folders which are derived from the assignment. These are titled using a date and a name. An example would be “2009-09-28 Gordon Brown”. These Projects (with consolidated masters) are then backed up onto a Sonnet Technologies external SATA RAID box (more on which later). The same Project is also backed up onto an external hard drive which is kept off-site. It’s important to have off-site storage to secure the safety of data in the event of fire or theft. Lastly, important images, documents, video and audio are backed up to “The Cloud” (more on which later).

What’s New?

The old ways to back up were CDs and then DVDs. Blu-Ray doesn’t seem to have caught on, even though a double sided disc offers 50Gb of storage. The problem with optical storage, apart from the slow speed of writing, is that they don’t seem to have the longevity needed. I’ve had top brand CDs, kept upright in their cases and stored in cool, dark and dry conditions become unreadable after a few years. There is a 500Gb optical disc that’s being talked about, but again, how long is any media stored on that going to last? I for one certainly hope that it has proper archival stability, as it really would be useful to have.

My thoughts are that using several hard drives which are swapped out every three to four years, is the best method available. This provides speed and security. It also offers value for money, as drive prices continue to fall, with speed and capacities rising.                 

If you’re wondering why I seem to have an obsession with several hard drives, and a golden rule that everything is kept on three drives at least, it’s because hard drives fail. It’s an absolute given. Anyone in IT will tell you that hard drives fail, it’s just a question of when. I’ve personally had a major brand name drive fail after 3 months of use whilst sat on a desk, but had others which have worked solidly for many years. It’s always a gamble, and you should be well prepared.

At the moment, I have 12 external drives in my office and all the associated cabling and power bricks cluttering up the floor area. After some research, I recently decided to go for a Sonnet D800 Fusion RAID ( www.sonnettech.com ). This is an external SATA RAID box with eight drive bays. This means that it will hold eight hard drives, connect to my MacPro (its multi platform) using eSATA cables (connecting to its own controller card which installs inside your computer) which means that it’s blisteringly fast, and has only three cables; two eSATA cables and one power cable. To say that it’s a neat and tidy solution would be an understatement. I’ve recently finished moving my archive over to this system and will do away with most of the external drives. This also makes the office more quite and power-efficient.

I have the drives in the D800 set up in pairs of RAID 1 (also known as Mirror RAID). This basically means that everything that is saved on the drive (and you will only see one of the drives on your computer), is automatically copied onto its partnered drive. This is transparent, automatic and at the same speed. This protects the data from a hardware failure. I then manually back up data from this unit onto an external drive via FW800. This ensures that if by accident I erase an image, or an image gets corrupted, I can get it back from this back-up. These back-up drives are then stored off-site.

In use the system is amazing. Accessing images or video on the unit is blisteringly fast as it’s working over SATA, which is much quicker than even FW800. I’ve had the unit running for weeks without powering down and it’s been absolutely stable on the Mac Pro running Leopard.

The other new term you may have heard is “The Cloud”. This is virtual storage that’s kept on servers, somewhere in the internet, sometimes even in different countries. The Cloud’s not such a new thing, but with faster broadband, it’s now becoming more usable. Apple has had “.Mac” (now called “Mobile Me” www.me.com ) for years. Part of this service has been the iDisk which has been a virtual drive, available for use by Mac and PC users. I’ve been using this system for many years, and although a little slow, it’s been solid and stable.

The other Cloud system I’ve been using for around a year is Amazon’s S3 via Jungle Disk ( www.jungledisk.com ). This mounts a virtual drive onto your desktop (it’s multi platform) and allows you to use it like any other drive. It’s a faster system than iDisk and allows you to pay for the storage you use. At the time of writing, this is $0.15 per Gb per month. Your data is then saved on Amazon’s servers either in the USA or in Europe at locations which are not disclosed.

I’ve been using the Cloud in two ways. When on assignment, if I’ve shot a particularly important image, I’ve been saving them immediately to my iDisk. This has been for back-up purposes. Also, if I’ve got documents to which I need access, as well as having copies with me, I’ve also got them on my iDisk. This also includes email and FTP account details, which means if my laptop gets stolen, I can still function by accessing this information from another machine.                                                                                              

As my working year continues, every few weeks, I upload the edited pictures from assignments to my Amazon S3 drive. At the end of the year, like most photographers, I look through that year’s work and select my best work for competitions. Once I have this edit sorted, and the images processed to perfection, I take the contents of this folder and also upload it to Amazon S3. This gives me an off-site back-up of the year’s de facto most important and best work.

Lastly, a word on automated back-ups. I use Apple’s Time Machine to back up everything apart from my work images. This includes emails, invoices, letters, music, family snaps and so on. It’s saved me twice so far after I accidentally deleted important information. You can get back-up software for any platform, and I urge you to also have this system in place. For me, I use a separate FW800 drive for this purpose.

One thing’s for sure; as prices tumble for memory cards and hard drives, there’s no excuse not to have a solid and dependable back-up strategy. A little time spent planning and executing this strategy will save much stress and tears; trust me, I’ve seen enough colleagues suffer.

This article was originally published in the BJP on October 07, 2009.