Storage For Photographers

Your Hard Drive Will Fail;

It’s Just A Question Of When


A yawn is the typical reaction from my colleagues when the topic of digital storage, archiving and backing up is talked about. That is, until one of them loses their work, because they weren’t paying attention to the need for a reliable back-up strategy. The yawns are then replaced by much shouting of obscenities and tears.

In this day and age of digital photography, the obsession of ever increasing megapixels, the need to shoot video and record audio, one thing is for certain; we need somewhere to store it all. The lazy and stupid will leave it all on their laptop until the disk’s full and then drag it off in a hurry, onto an external drive, usually losing stuff. Sometimes they’ll even be stuck on a job with no choice but to delete older stuff just so they can download the latest job and process it. I’ve even seen colleagues work straight of a CF card when in a hurry send a low res jpeg, forget to download the card and then format it. Disaster.

Sonnet Tech Fusion D800 RAID Sata 8 drive external box. Photo: Edmond Terakopian

Although it’s a pain, at the end of a long day, it still pays to have a system. My personal way is to never format my CF cards until I have the same work on at least two hard drives. If it’s just on my MacBook Pro, then the CFs remain in my belt pouch until this has been backed up at home. I have a ThinkTank Photo Pixel Pocket Rocket which is always full of CF cards. I make it a point to have more than enough cards with me. I always have one small external hard drive with me too, so I might back up onto this. When on foreign assignments, I carry two external drives and back up onto these (these portable drives are never stored with my laptop. I’ll leave one in my luggage and have the other with me if leaving the laptop in a hotel or car). Once home, after a typical day in town or a trip, I’ll back up the assignment onto my Mac Pro’s Aperture Library.

I’m a big fan of Aperture and have it on all my machines. Apart from RAW processing, Aperture also acts as a fully searchable image database. At the moment I have more than 385,000 images in my Aperture library, and this number is constantly growing. RAW images are stored on an internal drive, in separate Project folders which are derived from the assignment. These are titled using a date and a name. An example would be “2009-09-28 Gordon Brown”. These Projects (with consolidated masters) are then backed up onto a Sonnet Technologies external SATA RAID box (more on which later). The same Project is also backed up onto an external hard drive which is kept off-site. It’s important to have off-site storage to secure the safety of data in the event of fire or theft. Lastly, important images, documents, video and audio are backed up to “The Cloud” (more on which later).

What’s New?

The old ways to back up were CDs and then DVDs. Blu-Ray doesn’t seem to have caught on, even though a double sided disc offers 50Gb of storage. The problem with optical storage, apart from the slow speed of writing, is that they don’t seem to have the longevity needed. I’ve had top brand CDs, kept upright in their cases and stored in cool, dark and dry conditions become unreadable after a few years. There is a 500Gb optical disc that’s being talked about, but again, how long is any media stored on that going to last? I for one certainly hope that it has proper archival stability, as it really would be useful to have.

My thoughts are that using several hard drives which are swapped out every three to four years, is the best method available. This provides speed and security. It also offers value for money, as drive prices continue to fall, with speed and capacities rising.                 

External Seagate hard drives, with Western Digital drives in background. Photo: Edmond Terakopian

If you’re wondering why I seem to have an obsession with several hard drives, and a golden rule that everything is kept on three drives at least, it’s because hard drives fail. It’s an absolute given. Anyone in IT will tell you that hard drives fail, it’s just a question of when. I’ve personally had a major brand name drive fail after 3 months of use whilst sat on a desk, but had others which have worked solidly for many years. It’s always a gamble, and you should be well prepared.

At the moment, I have 12 external drives in my office and all the associated cabling and power bricks cluttering up the floor area. After some research, I recently decided to go for a Sonnet D800 Fusion RAID ( www.sonnettech.com ). This is an external SATA RAID box with eight drive bays. This means that it will hold eight hard drives, connect to my MacPro (its multi platform) using eSATA cables (connecting to its own controller card which installs inside your computer) which means that it’s blisteringly fast, and has only three cables; two eSATA cables and one power cable. To say that it’s a neat and tidy solution would be an understatement. I’ve recently finished moving my archive over to this system and will do away with most of the external drives. This also makes the office more quite and power-efficient.

I have the drives in the D800 set up in pairs of RAID 1 (also known as Mirror RAID). This basically means that everything that is saved on the drive (and you will only see one of the drives on your computer), is automatically copied onto its partnered drive. This is transparent, automatic and at the same speed. This protects the data from a hardware failure. I then manually back up data from this unit onto an external drive via FW800. This ensures that if by accident I erase an image, or an image gets corrupted, I can get it back from this back-up. These back-up drives are then stored off-site.

In use the system is amazing. Accessing images or video on the unit is blisteringly fast as it’s working over SATA, which is much quicker than even FW800. I’ve had the unit running for weeks without powering down and it’s been absolutely stable on the Mac Pro running Leopard.

The other new term you may have heard is “The Cloud”. This is virtual storage that’s kept on servers, somewhere in the internet, sometimes even in different countries. The Cloud’s not such a new thing, but with faster broadband, it’s now becoming more usable. Apple has had “.Mac” (now called “Mobile Me” www.me.com ) for years. Part of this service has been the iDisk which has been a virtual drive, available for use by Mac and PC users. I’ve been using this system for many years, and although a little slow, it’s been solid and stable.

The other Cloud system I’ve been using for around a year is Amazon’s S3 via Jungle Disk ( www.jungledisk.com ). This mounts a virtual drive onto your desktop (it’s multi platform) and allows you to use it like any other drive. It’s a faster system than iDisk and allows you to pay for the storage you use. At the time of writing, this is $0.15 per Gb per month. Your data is then saved on Amazon’s servers either in the USA or in Europe at locations which are not disclosed.

I’ve been using the Cloud in two ways. When on assignment, if I’ve shot a particularly important image, I’ve been saving them immediately to my iDisk. This has been for back-up purposes. Also, if I’ve got documents to which I need access, as well as having copies with me, I’ve also got them on my iDisk. This also includes email and FTP account details, which means if my laptop gets stolen, I can still function by accessing this information from another machine.                                                                                              

Screenshot from an Apple Mac running OS X showing Jungle Disk which is used for accessing Amazon S3’s Cloud service.

As my working year continues, every few weeks, I upload the edited pictures from assignments to my Amazon S3 drive. At the end of the year, like most photographers, I look through that year’s work and select my best work for competitions. Once I have this edit sorted, and the images processed to perfection, I take the contents of this folder and also upload it to Amazon S3. This gives me an off-site back-up of the year’s de facto most important and best work.

Lastly, a word on automated back-ups. I use Apple’s Time Machine to back up everything apart from my work images. This includes emails, invoices, letters, music, family snaps and so on. It’s saved me twice so far after I accidentally deleted important information. You can get back-up software for any platform, and I urge you to also have this system in place. For me, I use a separate FW800 drive for this purpose.

One thing’s for sure; as prices tumble for memory cards and hard drives, there’s no excuse not to have a solid and dependable back-up strategy. A little time spent planning and executing this strategy will save much stress and tears; trust me, I’ve seen enough colleagues suffer.

This article was originally published in the BJP on October 07, 2009.

11 responses to “Storage For Photographers

  1. Erm, when did you set up a WordPress blog in addition to your Blogger one? I came back to this post, via Google Search not not Reader, as I wanted to know if you have an opinion on the Hitachi G-Raid mini? http://www.g-technology.com/products/g-raid-mini.cfm
    I just changed over to an iMac 27″ from a PC laptop and am now looking at redundant storage options for real, not just thinking about it.

    All the best with the Ascough/Terakopian workshop!

    • Well, this is probably going to be the new platform; still trying things out. Naturally I’ll publicise it when it happens 🙂
      DO NOT GET THE G-RAID Mini!! It’s a RAID 0 which is the worst thing you can use for backing up. These are designed for video use and speedy access for video editing. For storage you need RAID 1, also called Mirror RAID.

  2. Thanks, I’ll avoid, although I read in the specs you could switch it to RAID1. Still doesn’t seem to be easy to change or swap out HDs. Your RAID1 system would be overkill for me, is there a smaller one you can recommend.

    I like WordPress a lot, it’s what we are using. I just didn’t find Blogger that good of a platform. But with WordPress.com you can’t use Google analytics.

  3. Pingback: Backup Workflow For Photographers « Photo This & That

  4. Pingback: Excellence In Customer Service | Photo This & That

  5. For those of you who have a budget, dlink makes a good line up of very easy to use small 2 and 4 bay NAS devices with raid 01 (or 05 on the 4 bay systems).
    Backing up to them is simple as drag and drop, or use of a script if you store your files exactly the same way every time.
    They start at about 100 dollars for the empty 2 bay devices and go up from there, but this means you can get yourself into a 2 Tb storage system with RAID1 (mirroring) for around 400 dollars (CAN)

    • Absolutely; however, I do prefer DAS (Direct Attached Storage) as this is much quicker for large photo files and even more important for huge video files. I did try the NAS route but things were just too slow. I think for low cost, the Western Digital MyBook Studios are great; two drives inside which can be set to RAID 1 and they have some pretty big capacities too!

  6. Pingback: Mirrored? Single? Manoeuvre! | Pointless vanity project of the former Tabascokid

  7. Great post on backup strategy. I use a similar plan although I least each job upload automatically to JungleDisk when I add it to my jobs folder.

    The comments about wordpress are interesting too. There’s essentially two types of WordPress installs. One on the wordpress servers which I think limit you in some ways to what you can do, or secondly install WordPress on your own hosting which allows a lot more flexibility and allows Google Analytics etc.

    • Thanks 🙂 I hope you still have physical backups as relying on the Cloud can have it’s own difficulties. For me, as I mentioned, it’s the fourth place for important stuff.
      Thankfully WP has improved and plays nicely with Vimeo!!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s