Outgrown image storage strategy, ready to bite the bullet and do it right

[Steve Uffman]

On a chassis for housing multiple hard drives for backup,

a techie told me of this: http://www.datoptic.com/usb3-esata-f...enclosure.html It is described as Five SATA tray-less hardware RAID5/JBOD enclosure with USB3.0/USB2.0/eSATA/FireWire400/800

option to have "Just a bunch of drives" appearing as one logical sounds "real" interesting to me as well as running USB 3.0 I have to think it through but might be real attractive backup without using proprietary software

Not the cheapest solution but pretty scalable....hmmm

[Steve Uffman]

Looks impressive, Steve.

I've been using Coolermaster chassis with 6 to 7-inch fans. The larger fans are much quieter. The cases that hold 4 or so drives with big fans are quite large though. The ones I get can hold 6 hard drives.

What motherboard? Impressive to get up to 64GB.

Roger

For anyone interested, I ordered my machine from a custom build shop today. The price delta of building it myself versus the custom shop building was not enough for the headache. And the shop regularly builds custom systems for Post processing at far higher levels than I expect to need.


First, this is not a machine for everyone. I am at the point in life where I measure my active lifespan in years rather than decades so no doubt I was indulgent. Still the ownership cost will be less that the two year desktop upgrade cycle I have been on.

There are a few spec changes from what I posted as I researched it more.

My criteria was to build a machine that had a high level of product fitness for my tasks at hand-which processing images is the most intensive. I also wanted a box that would allow me to save more PSD files with layers intact. ..and I want a solution that was scalable. As I mentioned earlier, the seemingly two year cycle of upgrading hardware is exhausting ...and in my former corporate life, machines had longer useful life if they still met appropriate levels of product fitness. This machine hopefully will do it.

So the new desktop has room for 5 permanent drives including the SSD. Operating system and all programs are going on the SSD. I am installling 3 3tb Seagate Barracuda drives at 7200 rpm with 64mb cache for the data.

And I wanted a better, faster backup scheme that I had now for the over 80K images, most of which are family) than I had now with the external drives that were USB 3.O and were 5400 rpm devices.

For backup I did significant research and interviewing. I ended up installing an integral Kingston 3 in two chassis that is Sata III which give me 9tb of swappable backups at 7200 Rpms running at 6gb/sec. Per a noted high tech but distinguished pro photographer, I will use MS synchtoy to do the backups. This system allows for me to easily make my off site backups with tool less swaps. Because it is integral, the footprint is only that of my desktop. And the unit has its two cooling fans which are in addition to the heavier cooling that I am having put into the desktop. Roger Clark described his cooling solution and mine will be similar.

To power all this I beefed up the power supply to 1200watts which will carry me to full expansion.

Another luxury, was that I am installing the full 64gb of memory...More than the 32Gb I had planned but for $350 more, it was worth the experiment even if there was no apparent yield.

Anyway felt I owed a report back for all the help so many here provided

Thanks again

Steve

[David Stephens]

What will you do for off site backup?

[Steve Uffman]

Belt and Suspenders philosophy

Plus one strategy which is weekly rotation of a hard drive offsite and bringing the oldest of the two offsite hard drives in to reuse...in other words ,always two offsite representing different backup periods will be kept locally.....

Preponderance of family images are jpeg and are currently stored offsite with smugmug. Those as you might imagination have high emotional value and could not be replaced as easy as a picture of a bear-especially since I don't use any of our other images as income producing assets.

Old drive from Dell desktop to be kept as archive per Roger Clarks strategy..I am not reusing the old 2tb internal on the new system. My existing system was attacked with a root kit and it was good advice to try to not risk "transmission" by reusing the spindle. But could be an option if catastrophic happens..

.Non proprietary backup of the archive using one of the external 3tb My books that I own will be kept offsite locally...


Will look at online backup as the plan of last resort...Relatively Cheap insurance but unless a case of catastrophe, a poor option to use to restore significant amounts of data..

Smug vault is an option which is Amazon for any high value images.. But will look at others such as Carbonite...

[David Stephens]

I think that having a offsite archive that's synchronized often is the sinle most critical procedure, no matter what hardware is used. I've got two 2TB external drives, one at home and one at my highrise office, that I keep synchronized. My weakness there is that I physically bring the drive from home and synch up monthly and that really should happen automatically daily. I need to look into software to do that.

I think that viewing Flickr or SmugMug or one of those types of sites as a jpeg archive is a valid assumption. I upload my Original size jpegs to Flickr. I've historically only uploaded what I wanted to "share", but I may rethink that and upload anything that I keep and just limit who can see it. There's a Flickr app where you can back up your Flickr account, just in case you want a backup of a backup of a backup. I suspect that there are similar apps for other providers.

[Steve Uffman]

I think that having a offsite archive that's synchronized often is the sinle most critical procedure, no matter what hardware is used. I've got two 2TB external drives, one at home and one at my highrise office, that I keep synchronized. My weakness there is that I physically bring the drive from home and synch up monthly and that really should happen automatically daily. I need to look into software to do that.

I think that viewing Flickr or SmugMug or one of those types of sites as a jpeg archive is a valid assumption. I upload my Original size jpegs to Flickr. I've historically only uploaded what I wanted to "share", but I may rethink that and upload anything that I keep and just limit who can see it. There's a Flickr app where you can back up your Flickr account, just in case you want a backup of a backup of a backup. I suspect that there are similar apps for other providers.

David,

One lesson learned about synchronization is a corrupt image ( or corrupt source code in my old business) can be synchronized in both places....also a limitation of many raid implementations..That is why I try to keep backups of at least two previous backups so hopefully I have not overwritten a good file with a corrupt file....the belt and suspenders..... (in my old business, we would keep at least 4 iterations of our weekly backup periods) Frankly here I will keep an archive up to a certain period like my Africa trip so I can always at worst go back to that trip and all before.

On your second point, I have two websites now ..one for family and one for hobby images.....On the family, I have all the family images that we want to keep...they are password protected in the galleries I want family to see...I also have some hidden galleries there as well that only I have access to.....
In fact my family website was done first in response to my son losing all of their family images due to Hurricane Katrina flooding in New Orleans. Pointed out why we needed an offsite backup strategy in a far away place.

Now something that I like about Smugmug is that I can download very large original images ( like you do) and Smugmug will do a rendering for the web. By doing so I know I have the original quality jpeg that is off site plus when folks want copies of the family images, I direct them to the imbedded printing service through smug mug- so they can get their prints but off of the better quality original image that Smugmug archives. Of course I have the print option off on my hobby site.

The Smugvault will take the raw files but I have not chosen to do that yet.

[Steve Uffman]

not sure if anyone is interested but just in case for reference purposes in case someone in the future is looking for a machine that is

-windows based but has a signature specially engineered for image processing using Photoshop, Lightroom and Nik to shorten image processing cycle times (processor, memory, ssd for programs and etc.)

- includes large internal storage capacities to store all the PSD layered files with layers intact that I would want (multiple HDD each 3tb at 7200rpm)

-but has scalable internal storage beyond my initial implementation

-has internal backup strategies that are at SATA III speeds but can be" toollessly" rotated and removed for multiple backups (ie. offsite)

-card readers at USB 3.0 and much more.

-6+ usb 3.0 ports for external devices if needed

-All within a single desktop footprint

-that is properly cooled and provisioned for future scalability

- and contains a power supply sufficient to allow the desired scalability......


I have detailed benchmarking if the scientists here are interested....but in its simplest form, the machine offers a top shelf Windows experience index of 7.8 out of a max of 7.9.

Image of the system and final specs which were revised from previous post are attached..