PDA

View Full Version : Rethinking DAM for 2009



Jim Poor
12-23-2008, 01:52 PM
Well, I think I just finished my last photo event of 2008, barring any last minute bookings. I'm taking a breather and trying to re-think my Digital Asset Management in terms of both hardware and file management.

I'll warn you, this is long, so thanks in advance if you read the whole thing. Even more thanks if you point out what you see as weaknesses.

This is what I have on my main workstation which is only important so that you how I'm storing what.
4TB internal - 1TB partitioned into 250GB system and 750GB data. The other 3TB are RAID which only gives a 2TB capacity.

20TB External - Two 3TB Concatenated disk sets and 14 other 1TB drives in a total of three external enclosures.

A few 750GB drives and smaller sitting around doing not much, but soon to be put into my secondary machine.

This is what I'm considering, though I'm open to suggestions.

This is my plan.

Swap out the 1TB internal drive for a smaller (32GB or 64GB) Solid State Drive for system & applications.
Break the internal RAID and make it a concatenated disk set to get the extra 1TB back. I haven't noticed much speed improvement with the RAID set. I may swap out those three disks for 1.5TB disks, but I’m not sure yet.

I had initially reorganized my Lightroom catalogues by year, but I have well over 100,000 RAW files from 2008, so I went to a current and past set up.

I'm not real happy with that either so I'm thinking about this in terms of file structure and breaking out catalogues:


Photos (Top Level on the volume)

YEAR

>Nature

>Pets
-Volunteer
>SPCA
>HART
>ETC
-Paid
>Private
>Event-Fundraiser

>Church Directory Project

>Everything else



The LR Catalogues will be YYYY-Nature; YYYY-Pets-Paid; YYYY-Pets-Volunteer; YYYY-Everything-Else; Church Directory Project


As far as managing actual shoots, I have been using this structure for the files within LR:

Event
>Last Name (RAW with the occasional Tiff composite)
>Last Name Proofs (Watermarked, low-res. JPGs)
>Last Name Prints (Actual ordered prints in full sized JPG, sharpened and ready to send to the lab, named with the image size inserted in front of the file name)

All RAW files are backed up during download in folders by date.
All LR catalogues are backed up in two external locations
All Photos from the "Photos" drive(s) are also backed up in their "organized" file trees in two external locations.

One copy of all RAW files and one copy of the file tree including the accompanying LR catalogues will be backed up off-site.

Thoughts?

Alfred Forns
12-31-2008, 08:47 AM
Hi Jim The best suggestion I can give you is to cut down on the keepers? Not sure how are you doing it but keeping 100,000 images from only one year will become unmanageable.

The biggest hurdle I had to overcome was editing. For some reason I wanted to keep even marginals which I would never use for any purpose. This goes back to the slide days.

Organization wise I wouldn't go by the year but by event or similar. Would like to see same events form different years like all your dog portrait sessions on one drive or all your rescue images in one drive. Would be difficult and inconvenient having to open and close catalogs.

One thing you might consider is going to DNG? One advantage I can see is the having no side cart so they will be easier to move around.

Hardware wise looks like you have a good plan, I like the redundancy !!!

Jim Poor
12-31-2008, 09:27 AM
Hiya Al,


Thanks for the tips. I do need to cull a bit more strictly for sure.

Even with grouping related photos on separate drives, I have to split the catalogue. LR bogs down terribly at about 30,000 files.

It sounds funny, but the wildlife & nature images are easier to cull than the pets. I only have to worry about pleasing myself for them, but with the pets, I have to think about what the owners like and may want to order later. I have considered only keeping the files that the owners actually bought the first time around, but I've had a few re-orders that included more pictures that they couldn't budget for the initial purchase.

I'm still looking for a good way to keep those numbers manageable.

Amazing how quickly it adds up.

Ed Cordes
12-31-2008, 10:31 AM
More and more this is becoming an issue for all of us. I have also had "issues" with culling out the marginal files. My wife is better at it than me!!

I don't want to hijack your thread Jim, but what drive enclosures are you using for managing multiple drives? I have looked ar Drobo but I am not sure I want to only have 3 TB of storage on 4 TB drives. However, the redundancy is nice.

Jim Poor
12-31-2008, 10:47 AM
Hi Ed,

Not hijacking at all. I think it is a natural flow to go that direction.

Anyway, I am using Burly enclosures purchased from Macgurus.com.

I've re-arranged things (yet again) since I posted this initially.

I switched to a Solid State Drive (SSD) for my system drive. I didn't get the Intel X25-m like I wanted (too pricey right now), but the one I got made a huge difference in terms of speed, heat, and sound.

That left me with only three open bays inside the machine, but as luck would have it, I had three 750GB drives that were doing basically nothing. So I moved my main image storage to the eSATA enclosure (just as fast as internal disks) and use the three 750GB drives as a back up. That keeps them from spinning up much and keeps my machine quite and cool.

Here is a grab shot of what they look like. The Burly enclosures are the black ones on either side of the printer and the little on on top of the computer:
http://www.pbase.com/jimpoor/image/104313548/original.jpg



Some day, when I'm feeling more adventurous and less thrifty, I would like to explore doing this modification:
http://echeng.com/journal/2007/08/30/6-terabytes-in-a-mac-pro/ and going to the more costly SSD, but it will have to wait.

Ed Cordes
01-01-2009, 06:48 PM
Jim, Thanks for the heads up on Macgurus. I have a PC and corresponded with them and they directed me to their PC site. I am currently researching the options. The Staff at the company seem very helpful. I will probably make a decision and set up early in the second quarter of 2009. Thanks for the input.

BTW, your work station look entirely too neat for me! LOL ;)

Jim Poor
01-01-2009, 06:53 PM
You're welcome. They have always been really great to deal with.

Don't worry, it hasn't looked that neat since the day I took the picture ;)