
Originally Posted by
Roger Clark
Hi Guys
I've been out hiking.
Steve, my first suggestion is why not upgrade your 1 TB drives to 2 TB? That solves your immediate space problem.
John, I heard reports of some OS's having trouble beyond the 2 TB boundary. I'm surprised Mac is one of them. XP apparently is one (perhaps depending on version). Windows 7 seems OK, although I have not used windows 7 much with 3 TB drives. I have been using dozens of seagate 3 TB drives under linux with zero problems.
I built the following system partly to solve backup issues:
I put together 2 I7-950 systems (quad core), one with 12 GBytes ram, the other with 6. I bought a case that can hold 6 drives, and with some adapters, 8. I have 8 USB ports for disks on the motherboard. For my main (the 12-GB ram) desktop, I have a 1.5TB system drive, two 2-terabyte data drives, and two 2-terabyte backup drives. Both systems were under $3K not including the data disks. For the backup system, I have a 1.5 TB system drive and two 2-TB backup drives. Then I have 3 sets of 2-TB USB drives for backup, two of which are kept off site. So on site I have the main system and 3 backups. I keep the backup computer off most of the time to protect it from power surges, especially in summer asd there is a lot of lightning here, and protection against internet attacks. First backup is the the second set of drives in the desktop. Then occasionally I turn on the backup computer and backup to it. I rotate the 3 USB backup sets (more when I have time than a regular backup). I do tend to do a USB backup cycle after a big trip. During photo trips I backup to portable USB drive and that I consider an archive. When the drive is full, I never touch it again--it stays on a shelf, so that is another backup of my photos from my cameras. This USB backup is the dumps from the memory cards on the trips, with few images deleted.
OK, so now how to manage this. Before my dual system put in place this year, I was running XP. I would do a hand copy of directories of files to a temp USB backup. My 3 backup sets: I would erase one and completely rewrite it for each cycle. A pain and took overnight per disk.
I'm now in the process of retiring the 2 TB USB drives for 3TB USB3 drives. I have had no problems, and between work and home I have had no problems with any 3 TB drive. I've moved to USB3 for the faster transfer speeds. On XP with USB2, I was getting about 18 megabytes/second. Under linux, I would get 32 megabytes/second (on the same hardware--dual boot). With USB3, and fast disks with linux I get about 96 megabytes/second transfer rates. So I've improved throughput about 5x what I was getting last year.
Data management is an issue and that added to the reasons for me to move to Linux. For photoshop (since it is not yet on linux) I installed virtualbox and windows 7 (natively) under linux. This gives me the best of both worlds (which for me is Photoshop CS5, ImagesPlus, and PTGui under windows) and everything else under linux.
I use rsync to backup from my main two drives to my backup drives on my desktop, use rsync to backup to my backup server, and use rsync to backup to my USB drives. It is simple, and very fast, and very very robust. Mac users can get rsyn to run under unix (may already be there). Windows users might try synctoy, which I understand is free from microsoft but I have not tried it.
An alternative for windows users. When you buy a new PC, convert your old PC to Ubuntu linux and make it a backup server. You can mount the linux disk on your windows machine using Samba services (I did that for many years). You can then easily copy files to the backup server from your windows machine and have better security. You may be surprised how fast linux runs on an old PC.
Another advantage I have found under linux is the disk health monitoring. The system monitors and reports how each disk is performing, including bad sectors. I have had (between work and home) 2 disks beginning to go bad: increased retries, more bad sectors, but no actual failures. The data presented to me allowed me to make decisions on when to pull a drive out of service before a hard failure. Very impressive. I have pulled 2 (2TB) drives out of about 20 that I use for various projects.
Note that if your machine is attached to the internet, it is likely getting thousands of attacks per day! A determined hacker might find a way in unless you are diligent about security. Under linux there are tools and firewalls to monitor attacks and block them. And it is all free.
Roger