The GetDPI Photography Forum

Great to see you here. Join our insightful photographic forum today and start tapping into a huge wealth of photographic knowledge. Completing our simple registration process will allow you to gain access to exclusive content, add your own topics and posts, share your work and connect with other members through your own private inbox! And don’t forget to say hi!

RAID Storage ideas

GrahamWelland

Subscriber & Workshop Member
Another :thumbdown: for Lacie here too. I have a pile of Lacie disk units that eventually failed.

I personally use a combination of internal RAID using eSATA drives and an Apple RAID controller inside my MacPro, plus a secondary external RAID (AMCC/3WARE Sidecar) which backs up the internal drives.

Important other considerations - don't rely on RAID as a 100% reliable storage solution. You need to maintain a backup of the RAID!

If you go with a RAID solution I HIGHLY recommend finding one that can support 2+ parity drives such as RAID 6 or mirrored solutions like RAID 10. After having had a RAID power bus failure that lost 2 drives I can attest to how huge a pain in the rear it is to rebuild and recover. If I hadn't had an external backup of the RAID I'd have been totally screwed.
 

Dustbak

Member
I have specifically opted for the highest speed RAID I could get (RAID0). I need to process lots of files often and they are big files.

I have taken the calculated risk this will crash one of these days which is why I have another RAID0 (External) for backup).

Besides this I backup to separate drives which I take to another location.

RAID is indeed not a substitute for a backup!
 

ustein

Contributing Editor
On the Macs I am not a big fan of RAIDs (for archiving at least). I had once a file system fail on a raid (Drobo that is). Disk Warrior could find all files but not repair because the RAID put a layer between the OS and the disk level OS.

I think in most cases 2-3 TB single disks work. Raid 0 for performance is a different story though.
 

narikin

New member
I have specifically opted for the highest speed RAID I could get (RAID0). I need to process lots of files often and they are big files.

I have taken the calculated risk this will crash one of these days which is why I have another RAID0 (External) for backup).

Besides this I backup to separate drives which I take to another location.

RAID is indeed not a substitute for a backup!
Raid 0 whilst technically a Raid level, is raid Zero because there is Zero protection. one drive fails, its all gone. backup is your only hope at that point.

I am getting 650 mb/sec on my Raid array, sustained sequential read. See below. 1107mb/sec burst. That's 1.1GB per second. I doubt you could seriously want more. This was only 6 regular 2Tb drives in Raid 5. 8 drives would be even faster. That is so far above the drive speeds of even the fastest SSD's that I cannot think any speed gain is imaginable, and if so, certainly a foolish trade for zero protection.

 

Dustbak

Member
Yes, I know. Read on and see I have another RAID0 that I use as a backup, I use spare drives for off-site location. At any point in time I can loose either the off-site drives, one or both RAID sets and still recover.

Try processing 400files that are 2GB on average. You will definitely appreciate all the write/read speed you can get.

BTW, my 4 drive RAID0 get about the same speeds (slightly below) as your 6 drive RAID5 which is kind of in-line with what you could expect.
 

narikin

New member
Why move the thread?

ACK - why did you move this thread administrator? Do you really think people with MF camera systems don't need large storage systems, and its not part of the discussion there? Of course they do.

Almost nobody will see it in this dusty corner of the website, and many MF users would have found it useful. To me this seems like over-policing, considering how far OT or plain silly some threads get on the MF line.
 

narikin

New member
Yes, I know. Read on and see I have another RAID0 that I use as a backup, I use spare drives for off-site location. At any point in time I can loose either the off-site drives, one or both RAID sets and still recover.

Try processing 400files that are 2GB on average. You will definitely appreciate all the write/read speed you can get.
ok, run HD Tach, and show me what you get, please.

'Long test' thanks.
 

ustein

Contributing Editor
>Read on and see I have another RAID0 that I use as a backup

I would never ever use a RAID 0 for backup. Speed should be no big issue for pure backup.
 

Dustbak

Member
ok, run HD Tach, and show me what you get, please.

'Long test' thanks.
HDTach is windows only. Running xbench (the only test I have available) I get around 450Mb/sec. sustained max.

I know RAID0 is not ideal for backup, nor do you need the speed for that. I would not opt for it were it not that in this case I wanted the max size with drives I had available. It is also not a very big deal when 1 of the Raid sets fail.
 

narikin

New member
HDTach is windows only. Running xbench (the only test I have available) I get around 450Mb/sec. sustained max.

I know RAID0 is not ideal for backup, nor do you need the speed for that. I would not opt for it were it not that in this case I wanted the max size with drives I had available. It is also not a very big deal when 1 of the Raid sets fail.
I really can't understand the reason set things up like this

OK, let's assume the most flattering scenario for you: with 2 x2tb drives in Raid 0, you get 4tb data at 450mb/s max. 'Backup' is another 2 drives, also in raid 0. That's 4 2tb drives only giving you 4tb data storage, and allowing just 1 drive to fail.

With those same 4 drives in Raid 5, you would have 6Tb of data, and the same 1 drive failure to be safe, but get 650mb/sec, which is a full 50% faster than Raid 0 configuration. (you said "speed" is paramount for you). 50% more capacity and 50% more speed with the same 4 drives seems a good deal to me.

Anything more than 2 drives and the numbers get worse for the Raid 0 concept. Or with those same 4 drives you could use Raid 6 and have the identical 4Tb final capacity but allowing extra security above what you have. (3 drives would have to fail at same time for you to loose your data)

as it is with Raid 0:
your speed is slower than Raid 5.
your data fragility is same as Raid 5
your usable capacity is less than Raid 5.

The only scenario I can imagine this is that you only have space for exactly 2 drives in your machine, and must have 4tb. Otherwise this is not an advantageous setup. You do know you can fit 3 extra hard drives in something like this, which fits into in 2 Optical drive bays of most machines: http://www.newegg.com/Product/Produ...cm_re=icydock_3_into_2-_-17-994-097-_-Product There is also a bay for one into one if you only need that.

It is however your data, and therefore your right to do with it as you wish! :)

With this, I feel embarrassed that I've hijacked this thread, and will sign off for a good while.
 

Dustbak

Member
I think you have not read what exactly my setup is.

4x2TB INTERNAL drives (I cannot add any more) in RAID0. No way I can get it faster than this internally. 4 drives in RAID5 are slower than 4 drives in RAID0! My Areca card only has 4 ports, my MacPro can only host 4 drives internally. The other 2 slots I used 1 OWC SSD for the OS and the other for the DVD drive.

4x1TB EXTERNAL RAID0 (these were my old drives) this will give me 4TB of backup which is just enough (also considering I try not to fill more than 50% on the internal RAID). I totally agree I could have used 2TB drives in RAID5 here which for backup purposes would have made more sense.

I use drives a lot and most of the time I upgrade before they fail. When 3TB drives get cheaper (and more reliable), the 2TB's go to the backup unit and the 3TB internally.

I might have upgraded the MacPro by than since it is also from 2009.

RAID0 with the same number of drives is faster than RAID5.
 

gazwas

Active member
I appreciate all your replies but I feel your getting a little obsessed by speed. My main wish is data safety over speed and as kindly explained by narikin RAID 5 or 6 seems like the best deal to me.

Sorry for the very basic question but never even considered NAS before today, (thought it was for xbox/TV etc) If I opt for a NAS system why do they come with their own memory and processor and what effect does the processor speed have on the unit?
 

gazwas

Active member
Re: Why move the thread?

ACK - why did you move this thread administrator? Do you really think people with MF camera systems don't need large storage systems, and its not part of the discussion there? Of course they do.

Almost nobody will see it in this dusty corner of the website, and many MF users would have found it useful. To me this seems like over-policing, considering how far OT or plain silly some threads get on the MF line.
That was my reason for posting in the MF forum as my storage needs had increased massively with the purchase on my Phase P65+ and I thought it was a very relevant topic considering all the nice new storage hungry IQ180's have started arriving.

I didn't even know there was a gear garage on this forum. :(
 

Wayne Fox

Workshop Member
I use this one:

http://eshop.macsales.com/shop/hard-drives/RAID/Desktop/

Bought the one without drives and added my own. Very happy with it. Does pretty much everything you might wish for. Best part of it is that it sleeps when you put your machine to sleep (no noise!)! I use it with the eSata port.
I bought the same one with 4 2tb drives. Under $900, been running for over a year with no problems. End up with 6TB's of Raid5 storage.

I appreciate all your replies but I feel your getting a little obsessed by speed. My main wish is data safety over speed and as kindly explained by narikin RAID 5 or 6 seems like the best deal to me.
Any raid is not really appropriate as an archival solution (well maybe a raid 1 sort of). Even they can fail you, and if they do you lose far more data. Don't put all your eggs in one basket. The main purpose of a raid 5 more about letting you continue to work despite the failure of a drive. Yes, most of the time when a drive dies you'll be able to get back to where you were, but I have been using various raid setups for a lot of years, including some high end hardware raids, and have had failures on several occasions.
 
Last edited:

ustein

Contributing Editor
>The main purpose of a raid 5 more about letting you continue to work despite the failure of a drive.

Yes, it is not a backup scheme. It actually adds the potential of RAID electronics failure.
 

Ben Rubinstein

Active member
I'm looking into a couple of RAID setups soon for a project, do any of them have a plug and play replaceable controller? I've already had RAID setups die when the controller went...
 

gazwas

Active member
So if RAID is unreliable (and expensive) is the best option to have multiple single drives of data then?
 

narikin

New member
Raid is extremely reliable. It is used in nearly every server in the world.

Most hardware controllers are plug and play. I have move raid arrays (6, 8 or 12 drives) from one controller card to another, and everything appeared perfect. I have never had a controller go down, in 8 years, but of course that might happen. However in that same time I have had 5 disks fail, and never lost a single byte.

No it is not 'backup' but it does protect you from the single most likely (inevitable) danger to your data - Disk failure. This is far and away the biggest problem for all of us here, and this is what Raid protects very well against.

It is not so much a question of 'carrying on working' as being able to auto-recover from a disk crapping out, without digging up some old backup copy somewhere, that may be out of date. Just put in a new disk, and it rebuilds in hours (while you can keep working if you wish) It's that simple.
 
Top