Site Sponsors
Page 1 of 2 1 2 LastLast
Results 1 to 50 of 58

Thread: RAID Storage ideas

  1. #1
    Senior Member
    Join Date
    Sep 2010
    Location
    UK
    Posts
    1,387
    Post Thanks / Like

    Question RAID Storage ideas

    Sorry if this is in the wrong place but having just bought a P65+ my increased data usage means I'm in search of an external RAID mirrored storage solution for Mac and was wondering what people here are using. I instantly thought of Lacie 2big and Drobo but hear horror stories about both units. I don't like the sound of the Western Digital units having to install their software to use the drives as I just want it to appear as one large external drive that I can then use with Media Pro to catalogue my sessions. Also being based in the UK, we don't have access to some of the specialist Mac shops in the US so ideally it will be an "off the shelf" solution rather then build your own solution.

    I'm leaning towards a Lacie 2big 4TB or 6TB as it looks a really well made unit but obviously looks aren't everything.

  2. #2
    Senior Member
    Join Date
    Jan 2010
    Posts
    528
    Post Thanks / Like

    Re: RAID Storage ideas

    I use this one:

    http://eshop.macsales.com/shop/hard-.../RAID/Desktop/

    Bought the one without drives and added my own. Very happy with it. Does pretty much everything you might wish for. Best part of it is that it sleeps when you put your machine to sleep (no noise!)! I use it with the eSata port.

  3. #3
    Senior Member
    Join Date
    Aug 2009
    Posts
    314
    Post Thanks / Like
    Images
    3

    Re: RAID Storage ideas

    Don't go for the Lacie solution. They will fail, it is only a question of when it will happen. I had a lot of lacie disks, they all crashed.... so i asked some colleagues about their experiance: same thing over there. I think they are not up to " hard labour". I went for some simple external harddisks and now i am about to order some Raid disk's from OWC. OWC is probably the world leader in good memory and storage solutions for Mac.

  4. #4
    Senior Member
    Join Date
    Aug 2009
    Posts
    314
    Post Thanks / Like
    Images
    3

    Re: RAID Storage ideas

    Tahts the one, you were just a bit faster then me Ray ;-)

  5. #5
    Senior Member
    Join Date
    Oct 2008
    Location
    Munich
    Posts
    876
    Post Thanks / Like
    Images
    1

    Re: RAID Storage ideas

    Sorry but OWC world leader ? Perhaps in the US, nobody even knows them here in Germany. ;-)

    There are lot's of option. My current second backup is a case from Lian Li.

  6. #6
    Senior Member
    Join Date
    Apr 2009
    Location
    Sydney, Australia
    Posts
    581
    Post Thanks / Like
    Images
    2

    Re: RAID Storage ideas

    +1 for OWC, and agree about LaCie. I got burned with them a few times. The support was as good as the product. I have found WD to be good too.

    I have also sworn never to be a Linux based one again, having been left with useless disks when the controller has died. If my Mac can't read the data, I don't buy it.
    Cheers,
    Jeff
    www.jeff-grant.com

  7. #7
    Senior Member
    Join Date
    Mar 2010
    Location
    Germany
    Posts
    253
    Post Thanks / Like

    Re: RAID Storage ideas

    If possible, I would avoid a (hardware-based) RAID solution unless you are willing to invest in rather expensive hardware: As soon as you do more than plain disk mirroring, you are vitally dependent on the controller of your system. If the controller dies and your replacement controller comes with a different firmware, you are out of luck.

    I guess it depends (i) on the data volume you have to manage and (ii) on the way you backup you data, but if you can break it up into 2-3 TB chunks I'd go for a software-based mirroring of HDs. (But: Breaking it up might make it more difficult to run proper backups because doing it automatically get's more difficult.)

    BTW: Being based in Europe doesn't mean you can't order from OWC. Shipping time is a few days.

    Chris

  8. #8
    Member
    Join Date
    Sep 2008
    Posts
    245
    Post Thanks / Like

    Re: RAID Storage ideas

    Another vote NOT to go with Lacie
    This is my solution and I'm very happy with it so far.
    http://www.bhphotovideo.com/c/produc..._External.html
    am

  9. #9
    Workshop Member Woody Campbell's Avatar
    Join Date
    Dec 2007
    Location
    New York
    Posts
    2,120
    Post Thanks / Like
    Images
    66

    Re: RAID Storage ideas

    Many of the folks here use Drobo. Very reliable and cheap, especially if you buy it with no drives and populate it yourself.

  10. #10
    Super Duper
    Senior Member

    Join Date
    May 2008
    Location
    32 31' 37.06" N, 111 6' 0.9" W
    Posts
    4,333
    Post Thanks / Like

    Re: RAID Storage ideas

    Another vote againt Lacie and another vote for Drobo. I've been using 2-Drobos now for a couple years without a hitch.
    Don Libby
    Iron Creek Photography
    Blog
    Tucson AZ

  11. #11
    Senior Member
    Join Date
    Jan 2010
    Posts
    431
    Post Thanks / Like

    Re: RAID Storage ideas

    Another -1 for LaCie and (sorry) Drobo.
    LaCie's just break all time, Drobo's are very slow, plus neither of these are really professional solutions.

    My main solution is an internal Areca ARC-1222 Raid Card, currently 6x 3Tb drives =18Tb internal, in Raid 6, which is amazingly reliable, huge (up to 24Tb with 8 disks) flexible (array roaming, hot spares, capacity changes, multiple raids/volumes, etc) and above all *fast* at 700mb/sec sustained read, and 1050mb/s burst. BUT... Mac's don't have the internal room for enough drives inside the machine -it's one of their handful of major negatives - so that's not on your list.

    This is backed up onto an external Areca ARC-8040 also an 8 drive unit, but in a self contained box. This is my main backup of my internal raid array. I would recommend that to you, but am not sure of the Mac compatibility. Otherwise it is pretty perfect with USB 3.0, e-SATA, Firewire, USB 2.0, Ethernet, etc, etc. Rock solid. Nowhere near as fast as the internal one, but a nice solution.

    My best advice would be to get a QNAP NAS machine. This can be anything from 4 to 8+ drives - whatever you require. Set it up in Raid 5 (or 6). I have a 5 drive one in Raid 5, and back my main array up onto it also. (thats 3xbackup, I'm neurotic) It is of course a NAS system, so everything is online. That has been a life saver a couple of times. Not remotely as fast as the internal Areca, but amazing machines with a huge online community that solves any setup issue. It can also act as a Time Machine backup as well as being your raid array + your files are on the network, so any computer can access them, in your home, or anywhere else in the world, with a password. Great reliable machines, and fast becoming the leading NAS system/community in the world.

  12. #12
    Senior Member
    Join Date
    Jan 2010
    Posts
    431
    Post Thanks / Like

    Re: RAID Storage ideas

    Quote Originally Posted by cly View Post
    If possible, I would avoid a (hardware-based) RAID solution unless you are willing to invest in rather expensive hardware: As soon as you do more than plain disk mirroring, you are vitally dependent on the controller of your system. If the controller dies and your replacement controller comes with a different firmware, you are out of luck.
    This seems odd advice. All the professional systems are hardware based (Raid cards with dedicated I/O processors) to say this is insecure flies in the face of the entire industry. Cards from one manufacturers are inter-operable. I've put old arrays into new cards and read everything.

    OK, the advice is sound if you are a small, low volume user. If everything you have already, and are likely to need in the future fits into a single 2Tb or 3Tb drive, which you just mirror every so often, then kind of fine. But that's a Raid system (raid 1 - mirroring) just a home made one.

  13. #13
    Senior Member
    Join Date
    Sep 2010
    Location
    UK
    Posts
    1,387
    Post Thanks / Like

    Re: RAID Storage ideas

    Thanks for your replies. So the Lacie is a big no, no! I've spoken to a couple of others and tey have also said avoid any Lacie products.

    The G speed looks very interesting as still does the Drobo S. I want the unit as my master storage and backup backup hence wanting the mirrored RAID.

    My only worry with the Drobo is thd reports of how slow it is in operation however it seems a lot cheaper than the G Speed. I will use via FW800. Can any Drobo users coment on this and which unit your using.

  14. #14
    Senior Member
    Join Date
    Jan 2010
    Posts
    528
    Post Thanks / Like

    Re: RAID Storage ideas

    I use an internal Areca 1220 Raid Card (4 ports internal) with 4 WD drives. This rocks and is faster than a software RAID and faster than the Mac RAID Card which is maxed out at 500MB/s. The Areca RAID is much faster than my OWC SSD. I could install up to 6 drives (even 8 with some modification) in my MacPro but the Areca only has 4 ports anyway.

    You have to install the latest Areca firmware for the Mac and it works just fine on my MacPro with 10.6.x

    I use the external OWC RAID case for Backup. I can definitely advise you to not use the firewire port but instead find something that supports eSata.

  15. #15
    Senior Member
    Join Date
    Mar 2010
    Location
    Germany
    Posts
    253
    Post Thanks / Like

    Re: RAID Storage ideas

    Quote Originally Posted by narikin View Post
    This seems odd advice. All the professional systems are hardware based (Raid cards with dedicated I/O processors) to say this is insecure flies in the face of the entire industry.
    Narikin, you didn't read my posting. I wrote "unless you are willing to invest in rather expensive hardware".

    You don't get a professional systems for the price of a LaCie, Taurus or whatever consumer-level RAID solution one might have in mind. I'd never go with the latter but I wouldn't hesitate going with an industry-proven solution.

    Chris

  16. #16
    Senior Member
    Join Date
    Sep 2010
    Location
    UK
    Posts
    1,387
    Post Thanks / Like

    Re: RAID Storage ideas

    And I thought buying an MF sysytem was an education.

    Thanks for your recommendation narikin, the QNAP does look a very robust storage solution and can also grow with my needs. The QNAP TS-659PRO+ Turbo NAS 6 Bay NAS looks very interesting and looks like it could last many years or are these like computers that get outdated very quickly and need replacing?

    Also as all this is very new to me I'm a little confused. An I correct in thinking being a NAS basically means its accessed over a network either wireless or ethernet? So for best performance ethernet would be the main connection to my master computer.

  17. #17
    Senior Member
    Join Date
    Jan 2010
    Posts
    431
    Post Thanks / Like

    Re: RAID Storage ideas

    Quote Originally Posted by cly View Post
    Narikin, you didn't read my posting. I wrote "unless you are willing to invest in rather expensive hardware".

    You don't get a professional systems for the price of a LaCie, Taurus or whatever consumer-level RAID solution one might have in mind. I'd never go with the latter but I wouldn't hesitate going with an industry-proven solution.

    Chris
    Internal Areca cards start around $325 - cheaper than Drobos or LaCie's. And vastly superior. I wouldn't call that "rather expensive hardware".

    All you need is to add the drives of your choice/capacity.

  18. #18
    Senior Member
    Join Date
    Jan 2010
    Posts
    431
    Post Thanks / Like

    Re: RAID Storage ideas

    Quote Originally Posted by gazwas View Post
    And I thought buying an MF sysytem was an education.

    Thanks for your recommendation narikin, the QNAP does look a very robust storage solution and can also grow with my needs. The QNAP TS-659PRO+ Turbo NAS 6 Bay NAS looks very interesting and looks like it could last many years or are these like computers that get outdated very quickly and need replacing?

    Also as all this is very new to me I'm a little confused. An I correct in thinking being a NAS basically means its accessed over a network either wireless or ethernet? So for best performance ethernet would be the main connection to my master computer.
    They are very reliable and no, you don't need to update or change them much. They are like little self-contained computers. Linux kernel, I believe.

    NAS is Network Attached Storage, so yes, it connects via your Ethernet, usually just plug it into the router. Shows up on your network. You control it by entering its I.P. address (shown on the display panel) into your browser. amazingly easy. I have the TS-509Pro, but that is 4 years out of date now. works perfectly though.

    You can get good quality 2Tb drives as low as $70 each (try Samsungs - they are excellent), so I'd go for that size. Put 4/5 of them in Raid 5 and you'll be happy for a decade probably. If you have the 6 bay system, you can do that right away, and add extra drives later to expand the capacity. Or have the 6th one in there acting as a 'hot spare' should a drive ever fail on you.

    This will not be as fast as an internal Areca system, but will compare well with most any other raid box. If ultra high speed is your requirement, then you have to go internal raid. If not, then this is an excellent answer.

  19. #19
    Senior Member
    Join Date
    Jan 2010
    Posts
    431
    Post Thanks / Like

    Re: RAID Storage ideas

    The bit no-one wants to hear:

    A raid system is not strictly 'backup'. It does protect you against the most common form of data loss - drive failure (a WHEN scenario, not an IF scenario). But - something like a flood/fire/theft and you loose protection.

    So make a backup, and keep it elsewhere. Transfer data to backup disks, label and wrap them securely and store off-site. Disks are so cheap now, its better to keep the old ones in case you make a mistake and delete something, than have a backup overwrite the old, 'safe' copy.

  20. #20
    Senior Member
    Join Date
    Jan 2010
    Posts
    431
    Post Thanks / Like

    Re: RAID Storage ideas

    Here is a snapshot of my Raid systems:

    'Local Disk C' is the OS disk -a SSD 240Gb drive. very fast and reliable
    'Internal Raid R' is main ultra fast internal storage raid 11Tb usable out of 17.5Tb total, in Raid 6.
    'External Raid T' is external Areca Raid box as daily backup for the internal array, in Raid 5.
    'Locked Backup Raid Z' is again the external Areca box with a locked 5Tb volume, where I keep a fixed backup from a year ago. Also Raid 5.

    Remember the capacities showing are what is usable after the Raid array has protected itself against disk failure. Actual total disk capacity is larger.

    There is also the Qnap showing on the Network in bottom left, for my NAS storage.

    I should add that this is overkill, but my work is my life.

    Last edited by narikin; 30th May 2011 at 11:00.

  21. #21
    Subscriber & Workshop Member GrahamWelland's Avatar
    Join Date
    Nov 2007
    Location
    Vancouver, WA
    Posts
    5,802
    Post Thanks / Like
    Images
    564

    Re: RAID Storage ideas

    Another for Lacie here too. I have a pile of Lacie disk units that eventually failed.

    I personally use a combination of internal RAID using eSATA drives and an Apple RAID controller inside my MacPro, plus a secondary external RAID (AMCC/3WARE Sidecar) which backs up the internal drives.

    Important other considerations - don't rely on RAID as a 100% reliable storage solution. You need to maintain a backup of the RAID!

    If you go with a RAID solution I HIGHLY recommend finding one that can support 2+ parity drives such as RAID 6 or mirrored solutions like RAID 10. After having had a RAID power bus failure that lost 2 drives I can attest to how huge a pain in the rear it is to rebuild and recover. If I hadn't had an external backup of the RAID I'd have been totally screwed.
    Remember: adventure before dementia!

    As Oscar Wilde said, "my tastes are simple, I only like the best"

  22. #22
    Senior Member
    Join Date
    Jan 2010
    Posts
    528
    Post Thanks / Like

    Re: RAID Storage ideas

    I have specifically opted for the highest speed RAID I could get (RAID0). I need to process lots of files often and they are big files.

    I have taken the calculated risk this will crash one of these days which is why I have another RAID0 (External) for backup).

    Besides this I backup to separate drives which I take to another location.

    RAID is indeed not a substitute for a backup!

  23. #23
    Contributing Editor ustein's Avatar
    Join Date
    Dec 2009
    Location
    Germany
    Posts
    3,658
    Post Thanks / Like

    Re: RAID Storage ideas

    On the Macs I am not a big fan of RAIDs (for archiving at least). I had once a file system fail on a raid (Drobo that is). Disk Warrior could find all files but not repair because the RAID put a layer between the OS and the disk level OS.

    I think in most cases 2-3 TB single disks work. Raid 0 for performance is a different story though.
    Uwe Steinmueller
    -------------------

    Editor&Owner of Digital Outback Photo
    http://www.outbackphoto.com

  24. #24
    Senior Member
    Join Date
    Jan 2010
    Posts
    431
    Post Thanks / Like

    Re: RAID Storage ideas

    Quote Originally Posted by Dustbak View Post
    I have specifically opted for the highest speed RAID I could get (RAID0). I need to process lots of files often and they are big files.

    I have taken the calculated risk this will crash one of these days which is why I have another RAID0 (External) for backup).

    Besides this I backup to separate drives which I take to another location.

    RAID is indeed not a substitute for a backup!
    Raid 0 whilst technically a Raid level, is raid Zero because there is Zero protection. one drive fails, its all gone. backup is your only hope at that point.

    I am getting 650 mb/sec on my Raid array, sustained sequential read. See below. 1107mb/sec burst. That's 1.1GB per second. I doubt you could seriously want more. This was only 6 regular 2Tb drives in Raid 5. 8 drives would be even faster. That is so far above the drive speeds of even the fastest SSD's that I cannot think any speed gain is imaginable, and if so, certainly a foolish trade for zero protection.


  25. #25
    Senior Member
    Join Date
    Jan 2010
    Posts
    528
    Post Thanks / Like

    Re: RAID Storage ideas

    Yes, I know. Read on and see I have another RAID0 that I use as a backup, I use spare drives for off-site location. At any point in time I can loose either the off-site drives, one or both RAID sets and still recover.

    Try processing 400files that are 2GB on average. You will definitely appreciate all the write/read speed you can get.

    BTW, my 4 drive RAID0 get about the same speeds (slightly below) as your 6 drive RAID5 which is kind of in-line with what you could expect.

  26. #26
    Senior Member
    Join Date
    Jan 2010
    Posts
    431
    Post Thanks / Like

    Why move the thread?

    ACK - why did you move this thread administrator? Do you really think people with MF camera systems don't need large storage systems, and its not part of the discussion there? Of course they do.

    Almost nobody will see it in this dusty corner of the website, and many MF users would have found it useful. To me this seems like over-policing, considering how far OT or plain silly some threads get on the MF line.

  27. #27
    Senior Member
    Join Date
    Jan 2010
    Posts
    431
    Post Thanks / Like

    Re: RAID Storage ideas

    Quote Originally Posted by Dustbak View Post
    Yes, I know. Read on and see I have another RAID0 that I use as a backup, I use spare drives for off-site location. At any point in time I can loose either the off-site drives, one or both RAID sets and still recover.

    Try processing 400files that are 2GB on average. You will definitely appreciate all the write/read speed you can get.
    ok, run HD Tach, and show me what you get, please.

    'Long test' thanks.

  28. #28
    Contributing Editor ustein's Avatar
    Join Date
    Dec 2009
    Location
    Germany
    Posts
    3,658
    Post Thanks / Like

    Re: RAID Storage ideas

    >Read on and see I have another RAID0 that I use as a backup

    I would never ever use a RAID 0 for backup. Speed should be no big issue for pure backup.
    Uwe Steinmueller
    -------------------

    Editor&Owner of Digital Outback Photo
    http://www.outbackphoto.com

  29. #29
    Senior Member
    Join Date
    Jan 2010
    Posts
    431
    Post Thanks / Like

    Re: RAID Storage ideas

    Quote Originally Posted by ustein View Post
    >Read on and see I have another RAID0 that I use as a backup

    I would never ever use a RAID 0 for backup. Speed should be no big issue for pure backup.
    agree.

  30. #30
    Senior Member
    Join Date
    Jan 2010
    Posts
    431
    Post Thanks / Like

    Re: RAID Storage ideas


  31. #31
    Senior Member
    Join Date
    Jan 2010
    Posts
    528
    Post Thanks / Like

    Re: RAID Storage ideas

    Quote Originally Posted by narikin View Post
    ok, run HD Tach, and show me what you get, please.

    'Long test' thanks.
    HDTach is windows only. Running xbench (the only test I have available) I get around 450Mb/sec. sustained max.

    I know RAID0 is not ideal for backup, nor do you need the speed for that. I would not opt for it were it not that in this case I wanted the max size with drives I had available. It is also not a very big deal when 1 of the Raid sets fail.

  32. #32
    Senior Member
    Join Date
    Jan 2010
    Posts
    431
    Post Thanks / Like

    Re: RAID Storage ideas

    Quote Originally Posted by Dustbak View Post
    HDTach is windows only. Running xbench (the only test I have available) I get around 450Mb/sec. sustained max.

    I know RAID0 is not ideal for backup, nor do you need the speed for that. I would not opt for it were it not that in this case I wanted the max size with drives I had available. It is also not a very big deal when 1 of the Raid sets fail.
    I really can't understand the reason set things up like this

    OK, let's assume the most flattering scenario for you: with 2 x2tb drives in Raid 0, you get 4tb data at 450mb/s max. 'Backup' is another 2 drives, also in raid 0. That's 4 2tb drives only giving you 4tb data storage, and allowing just 1 drive to fail.

    With those same 4 drives in Raid 5, you would have 6Tb of data, and the same 1 drive failure to be safe, but get 650mb/sec, which is a full 50% faster than Raid 0 configuration. (you said "speed" is paramount for you). 50% more capacity and 50% more speed with the same 4 drives seems a good deal to me.

    Anything more than 2 drives and the numbers get worse for the Raid 0 concept. Or with those same 4 drives you could use Raid 6 and have the identical 4Tb final capacity but allowing extra security above what you have. (3 drives would have to fail at same time for you to loose your data)

    as it is with Raid 0:
    your speed is slower than Raid 5.
    your data fragility is same as Raid 5
    your usable capacity is less than Raid 5.

    The only scenario I can imagine this is that you only have space for exactly 2 drives in your machine, and must have 4tb. Otherwise this is not an advantageous setup. You do know you can fit 3 extra hard drives in something like this, which fits into in 2 Optical drive bays of most machines: http://www.newegg.com/Product/Produc...-097-_-Product There is also a bay for one into one if you only need that.

    It is however your data, and therefore your right to do with it as you wish!

    With this, I feel embarrassed that I've hijacked this thread, and will sign off for a good while.

  33. #33
    Senior Member
    Join Date
    Jan 2010
    Posts
    528
    Post Thanks / Like

    Re: RAID Storage ideas

    I think you have not read what exactly my setup is.

    4x2TB INTERNAL drives (I cannot add any more) in RAID0. No way I can get it faster than this internally. 4 drives in RAID5 are slower than 4 drives in RAID0! My Areca card only has 4 ports, my MacPro can only host 4 drives internally. The other 2 slots I used 1 OWC SSD for the OS and the other for the DVD drive.

    4x1TB EXTERNAL RAID0 (these were my old drives) this will give me 4TB of backup which is just enough (also considering I try not to fill more than 50% on the internal RAID). I totally agree I could have used 2TB drives in RAID5 here which for backup purposes would have made more sense.

    I use drives a lot and most of the time I upgrade before they fail. When 3TB drives get cheaper (and more reliable), the 2TB's go to the backup unit and the 3TB internally.

    I might have upgraded the MacPro by than since it is also from 2009.

    RAID0 with the same number of drives is faster than RAID5.

  34. #34
    Senior Member
    Join Date
    Sep 2010
    Location
    UK
    Posts
    1,387
    Post Thanks / Like

    Re: RAID Storage ideas

    I appreciate all your replies but I feel your getting a little obsessed by speed. My main wish is data safety over speed and as kindly explained by narikin RAID 5 or 6 seems like the best deal to me.

    Sorry for the very basic question but never even considered NAS before today, (thought it was for xbox/TV etc) If I opt for a NAS system why do they come with their own memory and processor and what effect does the processor speed have on the unit?

  35. #35
    Senior Member
    Join Date
    Sep 2010
    Location
    UK
    Posts
    1,387
    Post Thanks / Like

    Re: Why move the thread?

    Quote Originally Posted by narikin View Post
    ACK - why did you move this thread administrator? Do you really think people with MF camera systems don't need large storage systems, and its not part of the discussion there? Of course they do.

    Almost nobody will see it in this dusty corner of the website, and many MF users would have found it useful. To me this seems like over-policing, considering how far OT or plain silly some threads get on the MF line.
    That was my reason for posting in the MF forum as my storage needs had increased massively with the purchase on my Phase P65+ and I thought it was a very relevant topic considering all the nice new storage hungry IQ180's have started arriving.

    I didn't even know there was a gear garage on this forum.

  36. #36
    Workshop Member Wayne Fox's Avatar
    Join Date
    Nov 2009
    Location
    Draper, Utah
    Posts
    871
    Post Thanks / Like
    Images
    134

    Re: RAID Storage ideas

    Quote Originally Posted by Dustbak View Post
    I use this one:

    http://eshop.macsales.com/shop/hard-.../RAID/Desktop/

    Bought the one without drives and added my own. Very happy with it. Does pretty much everything you might wish for. Best part of it is that it sleeps when you put your machine to sleep (no noise!)! I use it with the eSata port.
    I bought the same one with 4 2tb drives. Under $900, been running for over a year with no problems. End up with 6TB's of Raid5 storage.

    Quote Originally Posted by gazwas View Post
    I appreciate all your replies but I feel your getting a little obsessed by speed. My main wish is data safety over speed and as kindly explained by narikin RAID 5 or 6 seems like the best deal to me.
    Any raid is not really appropriate as an archival solution (well maybe a raid 1 sort of). Even they can fail you, and if they do you lose far more data. Don't put all your eggs in one basket. The main purpose of a raid 5 more about letting you continue to work despite the failure of a drive. Yes, most of the time when a drive dies you'll be able to get back to where you were, but I have been using various raid setups for a lot of years, including some high end hardware raids, and have had failures on several occasions.
    Last edited by Wayne Fox; 30th May 2011 at 20:24. Reason: clarification
    wayne
    My gallery

  37. #37
    Contributing Editor ustein's Avatar
    Join Date
    Dec 2009
    Location
    Germany
    Posts
    3,658
    Post Thanks / Like

    Re: RAID Storage ideas

    >The main purpose of a raid 5 more about letting you continue to work despite the failure of a drive.

    Yes, it is not a backup scheme. It actually adds the potential of RAID electronics failure.
    Uwe Steinmueller
    -------------------

    Editor&Owner of Digital Outback Photo
    http://www.outbackphoto.com

  38. #38
    Super Duper
    Senior Member

    Join Date
    May 2008
    Location
    Manchester/Jerusalem
    Posts
    2,652
    Post Thanks / Like
    Images
    290

    Re: RAID Storage ideas

    I'm looking into a couple of RAID setups soon for a project, do any of them have a plug and play replaceable controller? I've already had RAID setups die when the controller went...
    I am not a painter, nor an artist. Therefore I can see straight, and that may be my undoing. - Alfred Stieglitz

    Website: http://www.timelessjewishart.com

  39. #39
    Senior Member
    Join Date
    Sep 2010
    Location
    UK
    Posts
    1,387
    Post Thanks / Like

    Re: RAID Storage ideas

    So if RAID is unreliable (and expensive) is the best option to have multiple single drives of data then?

  40. #40
    Senior Member
    Join Date
    Jan 2010
    Posts
    431
    Post Thanks / Like

    Re: RAID Storage ideas

    Raid is extremely reliable. It is used in nearly every server in the world.

    Most hardware controllers are plug and play. I have move raid arrays (6, 8 or 12 drives) from one controller card to another, and everything appeared perfect. I have never had a controller go down, in 8 years, but of course that might happen. However in that same time I have had 5 disks fail, and never lost a single byte.

    No it is not 'backup' but it does protect you from the single most likely (inevitable) danger to your data - Disk failure. This is far and away the biggest problem for all of us here, and this is what Raid protects very well against.

    It is not so much a question of 'carrying on working' as being able to auto-recover from a disk crapping out, without digging up some old backup copy somewhere, that may be out of date. Just put in a new disk, and it rebuilds in hours (while you can keep working if you wish) It's that simple.

  41. #41
    Senior Member
    Join Date
    Jan 2010
    Posts
    431
    Post Thanks / Like

    Re: RAID Storage ideas

    Quote Originally Posted by gazwas View Post
    I appreciate all your replies but I feel your getting a little obsessed by speed. My main wish is data safety over speed and as kindly explained by narikin RAID 5 or 6 seems like the best deal to me.

    Sorry for the very basic question but never even considered NAS before today, (thought it was for xbox/TV etc) If I opt for a NAS system why do they come with their own memory and processor and what effect does the processor speed have on the unit?
    Yes Gazwas, we got a little sidetracked. apologies.

    NAS are completely self contained little computers, with processors, power supplies, etc. and as they don't really do anything but control a bunch of disks, and connect with the Ethernet, they are very happy with low power processors. I have never had one break, or noticed it as being 'slow'.

    You don't sound uber-technical (and nor am I despite my modest Raid experience) so just be sure you are happy setting up a NAS. It's really not that complicated, unless you want to do something more fancy, like setting up your NAS to be accessible from outside the home (eg for a traveling photographer who needs to see files sometimes from far away). I have mine set up for that, and to act as time machine backup for girlfriends Powerbook. works perfectly. I am extremely happy with my Qnap.

    If its just internal home network use, and RAID protection of your data, then its kind of perfect. Raid 5 is the de-facto standard for most people. Raid 6 is for the paranoid.

    Good luck.

  42. #42
    Senior Member Lars's Avatar
    Join Date
    Nov 2007
    Location
    Sunnyvale, California
    Posts
    1,811
    Post Thanks / Like
    Images
    19

    Re: RAID Storage ideas

    Quote Originally Posted by narikin View Post
    If its just internal home network use, and RAID protection of your data, then its kind of perfect. Raid 5 is the de-facto standard for most people. Raid 6 is for the paranoid.

    Good luck.
    With today's disk sizes, rebuild time for a RAID5 array after a disk failure is long enough that the probability of another disk failure and total data loss becomes significant. Throw some reasonable probability estimates into the equation and you can find a probability in the single-digit percents for a total loss at any given year for a 4-disk RAID5. Move up to more disks and the probability increases.

    On top of that is the possibility of other hardware failure, like RAID card or the NAS box, the only way to protect against that is to purchase an identical backup box at the time you buy your RAID hardware, so you can move the disk array over to identical hardware. Don't expect to be able to read the disks anywhere else.

    Furthermore is the uncertainty around if your RAID hardware is actually able to use a new disk that you purchase down the road, and successfully rebuild the array. This needs to be tested _before_ you fill up the array with vital data. Unless you get paid by the hour as an IT pro, I think life is too short for weeks of disk testing.

    Yeah, good luck indeed. My recommendation is frequent backups if you consider RAID5. And if you go above 4 drives, use RAID6 if you want to sleep well. Or get enterprise-quality drives for 3x the cost.

    The most important rule of thumb here is worth reiterating: your data isn't safe unless it is stored at two geographically separate locations.
    Monochrome: http://mochro.com

  43. #43
    Senior Member
    Join Date
    Jan 2010
    Posts
    431
    Post Thanks / Like

    Re: RAID Storage ideas

    Quote Originally Posted by Lars Vinberg View Post
    With today's disk sizes, rebuild time for a RAID5 array after a disk failure is long enough that the probability of another disk failure and total data loss becomes significant. Throw some reasonable probability estimates into the equation and you can find a probability in the single-digit percents for a total loss at any given year for a 4-disk RAID5. Move up to more disks and the probability increases.

    On top of that is the possibility of other hardware failure, like RAID card or the NAS box, the only way to protect against that is to purchase an identical backup box at the time you buy your RAID hardware, so you can move the disk array over to identical hardware. Don't expect to be able to read the disks anywhere else.

    Furthermore is the uncertainty around if your RAID hardware is actually able to use a new disk that you purchase down the road, and successfully rebuild the array. This needs to be tested _before_ you fill up the array with vital data. Unless you get paid by the hour as an IT pro, I think life is too short for weeks of disk testing.

    Yeah, good luck indeed. My recommendation is frequent backups if you consider RAID5. And if you go above 4 drives, use RAID6 if you want to sleep well. Or get enterprise-quality drives for 3x the cost.

    The most important rule of thumb here is worth reiterating: your data isn't safe unless it is stored at two geographically separate locations.
    amazing how anyone sleeps at night isn't it?

    I had a 3Tb disk fail in my 6 disk Areca array last month. Insert a new disk, and it rebuilt in 5 hours. (Raid 6)

    once again: Raid is not backup.
    backup is backup (having a separate copy of your data stored elsewhere)
    raid is raid (protection against disk failure, speed, and multi Tb ability)

  44. #44
    Sr. Administrator Jack's Avatar
    Join Date
    Oct 2007
    Location
    Los Altos, CA
    Posts
    10,486
    Post Thanks / Like
    Images
    1031

    Re: RAID Storage ideas

    My .02 on the entire issue:

    RAID5: With current disk prices and sizes, R5 has outlived it's usefulness --- as others have indicated, it can be more hassle to keep an R5 up and take more time to rebuild a single drive than it takes to copy an entire archive.

    RAID6: Ditto R5, except it rebuilds about 2x to 4x faster due to the dual redundancy, but a bit more costly due to drive overhead.

    DROBO: While a RAID-5 or 6 in principle and depending on model, it is an automated R5/6 and can used mixed disks so this is one unit I think is viable. It's downsides are I/O speeds are slow and the time it takes to rebuild the array or an added drive. It takes far less time to wipe and re-copy an entire archive than for DROBO to rebuild the failed or added drive. On the upside, DROBO remains accessible while this is going on, but is not considered "safe," and really needs to be backed up or be the back-up array in the first place. In use, it is really more a sophisticated JBOD than a true RAID device.

    JBOD: Slow, but very easy and most economical. Can use bare drives or boxes of various sizes in almost any connection interface, allowing you to add to your storage requirements as needed and as cost-effectively as possible. If access speed is non-critical, this is an excellent and inexpensive option. But it is not redundant, so everything needs to be backed up.

    RAID0: Fast and easy to maintain via system software, but theoretically less safe than any single drive and not redundant, so back-up is mandatory. Theoretically the most expensive option because you need 100% redundancy of drives for back-up, but the back-up drive array does not need to match the original drive array.

    RAID-0>1, What I do: Since drives (and simple enclosures) are so cheap, I now keep my main archive on a 4-drive R-0 array inside my MacPro. That is on-site copied hourly to external 2-drive 4TB eSATA R-0 units, which are also very fast access and write. This effectively makes this a RAID0-1 or RAID10 configuration. The archive is again backed-up offsite to matching size JBOD devices after every major shoot, and at least monthly. In this fashion, I always at least two back up copies of critical data onsite, and have my critical image inventory intact even if a total physical loss strikes at either storage location. As my storage requirements grow, I simply add a pair of the 2-drive R-0 boxes (they're cheap at around $110 from OWC) one for on-site and the second for of-site.
    Jack
    home: www.getdpi.com

    "Perfection is not attainable. But if we chase perfection, we can catch excellence."

  45. #45
    Googaliser
    Guest

    Re: RAID Storage ideas

    I love the speed of RAID0 but would never use again for large arrays (not even with SSDs). An obvious, but in my case overlooked point is the following:
    If each drive has a 5% chance of failing in a particular year, then in a 6 drive RAID0 array, there is a 30% chance the array fails in a year - its additive.

    RAID 5 is scary. Its really nail-biting watching it rebuild since this puts a lot of pressure on the drives and could trigger another failure (especially if they are from the same batch). I have had 2 drives fail simultaneously in a RAID 6 with SMART errors on a 3rd drive - see above for nervousness.

    I now use RAID 1 for primary and RAID 6 for back-up (offsite). An Areca 1880 controller assures this is still fast enough for my needs. I use different batches of disks throughout.

    My 2c

  46. #46
    Senior Member
    Join Date
    Jan 2010
    Posts
    431
    Post Thanks / Like

    Re: RAID Storage ideas

    Quote Originally Posted by Jack Flesher View Post
    My .02 on the entire issue:

    RAID5: With current disk prices and sizes, R5 has outlived it's usefulness --- as others have indicated, it can be more hassle to keep an R5 up and take more time to rebuild a single drive than it takes to copy an entire archive.

    RAID6: Ditto R5, except it rebuilds about 2x to 4x faster due to the dual redundancy, but a bit more costly due to drive overhead.

    DROBO: While a RAID-5 or 6 in principle and depending on model, it is an automated R5/6 and can used mixed disks so this is one unit I think is viable. It's downsides are I/O speeds are slow and the time it takes to rebuild the array or an added drive. It takes far less time to wipe and re-copy an entire archive than for DROBO to rebuild the failed or added drive. On the upside, DROBO remains accessible while this is going on, but is not considered "safe," and really needs to be backed up or be the back-up array in the first place. In use, it is really more a sophisticated JBOD than a true RAID device.
    .
    Sorry, not my experience, or the tens (hundreds?) of thousands of people out there with Raid 5 / Raid 6 boxes.

    Why is a Raid 5 array rebuilding a single lost drive working harder than your backup drive rebuilding a lost drive? Why is the raid going to break down in a rebuild and your backup drive is not? Can anyone point me to a professional IT storage study done that shows this to be a fact? If not, then its just here-say, and needless scare-mongering, that will stop people from protecting themselves properly.

    also FWIW, my experience of Drobo is dreadful. they are slick looking but slow as heck. underpowered pretty boxes. I posted the transfer rates of my Raid 6 array above, as hard facts for anyone to see. Compare that to a Drobo and weep. They cost about the same.

  47. #47
    Super Duper
    Senior Member

    Join Date
    May 2008
    Location
    Manchester/Jerusalem
    Posts
    2,652
    Post Thanks / Like
    Images
    290

    Re: RAID Storage ideas

    As a matter of interest as I'm about to buy two 8tb RAID boxes, what companies should I be looking at? I was quoted for a Lacie 4big Quadra Enterprise Class 8TB RAID 5 protected system and a Promax SataMax FX4 8TB RAID 5 protected system. I asked for quotes on Raid 6 boxes as well but I need two different types as one of these will be backup for the other and I know Murphy far too well to buy two of the same make when one is backup...

    Suggestions on what types or where I can learn about this stuff enough to make my own choice?
    I am not a painter, nor an artist. Therefore I can see straight, and that may be my undoing. - Alfred Stieglitz

    Website: http://www.timelessjewishart.com

  48. #48
    Senior Member
    Join Date
    May 2009
    Location
    Marseille, FRANCE
    Posts
    972
    Post Thanks / Like

    Re: RAID Storage ideas

    i'm in a process to buy a back up solution... finally, i won't go with a RAID...
    i will buy internal HD and put them in a box... and i will make a clone on an other one, but an other model of HD... and store them in two locations.
    I think i will buy HD that are sold for Server, they cost nearly the double price, but they are supposed to have a longer life time !

    Not a good idea ?

  49. #49
    Subscriber Member Jorgen Udvang's Avatar
    Join Date
    Nov 2007
    Location
    Pratamnak
    Posts
    9,344
    Post Thanks / Like
    Images
    2157

    Re: RAID Storage ideas

    I've been following this thread for a while, and the more I read, the happier I am for not having chosen a RAID solution. The complications and costs seem to outweigh the advantages by a solid margin. If anybody are interested, here's my setup:

    I use Probox units (similar to Drobo, but cheaper and easier to find, at least here) with 4 x 2TB in each, two storage disks and two backups. Backup run automatically every second day on the current disk set and weekly on the older archives.

    Every week, I back up the most recent data (usually the last six months) to a 1TB Samsung portable 2.5" and take it to an identical setup at a different location where I mostly spend my weekends anyway. If I do work while staying there, which I usually do, I follow the same routine when going back.

    I use Carbon Copy Cloner (Mac only, I believe) for all backups, and since the program only backs up data that have been changed or added, the process is rather swift. I have set CCC to not delete data on the target disk if deleted on the source disk. That way, if data that have previously been backed up, they will automatically come back to life again after one weekly cycle should I delete something by mistake. If I don't know that I have deleted it, I won't even notice.

    This system might not be the ultimate for speed, but so far, I've found it to be more or less fail safe. It also means that I can work at any of two or more locations and always have multiple backup of any changes.

    Just my 2c

  50. #50
    Senior Member
    Join Date
    Jan 2010
    Posts
    431
    Post Thanks / Like

    Re: RAID Storage ideas

    Quote Originally Posted by Jorgen Udvang View Post
    I've been following this thread for a while, and the more I read, the happier I am for not having chosen a RAID solution. The complications and costs seem to outweigh the advantages by a solid margin. )
    So the scaremongers here have convinced you that 99% of the world, and hundreds of thousands of users over decades are also wrong, and that Raid 5 is dangerous. This is just plain risky, but it is your work and your right to ignore the happy majority, and the entire data storage industry, who say otherwise. It is not complicated it is same or cheaper than your setup, faster and safer, assuming you complement it with a good backup regime. (like you have already)

    Quote Originally Posted by Jorgen Udvang View Post
    I use Probox units (similar to Drobo, but cheaper and easier to find, at least here) with 4 x 2TB in each, two storage disks and two backups. Backup run automatically every second day on the current disk set and weekly on the older archives.
    If I understand this the way it is written, then if you have an important shoot, put it onto your storage system (no Raid) and a single disk crashes between that shoot and "every second day" - then what? You have lost your shoot. Not Great.

    One of your disks will crash at some point. let me repeat: It. Will. Crash. And you will loose work, because you only have protection "every second day". Why are you risking this, when simple protection for the most likely event, the inevitable event of disk failure, is easily adopted?

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •