The GetDPI Photography Forum

Great to see you here. Join our insightful photographic forum today and start tapping into a huge wealth of photographic knowledge. Completing our simple registration process will allow you to gain access to exclusive content, add your own topics and posts, share your work and connect with other members through your own private inbox! And don’t forget to say hi!

Computer question for Medium Format photography only

jagsiva

Active member
Used Macs for a long time, but switched to Windows after the last Mac Pro was released. Just did not fit my storage needs. Still continue to use a Macbook Pro as a laptop.

PC - This thing just flies through CS/PS/LR/C1:

i7 4930K
64GB RAM
Dual AMD R9290X
2 NEC PA27
1 Dell 4K 3214Q

OS Drive - 2 striped SSDs - about 900MBps R/W speeds
Images - 24 SSDs on adaptec RAID Controller, total approx 5TB - about 3GBps to 4GBs R/W (yes this part is nuts)
Primary Backup - RAID 10 External Mini-SAS (I do this near real-time after a few disasters)
Secondary Backup - RAID 10 on separate server with Mini-SAS
Off-site - Just copy my disks every month and keep them elsewhere.


Windows has it's own issues. It is fast, and I can upgrade it easily. It also can heat my office quite well if needed:) The Mac was certainly nicer in some aspects.
 

Egor

Member
satybhat, you will want to work with R-0 configs of SSD drives for day2day work, and then have it backed up/transferred to a large inexpensive R-5 config using either carbon copy cloner, Super Duper!, or CrashPlan. If you need real time incremental, then Apple TimeMachine, crash plan or even retrospect work well for that purpose. On some machines that are working on critical files I have written a script that just auto-copies all data as it is saved to a remote R-5 array, but that is overkill, imo. Never in over 10 years has it been an issue. I even have some R-0 configs dating back to SCSI drives that have never crashed in over 12 years...still running strong and stable!

Interesting afternote: All multi-platter hard drives (which is all hard drives) are R-0 configs within their enclosures. Just food for thought.
 
Last edited:

GrahamWelland

Subscriber & Workshop Member
If you go with RAID it's imperative that you also buy RAID certified drives when building your own. It's not a case of if a raid drive will fail, it's a case of when. Also, make sure that whatever you use, make sure that it has a very robust power supply - I had a raid 5 4x drive setup die a while back when the power supply browned out and two drives dropped. That was he end of that array - luckily it was a working raid and backed up elsewhere but it took a LONG time to rebuild it.
 

jagsiva

Active member
+1 on what Graham said on RAID. RAID is not a backup solution, always make sure you have proper backups.

HDD are so cheap now that it doesn't make sense to not do it, or use anything else.
The new cloud based stuff is OK, but I am not sure performance an security are there yet for me.
 

dougpeterson

Workshop Member
Here is an article I wrote about Computer Specs for Capture One.

And an article about Capture One v8 vs Capture One v7 which contain some useful tidbits.

Most of this info is specific to working with raw files in Capture One, but should be generally applicable.

Bottom line (if you want to skip the articles)
- newer versions of C1 are faster than older versions
- video card (GPU) is more important now than in the past
- only the D700 graphics card (on the mac side) supports dual-GPU use in C1, as a result it's much faster than the D500 card version of the mac pro
- SSD is much faster than HD, especially within C1
- more CPU cores are mostly useful for batch processing to TIFF, not for working with raws in raw form where clock speed of the CPU, and the power of the GPU is most important
 
Most of this info is specific to working with raw files in Capture One, but should be generally applicable.

Bottom line (if you want to skip the articles)
- newer versions of C1 are faster than older versions
- video card (GPU) is more important now than in the past
- only the D700 graphics card (on the mac side) supports dual-GPU use in C1, as a result it's much faster than the D500 card version of the mac pro
- SSD is much faster than HD, especially within C1
- more CPU cores are mostly useful for batch processing to TIFF, not for working with raws in raw form where clock speed of the CPU, and the power of the GPU is most important
Good breakdown. On a more general image processing level:
- a faster CPU will aid in image processing and conversion speed
- a faster storage system will aid in opening/saving/moving files and generating previews or cache data
- a faster video card will enhance realtime photo manipulation and auxiliary processing (depending on program)
- larger amounts of memory let you have larger/more files and programs open at once without the OS having to page memory to disk

Don't know about anyone else, but I have a tendency to have like half the Adobe creative suit open at once along with other stuff... never gauge how fast a computer you need based on one application, because you'll eventually want to see if you can get away with running all of them at once. :chug:
 

Wayne Fox

Workshop Member
as one who just experienced a catastrophic failure of a raid 5 “backup” I would offer that rendundant copies using raid 0’s is probably more reliable. The odds of a second drive failing in a raid 5 while rebuilding one that has failed grows dramatically as drive sizes increase, and in fact most data centers have/are moving to raid 6 for that reason - and predicting that within a few years even raid 6 will be problematic. While theoretical estimates probably overstate the problem, it is a problem. A raid 6 can lose 2 drives, so if another fails during the rebuild you are OK. The main advantage of a raid 5 is the data can still be accessed while it rebuilds and it is cheaper than the same amount of storage setup as a raid 1 or using full duplicate/cloned backups. Other than data centers or large corporate servers, most don’t need that feature, and raid 5 is not a backup solution. Another problem with raid 5 is rebuild times are very long ... we’re talking many days. Lower priced raids take 24-36 hours per terabyte of data to rebuild. (here is one article discussing these issues with raid 5 )

Additionally, hardware raids while they sound great, also have caveats (an “error” of a single block on a raid 0 on one of the drives can mark the entire drive as bad, resulting in total loss). A software raid seems to handle this better. There is a slight speed hit and possibly a little extra processor overhead, but neither are noticeable. Certainly there are downsides, but after some experience I have opted for software raids. (as an example, see this article)

I had a 4 drive raid 5, and one drive failed. Fortunately I learned long ago that raid 5’s and their redundancy is not intended as a backup method but as a keep the data available solution, so that raid 5 was cloned to a second raid 5 each night. Anyway, a new drive was inserted and somewhere during the rebuild a second drive failed, because the unit now is stuck in rebuild mode, and it doesn’t appear as the drive will be recoverable.

I already had a LaCie Big 5 unit, so I bought 2 OWC JBOD thunderbolt 2 cabinets along with eight 4 TB HGST(Hitachi) 7200 RPM drives configured each box using OS X soft raid as a set of raid 0’s. (some more research I did seems to show the hitachi/HGST drives are the most reliable, see this.)I debated on doing a raid 10 so that all changes are reflected to both drives, but decided an issue with that could be a corruption which gets mirrored. I opted instead to set the system up as a daisy chain backup, my main raid 0 is cloned over to the backup #1 at 3:00 am using Carbon Copy Cloner, when that finishes it triggers a clone from backup 1 to backup 2 using carbon copy cloner. So each day I start with 3 identical copies of the same data, and by changing a few symlinks on the SSD startup volume which maps some of my home folders over to the raid, I can start up with any 3 of the drives if necessary . The downside is if my main raid goes down during the day I lose anything I have done that day. Normally that isn’t that much, since I’m working at a second location with clones of various folders, which are then cloned over to the main drive when I get to my Mac Pro workstation.

The OWC thunderbolt 2 JBOD box is about $100 less than the one with the raid hardware built in. I’m much more comfortable with my setup now, although I may move the 3rd backup to another site, and try to setup my own internet backup scheme to keep it current. Haven’t decided if that is practical or not yet. (also looking at various cloud/online options to backup just some critical folders). I’m also planning on setting up a backup to an SSD of all my critical data which can be stored at a second location. I believe SSD’s which see very little data change offer the best long term backup solution, since they have no moving parts. And as SSD’s get cheaper I may look at creating a SSD raid 0 volume which can be a main work volume (which of course would be cloned to my main raid 0, and so on ). But right now I’m seeing 600MB/sec throughput on my 4 disk raid 0 ... I’m not sure I would see any really improvement in LR or PS with this.

So this may be beyond the budget of most, but the basic idea to me is raid 0’s when you need speed, redundant copies rather than parity raids (5 or 6) is probably more reliable and less troublesome if the data doesn’t have to stay live 24/7.
 

Wayne Fox

Workshop Member
- only the D700 graphics card (on the mac side) supports dual-GPU use in C1, as a result it's much faster than the D500 card version of the mac pro
I find this puzzling, wondering why all 3 versions of the GPU’s can’t boost the processing.

I’ve been told the GPU’s sort of work as “parallel” processors, which implies some of the load should be automatically shared. Maybe this is only for video output. I do know that only GPU2 drives any attached displays, and GPU 1’s role is to boost GPU processing.
 

jagsiva

Active member
Wayne, it is likely that the GPU's in the MP are setup as SLI, or in AMD world Crossfire. This would mean a couple of things.

Firstly, as far as I understand from the PC side, the support needs to be from the application, and NOT at the OS level. Phase could simply have decided to go for what they felt was the most common configuration for their users.

Secondly, the memory is essentially mirrored, so you get 1/2 of the total memory in both cards. The larger capacity of the 700 series may be better suited for this. On the 300/500, it could be memory limited before any performance improvements are scene, but this is just a guess.
 

dougpeterson

Workshop Member
The limit on simultaneous use of the dual GPUs is at the OSX level. On the D300 and D500 the appropriate hooks are not provided for concurrent use. The cynic in me assumes this is a purposeful limit by Apple to sell more of their high-end configuration.
 

GrahamWelland

Subscriber & Workshop Member
The limit on simultaneous use of the dual GPUs is at the OSX level. On the D300 and D500 the appropriate hooks are not provided for concurrent use. The cynic in me assumes this is a purposeful limit by Apple to sell more of their high-end configuration.
The Apple realist in me says that's almost certainly why it is so ... it wouldn't be the first or last time.
 

dougpeterson

Workshop Member
RAID is for uptime, not backup.

A good backup system protects against:
- user error (accidental erasure, overwriting a folder, restoring a previous state accidentally etc)
- software/OS error
- hardware failure of the disk platter (e.g. data rot)
- hardware failure of the enclosure
- catastrophic loss of the environment (e.g. flood, fire, hurricane etc)

RAID5 protects against exactly one of those (hardware failure of the disk platter)
RAID0 protects against two of those (hardware failure of the enclosure or disk platter)

Hardware failure definitely gets the most attention in discussions about backups, but in my experience it's a less likely culprit for significant data loss among individual photographers than user error.

Notably with either RAID, if you as the user do something stupid, the stupidity is copied across all disks immediately.
 

jagsiva

Active member
The limit on simultaneous use of the dual GPUs is at the OSX level. On the D300 and D500 the appropriate hooks are not provided for concurrent use. The cynic in me assumes this is a purposeful limit by Apple to sell more of their high-end configuration.
My wrong assumption then, as in Windows, the app needs to support Crossfire.

Makes sense for Apple....been paying the Apple tax for some time now, but heck, can still pretty much have any Mac for the price of a good tech lens:)
 

dchew

Well-known member
Several years ago I lost my PODAS raw images. It was the first time I used C1 and the images landed in the Pictures folder. Two years later, long after the RAWS were processed and output images placed in my master catalog, I got rid of all the backups associated with that old computer.

I have a robust backup system, but I am still the weakest link in the chain.

Dave
 

Wayne Fox

Workshop Member
RAID is for uptime, not backup.

Notably with either RAID, if you as the user do something stupid, the stupidity is copied across all disks immediately.
sort of my logic. Raid 0 is simply for speed and large disk size. Instead of raid 10 (probably better called a raid 1+0) I opted for redundancy and backup by automated backup software. I debated on whether to use raid 0’s for those backups to get enough capacity, or whether to use a simple span of multiple drives to get the backups large enough. Since loss of any drive in either system kills the whole thing, and because I had another raid, i decided that double redundant backups would be a good option, and by using raid 0’s it also allowed me to get back up and running in a few minutes.

Because the backups only occur nightly, if I do something stupid on the main raid I can use the backup to fix it. Additionally, the backup raids use Carbon Copy Cloner with a safety net, so any file that is modified on my main raid doesn’t just copy over the same file on the backup ... that file is moved to an archive and kept for a period of time.
 

Wayne Fox

Workshop Member
The limit on simultaneous use of the dual GPUs is at the OSX level. On the D300 and D500 the appropriate hooks are not provided for concurrent use. The cynic in me assumes this is a purposeful limit by Apple to sell more of their high-end configuration.
So if there are no hooks to the second GPU, what exactly is it doing?
 

satybhat

Member
Thanks for the information Wayne, Doug and everyone.
I think the high end dual GPUs were put in with scalability in perspective. Agreed that OSX will limit their utility ( as well as being limited by individual software efficiency ), however they do remain available for future versions of OSX ( and software upgrades ).
Even assuming planned dehiscence in mind, there is speculation that apple is actually not making any money on the AMD chips. See here.
 
So if there are no hooks to the second GPU, what exactly is it doing?
Only Apple knows. The Mac Pro is clearly aimed at Final Cut users, and to my knowledge Final Cut can use all the processing that's on offer, so they could just be keeping the secret sauce to themselves unless you buy the top-end configuration.

As a side note - a Crossfire/SLI link is NOT required for multi-GPU processing, because modern GPUs are fully virtualized devices that may be treated as separate resources for GPU compute, meaning you can even have a rack of GPUs on a network and still utilize them for processing from a central system; this is how modern supercomputers are now built. Having the SLI/CF bridge attached to your cards while doing GPU compute may even incur a performance penalty if the software accidentally treats both cards as "one" card the way videogames do. Data between graphics RAM is only cloned when two or more cards are working on the same data set, but not when the two cards are applied for different purposes, for example in FCPX one card could be handling the real time playback and effect rendering while the second card does transcoding, assuming the application is threaded in a way to allow that.
 

Egor

Member
R-5 arrays use parity striped across all drives in the set. Therefore, its the best compromise between speed and span size. Yes, you can lose 2 drives or more (depending on R5 config) simultaneously and not lose any data. Yes, a new drive(s) must be inserted and it takes time to rebuild, but its the best compromise between speed and size and is why I choose it for most uptime and backup configs.
Even if they made a 32TB single platter drive (they don't), I doubt it would be a cost effective solution.
R-0 I use for speed, size after 1TB is not as important to us, because any given day's work is transferred to the big arrays every night, and the R-0's wiped clean for the next day's work. We only need SSD R-0 to shave an hour or so off each day's post work on large quantities of BAF (BiggAssFiles).
Boot drives are also SSD but not R-0, just not necessary.
 

torger

Active member
The 5K screen is just awesome, looked at some of my images on the 5K mac last weekend. My old 100ppi screen looks very blocky now. I want 200ppi. On the other hand great screens tend to survive one or two computers, so I don't really like the concept of having computer and screen in the same unit. I also like the 16:10 format better than 16:9, so I hope a 30 inch 16:10 200ppi screen will come out soon(ish) and not cost too much.

I'm quite patient when it comes to a computer being a bit slow -- as long as it doesn't run out of RAM. Therefore I like computers that can hold lots of RAM, it prolongs the life of the machine. I have 64 gig in my box now. As far as I know Apple boxes can be a bit limited concerning RAM though, as it seems to be one of the main things they use to differentiate their products.
 
Top