Need Apple Computer advice

[ndwgolf]

Guys
I currently have a 2012 Apple MacPro that has been upgraded to be able to run OS Mojave

The problem I have is that I am using a Hasselblad H6D100c and when I try and process the files its painfully slow (each file is ~267MB).

I have 3 x 6TB internal drives that store all my files and a 2TB SSD Drive as the main working drive.

I also have a new MacMini that is hooked up to my Smart TV and I use that sat on the sofa surfing the web and emails.

So my question to the gang is what would be the best Apple computer to buy to run Phocus/PScc software for all my photo editing. I am thinking another MacMini with a bunch of superfast external hard drives (if there's such a thing). Im retired now so need to watch my pennies but I don't want to have to by twice.
I appreciate any feedback

Neil

 

[RobbieAB]

Guys
I currently have a 2012 Apple MacPro that has been upgraded to be able to run OS Mojave

The problem I have is that I am using a Hasselblad H6D100c and when I try and process the files its painfully slow (each file is ~267MB).

I have 3 x 6TB internal drives that store all my files and a 2TB SSD Drive as the main working drive.

I also have a new MacMini that is hooked up to my Smart TV and I use that sat on the sofa surfing the web and emails.

So my question to the gang is what would be the best Apple computer to buy to run Phocus/PScc software for all my photo editing. I am thinking another MacMini with a bunch of superfast external hard drives (if there's such a thing). Im retired now so need to watch my pennies but I don't want to have to by twice.
I appreciate any feedback

Neil

Could you temporarily borrow the current Mac Mini from the tv to test the software on it?

Do you have any insight into what is the limiting factor for the current MacPro performance? I am assuming with 32GB of RAM, you are not memory starved, so are you CPU bound or storage bound?

Does the software support openCL acceleration? If so, is it currently enabled?

 

[ndwgolf]

Could you temporarily borrow the current Mac Mini from the tv to test the software on it?

Do you have any insight into what is the limiting factor for the current MacPro performance? I am assuming with 32GB of RAM, you are not memory starved, so are you CPU bound or storage bound?

Does the software support openCL acceleration? If so, is it currently enabled?

My MacMini downstairs only has 8GB of ram so I think that would be an issue. I use Clean my Mac and that is forever popping up saying my memory is almost full. I am not sure about the Open CL function

I have just imported 525 fff pictures from one of my internal hard drives to Phocus and 45 minutes later it still has 337 pictures to go.

 

[Boinger]

Guys
I currently have a 2012 Apple MacPro that has been upgraded to be able to run OS Mojave

The problem I have is that I am using a Hasselblad H6D100c and when I try and process the files its painfully slow (each file is ~267MB).

I have 3 x 6TB internal drives that store all my files and a 2TB SSD Drive as the main working drive.

I also have a new MacMini that is hooked up to my Smart TV and I use that sat on the sofa surfing the web and emails.

So my question to the gang is what would be the best Apple computer to buy to run Phocus/PScc software for all my photo editing. I am thinking another MacMini with a bunch of superfast external hard drives (if there's such a thing). Im retired now so need to watch my pennies but I don't want to have to by twice.
I appreciate any feedback

Neil

External HDs are the worst thing for photo editing.

You want an internal ssd preferrably on a U.2 port.

I don't know much about macs so I don't know what their storage architecture is in the newest ones. In general Macs are always a generation or two behind windows pcs.

Unfortunately Phocus is a really awful software in terms of its optimizations. I have a very powerful machine it is extremely slow on that as well.

How I configure my storage is I have a Raid array for storage. And I have a working drive that I put my most current pictures that I am working on, generally speaking this year's files.

I archive my previous years photos. This helps speed a lot.

I would try your mac mini first to see how it works. Importantly put your working pictures on the internal drive not external HDs. It has a more modern cpu and it should help significantly.

 

[Pelorus]

I think the first thing to do is to try and understand where the bottlenecks might be. Have you used Activity Monitor before? If not you need to go to Finder -> Go Menu -> Utilities and find and launch Activity Monitor.

The app has a number of tabs CPU, Memory, Energy, Disk, Network. Go into each of these tabs and click at the top of the relevant column (for instance CPU %) so that the hungriest app is sorted to the top. Also then note the table and graph at the bottom of each page. This should give you a guide to what's going on. Launch a big PHocus job and inspect activity in Activity Monitor for information.

The Macs use the fast SSD drive as a memory swap. My guess is you are maxing out your physical memory and may be struggling to get enough virtual memory. You'll find out about that by careful use of Activity Monitor. It's not the most complete analytics but it will tell you most of what you need to know.

Please report back

Guys
I currently have a 2012 Apple MacPro that has been upgraded to be able to run OS Mojave

The problem I have is that I am using a Hasselblad H6D100c and when I try and process the files its painfully slow (each file is ~267MB).

I have 3 x 6TB internal drives that store all my files and a 2TB SSD Drive as the main working drive.

I also have a new MacMini that is hooked up to my Smart TV and I use that sat on the sofa surfing the web and emails.

So my question to the gang is what would be the best Apple computer to buy to run Phocus/PScc software for all my photo editing. I am thinking another MacMini with a bunch of superfast external hard drives (if there's such a thing). Im retired now so need to watch my pennies but I don't want to have to by twice.
I appreciate any feedback

Neil

 

[Pelorus]

Please also in the About This Mac click on the Storage tab and show us what it says.

Guys
I currently have a 2012 Apple MacPro that has been upgraded to be able to run OS Mojave

The problem I have is that I am using a Hasselblad H6D100c and when I try and process the files its painfully slow (each file is ~267MB).

I have 3 x 6TB internal drives that store all my files and a 2TB SSD Drive as the main working drive.

I also have a new MacMini that is hooked up to my Smart TV and I use that sat on the sofa surfing the web and emails.

So my question to the gang is what would be the best Apple computer to buy to run Phocus/PScc software for all my photo editing. I am thinking another MacMini with a bunch of superfast external hard drives (if there's such a thing). Im retired now so need to watch my pennies but I don't want to have to by twice.
I appreciate any feedback

Neil

 

[Pemihan]

I have a setup almost identical to yours, the only difference being I have two SSD's in PCIe adapters and my main internal storage HDD drives are 10TB. The system is on one SSD and the other is used as scratch disk. I also have Capture One and lightroom catalouges etc. on that SSD. Makes quite a difference.

I process 100MP Phase One files in Capture One, sometimes a little Lightroom or Photoshop and have no problem at all processing the files.
I don't think me having the SSD's in PCIe slots should make a big difference but how much free space do you have on your harddrives? If you fill them up that will slow things down considerably, so leave plenty of free space like 25 % or get bigger drives.

Don't know about Phocus but in Photoshop you can designate what you want to use as scratch disk. Also make sure to enable OpenCL in Phocus if it supports that. In Capture One it makes a big difference.

You might benefit from a second SSD dedicated for scratch disk. You can plug it in beneath the CD ROM drive. Pull out the tray and there's a free SATA cable. Just connect the SSD and leave it beneath the CD ROM drive, no need to mount it.

How are the Graphics Card connected to power? The best is to use a power cable like this so the card gets power from both connectors on the board. DUAL MINI 6 PIN male to 8 PIN male PCI-E power cable FOR Mac-Pro MADE IN USA

If you're on Facebook, join the MacProUpgrade group. The group are full of very knowledgeable and helpful people.

 

[RobbieAB]

I don't think me having the SSD's in PCIe slots should make a big difference

I am going to disagree here. If his work is storage bound (which I suspect importing is!) a storage system running as fast as the PCIe bus is going to be dramatically faster than than any storage device hidden behind a SATA controller.

This is, in fact, was where I was heading with my questions: Would adding an NVMe SSD to the Mac Pro give a bigger performance boost than replacing the system with a new system?

That said, I realize now that the 2012 Mac Pro was the first "Trash can" version, so it may not be practical to do much internally thanks to the beautiful design of the system.

External HDs are the worst thing for photo editing.

You want an internal ssd preferrably on a U.2 port.

I don't know much about macs so I don't know what their storage architecture is in the newest ones. In general Macs are always a generation or two behind windows pcs.

While in general I agree that external storage is not ideal, this might be one of those rare exceptions: A ThunderBolt disk enclosure might actually be a viable option here, if one can be sourced for sane prices.

My MacMini downstairs only has 8GB of ram so I think that would be an issue. I use Clean my Mac and that is forever popping up saying my memory is almost full. I am not sure about the Open CL function

I have just imported 525 fff pictures from one of my internal hard drives to Phocus and 45 minutes later it still has 337 pictures to go.

Full memory? That sounds more like full disks to me. If that's the case, a major analysis of your storage usage, and even a "slow" external enclosure for archiving off to might help a lot as well. Can you get a review of your disk usage?

The symptoms you are describing sound like a storage issue more than a memory or CPU issue. How long did it take to import a single image when new? Even if it was only 50Mpixel, it's still a useful point of reference.

 

[Boinger]

While in general I agree that external storage is not ideal, this might be one of those rare exceptions: A ThunderBolt disk enclosure might actually be a viable option here, if one can be sourced for sane prices.

Thunderbolt is never going to match a pcie based ssd.

 

[Pemihan]

"I don't think me having the SSD's in PCIe slots should make a big difference"

I am going to disagree here. If his work is storage bound (which I suspect importing is!) a storage system running as fast as the PCIe bus is going to be dramatically faster than than any storage device hidden behind a SATA controller.

This is, in fact, was where I was heading with my questions: Would adding an NVMe SSD to the Mac Pro give a bigger performance boost than replacing the system with a new system?

What I was referring to was SSD's used for the system not storage. Of course a PCIe SSD storage system would be ideal (and very expensive if you like me have almost 6TB of files)
That said up until a short while ago I was running a single Sata SSD and I didn't have any issues.

I think Neil's problem is caused by one or more of the following reasons:

Phocus.
Almost full storage hard drives.
Not having OpenCL enabled in the apps that support it.
Graphics Card not getting enough power.


Getting a second SSD and move both to PCIe slots would probably improve overall performance and of course NVMe SSD's would be even better. That requires a firmware update of the Mac Pro though in order to use NVMe SSD for the system.

That said, I realize now that the 2012 Mac Pro was the first "Trash can" version, so it may not be practical to do much internally thanks to the beautiful design of the system.

The 2012 model was the last of the good old classic Mac Pro tower. Even today it is a very powerfull machine when you put some upgrades into it like new CPU(s), SSD's or NVMe SSD, better graphics cards, more memory, PCIe USB3 ports etc.
I'm in the process of upgrading a dual CPU 2012 Mac Pro with all of the above and when done it will be screaming and ready for several more years of service for my use. I take them apart and clean everything and add fresh cooling paste where needed.

 

[Pemihan]

I use Clean my Mac and that is forever popping up saying my memory is almost full

Could you post a screenshot of the message window?

 

[Pemihan]

I have just imported 525 fff pictures from one of my internal hard drives to Phocus and 45 minutes later it still has 337 pictures to go.

I have no clue to how Phocus works but here is your Phocus archive located?

 

[JaapD]

Let’s do a quick calculation:
(1) 525 fff pictures and 45 minutes later it still has 337 pictures to go => 188 files in 45 min.
(2) each file is ~267MB.

(1) +(2) combined: 50196 Mb in 2700 sec. => 18.6 Mb/sec.

CONCLUSION: This is slow! The only thing that could be this slow is not the CPU, not the SSD, not the PCIe, not the memory but only the hard drives. It very much looks like the HDDs are way too full, slowing down your complete system. The solution would be to purchase latest gen huge capacity HDD’s.

Also be aware that copying large amounts of data within the same drive is so much slower that copying from one drive to another (read/write cycles, the head can only be at one place at a time). So my suggestion is to have a separate image storage disk and a separate image work disk, the latter one preferably a fast PCIe SSD.


Regards,
Jaap.

 

[RobbieAB]

Thunderbolt is never going to match a pcie based ssd.

No, but if it was a trash can Mac Pro (Apparently I have the model years wrong here, so...), it may be a better option than anything else once budget is considered as fitting NVMe devices to those is, AIUI, an interesting challenge.

If it is the last old desktop Mac Pro, I completely agree, PCIe storage, possibly on a multi-port PCIe->M.2 expander, is going to be much much faster.

What I was referring to was SSD's used for the system not storage. Of course a PCIe SSD storage system would be ideal (and very expensive if you like me have almost 6TB of files)
That said up until a short while ago I was running a single Sata SSD and I didn't have any issues.

Expensive compared to what?

I think Neil's problem is caused by one or more of the following reasons:

Phocus.
Almost full storage hard drives.
Not having OpenCL enabled in the apps that support it.
Graphics Card not getting enough power.

Agreed, and I suspect the second line to be the most important issue.

Getting a second SSD and move both to PCIe slots would probably improve overall performance and of course NVMe SSD's would be even better. That requires a firmware update of the Mac Pro though in order to use NVMe SSD for the system.

Why bother using it for the system? Use it for the catalog, and the working set storage, expand the disk arrays to hold the image archives.

The 2012 model was the last of the good old classic Mac Pro tower. Even today it is a very powerfull machine when you put some upgrades into it like new CPU(s), SSD's or NVMe SSD, better graphics cards, more memory, PCIe USB3 ports etc.
I'm in the process of upgrading a dual CPU 2012 Mac Pro with all of the above and when done it will be screaming and ready for several more years of service for my use. I take them apart and clean everything and add fresh cooling paste where needed.

My mistake, my google foo let me down. If that is the case, than I completely agree, it should be possible to make it fairly fly for the use-case here.

My overall suggestion, assuming it is storage exhaustion, would be some form of external storage enclosure, probably with 8 disk slots, for bulk storage, because there really isn't going to be that much more headroom if 3x 6TB disks are already in use. This can possibly be combined with an M.2 NVMe expander card (which may need to include a PCIe switch chip, I don't know the specifics of the Mac Pros ability to bifurcate it's PCIe lanes).

A fully kitted out Mac Mini is going to be well over $1000, which will go a long way if invested purely in storage.

 

[Pemihan]

Expensive compared to what?

2 x 10TB HDD in Raid 1 (I prefer instant redundancy and have several backups both on and off site) So to have the somewhat same setup I would need 6 x 4TB SSD's which is expensive and space in the machine would be an issue too. Of course I could get two 16TB SSD's at 10K a piece :banghead:
But for my usage editing and processing one image at a time ordinary hard drives works just fine.

Why bother using it for the system? Use it for the catalog, and the working set storage, expand the disk arrays to hold the image archives.

I might be wrong but having the system and apps on SSD is part of speeding things up. And you are absolutely right about the rest, that's why I have two SSD's and plenty of free space on my hard drives.

My mistake, my google foo let me down. If that is the case, than I completely agree, it should be possible to make it fairly fly for the use-case here.

My overall suggestion, assuming it is storage exhaustion, would be some form of external storage enclosure, probably with 8 disk slots, for bulk storage, because there really isn't going to be that much more headroom if 3x 6TB disks are already in use. This can possibly be combined with an M.2 NVMe expander card (which may need to include a PCIe switch chip, I don't know the specifics of the Mac Pros ability to bifurcate it's PCIe lanes).

If you use two PCIe SSD's one for system and one for catalog etc. you have room for up to 5 SATA hard drives (6 if you remove the CD Rom drive) which should give you plenty of storage including redundancy if you wish. Or to setup RAID 0 or 5 for increased speed, or 1 + 0 for both speed and redundancy.

 

[Wayne Fox]

Spinning hard drives just don’t cut it. even if they’re in a raid.

You might find adding sufficient internal PCI based SSD (https://eshop.macsales.com/shop/ssd/owc/mac-pro/2009-2012) and keeping everything on SSD’s would be the most effective way to move forward.

another problem is Phocus might not be able to utilize multi cores very well, if so your 7 year old Mac Pro is performing about 50% of the speed of a 2018 MacMini in single core tests.

If your MacMini is a 2018, I’d beef up the memory, and throw some ThunderBolt 3 SSD’s on it. That’s basically what I use now , except I have the 8 core MacBook Pro. It’s noticeably faster than my 2013 Mac Pro was.

If you're getting constant memory messages, make sure it’s not because your startup drive is too full. You need 50-80 GB’s of free space on your startup drive for the OS to run efficiently.

 

[ndwgolf]

External HDs are the worst thing for photo editing.

You want an internal ssd preferrably on a U.2 port.

I don't know much about macs so I don't know what their storage architecture is in the newest ones. In general Macs are always a generation or two behind windows pcs.

Unfortunately Phocus is a really awful software in terms of its optimizations. I have a very powerful machine it is extremely slow on that as well.

How I configure my storage is I have a Raid array for storage. And I have a working drive that I put my most current pictures that I am working on, generally speaking this year's files.

I archive my previous years photos. This helps speed a lot.

I would try your mac mini first to see how it works. Importantly put your working pictures on the internal drive not external HDs. It has a more modern cpu and it should help significantly.

Okay so I was busy last night. I moved 90% of what I had in the pictures folder into one of the 6TB internal drives. I now have 1.75TB spare space in my main SSD drive. Phocus is loaded in the applications folder in that same SSD drive. In my pictures folder I now have a new folder for initial imports , so that when I import new picture files they are put into the same SSD drive eliminating Phocus having to pull them from one of the internal hard drives. I did two imports today and they were so so...........better than before but still so so. I went to disc utility and to my horror I noticed that it is showing that my New SSD drive (Main Drive) is an external drive not an internal drive.............could that be where the issue is?
Note my new SSD drive replaced one of the internal 6TB drives so its not an external drive but internal

 

[ndwgolf]

Okay so I was busy last night. I moved 90% of what I had in the pictures folder into one of the 6TB internal drives. I now have 1.75TB spare space in my main SSD drive. Phocus is loaded in the applications folder in that same SSD drive. In my pictures folder I now have a new folder for initial imports , so that when I import new picture files they are put into the same SSD drive eliminating Phocus having to pull them from one of the internal hard drives. I did two imports today and they were so so...........better than before but still so so. I went to disc utility and to my horror I noticed that it is showing that my New SSD drive (Main Drive) is an external drive not an internal drive.............could that be where the issue is?
Note my new SSD drive replaced one of the internal 6TB drives so its not an external drive but internal

More info on that Main SSD Drive

 

[k-hawinkler]

External spinning hard drives in RAIDs are perfect for backups. :grin:

I have 28 HDDs (useable 106TB, redundant TB not counted) attached to my late 2013 trash can Mac Pro via four TB2s. Oh I also have five 2TB Samsung SSDs attached, 4 in a RAID 5 configuration for data storage processing, 6TB useable, the fifth 2TB SSD to boot Catalina from. The internal 1TB Apple SSD boots Mojave. :shocked: :loco:

I need to keep Mojave to run the old PS CS6, an ancient Nik, and Helicon Focus with CS6 Plugin. :facesmack: BTW one has to keep CS6 going as one cannot install it anymore from scratch. That simply does not work since some Mac OSs back. :banghead:

C1Pro12 runs great in Catalina. :thumbs:

 

[Wayne Fox]

More info on that Main SSD Drive

It says it’s connected with SATA, so it should be running at max speed. Not sure what type of SSD it is. I would run BlackMagic Disk Speed Utility on it and see what kind of speed you are getting. Not sure why it’s listed as external, although I know my old tower Mac Pro had to use a PCI interface card and the SSD plugged into that, which would make it the equivalent of an external drive.

Again if it’s running slower than you expect it might be Phocus cannot leverage the cores and is running basically in single core mode. As a comparison, when running a Lightroom import, all 8 cores of my MacBook Pro are running at pretty high levels. You can use Activity Monitor and show the CPU usage while importing into Phocus to see how well it uses the cores.