Some Mac Pro performance tips...

[Jack]

Indeed... The code Adobe uses for CS scratch must have been designed like 15 years ago -- it is really inefficient.

I was wondering if I needed to push the file sizes up to over 4G to force a true I/O on scratch. It seems maybe we need to add in one more sizing step or a few dupe layers to that action to get us over the 3G ram limitation...

 

[Bob]

Might as well use a thumb drive for scratch...
Yup 57 secs using a CF card formatted to Mac OS Extended as scratch
-bob

 

[Jan Brittenson]

How large ws that file?

160MB, 16-bit RGB, JH Chrome Space 100.

 

[Jan Brittenson]

Are you sure the gain was due to using the disk image and not just because you had a smaller partition limiting seek times?

Well, that was the idea - I don't have a small partition but am using a disk image instead. I was comparing putting the scratch on a large stripe consisting of two drives vs putting it on a small image on this stripe - precisely to constrain drive seeks.

 

[Bob]

Indeed... The code Adobe uses for CS scratch must have been designed like 15 years ago -- it is really inefficient.

I was wondering if I needed to push the file sizes up to over 4G to force a true I/O on scratch. It seems maybe we need to add in one more sizing step or a few dupe layers to that action to get us over the 3G ram limitation...

What it LOOKS like it is doing is to keep enough memory free to duplicate a layer. It starts writing to scratch even before the 3G limit is near. What it doesn't do is read from scratch unless some transformation occurs in that part of the image that has already been written to scratch and purged from physical memory. Try making a 6G file then rotate it. That ought to about max out scratch i/o. The gaussian blur is a piece of cake compared to that.
-bob

 

[Jan Brittenson]

I was wondering if I needed to push the file sizes up to over 4G to force a true I/O on scratch. It seems maybe we need to add in one more sizing step or a few dupe layers to that action to get us over the 3G ram limitation...

I noticed it's memory bound up to the rotation stage in my test, after that it's I/O bound. It uses about 50-60% across all 8 cores up to this point with a load average of 8 (one compute thread per CPU is fairly standard practice) - which is indicative that it's bottlenecking around memory. Then sometime during the rotation step this changes and CPU usage drops to noise level with occasional spikes on one or two cores. The time up to this point is a relatively short part of the test, so I don't think it really impacts the test that much.

 

[Jan Brittenson]

Keep in mind that HFS+ doesn't support sparse file allocation. If you create a file and extend it to some particular size, like 3GB, the file system will go off and start allocating and writing blocks to bring it to this size. You can still do random I/O in the file during this; it will prioritize over the background fill. (When you close the file you will block waiting for any remaining fill to complete.) So what you're looking at could just be PS predicting it will need scratch soon and allocating a file large enough - and HFS going off to do its background allocation. If the scratch is never used you won't see any reads from it.

 

[Bob]

Jan,
I am getting with that benchmark:
54.2 sec, scratch on boot vol hardware (raid0+1)
57.5 sec, scratch on 32G disk image (HFS+ no journal) allocated on same boot vol (raid0+1)
these are averages over three runs.
I have only four cores, and interestingly enough, although I have told PS to use all 3G, PS is using a total of 2.65G real, 3.49G virtual.
IO rates were interesting,
The physical read to write ratio favored writes considerably, maybe 4:1 in general. The peak write rate was about 60MB/sec with a read rate of 16MB/sec. I am presuming that this is due memory based filesystem cache. This ratio (with this benchmark) seems to be consistent via dtrace or activity monitor. PS utilized 11 threads, and in my 4-core ran at almost 100% on all four during resizing and almost zero during rotation, similiar to your observation. How many total PS threads did you observe?
In my stand-alone testing, I can sustain with my setup a disk write rate of 120MB/sec, such as what you might see when creating a disk image(non-sparse) or in sequential write loops. I presume that by memory limited, you mean memory bandwidth limited? I am courious if there is a possibility that PS might be thread starved in resize operations.
thanks
-bob

 

[Jack]

This may be old news, but I had missed it so am sharing it here... I just found this nifty little-known plug-in from Adobe that turns off file compression for scratch. Seems that for some time now Adobe has defaulted to compressing scratch files to save write time to drives, since historically they were relatively slow. Obviously the tradeoff was the compression requires time to perform, but apparently the drive writes were even slower. In today's world of fast drives, the compression probably takes more time than the writes, ergo this plug-in. FWIW Adobe does recommend it for systems with fast drives. It goes into your Plugins/Extensions folder.

I cannot upload it here and it is tough to find online. I found an obscure reference to it in a CS4 readme file, but could not find the plugin anywhere in CS4, but I did locate it in my CS*3* extra goodies folder(!) How's that for obscure?

Anyway, it works and seems to boost performance on my system running SATA2 drives in CS4 -- I cannot say how much better it is since I've only run in 4 and all my other benchmarks were done in 3...

Found them. Here is the whole enchilada of optional plug-ins:

For Mac: http://www.adobe.com/support/downloads/detail.jsp?ftpID=4049

For Win: http://www.adobe.com/support/downloads/detail.jsp?ftpID=4050