Stitching multiple 40MP files - computer 'grunt' required

[Ed Hurst]

Hello all,

I am about to embark on a pretty ambitious task - creating a large panoramic file using about 24 files from my 40 megapixel 645D. I have a Mac Pro with 16gb of RAM and am about to install Photoshop CS5 so that I can use 64-bit processing's ability to allocate most of the RAM to Photoshop for the task. I can't test thins yet - because I don't yet have the software. But I know my current set up with CS3 (with its associated RAM limitations) won't even come close to performing the task. Just for giggles, I tried a few things in CS3 to do a quick preview of how the full file will look. I created smaller JPG versions of the files; I stitched pairs of files together, then joined those files together; etc.. All of this had problems - the smaller JPGs would not stitch together unless they were very small indeed! Stitching in pairs created very odd perspective artefacts that were hard to fix later (I am assuming that stitching all of the files at once will not have this problem, but maybe I am wrong about that...).

Anyway, I have decided to try and use CS5 (with 15gb of RAM) to stitch all of the files together at once, hopefully as 16-bit TIFFs. This is asking a lot, of course. So my question is - how much RAM do you think I will need to do such a thing (joining 24 40MP 16-bit TIFF files together as one operation)? Will 15 be enough or am I, as I suspect, not even close? I could get even more RAM of course, but not sure if such a thing is needed. And will joining all files together at once create a different (and better) perspective result than joining files together in pairs and doing it progressively?

Many thanks!

Ed

 

[PeterL]

Is there not a 30 day free "try it" feature with CS5? Just download and try it out, if it fails try 8 bit tiff.

Cheers, -Peter

 

[Ed Hurst]

Thanks Peter. Not sure if that is available as an option, but will look into it. However, it anyone has any experience of how much RAM I will require, such advice would also be most welcome.

Best wishes,
Ed

 

[mvirtue]

This might or might not help. I use a 1st gen MacPro w/ 8GB and have stitched 10 33MP 16 bit TIFFs in CS5. It was not exactly pleasant, but it did it. Once assembled it handled the file just fine.

 

[PeterL]

Another thought, maybe try AutoPanoPro (I know they have a 30 day trial). It works very well, however I have never tried a job as large as the one you suggest. In any case, it may just take longer - even over-night, but it should complete it even though it may be RAM starved.

Cheers, -Peter

 

[Christopher]

It dös Not matter how much ram you have. It will just take longer. The more ram you have the quicker the process. I currently have 48gb and even 20 images shot with the P65 stitch easily in PtGui or AutoPano. However, opening them in CS4/5 is another story. It is sad, that as far as I know Adobe still can't use a lot of CPUs cores for this purpose.

 

[Ed Hurst]

Thanks for that guys.

Christopher - it appears that this is one operation for which the amount of RAM does matter (at least when it's done in Photoshop). With my current set-up, which is rather RAM limited, I get an error message saying that the operation cannot be performed due to a lack of RAM. So it's not just a speed issue. Even the fact I have a large scratch disc does not prevent this from happening.

My suspicion is that this is the case because comparing files and putting them together is something that requires a certain amount of data to actually be held in RAM at once (passing data back and forth to a scratch disc or just doing it slowly using a small amount of RAM seems not to be a substitute) - if it can't 'hold the information in its head' the operation cannot be performed at all. Now, I am no expert in this area, but that's my read on things.

If it it just a case of being patient with a slow operation, I would put up with that for such a relatively rare task. But sadly sufficient RAM seems to be required for it to be done at all.

 

[Graham Mitchell]

If you have installed maximum RAM and allocated it to CS5 in 64-bit mode then there's not much else you can do except use a fast SSD as a scratch disk.

 

[Ed Hurst]

Hi Graham,

I have not yet done that because I have not yet installed CS5 (am away from home for a few days and will do so soon). I also have not installed maximum RAM because I have not yet purchased more. In fact one of the goals of my question was to establish how much RAM I might need to purchase for such a task. At present, your statement 'maximum RAM' refers (I think) to 96gb if I chose to spend a lot to purchase such a thing, but I would rather spend less! So some indication of what is required to enable my computer to perform the task in the first place is really what I am after.

Also, re. scratch disks, as indicated above, this seems to be an operation that does not make use of them (I already have a huge scratch disk, but the computer stops the operation due to lack of RAM).

 

[Graham Mitchell]

Hi Ed, I don't know for sure but I imagine that your 'tiles' are about 225MB each, which for 40 tiles should be in the vicinity of 9GB, and the resulting layered file should be similar in size (you could check these file sizes yourself with a smaller test). So I would estimate that 18GB would be enough RAM to avoid scratch disk usage. The rest should be down to processor and disk performance. This is all just my 'hunch' and shouldn't be taken too seriously

 

[Ed Hurst]

Thanks Graham - I can see the logic! I'll try it once I have installed CS5 with my existing 16gb of RAM (allocating 15) and see what happens. If that's not enough, I'll upgrade... Either way, I shall share the outcomes here

 

[dick]

Thanks for that guys.

Christopher - it appears that this is one operation for which the amount of RAM does matter (at least when it's done in Photoshop). With my current set-up, which is rather RAM limited, I get an error message saying that the operation cannot be performed due to a lack of RAM. So it's not just a speed issue. Even the fact I have a large scratch disc does not prevent this from happening.

It does seem incredible that Photoshop cannot use a scratch disc when it runs out of RAM .... what do we pay all that money for? Is 5.5 any different?

I just got a 4TB 3 Gigabit e-SATA external disc for my Mac Book Pro ¿but would it help?

I am expecting to get a £10K Mac Pro sometime, when I need it.

 

[Ed Hurst]

Hi Dick,

Apologies if I have confused matters! PS does indeed use a scratch disc (and there are lots of settings etc. you can choose related to how this works). Thing is, from experience seems to suggest that doesn't help with the specific operation of stitching. I would be delighted to be shown to be wrong about this btw!

 

[tsjanik]

Ed:

I don't think it will be a problem, just slow. Here's a 7 exposure stitch of 645D files done on my laptop with only 6GB RAM and no separate scratch disk. (CS4, Win 7, 64 bit)

Tom

 

[dick]

Hi Dick,

Apologies if I have confused matters! PS does indeed use a scratch disc (and there are lots of settings etc. you can choose related to how this works). Thing is, from experience seems to suggest that doesn't help with the specific operation of stitching. I would be delighted to be shown to be wrong about this btw!

Stitching is one of the thing I want to be able to do - another is dof/focus stacking ,,, are the problems similar?

 

[aboudd]

Hey Ed, I have been doing a few of these panos with my 645D. A couple of suggestions. Reduce your files to 8 bit, you won't see the difference. If you are going to do any processing in ACR, be sure to save notes on the settings so all of the images have the same factors. Then create parts of the pano is sections. I have 10 gigs of RAM on my MacPro and I process 6 or 8 captures at a time so I don't have to go plow, plant, grow, and harvest zucchini before the file is processed. Just name them pano 1,2,3, etc. Then after you have the set, merge those. After you stitch the parts together, do your final post processing.

 

[gimlor]

I believe you need to enable the 3D hardware (GPU) support (within CS5) which might speed things up. If you have a good card and its supported for that feature it'll help . Then you can go finetune the GPU support options which become availalbe.

 

[Ed Hurst]

Hey guys - thanks for all the suggestions and ideas.

I shall certainly use 8 bit files if I need to (with my current CS3 set up and its lack of access to RAM, that made no difference with 23 files, but maybe it could allow me to do it with CS5 and about 15gb of RAM, if it won't do it with 16 bit files). As for doing the stitching in sections, see my notes on this above - I have found that doing this messes up the perspective in a complicated way that is hard to fix later; it creates 'bowing' around the centre of each of the frames I create as an intermediate step which then means that the overall file at the end has a strange set of multiple distortions (if you know what I mean). I am hoping to be able to do the thing in one go and avoid this problem; I know there will still be distortion, but am hoping it will be a single sweep of distortion that can be more easily fixed. Do you know if doing it in one go achieves this (i.e. simpler distortion compared to the multiple distortions that occur with doing the files in stages)?

Thanks also for the idea about enabling 3D hardware (GPU) support. I do have a good card, so will try this (though I have read conflicting accounts of whether this actually helps or not)...

Once I get CS5 installed, and perhaps more RAM (if needed), I will report back!

Ed

 

[engel001]

May I ask a question related to the above? Early versions of Photoshop limited the size of a file that PS3 and earlier could open to 30,000 pixels wide or less. Now with PS5, I can open and edit much larger panoramas with relative ease on my i7 Windows 7 64bit machine (some third-party plugins refuse to work properly, however). When it comes to saving and uploading for printing, I need to reduce the pixel dimensions to less than 30,000 wide again, otherwise PS5 won't let me save as a JPEG. My last steps in PS5 are sharpening, convert to sRGB and reduce bit depth. But saving as a JPEG doesn't seem to be possible. Where is the limitation, how can I circumvent this? I gave up in-house printing some years ago.
Thanks, -Christopher

 

[etrump]

My suggestion is to convert to 8 bit tiff, stitch, then back to 16 bit. Any manipulation after you stitch will be much smoother using 16 bit.