Raw workflow with the E-P1?

[Streetshooter]

Jono,
So what is the best solution at the moment.
The Oly software is, well....
Any ideas......
Don

 

[jonoslack]

Jono,
So what is the best solution at the moment.
The Oly software is, well....
Any ideas......
Don

HI Don
Best results? Oly software . . . . if you have the fastest computer in the world it might even be bearable!

Me - I'm using Capture One - I'm pretty sure there will be full support very soon. I guess there will also be ACR support in the next few weeks (together with Lightroom and CS4, but not, one assumes CS3).

I should say that my opinion that Adobe don't do a good job with 4/3 files is very much that (i.e. my opinion), I'm sure others will disagree. My personal feeling is that they don't try so hard with second tier cameras (the A900 is an obvious case in point).

If you aren't comfortable with changing Capture One, then why not shoot RAW+jpg superfine for a week or so, and save the ORF files of the best shots so you can re-do them when the right converter comes along?

 

[jonoslack]

But then there are the non-standard parts of dng files, the "maker notes" section.
Picking apart the dng standard and low and behold it has at least one area that requires specific camera by camera support.
The next issue is what revision of dng?
The latest supports all sorts of lens correction goodies, but as far as I know, is not yet implemented by any camera manufacturer or raw processor for that matter.
-bob

HI Bob
I quite agree, but at least the files can be read. For instance, Aperture reads .dng files - if Apple have specific camera information for that file (as they do for the M8 for instance) then they will apply it. If they don't, then they'll ignore it.

It might not be perfect, but in effect it means that when a new camera comes out which uses DNG, then you'll be able to process raw files immediately with any converter which supports DNG properly (which, in this context, Aperture seems to do) (worth noting that as Godfrey has pointed out, Apple do have shortcomings with respect to linear DNG).

When they DO add the camera support, you can apply that to existing files, or not, as you wish.

Robert's question is pertinant, as I've found a very good 'interim' solution is to convert files to .DNG using Adobe's free dng converter, and then opening the subsequent file in Aperture - again, when camera specific support is added to Aperture it's picked up.

Part of me feels that I'd rather do that anyway, as I'm quite suspicious about the future of camera specific RAW files.

Wouldn't it be nice to just use jpg!

 

[Streetshooter]

Jono,
That appears to be the solution for the moment.
Thanks for the advice....if I start to hate the IQ,
now I can blame you......
Your other post are stimulating to say the least.
Thanks again..... Don

 

[Godfrey]

But then there are the non-standard parts of dng files, the "maker notes" section.
Picking apart the dng standard and low and behold it has at least one area that requires specific camera by camera support.
The next issue is what revision of dng?
The latest supports all sorts of lens correction goodies, but as far as I know, is not yet implemented by any camera manufacturer or raw processor for that matter.

Camera Raw/DNG Converter v5.4 and Lightroom v2.4 are both supporting DNG Specification v1.3 right now. ...

Since this new specification is only months old at most, I expect it will take other vendors a short while at least to incorporate the changes it includes.

 

[Godfrey]

Thanks Robert - but they always do such a bad job with 4/3 sensors (this is just my humble opinion).

Which, needless to say really, I completely disagree with.

I've tested every RAW processor with every camera I use and I find I get the same or very similar results out of all of them with suitable understanding and skill applied. My results are shown in the photos I post, that's the evidence supporting my opinion.

I just wish that all these companies would use dng files (As Leica did with the M8 ricoh do) then we wouldn't have to have this struggle each time there's a new camera.

Pentax has done so since the K10D also.

 

[jonoslack]

HI There Don

Jono,
That appears to be the solution for the moment.
Thanks for the advice....if I start to hate the IQ,
now I can blame you......

Be my guest . . . as long as you don't know where I live I'll be okay :ROTFL:

Your other post are stimulating to say the least.
Thanks again..... Don

Hmm - is that meant to be a compliment - if so, thank you , I'm afraid some people are finding them irritating

Truth is that I like taking pictures - I don't really like doing PP. I made a pact with myself that I would never again buy a camera not supported by Aperture . . . I'm not sure what went wrong

I seem to have made a concession for Capture One for those other cameras - I like the way the batch processing works, just not the requirement for having to DO batch processing!

 

[jonoslack]

Which, needless to say really, I completely disagree with.

Of course :ROTFL:

Pentax has done so since the K10D also.

There ARE lots of tempting things about Pentax aren't there - the K7 is a nice looking camera.

 

[Godfrey]

As far as I understand it (Godfrey might correct me here ), the answer is:

1. If they are linear DNG then Adobe is doing the demosaic, so YES (they will still be bad)

So what you're saying is that in your opinion Adobe's demosaic algorithm is deficient?

How do you tell this? What measurement methodology are you using?

In my opinion, having evaluated ten or twelve different RAW converters over the past several years, while all demosaic algorithms are not equal the range of variation is indeed pretty small in practical terms. Other parts of the rendering process end up making a bigger difference in the quality of the final image.

2. If they are not linear DNG then Adobe isn't so the answer is NO. (they will be as good as the other converter).

At its base, DNG is simply an alternative and standardized way for the RAW data file to be structured and written to disk. It is a container format, specialized from TIFF. Any application that knows how to read the data it contains will be able to process the data with any algorithm that is preferred, up to the limit of the application's support for the types of data that are contained.

 

[jonoslack]

I've tested every RAW processor with every camera I use and I find I get the same or very similar results out of all of them with suitable understanding and skill applied. My results are shown in the photos I post, that's the evidence supporting my opinion.
.

The trouble is that every month there is a new iteration of one of the major raw converters, and each time that happens things change.

Added to which we don't all have as much understanding and skill as you do (I certainly don't).
We don't all take the same sort of subjects.
We don't all like the same kind of results (for instance I hate smudgy detail, but I never mind noise much).

You don't post all your pictures with different converters (or if you do I haven't seen it), so, at best the evidence is very circumstantial, and really only relevant to yourself . . . . which is fab. But your opinion is still only an opinion and probably worth almost as little as mine.

 

[Godfrey]

The trouble is that every month there is a new iteration of one of the major raw converters, and each time that happens things change. ...

In my opinion, constantly worrying about using only the best RAW converter is a notion of a nature similar to constantly worrying about having the best lens or the best body or the best camera bag, ad nauseam. It does more to get in the way of doing photography than it helps. No camera is perfect. No lens is perfect. No software is perfect. Nothing is perfect.

That said, LOTS of cameras, equipment, software are good enough. Pick a setup and work with it, learn to exploit it to do what you want. It will never be perfect, no matter how much change or improvement happens. The art lives in making what you have chosen produce what you imagine. Else you live forever on The Hamster Wheel of Progress.

I was not posing my 'evidence' formally. One of the reasons I don't post my intermediate test results is that they will all look the same since the processing quality differences are so small. I pick what I work with based on other criteria (usability and workflow integration, mostly).

 

[sizifo]

I hope nobody here considers it rude. But, I'd be super grateful if some of the RAW energy here could be channeled into the thread re: problems I've been having with DNG conversions following the upgrade to the most recent version of aperture http://forum.getdpi.com/forum/showthread.php?p=120358#post120358 . Even if it's just to say bad luck, and that you're not seeing anything similar :thumbdown: .

Cheers. I'm running out of options here

 

[jonoslack]

So what you're saying is that in your opinion Adobe's demosaic algorithm is deficient?

How do you tell this? What measurement methodology are you using?

In my opinion, having evaluated ten or twelve different RAW converters over the past several years, while all demosaic algorithms are not equal the range of variation is indeed pretty small in practical terms. Other parts of the rendering process end up making a bigger difference in the quality of the final image.

You may indeed be right that it isn't the de-mosaicing which is at fault - it could be some other part of the process - I'm not sure how it's possible to separate the functions. Maybe I was using the term too loosely, for which I apologise (like perhaps the term application code?).

As I've not found anything I use that can read linear DNG files except Adobe I have no way of comparing the results.

As for my methodology:
I take a number of photos of different types and process them in whatever converters I'm looking at. I then use my eyes - because that's the measuring equipment I use to analyse all parts of my pictures - both on screen, and particularly in print. Numbers are more scientific, but, IMHO have less to do with real life situations.

Unlike you I haven't compared many cameras this thoroughly, and not for some time. However the Olympus E3 the Leica M8 and the Sony A900 are three where I found that Aperture got more detail with less noise and better colour (in my very subjective opinion of course). The Nikon D700 and D3 seemed to be better with Lightroom - hence my lightly grounded suggestion that Adobe put more effort into the mainstream cameras.

At its base, DNG is simply an alternative and standardized way for the RAW data file to be structured and written to disk. It is a container format, specialized from TIFF. Any application that knows how to read the data it contains will be able to process the data with any algorithm that is preferred, up to the limit of the application's support for the types of data that are contained.

I think we know this don't we? However, as I understand it, with linear DNG files the demosaicing is already done, and cannot therefore be able to be processed again (unless, of course, the original file is also embedded in the container). If I'm wrong about this, please correct me. Mind you, I think it's rather beside the point as I don't think there is any conceivable reason for using a huge and unweildy linear DNG file now that the latest version of DNG converter will create normal ones. I'll put a note by someone else in the next message which you can comment on:

 

[jonoslack]

I hope nobody here considers it rude. But, I'd be super grateful if some of the RAW energy here could be channeled into the thread re: problems I've been having with DNG conversions following the upgrade to the most recent version of aperture http://forum.getdpi.com/forum/showthread.php?p=120358#post120358 . Even if it's just to say bad luck, and that you're not seeing anything similar :thumbdown: .

Cheers. I'm running out of options here

HI There
I'm really sorry - It was me being rude - apologies.

Right - my unfortunate answer is that I've been using 2.1.3 for some time now, and I didn't notice what you've indicated with the DNG conversions. However, I suspect that the real truth is that once an import / wedding / holiday / party / month is done, I tend not to revisit it unless I want to use shots for a different album, and when I do that I tend to make some more adjustments, so I don't think that I would have noticed the change.

So, I suppose the bottom line is to say 'hard luck'

 

[jonoslack]

At its base, DNG is simply an alternative and standardized way for the RAW data file to be structured and written to disk. It is a container format, specialized from TIFF. Any application that knows how to read the data it contains will be able to process the data with any algorithm that is preferred, up to the limit of the application's support for the types of data that are contained.

Last word from . . .


Jeff Schewe, writing in a 2005 issue of PhotoshopNews.com

reported that Adobe's DNG converter application "allows you to either keep the DNG in the un-demosaiced form or to demosaic the file and convert to a linear file. Generally, you'll want to keep the file in its un-demosaiced form as there is a space savings. Linear DNGs have already been converted to RGB files and therefore are larger."

The article as presented includes an inserted editor's note:

"[Digital imaging expert] Bruce Fraser sent the following comment regarding Linear DNG: 'The only reason to use Linear DNG is to feed the file to a DNG reader that can't understand the particular flavor of DNG that DNG Converter creates for the specific camera, or to act as an interchange format—e.g., lens correx with DxO can write out a linear DNG that ACR can read. This is typically NOT an option people want, not because of size considerations, but because it's no longer really raw—it's half-baked. All the operations that take place during demosaicing are set in stone and can't be redone.'"

I'm sorry for that interjection Sizifo - It's not my words, but I think it closes the conversation with respect to comments on linear / 'normal' dng.

I really will shut up now.

 

[tashley]

In my opinion, constantly worrying about using only the best RAW converter is a notion of a nature similar to constantly worrying about having the best lens or the best body or the best camera bag, ad nauseam. It does more to get in the way of doing photography than it helps. No camera is perfect. No lens is perfect. No software is perfect. Nothing is perfect.

That said, LOTS of cameras, equipment, software are good enough. Pick a setup and work with it, learn to exploit it to do what you want. It will never be perfect, no matter how much change or improvement happens. The art lives in making what you have chosen produce what you imagine. Else you live forever on The Hamster Wheel of Progress.

I was not posing my 'evidence' formally. One of the reasons I don't post my intermediate test results is that they will all look the same since the processing quality differences are so small. I pick what I work with based on other criteria (usability and workflow integration, mostly).

Philosophically I agree with you but the question is inevitably one of degree. In other words, how much of an advantage can be gained from using one converter/lens/camera/etc rather than another.

I fought for ages against the conclusion that C1 was a visibly and significantly superior RAW converter to Lightroom, because I have a lot invested in the Lightroom workflow. However, I am totally convinced that at least with a 1DSIII, a 5DII, a P45+ back and an M8 I get results with C1 which are better than LR and which I cannot, with any amount of tweaking, replicate in LR.

Now I am am a man who could not de-mosaic an ancient Greek bathroom with a claw hammer so I can only judge by the results I get but I simply can't buy the argument that all RAW developers are within such close spitting distance of each other that there's no important difference between them... if that's roughly what I take you to be saying?

Best

Tim

 

[tashley]

Of course :ROTFL:



There ARE lots of tempting things about Pentax aren't there - the K7 is a nice looking camera.

I keep a K20D as spare and it has a wonderful user interface and a great 300mm lens. It's more fun to use than a 5DII. It's files aren't very good though. :-(

 

[nostatic]

I keep a K20D as spare and it has a wonderful user interface and a great 300mm lens. It's more fun to use than a 5DII. It's files aren't very good though. :-(

I guess that begs the question, "compared to what?"

The files off my K20d are certainly "better" than from the E-P1 (noise/resolution). But they aren't as good as I've seen from a900 (some of Jono's files). I haven't looked at 5D raw, but have looked at older Nikon (D70) and didn't think those were good at all by comparison.

 

[Godfrey]

You may indeed be right that it isn't the de-mosaicing which is at fault - it could be some other part of the process - I'm not sure how it's possible to separate the functions. Maybe I was using the term too loosely, for which I apologise (like perhaps the term application code?).

As I've not found anything I use that can read linear DNG files except Adobe I have no way of comparing the results.

It goes without saying that if you can't compare results between processes in a way that clearly separates the criteria you are trying to judge, making a useful assessment is impossible.

VueScan and dcraw, and most of the RAW conversion applications based on the distributed dcraw libraries, can all read DNG files with linearly represented data.

BTW, it is not only Bayer mosaic RAW data files from cameras that can be represented in DNG format. Some devices, for instance scanners, can output RAW data with the appropriate driving application (like VueScan). The data is acquired from the scanner in a linearly represented RAW (not gamma corrected) form ... VueScan can output that data either to a TIFF file or to a DNG file. Same goes for data captured by Fovean sensor equipped cameras ... it's not a Bayer mosaic. So a DNG file with linearly represented data is a container for another generic RAW data form, nothing more.

As for my methodology:
I take a number of photos of different types and process them in whatever converters I'm looking at. I then use my eyes - because that's the measuring equipment I use to analyse all parts of my pictures - both on screen, and particularly in print. Numbers are more scientific, but, IMHO have less to do with real life situations.

My methodology is to make a set of representative reference exposures, saving to RAW+JPEG so that I have both the RAW data and the in-camera rendered JPEG data (with the JPEG data set to maximum rez and minimum compression) that I will do testing with. I then spend time with each RAW converter ... typically up to a week's time depending on how much time I have and my level of interest ... learning how to get the most out of each converter application's capabilities.

THEN I do my comparisons using my eye and various tools in Photoshop, using the tools as analytic instruments and comparing the out-of-camera JPEG against the RAW conversions saved out as TIFF files. I compare on noise and resolution primarily, color rendering can be easily changed.

I test both default and customized settings in the RAW converter .. defaults vary a lot, some better than others, but with customized settings
they all come down to being very very similar, in my experience.

...However, as I understand it, with linear DNG files the demosaicing is already done, and cannot therefore be able to be processed again (unless, of course, the original file is also embedded in the container). If I'm wrong about this, please correct me. Mind you, I think it's rather beside the point as I don't think there is any conceivable reason for using a huge and unweildy linear DNG file now that the latest version of DNG converter will create normal ones. I'll put a note by someone else in the next message which you can comment on:

The DNG files from the G1, when lens correction metadata was in the file, had to be represented as a demosaicked linear form with the lens corrections embedded in order that the intent of the DNG format was met (that all DNG compatible applications would be able to read the data) as well as Panasonic's intent (lens correction injected by the lens applied to the RAW data during demosaic time). It was obviously not what Adobe wanted to do but this form of metadata to be applied at RAW conversion time was a new thing. They acted speedily to update the DNG spec in order to enable both saving disk space and the ability to retain the original mosaic data form with the DNG converted files, without losing the LC metadata, for all applications that are DNG capable. I applaud their efforts.

 

[jonoslack]

HI Godfrey

You are much much much more conscientious about your testing than I am; we both know that, but I've done enough to establish that Aperture suits me best for the cameras I use, both in workflow and results.

Changing is out of the question, but right now, as I have an E-P1 I do have to work out some way of dealing with the RAW files. Capture One is a backup I like (and one which I have used over the last few years in special circumstances) - as Tim says, it's hard to ignore the fact that it gets the best detail out of most cameras. It's very simple to get Capture One to get good results from the E-P1.

In my opinion, constantly worrying about using only the best RAW converter is a notion of a nature similar to constantly worrying about having the best lens or the best body or the best camera bag, ad nauseam. It does more to get in the way of doing photography than it helps. No camera is perfect. No lens is perfect. No software is perfect. Nothing is perfect.

Couldn't agree more . . . . erm, except that lenses really do matter, and I guess that the Leica 50mm f1.4 'lux and the 75 'cron are as close to perfect as one is likely to get. I fixed on Aperture a couple of years ago as my software of choice, and imperfect though it may be I've seen nothing to make me change my mind.

My methodology is to make a set of representative reference exposures, saving to RAW+JPEG so that I have both the RAW data and the in-camera rendered JPEG data (with the JPEG data set to maximum rez and minimum compression) that I will do testing with. I then spend time with each RAW converter ... typically up to a week's time depending on how much time I have and my level of interest ... learning how to get the most out of each converter application's capabilities.

But if you're spending weeks testing RAW conversion software, and, as you say, they're almost identical, it sounds like you aren't taking your own advice - sounds like ad nauseam to me.