Adobe dng converter: a request

[Robert Campbell]

I normally use a Leica CL which writes dng files.

I tried a Leica D Lux 7 as a lighter alternative. It writes rwl files — it's a Panasonic clone.
So, I used Adobe dng converter to change the rwl files to dng. This worked very well. As a side-effect, when working with original dng files, it crunches them to a size roughly a third smaller.

I hadn't used this dng converter before — there wasn't any need. I discovered I could add the date and camera designation as well. I have used unique file names for all images for many years, in the format:
year-month-day_[camera name]_[original file name], thus: 2020-08-09_CL_[original file name]. This has been very successful.

I used Faststone to do this; it could read the date from the Exif and add that automatically. It's only available for Windows. It meant I could take images over a few days, and change all the dates and names automatically. I'm now Apple based, and this was getting to be a hassle.

With Adobe dng converter I have to enter the date manually, so I have to be careful about choosing the right ones for any particular date.

I'd like Adobe to add the option to read the Exif date automatically and add this to the file name.

Anyone else? Any other thoughts?

 

[pegelli]

Do you use Lightroom?

I do and when importing you can change the name and insert a camera designation, read and add the date from the exif and convert to dng (and do more).
I've got this import script set up for all my cameras.

I don't convert to dng but use a similar naming convention to yours:

PEG_[camera]_X_01234_YYYYMMDD where X is the amount of times the camera reset its file number counter and 01234 is the file number given by the camera when taking the image (and PEG are my initials).

This is all handled automatically on import.

Might be helpful for people that use Lightroom, if you don't sorry for the diversion.

 

[Godfrey]

I use Lightroom (now Lightroom Classic) to manage my image files.

I don't normally convert to DNG format as it is unnecessary if LR already supports the camera, I only do that when the DNG format saves 30-50% of the file space on some older cameras' uncompressed raw formats. (Not that space is that expensive anymore, but it's just good practice to be aware of minimizing space consumption.)

I have two file name patterns that are used to set my image files' names:

YYMMDD-{original file number}
YYMMDD-{tag}-{original file number}

where YYMMDD are the month day and year noted in the files' metadata as the capture date, {tag} is a mnemonic that I might assign if I feel it important to keep track of the event that some photos were made at, and {original file number} is the numeric component at the end of the original file name. I'm not so concerned with keeping things in any given order (the date and time of the original capture in the EXIF data does that already), I use this portion of the original file name as a file identifier when doing a search with the file system outside of LR or with the metadata search inside LR.

I also use YYMMDD-{seq03} as a file name for scanned film, indication the year, month, and date of the scanning and the sequence of the scans I did that day.

There's little reason to encode camera name, etc, into the file name since LR reads this part of the image metadata and has many different display capabilities in the Library module to read that information out for you. For scanned film, I add annotation as IPTC keywords to indicate camera, lens, film, date of capture, etc, when apropos. Occasionally, I'll go to the trouble of using EXIFtool to add these tags to the EXIF data in the images, but most of the time I find it isn't particularly useful anyway, why waste the time to set it up?

G

 

[Robert Campbell]

Thanks for the comments.

I use Lightroom, and have done since Lr 1.0. I don't import files, rather I synchronize files in Lr with an external HDD, from which I can make two more HDD backups. I was originally on Windows, so using Faststone was just a part of the workflow, and took up very little time. I've been on Apple for the last decade, but have kept the convention.

I have a file for each year, with sub-files for the months; I just find it easy that way. I don't make such a huge number of images that I need to automate things, though Chronosync does the work between the backup HDDs.

One reason for this naming performance was when I discovered after upgrading a camera's software that it had reset the counter to zero, so I (potentially) had files with the same names but taken at different times. No doubt this was my error in allowing the file counter to reset.

Perhaps I'm over cautious, but I feel that dng is likely to be around a lot longer than some of the makers' proprietary files, hence the conversion.

 

[Godfrey]

Thanks for the comments.

I use Lightroom, and have done since Lr 1.0. I don't import files, rather I synchronize files in Lr with an external HDD, from which I can make two more HDD backups. I was originally on Windows, so using Faststone was just a part of the workflow, and took up very little time. I've been on Apple for the last decade, but have kept the convention.

I have a file for each year, with sub-files for the months; I just find it easy that way. I don't make such a huge number of images that I need to automate things, though Chronosync does the work between the backup HDDs.

One reason for this naming performance was when I discovered after upgrading a camera's software that it had reset the counter to zero, so I (potentially) had files with the same names but taken at different times. No doubt this was my error in allowing the file counter to reset.

Perhaps I'm over cautious, but I feel that dng is likely to be around a lot longer than some of the makers' proprietary files, hence the conversion.

I haven't seen a single camera that Photoshop Camera Raw or Lightroom supported be dropped from the support list yet, and that's almost 20 years now. There's little reason to do DNG conversion on that basis, imo.

I also build my standard directory tree of original captures by date. I do it by year, and have LR create folders for the month and day when it imports the files. BTW: whether the import includes moving the file from the card/camera or not, it is still importing. If you're moving the files with another tool and just adding the files to the LR catalog, that's still an import but with the files already in place. All of my original image files are on external drives too.

It doesn't matter if files have the same name as long as the file system keeps them separated in different directories. The EXIF data allows you to sort them by capture date/time inside Lightroom, regardless of file name and disk location.

Many ways through this business. Whatever works for you and lets you do your photography is good.

G

 

[pegelli]

Thanks for the comments.

I use Lightroom, and have done since Lr 1.0. I don't import files, rather I synchronize files in Lr with an external HDD, from which I can make two more HDD backups. I was originally on Windows, so using Faststone was just a part of the workflow, and took up very little time. I've been on Apple for the last decade, but have kept the convention.

I have a file for each year, with sub-files for the months; I just find it easy that way. I don't make such a huge number of images that I need to automate things, though Chronosync does the work between the backup HDDs.

One reason for this naming performance was when I discovered after upgrading a camera's software that it had reset the counter to zero, so I (potentially) had files with the same names but taken at different times. No doubt this was my error in allowing the file counter to reset.

Perhaps I'm over cautious, but I feel that dng is likely to be around a lot longer than some of the makers' proprietary files, hence the conversion.

I have a file tree structure by year/camera/month.

And while importing/renaming the photo's from my card I point Lightroom to put them in the correct directory.
I tried your workflow as well (first copy into the tree on the hard drive (doesn't matter if it's internal or external), then import into lightroom (can't rename in that step) and then rename after import. In the end I found the method I use now simpler (less steps) but every time I import a card I have to check and double check that I'm applying the correct rename (with the camera name) and put them in the correct folder. There's more room to make little mistakes and rename with the wrong string and/or put them in the wrong place. There's probably many more combinations/permutations in workflow, I don't think my method is "best", I'm just providing the info since it might give other people an idea what (parts) could or would not work for them.

I have no fear that in the end proprietary raw formats no longer get supported and if it happens I can still convert to dng at that time. And dng doesn't copy over some manufacturer specific exif data which is another reason for me to just stay with the manufacturer raw format.

 

[Godfrey]

...
I have no fear that in the end proprietary raw formats no longer get supported and if it happens I can still convert to dng at that time. And dng doesn't copy over some manufacturer specific exif data which is another reason for me to just stay with the manufacturer raw format.

This is a bit of a red herring. If you're not using the manufacturers' own raw processing solution (if they provide one), this manufacturer specific proprietary metadata is most likely of no particular value anyway, since being proprietary no other processing software will generally know what to do with it.

G

 

[pegelli]

This is a bit of a red herring. If you're not using the manufacturers' own raw processing solution (if they provide one), this manufacturer specific proprietary metadata is most likely of no particular value anyway, since being proprietary no other processing software will generally know what to do with it.

G

Sometimes yes but not always. For instance on some of my cameras the actual focus point used is lost when converting to dng. Not essential but I consider it nice to have and look back where the camera actually tried to focus.

The other thing that is stripped is the "built in lens profile" for lenses that have the profile stored in the firmware, no problem in case your raw converter (eg. Lightroom or C1) has the corrosponding profile in their software, but if not it can be tricky (especially when using C1, which has much less built in profiles as compared to Lightroom). Lightroom actually uses the "built-in profile" if it can find it and doesn't default to it's own profile.

I agree it's all far from essential but it's not a red herring either imho.

 

[Robert Campbell]

Thanks for the interesting thoughts and comments.

My anxiety with proprietary raw types disappearing stems from an unhappy experience around 15 years ago — it's not directly digital.

I decided I should get my slides in order, scan them, and import them into Lr so I could properly catalogue them. I had slides going back to the late 50s — not great works of art, more like youthful indiscretions, but interesting and really only of sentimental value.

Looking through them before scanning I found that around 80%+ were unusable; they had faded so badly that there only ghosts left. And of the rest, a good proportion showed colour casts, sometimes to blue, but mostly to purple. Most of the ones that had faded were Agfa, Ferraniacolor, and Ektachrome but many Kodachromes showed this as well. Strangely, there were some which had badly discoloured next to ones that were really good and showed minimal if any change. I have scanned all the "survivors", but for so many, I cannot easily get rid of the purple cast — others here might be able to do this. Realistically, I can only make monochromes from them. I'm left with around 1,500 out of 10,000 slides. [I did scan a lot of prints as well, they were much better preserved.]

I wondered if I can't use my own images from 50 years ago, will my descendants be able to do so in 50 years from now — assuming that they might want to? Will all these makers' raw formats be available then? It's a bit like trying to get 127, APS or Instamatic film today; it's not impossible but it is difficult. [I don't have a suitable camera.] My father had a Kodak "Autographic" camera, and the film for this is long gone.

I used to work with Parallels and Faststone for the file renumbering, and this was straightforward, but I'm not inclined to pay through the nose for the required updates for Windows and Parallels just so I can use one freeware program. I do have an ancient Windows netbook which still works well enough for this single task.

I think for the present I'll stick with my established workflow; there may be easier and better ways, but I'm very used to it and reluctant to change.

This train of thoughts started with my purchase of a s/h Leica D Lux 7; I was impressed with the results here on the Leica forum, and I thought it would be a good and light camera for me. I find even a Leica CL set up can be a bit heavy at times. But I got very frustrated with the D Lux; the previous owner seems to have had some very particular [peculiar] ideas about how cameras should function. It took a total reset and then a work through the menus to get things to my way of thinking.

 

[pegelli]

Interesting thoughts Robert, I've also been thinking about what and how to leave my photo collection for the next generation(s)

This happened mainly after my father passed away almost 2 years ago and he had a colour slide collection of close to 100.000 (you can guess how I got the photography bug).
My brother, sister and I went through it and basically threw away 99%+ of all his landscape, cityscape and macro shots. They were mostly taken on vacations and trips after we left home and while it were nice pictures we didn't have an emotional connection with them.
But we kept all the photos of family and friends and other events we were part of, in total less than 10% of the entire collection. He started with colour slides in 1948 or 1949 and surprisingly they held up pretty well, some faded colours but still "recognisable".
I guess my children and grandchildren would judge my collection the same. So at some point in time I'll give them a stick with exported jpg's of those kind of photo's and accept that my carefully built catalog of raw files will just be lost. It's infinitely valuable for me but probably close to worthless for almost everybody else.

I might just put one copy on a stand-alone laptop together with a perpetual version of Lightroom (I still have a licence for 6.14) so that if anyone is interested to go through and even change my processing (s)he can do that but I judge the chances of that ever happening less than 10%.

In the meantime I'm also slowly scanning my colour slides (~10.000), again the ones of family and friends might interest my heirs, but the others are just for me.

Sorry for rambling about this, everybody looks at these things differently so what I think about it might very well be very different for you.

 

[Robert Campbell]

When I was digitizing the slides and prints, I put keywords in for family members as applicable. I made contact sheets using the keywords and printed them off with the image file reference number below each image, and gave them out one Christmas.

 

[Godfrey]

For instance on some of my cameras the actual focus point used is lost when converting to dng. Not essential but I consider it nice to have and look back where the camera actually tried to focus.

The other thing that is stripped is the "built in lens profile" for lenses that have the profile stored in the firmware, no problem in case your raw converter (eg. Lightroom or C1) has the corrosponding profile in their software, but if not it can be tricky (especially when using C1, which has much less built in profiles as compared to Lightroom). Lightroom actually uses the "built-in profile" if it can find it and doesn't default to it's own profile.

Hm. If the focus point is a piece of proprietary raw metadata, how does something that doesn't understand that proprietary format of the data look back at it? Or do you mean that you go back to the native file and the proprietary software to read the focus point? (As I've mentioned many times, I am mostly a manual focus shooter. I figure if I can't look at a photograph and see where the camera was focused, what does it matter really? and particularly if I wasn't doing the focusing? I find AF an occasionally useful convenience, that's all, and what's in the image is all that matters when it comes to focus... )

As far as I'm aware, the most recent revision of the DNG specification includes rules for the translation of non-proprietary lens profile information stored in the raw file AND for private maker data ("makernotes" is what I think it's called in the spec) that is proprietary in nature. I could be wrong on that, but when I've compared the same images from Lightroom that were rendered using the native raw file and the DNG translation, for cameras/lenses that I *know* the lens profile was injected at capture time into the raw, the results look identical which indicates that the profile was used the same way in both circumstances.

...
Sorry for rambling about this, everybody looks at these things differently so what I think about it might very well be very different for you.

Indeed. I have no "heirs" nor am I concerned over what they might value. If I think any of my work will be of interest in the future, I put it into a book or album. Finished photographs that I've rendered and pronounced as "done" are all that matters ... what happens to the thousands and thousands of unrendered photographs in my library is of very little concern to me once I'm gone. I surely don't think much of it will be of any interest at all to anyone else.

What does this have to do with being able to process *raw* files from an old camera? What of your intent in making those exposures is there if you never made the effort to render them into finished photographs? Why bother worrying about stuff that you didn't worry about in the first place?

I live accepting that I will die, and when I die my existence will be forgotten but for the small number of people who care to remember or tell the stories of my interaction with them. Whether any of my work has value to last beyond that is up to the work itself—when that matters to me, I render and finish the work, and publish it. Whether that means making a bunch of prints and handing them out to my relatives who care about them, or publishing them in a book to be archived in the public record, or posting them to this and other forums for people to enjoy ... it's the same deal to me. The rest of my exposures have about as much substance as smoke in the wind: It's there but it passes swiftly into nonexistence.

This has gone rather far afield from asking for a file rename in DNG Converter... To return to that subject, perhaps tangentially:

...I used to work with Parallels and Faststone for the file renumbering, and this was straightforward, but I'm not inclined to pay through the nose for the required updates for Windows and Parallels...

BTW: If you're now running on macOS, you can create a workflow using the Apple-supplied free application Automator to manipulate filenames any way you want, en masse. It's very easy to learn and figure out how to do that. Once you have a workflow defined that does what you want, you can also convert it to a little application that you just drop the files onto and it does the job as you want. I use Automator frequently for doing lots of little file system tasks like this ... It's much more reliable and much better at it than any of the freeware I've found.

G

 

[pegelli]

Hm. If the focus point is a piece of proprietary raw metadata, how does something that doesn't understand that proprietary format of the data look back at it? Or do you mean that you go back to the native file and the proprietary software to read the focus point? (As I've mentioned many times, I am mostly a manual focus shooter. I figure if I can't look at a photograph and see where the camera was focused, what does it matter really? and particularly if I wasn't doing the focusing? I find AF an occasionally useful convenience, that's all, and what's in the image is all that matters when it comes to focus... )

I'm a manual shooter for a large part of my images also but when using AF I find it sometimes nice to go back and see what the camera tried to focus on as a way to understand how my camera AF system chooses certain AF points (in order to make better use of it a next time). If you convert to dng this info seems lost, if you keep the original raw file it's still there. This item might not matter to you, I understand that, but it might matter to other people (like me) who look at this differently from how you do. To each their own (on both sides of this equation).

As far as I'm aware, the most recent revision of the DNG specification includes rules for the translation of non-proprietary lens profile information stored in the raw file AND for private maker data ("makernotes" is what I think it's called in the spec) that is proprietary in nature. I could be wrong on that, but when I've compared the same images from Lightroom that were rendered using the native raw file and the DNG translation, for cameras/lenses that I *know* the lens profile was injected at capture time into the raw, the results look identical which indicates that the profile was used the same way in both circumstances.

I would need to test that, I did this test a few years ago and when (temporarily) removing the appropriate lens profile from Lightroom the native raw was still corrected by the profile provided in the metadata and the dng from the same file remained uncorrected. Maybe they fixed that in the meantime but since I'm not afraid that at some point native raws will no longer be supported I'm not converting to dng. So it's not a high priority for me, and probably will not get done until I maybe get more interested in dng in the future.

 

[Godfrey]

Regards focus setting info
.. To each their own (on both sides of this equation).

For sure!

I just figure that after a month or so of testing and learning a new camera, or another month or so of testing and learning at a later date, I understand enough about the AF proclivities of the camera that I know when I might find it useful and when not. I don't think about it after that other than to use what I learned as best I can manage, and look to the photograph to see how well focus was achieved in the particular image. Easy.

Regards lens profile metadata retained in DNG
...I would need to test that, I did this test a few years ago and when (temporarily) removing the appropriate lens profile from Lightroom the native raw was still corrected by the profile provided in the metadata and the dng from the same file remained uncorrected. Maybe they fixed that in the meantime but since I'm not afraid that at some point native raws will no longer be supported I'm not converting to dng. So it's not a high priority for me, and probably will not get done until I maybe get more interested in dng in the future.

I don't normally convert to DNG much any more since nowadays my Digital cameras are all supported and include some level of lossless compression on their raw file output ... I used DNG conversion for a long time to reduce disk space issues brought on by uncompressed raw files and to bring raw files from new cameras into use when LR had not yet been updated for the original raw file spec for a given camera.

The tests I did to ensure that the original raw and the DNG conversion matched were done with a variety of native Micro-FourThirds cameras and lenses to ensure that DNG conversion included the lens profile injected into the raw data since mFT lenses are designed with that lens profiling in mind. IIRC, that was somewhere around DNG v1.2 to v1.4 update time, when this style of lens profile metadata injection was a new thing and Adobe had to revise the DNG spec to accommodate it. Seems a while ago now, but exactly when is hard to say ... I know that the Panasonic LUMIX G I owned came out about 2008 and the current DNG spec is v1.4 dating from 2012. All the other cameras that use this facility basically use the same mechanism that mFT pioneered when it was introduced, including Leica's. (Of course, Leica M, CL/TL, SL series cameras all produce native DNG format raw files so however they do the lens profile injection is embedded into the DNG file from the start.)

It's easy enough to test: I have the Olympus E-M1 and a couple of native mFT lenses, I'll make a couple of exposures and see what a native raw vs DNG converted raw look like. I can test them in two different raw converters: Lightroom Classic and VueScan (since I know for sure the latter does not handle lens profile corrections at all).

Fun stuff.

G

 

[pegelli]

The tests I did were comparing to dng conversions from 2013/2014 with the latest dng converter at the time.
All I can tell is that as soon as the Lightroom profile was removed from the program the native raw file is still corrected (based on the baked in profile) and the dng was not.

Maybe it differs by camera/lens brand or the dng converter at the time wasn't updated to the latest spec yet but the difference between the files was remarkable, both for distortion as well as vignetting.

As soon as the lens profile was put back into Lightroom the converted files (from the native raw and dng) are identical to such an extent that if I put them on top of each other as photoshop layers and then looked at the "difference" blending mode I get a full black image, so apparently the Lightroom profile is identical to the baked in profile.

 

[Robert Campbell]

...

This has gone rather far afield from asking for a file rename in DNG Converter... To return to that subject, perhaps tangentially:

BTW: If you're now running on macOS, you can create a workflow using the Apple-supplied free application Automator to manipulate filenames any way you want, en masse. It's very easy to learn and figure out how to do that. Once you have a workflow defined that does what you want, you can also convert it to a little application that you just drop the files onto and it does the job as you want. I use Automator frequently for doing lots of little file system tasks like this ... It's much more reliable and much better at it than any of the freeware I've found.

G

Yet again I expose my distinct lack of knowledge of Apple and the inner workings!

That looks to be a very useful tip, Godfrey. Thank you!