The GetDPI Photography Forum

Great to see you here. Join our insightful photographic forum today and start tapping into a huge wealth of photographic knowledge. Completing our simple registration process will allow you to gain access to exclusive content, add your own topics and posts, share your work and connect with other members through your own private inbox! And don’t forget to say hi!

LR destroys your original DNG files

glenerrolrd

Workshop Member
:angry:

No one seems to be giving this issue the time of day ..so I thought I might be direct. LR alters your original DNGs even when referenced without moving.

My workflow is pretty conservative . I import to a folder structure under Mac OS and maintain backups of this folder structure. This is completely independent of LR or any other software.

When I import the folders to LR ...I specify reference the files without moving.

After a few days posting to both adobe and the usual forums....I understand that LR posts the sidecar information (normally contained in a .xmp file) directly into your original . There appears to be no doubt that it has altered your DNG that you assumed was safe. The recommendations I received indicated that to preserve the original out of the camera ..you have to back it up independent of LR. :wtf:

To make matters worse it appears that if you specified compressed DNG you just developed your original and baked in the profile . This kills taking the same file into C1 because it will not view it as an original raw and will not apply the camera profile. I have not done this .

The other issue for those of you using LR as your DAM system and exporting from LR the selects for reprocessing ..you will get the stuffed DNG . My tests show this works OK into C1 but others are stating on the LUF that the files are cooked.

Can this really be new information after more than 2 years of LR processing?
 

Bob

Administrator
Staff member
Nope,
LR has been doing this for awhile.
made some of my canon files unprocessable by anything but LR.
Just one of the reasons I won't use it anymore.
-bob
 

Dale Allyn

New member
I appreciate your aggravation, Roger. I use a similar file structure, in that I work from a root-level directory for image files and don't actually import into anything. One of the reasons that I'm not a LR user is that I had never liked the library methods of it, but I have other UI issues with it as well. However, I do keep two copies of my out-of-camera files just in case I do something stupid, or an app. breaks something.

To me, it's absurd to have any app. modify one's DNGs, rather than just read them (unless one instructs an app. to modify the file). I'm not one who has adopted the DNG format because I don't buy that TIFF is in danger of extinction, nor is .psd IMO. In fact, I went back to saving as .psd since C1 insists on labeling its completed raw conversions with ".tif" extensions.

It will be a real drag for users if Adobe don't offer a clean way to migrate from LR2 to LR3 (I suspect that they will), but the altering of one's DNG would be disturbing to me if I wanted to work the file in a different app and didn't feel it was the same as I intended from the raw conversion. Just another reason for me to avoid apps. like LR. I appreciate the heads-up on this.

Edit to add: Bob posted while I was typing. Adding Bob's info to the mix makes me even more certain that LR will not be a tool I choose.
 

Jeremy

New member
It will be a real drag for users if Adobe don't offer a clean way to migrate from LR2 to LR3 (I suspect that they will)
Tom Hogarty has said on the official LR blog that the release version of LR3 will import LR2 and LR3Beta catalogs. They do not want people importing LR2 catalogs into the Beta because it is not for production use.
 

Mike Hatam

Senior Subscriber Member
Roger,

Yes, there is a danger of LR modifying the original DNG files, and it's important to understand when/how this can happen, to avoid an unintended consequence.

You pointed out the two main culprits: saving LR edits into the DNG file as instructions (rather than in a sidecar file), and compressing the DNG file (affects the associated profile).

However, both of these are also completely controllable through settings, and it's possible to set up a workflow that prevents both from happening.

Here's what I do...

In the "Catalog Settings", under the "Metadata" tab, be sure that "Automatically write changes into XMP" is unchecked. During normal workflow, be sure to never go to the Metadata menu and select "save metadata to file". If you avoid that, your edits will be maintained in the LR library, but not in the actual DNG file, preserving the native DNG for other apps to use.

Likewise, avoid selecting "Update DNG Preview and Metadata" under the Metadata menu. That option will compress the DNG, saving file space, but introducing some color profile compatibility issues with C1 and potentially other converters. Avoid selecting this, and you won't have the issue.

So while it's true that LR CAN modify the DNG, it's also possible to work in LR in a safe completely safe mode, if you're aware and careful of these two options.

Mike
 

Jack

Sr. Administrator
Staff member
Roger,

Yes, there is a danger of LR modifying the original DNG files, and it's important to understand when/how this can happen, to avoid an unintended consequence.
Exactly what happened to me when I first tried it, along with a few unintended file deletes that were totally unrecoverable, and those two reasons are why I refuse to use the program any more...

Don't get me wrong, I understand my issues were "user error," but I am not a moron and the moves I made or didn't make were not intuitively wrong. IMO, for a company that espouses its software performs totally non-destructive edits, Adobe should have a more intuitive UI that helps prevent unintended screw-ups and in this case, loss of files...
 

Dale Allyn

New member
Adding to Jack's remark, it seems that the methods Mike recommends should be the defaults and that one should have to intentionally alter the settings to end up where the over-writes occur to the DNGs if one so desires.
 

Terry

New member
Adding to Jack's remark, it seems that the methods Mike recommends should be the defaults and that one should have to intentionally alter the settings to end up where the over-writes occur to the DNGs if one so desires.
I believe they are the defaults. I just went back and double checked my settings on 2.5 and they were as Mike suggested.
 

Bob

Administrator
Staff member
Well, somewhere along the line they changed on me.
Those options should just be removed.
The dng "standard" is a treacherous thing too. Essentially TIFF with a bunch of extensions none of which cannot be accommodated in tiff IMO.
It is a good strategy, for Adobe and them alone.
-bob
 

docmoore

Subscriber and Workshop Member
Adding to Jack's remark, it seems that the methods Mike recommends should be the defaults and that one should have to intentionally alter the settings to end up where the over-writes occur to the DNGs if one so desires.
Peter Krogh recommends a DAM workflow that imports files and duplicates them such that an archival file is left untouched and its duplicate is used to edit and process. Wonder why.

Storage is cheap...trust no one, be it with your original files or the key to the single malt cabinet.:ROTFL:

Bob
 

glenerrolrd

Workshop Member
Roger,

Yes, there is a danger of LR modifying the original DNG files, and it's important to understand when/how this can happen, to avoid an unintended consequence.

You pointed out the two main culprits: saving LR edits into the DNG file as instructions (rather than in a sidecar file), and compressing the DNG file (affects the associated profile).

However, both of these are also completely controllable through settings, and it's possible to set up a workflow that prevents both from happening.

Here's what I do...

In the "Catalog Settings", under the "Metadata" tab, be sure that "Automatically write changes into XMP" is unchecked. During normal workflow, be sure to never go to the Metadata menu and select "save metadata to file". If you avoid that, your edits will be maintained in the LR library, but not in the actual DNG file, preserving the native DNG for other apps to use.

Likewise, avoid selecting "Update DNG Preview and Metadata" under the Metadata menu. That option will compress the DNG, saving file space, but introducing some color profile compatibility issues with C1 and potentially other converters. Avoid selecting this, and you won't have the issue.

So while it's true that LR CAN modify the DNG, it's also possible to work in LR in a safe completely safe mode, if you're aware and careful of these two options.

Mike
Thanks Mike ..finally after dozens of posts over on the Lightroom forum ...the information thats needed. I knew somebody over on this forum must know the answers. Now I know what to do.

I really find no excuse for Adobe allowing this to occur...even if there is a work around. Most people assume that their raw files will never be modified unless they explicitly export a new version. The default settings should keep you out of trouble.

There appear to be several users on the Leica forum that have destroyed the original DNG and can not use C1 with its profiles. They can still import them but they appear as processed DNG with the adobe profile baked in.
 

glenerrolrd

Workshop Member
Peter Krogh recommends a DAM workflow that imports files and duplicates them such that an archival file is left untouched and its duplicate is used to edit and process. Wonder why.

Storage is cheap...trust no one, be it with your original files or the key to the single malt cabinet.:ROTFL:

Bob
Bob

I have that for 90% of my original files. The problem for me is that sometimes my back up will be my cards until I get home. So normally I have a copy (originals) on the MacBook and the cards. So my back ups would be made from what I thought were my unaltered raws on my MacBook. Lot of good that does?

I know how to design and install back up schemes as I did this professionally in situations where there was a lot at stake . The difference seems to be that Adobe decided it was ok to update the originals without making this clear.
 

dfarkas

Workshop Member
My settings are just like Mike's. When I import, it is usually directly from the card and I use the Copy method. I don't let LR make its own folder. Instead, I place the pictures in a folder that I create and name (ex. Seattle - Oct 2009). At that time I assign keywords that would apply to all images (ex. Seattle, LHSA, M9, travel, Pacific Northwest, etc). If you use the Copy method, there is a checkbox that says "Backup to:" and allows you to specify a directory of your choosing. So, if you wanted to, you could copy from your card into your selected and named directory (just like you are doing manually in Finder) and copy to a backup location (like an external HD) at the same time.

David
 

Steve Fines

Member
Imageingester does a great job of automatically generating the two paths.

Bob
+1 II is an excellent program with a lot of thought put into archival storage and backup.

One thing I found right away when importing M9 files with it, however, is that it cannot modify the Leica DNG file.

With Canon and Nikon I have always used II to 1. make an archival copy of the original and 2. Make a DNG with all the metadata as my 'working master' to develop derivative files.

With II, when adding metadata to the Leica DNG it cannot write to it, but instead generates an .xmp sidecar. It is only via by using Abode's DNG converter (which is what I assume Lightroom uses) that metadata can be written into the file.

To me it is a tad frustrating that I have to run my Leica DNG files through the Adobe DNG converter to insert metadata into them.
 

Godfrey

Well-known member
One of the values of DNG files, to me, is that all my metadata edits are stored in one file along with the original RAW data. Since I see absolutely no point to using other RAW conversion software if I'm going to use Lightroom, I really like the fact that it embeds in the DNG all the information, calibration, etc I've applied to the image.

If I wanted to use Capture 1 instead of Lightroom to process my RAW files and use Lightroom to manage them, I'd first bring the files into Capture 1, do whatever processing is required, output them as TIFFs, and import the TIFFs into the LR catalog. I do this using LR and Aperture today ... Aperture's book making facility is useful, I don't like its processing UI very much and it doesn't handle my RAW files, so the workflow moves the files from camera to Lightroom, to processed TIFFs to an Aperture project for book making.

YMMV, obviously.

Another way to keep your original files sacrosanct, if that's important to you, is to elect the option in the Import dialog to make a backup copy of the original files to a separate location from your working file repository. Then you don't have to be careful about using the "Save metadata to file.." command or any other embedding of data into the DNGs ... the originals are safely ensconced in a backup file repository completely separate from the files that Lightroom is using as the working file repository.
 

glenerrolrd

Workshop Member
Mike

After checking the Lr settings ....I don t think we quite have it yet. The options that are std have two settings that are relevant. On both my LR2 and LR3 versions.....the settings are not checked for exporting a .xmp. This has never changed for me . But there is another option to write the develop settings into jpeg,tiff and psd formats. This seems to be whats happening with the DNG formats.

The other issue brought up by Sandy(Corner Fix) on the LUF..is that when an Adobe product updates a DNG file ..it reformats the file to the full DNG std. This process can make it unrecognizable as a camera raw for some software products. I am pretty sure that with C1 we are OK except if you compress the metadata. My files look ok and are recognized as M9 files.
 

docmoore

Subscriber and Workshop Member
Bob

I have that for 90% of my original files. The problem for me is that sometimes my back up will be my cards until I get home. So normally I have a copy (originals) on the MacBook and the cards. So my back ups would be made from what I thought were my unaltered raws on my MacBook. Lot of good that does?

I know how to design and install back up schemes as I did this professionally in situations where there was a lot at stake . The difference seems to be that Adobe decided it was ok to update the originals without making this clear.

Roger,

I have about eight books on LR in all its multiwondered glory from 1 to 2 ... waiting for the deluge of LR 3 books to hit. On the whole all of them recommend writing to the file's XMP so that you have duplicate information in the file itself and in the catalog. None give more than a passing mention to this problem as the majority assume that you will use LR as a major player in the file-process-store sequence.

I had checked the preferences but have not used LR for much other than screening. I then edit in CS4 as a copy. If your files are unedited in LR you can export as original DNG without the changes appearing...still seen in C1.

After losing a few files and having untold versions of pics all over my personal universe I decided to read Peter Krogh's DAM...his best take-home message was parallel paths. I have implemented it on occasion but this thread is a clear key as to why it matters.

Most Macbook HD's are fairly small...it may be worth picking up a couple of Seagate 500 - 1Tb externals to run out multiple copies prior to returning home. My H3D is usually tethered to a HD Imagebank so it holds a lot of excess...either a good thing or you risk a lot. I do not travel with a large camera and use a small Vosonic reader to save backups from my DSLR or Small cameras.

Glad I occasioned this thread...thank you!:thumbup:

I have a hard time coming up with a more persuasive DAM software than LR as it recognizes almost all raw and DNG extant. I have a copy of Extensis Portfolio that I updated only to learn it cannot deal with many raw files. So I assume that I will make archival copies prior to LR and use it as a tool to store and only edit copies.


Bob
 

glenerrolrd

Workshop Member
There are of course alternative workflows ...but a good rule of thumb is to maintain an archival copy of your original camera raw. As raw convertors improve or specialized applications emerge going back to the original raw may be desirable. You are probably thinking.....no kidding?

But ..you choose LR as the core of your DAM . This is how you find things. If you are like me..you process 000s of images each year. I wanted to reprocess 30 of my street images . Its incredibly easy to find things in LR. So I do a select on the 30 images and export them to a new folder managed by Mac OS folder structure. I specified export originals . If these were .NEF s they would be originals and a sideccar .xmp would be created . But as DNG s they get the full treatment from LR . So for some applications no problem......but I didn t now the deal was re formating to adobe standards. Thats not my definition of non destructive.

So I will have my unaltered archive and no way to find anything?

I do know how to work around this but its not easy . This isn t a new problem with integrated software products.....but LR could have done this differently simply by treating the DNG file as a raw . They aren t messing with the .NEF files.
 

docmoore

Subscriber and Workshop Member
Mike

After checking the Lr settings ....I don t think we quite have it yet. The options that are std have two settings that are relevant. On both my LR2 and LR3 versions.....the settings are not checked for exporting a .xmp. This has never changed for me . But there is another option to write the develop settings into jpeg,tiff and psd formats. This seems to be whats happening with the DNG formats.

The other issue brought up by Sandy(Corner Fix) on the LUF..is that when an Adobe product updates a DNG file ..it reformats the file to the full DNG std. This process can make it unrecognizable as a camera raw for some software products. I am pretty sure that with C1 we are OK except if you compress the metadata. My files look ok and are recognized as M9 files.
Probably more a function of Adobe reformating to full DNG standard...

Martin Evening in his LR2 books says that the develop settings into jpg, tiff and psd do not affect the DNG file...which is why there are two choices. If you check write into j,t or psd when you open the file in Bridge it opens in Camera Raw...if you dont it opens directly into CS4 bypassing CR. His solution for redundency is to check XMP but not write into j,t or psd. Seems kinda backwards to me...he suggests more security and smaller file size for the j,t and psd's.

Bob
 
Top