Wednesday, November 4, 2015

Canon RAW Conversion Test–Nov 3, 2015

A reader was recently asking me about RAW vs JPEG for a Canon workflow and in the process of replying to those series of emails, I had an ADD moment.

For fun I started comparing the RAW conversion of a Canon CR2 raw file using the latest RAW conversion software from Adobe and Canon as of November 3, 2015. To make things more interesting, Canon actually has two versions of its Digital Photo Professional (DPP) software floating around for legacy reasons, so I was curious as to the result I’d get using what amounts various different conversion techniques.

This is actually an exercise I do time to time anyway because as software and firmware gets updated, the processing changes often yield different results (not always for the better). While this article is Canon specific the same concepts apply to any camera that creates a RAW image that is supported by computer based software and/or Adobe products. If you haven’t done this exercise on your own already, I’d encourage you to do so!

Default in-camera JPEG


In-Camera JPEG (downsized in Photoshop,watermarked by Zenfolio)

This is when the raw data is converted in-camera and applies all of the editing settings like color space, picture style, white balance, etc… to give you the image you see on the rear LCD panel of your camera. Even if you “only shoot raw” you see this image as the thumbnail to your RAW file and it’s the data that the camera’s histogram that you see is based on.

This raw processing is burned into the firmware of your camera and it will only get updated if something significant warrants a change. As a result, you’ll often fine that the in-camera RAW processing is very different from software processing – especially if the software is getting updated regularly.

Lightroom CC 2015.2.1 (ACR 9.2.486)

SNAGHTML16597e64
Lightroom CC 2015.2.1 / ACR 9.2.486

Lightroom uses Adobe Camera RAW (ACR) to convert your raw data in a CR2 file into something you can see and use. The develop tab allows you to tweak those settings, but Adobe also reverse engineers these files to try to create something you’d like as a starting point for your editing process (in the Camera Calibration section – see the “Process” option). The default is Adobe Standard which simulates Adobe’s ideal, but “camera” processing styles are designed to loosely mimic how your camera process images. In the case of Canon, they are trying to map to your Picture Styles.

Digital Photo Professional  (DPP)

DPP comes in the legacy version, DPP 3.x and under, which supports nearly every CR2 file Canon has made. Recent updates have added support for easily updating lenses without having to get a whole new version of DPP. Here’s what 3.x looks like:

DPP 3.15
DPP 3.15

It’s a crude and nasty UI that you’ll be tempted to dismiss. In fact, most people never bother to even install DPP off the disc so many don’t even know it exists. While the user interface does stink, the quality of the algorithms under that ugly UI are very powerful.

DPP 4.x

DPP 4.3.31.0
DPP 4.3.31.0

Someone at Canon got the bright idea to do a whole new DPP from the ground up that had much better support for things like printing, soft proofing, adjustments and more. However, getting parity with the previous version proved to be more than was expected so the new version only works with the latest cameras. What this means is that if you try to open up a CR2 file from an older camera, say a 5D Mark II or 1D Mark IV it may not be able to read the RAW file (although updates are promised – eventually).

While the new UI is better, it still is a far cry from Lightroom or ACR. As a result, I still rely on it for the underlying processing and not the user features.

In theory one should be able to “only shoot RAW” and generate the JPEG’s that would normally be rendered in-camera. Sadly this isn’t the case because the RAW processing done by DPP will use the latest technology vs the camera which uses its firmware – typically with much older processing techniques. However, it can also be frustrating when you see something you love on your LCD only to have to spend a lot of time getting your RAW image to match your CR2 thumbnail file. In fact, I’d argue that the quality of the firmware based processing is often superior to software alternatives.

Comparison

When you zoom in at 100% and compare the layers against each other you’ll notice that DPP 4.3 and Lightroom are pretty close to each other in the way that they process the raw data. The colors are a bit more rich and the highlights aren’t as blown on the DPP 4.3 version, but you have to look carefully to see the difference.

DPP 4.3.31.0 Raw Conversion 100% Crop
DPP 4.3.31.0

ACR 9.2.486 via LR CC 2015.2.1 Raw Conversion 100% Crop
Lightroom CC 2015.2.1 / ACR 9.2.486

In-Camera Default Raw Conversion 100% Crop
In-Camera JPEG

DPP 3.15 Raw Conversion 100% Crop
DPP 3.15

Where things get more interesting is that the old 3.15 version of DPP generates the file that is closest to the in-camera JPEG, but notice how none are identical. This is one of the big reasons why I try to shoot RAW+JPEG whenever I can as I like the in-camera processing since that matches what I saw on the rear LCD. It’s the softest of the image, but it’s typical Canon warm and vivid.

Click here to scroll through larger versions of the images converted in this article.

Notice the different results

If you have a good eye, you’ll notice that all methods result in different images – even if there are some similarities. This is a long running problem because the software in DPP is simulating what the camera does and its constantly being updated. The in-camera processing only changes if the firmware changes, and even then it’s rare for it to change unless there’s a problem. What this means is that you’ll never really be able to take the RAW and generate a JPEG that matches what you saw on your LCD (or CR2 thumbnail).

As a result, if you like what you saw when you were chimping the LCD and you want the option to use that photo, then you’ve gotta shoot RAW+JPG. If you don’t care then shoot straight RAW and just take whatever Adobe, or DPP gives you.

Advice for Canon Shooters

I love RAW and always shoot RAW, but I don’t always need to use the RAW file. For many images the default in-camera JPEG is a perfect starting point for editing. If I did my job right it’s 90%+ ready to publish in-camera. This means I can make some quick edits using my favorite plug-ins then print or publish to the web.

If I screwed up in camera, am printing large or doing commercial work where every pixel matters then to go to RAW. I always have that insurance on hand by shooting the RAW file, and I never delete my raw files.

If I’m not happy with how Lightroom or ACR processes the file so I start by processing the RAW in DPP and exporting it as a 16-bit TIFF using the Wide Gamut RGB color space (Edit | Work Color Space) using the File | Convert and save feature.

Some may also prefer to use other raw processors discussed in the next selection.

RAW Processing Alternatives

Of course there are other RAW converters out there that offer results that differ from the ones above. Some argue they are better, others suggest they are simply doing the technical equivalent of the processing above plus applying their own creative edits to an image up front. Two of the better RAW editors that I’ve used are Piccure+ and DxO OpticsPro.

For completeness I’ve included how a couple of them handled this file as well.

Piccure+ 2.5.0.55

Piccure+ 2.5.0.55
Piccure+ 2.5.0.55 (learn more)

Piccure+ is powerful, but it is painfully slow with the most awful user interaction experience I’ve seen in my 26 years of software development thanks to its lack of a proper save implementation.

I’m told by the founder that the design is optimized for batch operations whereby you choose your project folder up front and then when you work on the file the image is stored there. The problem is that the location where you set your project info isn’t visible when you are doing your file processing so it becomes a mystery as to where your file went. What’s more the way you get back is via the back button which is sometimes replaced by the cancel button which isn’t exactly intuitive either.

Eventually you get used to the user-interface quirks and while it is slow to work, it does a great job. It’s batch mode design can be helpful if you want to give it a bunch of files to process overnight while you sleep, so if you love its RAW conversion then there’s a way to love it without experiencing the pain of processing. 

Here’s how it handled this file:

Piccure+
Piccure+ created a darker, saturated and softer version

At first the image appears oversaturated, but if you look carefully it seems it has done a great job of color balancing the image for the maximum dynamic range and tonal range of each color. The net result is an image that looks visually pleasing without appearing to be overdone – in my opinion.

The default noise reduction and sharpness settings do leave the image softer than Adobe, and Canon typical do, but that can be adjusted to suit your desire. Personally I’d still rather leave the noise reduction to Noiseware after the fact.

DxO OpticsPro 10.5.1.848

DxO OpticsPro 10.5.1.848
DxO OpticsPro 10.5.1.848

DxO OpticsPro has a wealth of powerful features making it act like Lightroom and Perfectly Clear rolled into one. The advantage to this is that you end up with a lot of powerful editing done in one place and it can be saved as a preset for default processing of your RAW images. DxO’s experience with sensors and cameras comes into play to create corrections that promise to minimize the negative effects of noisy sensors and lens distortion / vignetting.

Here’s how it handled this file:

DxO OpticsPro
OpticsPro really brightened up the shadows

DxO does a lot of cool stuff and has noise reduction software that I think is superior to Lightroom and DPP, but still not as good as Noiseware. The image that gets created is pleasing but it is different size than the others so you’ll notice some pixels shifting around if you look at all of the images in the sample gallery.

Conclusion

Click here to scroll through larger versions of the images converted in this article.

Licensing restrictions for this file required me to use a watermark and limit the file size, but it’s still easy to tell what changes between each version when viewing larger files. All images are copyright Ron Martinsen and may not be redistributed, linked or repurposed without written permission.

Personally there’s no wrong way, so this isn’t really a call to action to change your workflow. Instead, it’s about awareness of some things to consider. What’s more, some of the “I only shoot RAW” zealots may find that life is much simpler when you use a little storage space to have the option of using the JPEG for non-critical scenarios (like the 200 shots of your kids or pets), and just process the RAW when they actually need to.

Where to order

SNAGHTML17b57743

I’ve got a special offer for Piccure where you can save 10% when you CLICK HERE and use the coupon code RONMART as shown above (available via credit card only).

DPP comes free with your camera on a CD, but you can also get it off Canon’s website using your camera serial number. Be sure to check out my article entitled Guest Blog: Five reasons to try out Digital Photo Professional by Andrew S. Gibson to learn how to master cool features with DPP.

Adobe Camera Raw can be free if you download the Adobe DNG Converter off their web site, but most people will use it via Lightroom or Photoshop.

DxO OpticsPro (available on Amazon) is the most expensive of the bunch, but it fancies itself as a Lightroom alternative with exclusive features that normally require plug-ins so some consider it a fair deal.

Other articles you may enjoy

If you enjoyed this article, you may also enjoy these:

Disclosure

If you make a purchase using links found in this article, I may make a commission. It doesn’t cost you a penny more, but it does help to support future articles like this.

If you enjoyed this article, please support future articles like this by making a donation or saving money by using my discount coupon codes. Either way, your support is greatly appreciated!

Click here to learn more about how this blog is funded. Residents from Arkansas, California, Georgia, Illinois, Kansas, Maine, Michigan, Minnesota, Missouri, Nevada, New Jersey, North Carolina, Ohio, Rhode Island, Tennessee, Vermont, Washington, West Virginia, and Wyoming (and where prohibited by law) may be prohibited from using the links to make purchases, so please consider making a donation instead. This blog is intended for freelance writing and sharing of opinions and is not a representative of any of the companies whose links are provided on this site.

The opinions provided are of Ron Martinsen alone and do not reflect the view of any other entity

6 comments:

Philip R Olenick said...

Interesting.

I'm assuming at the outset that the camera and all five RAW processors were set to default processing including lens geometry and color fringing correction.

It looks like the closest color match to the JPEG - and thus to the LCD review image - comes from the 32-bit "Ides of March" (3.15) version of DPP (followed closely by DXO 10.5.1), while the newer, 64-bit version of DPP (4.3.31.0) is fairly-well matched by ACR 9.2.486, but neither looks like the camera JPEG.

(The Piccure+ is a complete outlier, much darker and more saturated than any of the others.)

The main difference between the JPEG, DPP 3, and DXO is the differing amounts of distortion correction done by DPP 3.15 and DXO.

Since Canon knows its own lenses better than anyone else, if your goal is accurate distortion correction and faithful color rendition of Canon RAW files, I'd stick with the 32-bit DPP.

You could mimic the pre-Lightroom "Bridge image browser to Photoshop for editing with a stopover in ACR" workflow by using DPP 3.15 as the image browser and RAW file processor, and have it ship the image as a Wide Gamut TIFF into Photoshop." No stop in ACR needed, since DPP combines Bridge and ACR's capabilities.

For those coming from Bridge who hate Lightroom's requirement to explicitly import images into its library, that makes the 32-bit DPP 3.15 the winner.

For those who're comfortable with importing new images into Lightroom, in order to get Lightroom's library and virtual copy capabilities, DPP 3.15 is still the winner, since the main reason for using DPP in an Adobe workflow is to get color rendition that matches Canon's intent and distortion correction by those with access to the lenses' designs. Only DPP 3.15 meets both objectives.

I see little reason to bother with DPP 4 at all. DPP 4 may get you better lens correction than ACR but it doesn't get you accurate color as your starting point.

Adobe needs to make DPP 4's color rendition more like 3.15 - otherwise the only thing that recommends DPP 4 as a pre-processor is that it might be marginally-faster than the 32-bit DPP 3.15, but at the price of losing consistency with Canon's color intent.

PS For shots taken with non-Canon lenses, DPP, not having information about the lenses in its database, disables geometry and color fringing correction, so those will have to be done inside Lightroom or Photoshop.

Philip R Olenick said...

A short postscript to my comment: 32-bit DPP 3.15 supports all of my Canon lenses, including last summer's 10-18 EF-S IS STM, and my EOS 70D body, as well as all my older bodies and Powershots. 64-bit DPP 4 supports all my lenses, but snubs my XTi, G9, and G12 Powershots, displaying only the embedded JPEG thumbnails in those cameras' RAW files.

3.15 doesn't support the very latest xD cameras, stopping at the Mk II models of the 1D and 5D, but just about all other EOS bodies are supported, including the 7D II. Similarly, its support for the Powershots stops just short of the recently released G5X and G3X.

Canon needs to include its newer cameras in DPP 3.15 and/or make DPP 4 more color-consistent with its JPEGs and support its older cameras.

Or give Adobe the recipes to let ACR accurately mimic DPP 3.15 for all of its cameras!

Philip R Olenick said...

So I tried using DPP 3.15 as if it was Bridge + ACR.

Opened a standard 70D RAW file, used DPP's lens correction tools to remove geometry distortion and chromatic aberration, and on the Tools menu chose Transfer to Photoshop.

After a little while, I got a popup notifying me not that it couldn't find Photoshop but that Encoding has Failed. This happened repeatedly. Closing all other applications and all closable system tray processes made no differences.

So I had DPP 3.15 save it out as a TIFF - but there's a big problem: if you want it to save a 16-bit TIFF (to maintain the image's status as a RAW equivalent), it won't save any EXIF information. It will only do that for 8-bit TIFFs. So when I open the TIFF up in Photoshop, it has no shooting data: not the shutter speed, not the ISO, not the aperture or the focus distance, not even what lens and camera were used.

In frustration, after closing even my browser and email, I tried the Transfer to Photoshop menu item again, this time by means of the keystroke equivalent, Ctrl-P.

No error message, and Photoshop opened! The image came in as a TIF, and it even knew what kind of camera it came from, the shutter speed and f/stop, the ISO and metering mode,and that it was aperture priority - but not what lens was used!I suppose that since the lens corrections will have already been done by this point, it doesn't much matter - but it's galling - and it suggests that the transfer used an 8 bit TIFF, to include what EXIF info it did.

I tried the same thing with DPP 4 and ran into exactly the same shortcomings: the Convert and Save As dialog only saves EXIF with 8-bit TIFFs, and the Transfer to Photoshop routing omits the lens information.

Arrgh!

Philip R Olenick said...

My experiments with DPP last night showed me that neither version lets you create a 16-bit TIFF that includes the shot's EXIF information - that's only allowed if you let DPP knock the color bit depth down to JPEG's 8 bits.

That means you have to choose between Lightroom and Photoshop knowing the details about the shot's creation or giving them a full-quality image to work on. Until that Hobson's Choice is removed, DPP is disqualified from being anything more than a quick way to make JPEGs when you're in a hurry.

Since Ron's post showed that DxO does RAW file processing that preserves the colors just about as well as DPP, I downloaded the one-month trial of DXO Optics Pro 10 Elite today.

It's optimized to act as a plug-in alternative to Lightroom's Develop module. Seems comfortable enough if you've adapted to Lightroom's catalog approach, and would preserve the virtual copy system of Lightroom if you've implemented that (someday, someday).

You *can* open a RAW file directly with standard file tree navigation inside DXO, and then export the result to Lighroom (pre-configured) or, with some fiddling, directly into Photoshop (unfortunately, to preserve full-quality, you need to define that kind of TIFF save and then open the result in Photoshop - the "send to other program" option doesn't let you configure it to send a full quality TIFF).

However, using DXO as a front-end file browser and RAW file processor is hobbled by its lack of the contact sheet grid of lots of photos both Bridge and Lightroom have, which lets you scan through lots of pictures at once - so DxO really functions best as a Lightroom plug-in. That means the best way to use it is:
1. Import all your images into Lr's catalog,
2. Find a picture to work on with Lr's browsing tools,
3. Have Lr export it to DXO for basic RAW file development (it has most of the tools ACR has),
4. Have DXO ship the result back into Lr, and then
5. Do your fiddling with things like the healing brush, clone stamp, etc. in Lr (or have Lr send it to Ps for that, plus layer work).

I really, really, really have to take the time to read through the Andrew Gibson ebook I bought about Lightroom's advanced virtual image management!

Philip R Olenick said...

I'm buying both DXO OpticsPro Elite and Viewpoint 2, its plugin for wide-angle lenses. I get great RAW processing - and it has rescued my EFS 10-18 lens, which otherwise gave me so much geometric distortion that it wasn't very useful - this combo makes its output very nice.

I tried both choices in exporting DXO's output back to Lightroom, since you have the choice of TIFF or DNG. Both give you 16 bit RGB with full metadata (I had Lightroom send them into Photoshop to be sure), but the DNG result looks nicer to me - the TIFF looks a bit too contrasty, while the DNG has a smoother tonality, with more detail in both shadows and highlights.

To enable you to tell the various versions apart, it appends DXO to the end of the filename (before the dot DNG or TIF).

Philip R Olenick said...

On reflection, using a DNG to send the image back to Lightroom from DXO may have short-circuited the color rendition advantage, by bringing it in as a DNG to be interpreted by Lightroom!

But I do like what its output looks like.