Nikon responds to RAW file encryption controversy

Nikon has sort of gamely responded to the controversy from earlier this week about them encrypting white balance metadata in RAW image files for their for their D2X and D2Hs digital SLRs with a statement of "clarification" that does little to shed any light on the situation. They're not denying that the data is encrypted, but what they'd like to make clear is that they have a software SDK which "any legitimate software company" can get for free after filling out a "written application".

So we asked their rep a simple question: Can you name one software company that's used the SDK to build a software app that is compatible with NEF, Nikon's proprietary RAW digital photo format? His response: that information is confidential, we won't release the names of any licensees.  Ok, so even though Nikon refuses to give us a single example to back up their case, theoretically there are applications out there besides Nikon's Nikon Capture software that can read the data from encrypted RAW files. But why encrypt the data in the first place, especially since it's already a proprietary format? This one he couldn't answer at all, except to say something about how encrypting white balance metadata "protects the integrity" of the overall file. Uh, ok…

[Thanks, Brando]

[Justin @ Dec 19th 2005 12:13AM]

�protects the integrity�

Suuuuure... Because the white balance is the most important part of the image data (otherwise, the main image data would also be encypted).
If it wasn't encypted, then hackers would hack into our cameras and give the images a horrible colour-cast, like pink or green or something!!!

[Riskable @ Dec 19th 2005 12:13AM]

I've been maintaining a gadget never-buy list over the years. Time to add Nikon to the list.

[David @ Dec 19th 2005 12:13AM]

And the winner is... Canon

In other news Nikon has collected one of this weeks asshat awards.

[Steve @ Dec 19th 2005 12:13AM]

Nikon alway slikes to do things the hard way. So now besides the fact that their cameras have the same design as they did 20 years ago while competitors freshen up their products looks, Nikon makes it difficult to use the files taken with their cameras.

Nikon sucks.

[Sean Foushee @ Dec 19th 2005 12:13AM]

So let me see if i understand something, ALL camera manufacturers use a proprietary RAW format, Canon including, and Nikon is the only one that encrypts specific data, but provides an SDK for companies like Adobe to create plug-ins to read this format. So the question is why is this a story? Why does it matter if Nikon's proprietary format encrypts data and Canon's proprietary format doesn't? Since each manufacturer has there reason for using their RAW format over a proposed standard like Adobe's DNG you should be happy that Nikon has an available SDK for third-party solutions other than forcing Nikon shooters to purchase their software to use the camera.

This is a non issue, and as far as I'm concerned no camera manufacturer comes out of the RAW debate smelling like a rose, so choosing to purchase a photography 'tool' based on something like this is ridiculous.

[Brad @ Dec 19th 2005 12:13AM]

They're not going to give you the tools, nor do they give a shit that you can hack it. They just want to be able to sue Adobe if they try. This is about Camera Raw and the new digital negative format. They want pro photographers forced to keep Nikon software in their workflow, and forced to keep their originals in Nikon file formats. This will make migration away from Nikon harder, and enables their ultimate fantasy, an all-Nikon (or Nikon partner) image workflow from camera to web/print/etc. Right now pro workflow leads to Adobe; they're trying to use the digital copyright act to gain an advantage that they can't get through programming alone.

[Brad @ Dec 19th 2005 12:13AM]

Read the release more carefully. Nikon didn't say they would let Adobe or anyone else use the format -- they said they'd let them APPLY to do so.

[Sean Foushee @ Dec 19th 2005 12:13AM]

Brad get your head out of the sand, most companies that offer API or SDKs for further development want people to apply or register in order to download and use them. Do you think Microsoft just gives away their SDK to anyone with asking a few questions? Nikon has a format they believe is the best for their equipment, so let them, Canon does the same thing only they don't have encrypted information. This isn't a story, its just a group of people who are either die hard Canon fans or hate any and all encryption and proprietary formats going nuts over a non-issue. Nikon has their reasons for doing this, so let them and if their format turns out to be a thorn in their side costing them sales they'll change, but if you do any research into digital SLRs you'll find that each manufacturer's RAW format has their positive and negative aspects, so like I said none of them come out smelling like a rose on this issue.

[Brad @ Dec 19th 2005 12:13AM]

Okay, get my head out of the sand...what do YOU think are Nikon's reasons for the sudden encryption, and condescension (and "Nikon knows best") aren't answers.

[Nate MC @ Dec 19th 2005 12:13AM]

I have to agree with Sean over this..

"This isn't a story, its just a group of people who are either die hard Canon fans or hate any and all encryption and proprietary formats going nuts over a non-issue."

That was my sentiments exactly. I don't know why people are so rabid over cameras, they are tools to utilize in making art. Oi vey.

[jfb3 @ Dec 19th 2005 12:13AM]

How to get a Nikon SDK:

http://support.nikontech.com/cgi-bin/nikonusa.cfg/php/enduser/std_adp.php?p_faqid=376&p_created=1035551940&p_sid=Mc_WNHDh&p_lva=&p_sp=cF9zcmNoPTEmcF9zb3J0X2J5PSZwX2dyaWRzb3J0PSZwX3Jvd19jbnQ9MSZwX3Byb2RzPTAmcF9jYXRzPTAmcF9wdj0mcF9jdj0mcF9zZWFyY2hfdHlwZT1hbnN3ZXJzLnNlYXJjaF9ubCZwX3BhZ2U9MSZwX3NlYXJjaF90ZXh0PXNkaw**&p_li=&p_topview=1

In short:
1 Download the "Request for Nikon SDK form" from one of the links below:

http://www.nikonusa.com/kdb/sdk/nikon_SDK_request.doc
http://www.nikonusa.com/kdb/sdk/nikon_SDK_request.pdf

2 Complete the form and mail to:

SDK Applications
attn: JC � Electronic Imaging
Nikon Inc.
1300 Walt Whitman Road
Melville, NY 11747
USA
3 After the form is received the appropriate license agreement will be sent to you for signing
4 When we receive the license agreement the proposal will be reviewed. If approved, a complete Nikon Software Development Kit will be mailed to you by US Priority Mail
--------------
They offer the kit to "bona-fide software development companies or individuals"

I haven't read the application yet so I'm unusure of any licensing concerns...

[Slaven @ Dec 19th 2005 12:13AM]

"I have to agree with Sean over this..

"This isn't a story, its just a group of people who are either die hard Canon fans or hate any and all encryption and proprietary formats going nuts over a non-issue." "

People, you have to be really ignorant about DMCA to not understand why this IS a story. Lexmark SUED a third-party ink cartridge manufacturer as they broke DMCA in a similar fashion. This encryption gives Nikon ability to sue anyone who provides a way to process Nikon's RAW files. This is the real reason they added that encryption - as the story points out, what other purpose of this encryption is there????

[h0mi @ Dec 19th 2005 12:13AM]

Stolen from slashdot:


They point out that this 'proprietary' format is accessible through the use of their 'proprietary' SDK, which is freely available to 'bona fide software companies' on written application. In other words: open source coders can butt out."

[mickster @ Dec 19th 2005 12:13AM]

Nikon never claimed that they were suing Adobe. Adobe made it appear. Sounds like a bunch of fear, uncertainty and doubt FUD!

INikon says the list is confidential but I actually read that both Bibble & Adobe actually signed and have the SDK. (Thomas Knoll from Adobe posted it a few weeks ago that Adobe does have the SDK-so they must have signed it-seems very, very wierd).

[Sean Foushee @ Dec 19th 2005 12:13AM]

Slaven, again so what? Do you expect every company out there to just make open source software? Companies that feel they have an edge (whether they do or don't) have the right to protect their intellectual property however they choose.

Here's a tip for all of you who are so upset, DON'T BUY NIKON... but for goodness sake any professional who doesn't consider all options when selecting a tool for their trade is a moron, plain and simple, and as I've said this is a poor reason not to choose a Nikon camera. The people who are in the market for this camera most likely have multiple bodies and systems already in their studio so I imagine having to buy a small piece of Nikon software for their techs to read the NEF format properly isn't a problem. Besides, if any of you were a professional shooter you'd already have spent hundreds of dollars on software to compliment Adobe Photoshop, so Nikon's software isn't a shock... its the industry.

Give it up, this isn't a CoolPix issue, this is a non-issue.

Oh and BTW all SDK's can be considered proprietary.

[mickster @ Dec 19th 2005 12:13AM]

#15, Sean is right. Nikon never claimed to sue anyone or even mention the DCMA.

[Doug R. @ Dec 19th 2005 12:13AM]

In photoshp's forums, Thomas Knoll admits that Adobe signed Nikon's agreement... and their photoshop will support d2x files - can't figure out what the issue is here?!

[Doug R. @ Dec 19th 2005 12:13AM]

This on Bibble's website:

Updated Bibble 4.2.2 in download section.

Bibble 4.2 Ships with new features including:

New Camera Support for:
Nikon D2X, 8800, 8700, 8400
Canon Rebel XT 350D, G6, S70, S60, Pro1
Olympus E300 Evolt, 8080
Minolta Maxxum 7D
Pentax *ist DS
Full D2X support including camera white balance and tethering
Tethered support for Canon 1Ds2, 1Dm2 and 20D (Windows)
Revised and enhanced color and tone.
Enhanced cropping .
Streamlined browsing
Many bug fixes
Free update for all existing 4.x licensees
**NOTE: If you have already purchased any version of Bibble 4.x, then your license key will be valid for this new version. Simply download version 4.2 from our website and install it to your computer. There is no additional upgrade process necessary.

Complete list of changes in 4.2.2

They apparently use nikons sdk too!

[furtim @ Dec 19th 2005 12:13AM]

Geeks like their evil, monolithic corporate villains, is all.

Granted, DMCA is a bad law, but this is basically a drop in the ocean as far as that goes. It may not be the coolest thing Nikon's ever done, but I think all companies are allowed their fair share of boneheaded mistakes. No need to villify them over it unless they prove worthy of villification in the long run.

[gopi @ Dec 19th 2005 12:13AM]

Sean writes, "This is a non issue, and as far as I'm concerned no camera manufacturer comes out of the RAW debate smelling like a rose, so choosing to purchase a photography 'tool' based on something like this is ridiculous."

There is a fundamental, qualitative difference between "we're going to use our own format, you're on your own if you want to figure out how to use it" and "we're going to actively put effort into making sure that only the people we decide are worthy can use our format."

Nikon will only give the SDK to people they decide are worthy, and under terms that are not yet public. If I call Nikon, I doubt they're going to give me the SDK. With Canon, say, I'm sure I could figure out their format, and not have any fear of breaking the law by doing so.

If Nikon's new camera only, for example, worked with film that had a special proprietary chip on it, would you find that a problem? Don't worry, any major, reputable film company can get the chip design for free. You won't even realize it's there!

Nikon is demanding control in an area they have no rights to control. It's my camera. They're my photographs. I object to Nikon trying to control what I do with my photographs after I take them.

I want my camera to be a tool. I don't want somebody else to be secretly controlling what I can do when my work leaves the tool - and that's what Nikon's doing.

[gopi @ Dec 19th 2005 12:13AM]

My background: I'm a professional developer, and an amateur photographer, currently using Nikon film gear worth $2k or $3k. Not a lot by professional standards, but I have enjoyed my Nikon gear for many years.

Sean, I agree that companies have the right to protect their intellectual property. However, my photographs are not Nikon's intellectual property, they're my property. I'm not asking for Nikon's help in reading the files. Rather, I'm objecting to them deciding to take explicit action which has _no_ purpose other than making me get their permission to get my files off at their best quality.

I really do want my camera to just be a tool. That's why I don't want to have to get arbitrary permission from somebody to make it work.

File formats are not legally protected intellectual property - the term IP is a bad one, since there are many different laws about many different forms.

Copyright doesn't cover file formats. Trademark law doesn't. The only thing that could cover them is patents - you can use a patented technique in a file format, the way that GIF images can be compressed by LZW, which was patented.

Nikon's choice of encryption of a specific, small subset of the format doesn't protect their file format as a whole, and doesn't stop anybody from reverse engineering it - rather, it merely stops you from publishing a tool that lets other people read the format (in the US, under the DMCA.)

Let me say that again: This technique does _not_ stop Nikon's competitors from figuring out anything nifty or unique that Nikon's done in this file format. Canon can look at the format and figure out anything they want to about it. They merely can't distribute a tool to bypass the encryption.

The only effect of this is stopping third parties from working with the files without Nikon's permission. I consider it equivalent to requiring me to only use Nikon-approved film in my camera, for example.

When I need Nikon's blessing for the software on my computer, my camera seems a lot less like the general purpose tool I want it to be. While most people won't notice these limits, I do develop my own image processing software. Personally. I am not a "legitimate software company." I'm just a random guy, who wants to press a button on his camera and get out an image that he can use in his own software. Nikon's gone to a lot of effort to stop me from doing this. I don't want their help, I just don't want them to actively subvert me.

[ed @ Dec 19th 2005 12:13AM]

Besides being retarded, those people here who are jumping on the hate bandwagon are technical incompetent. Have you ever heard of Canon's RAW format?

[Robert Mohns @ Dec 19th 2005 12:13AM]

Question: Does Nikon's SDK just hand its RAW data over to the processing application, or does it do RAW to bitmap conversion itself, then give out the bitmap? If the latter, then there may be an issue. the point of third-party RAW processors is that they (potentially) do a nicer job than the camera developer's -- or provide extra features, or maybe even are faster. If the RAW data is not truly made available through the SDK, then Nikon has really crippled non-Nikon software. If it is, though, then the main harm will be to open source efforts, as noted by others, and (to a lesser degree) to shareware. Remember GIF: once Compuserv started demanding royalties for use of their suddenly popular format, open source projects dropped support because they had no way to pay the fees, and shareware authors had their costs go up.

There is active development of open source and shareware RAW viewers/processors:
http://freshmeat.net/search/?q=RAW+files

[gopi @ Dec 19th 2005 12:13AM]

ed: Yes, I have, as a matter of fact. I don't keep up to date on the newest version, since I am not in the market, owning only Nikon lenses.

Personally, I want to access the raw data because I would like to have access to the bayer-pattern sensor data. I work with wide angle and fish-eye lenses, which unfortunately frequently have chromatic aberration. Once an image has been converted into a different format, you lose the precise location of each original sensor.

Thus, what I plan to do - and there are of course others doing similar things, I'm not the only one - is use empirically measured chromatic aberration information to adjust the pixel positions of the raw sensor. In other words, at the edge, green light may be focused a few pixels away from where it should be, whereas red light is at the right place.

CCD sensors have narrow bands of sensitivity, so it's much easier to do this accurately than with film. Doing it properly really does require the raw sensor data - the camera's image processing steps will blur the pixel positioning data and make the calculations much less accurate.

I'm sorry that ed finds me to be "technical[sic] incompetent", but I think he's making a mistake by assuming that just because some people don't understand the implications, nobody does.

If somebody was, say, unhappy about the way car companies use proprietary OBD-II extensions for diagnostic codes, and require the purchase of trade-secret tools from the car manufacturer, would you say they needed to actually know how to repair a car to think this was a bad idea?

Sure, most camera users don't really need raw mode. However, I think people are justifiably unhappy that Nikon's decided to arbitrarily limit which software programs can access this information.

People here who focus on the idea that most consumers won't use third party apps are missing the point. Very few people in the US go to protests, right? So why shouldn't we just limit the right to assemble peacefully? Just because a right is taken advantage of by few people doesn't mean it's reasonable to restrict or eliminate it.

Perhaps some people will take offence at my comparison, but based on some legal opinions, writing software that reads Nikon's raw formats is illegal under penalty of prison. Maybe I'm a bit odd, but I am offended at the idea of having to risk arrest and prison to extract _my_ photos from _my_ camera.

[Jarle Aasland @ Dec 19th 2005 12:13AM]

Here's at least one application that uses Nikon's SDK:

http://www.rawphotodesk.com

[Slaven @ Dec 19th 2005 12:13AM]

"Perhaps some people will take offence at my comparison, but based on some legal opinions, writing software that reads Nikon's raw formats is illegal under penalty of prison. Maybe I'm a bit odd, but I am offended at the idea of having to risk arrest and prison to extract _my_ photos from _my_ camera."

Amen! If you're fine with what Nikon is doing then that's good for you. But accept there are definite downsides to this format "improvement". And if you think that Nikon didn't think of DMCA implications while drafting this format revision you are pretty naive.

[DjSpin @ Dec 19th 2005 12:13AM]

could it be that the raw is encrypted so that the images can be proven legitimate? I believe canon had a special software package and special flash card a few years back that would work with one of their DSLR's to encrypt the images so they could be verified as legit... and i believe this was mostly marketed to law enforcement agencies. That way if a photo is taken as evidence, the original encrypted file could be entered as evidence in court to prove that the image was never altered. I haven't used the raw feature on my canon 10D yet... but it sounds like if I took a photo then doctored it in photoshop...it might be possible with the right software to save it back to raw format so it looks like the image is straight out of the camera? Or if it's not possible w/ current software, I bet it's not an overly complicated feat to make software that could save a photoshopped image to RAW format from a particular manufacturer.

So if you go shooting all your photos in nikon raw and theres no available software that can 'save' to nikon raw format... then you're covered if you ever need to verify that your photos are legit and not photoshopped.


I'm not saying nikon is doing the right thing... maybe there should be an 'encrypted raw' option (essentially what canon did with their special software and flash card.)

[Boss Sauce @ Dec 19th 2005 12:13AM]

EXR is the answer.

[ed @ Dec 19th 2005 12:13AM]

"Thus, what I plan to do - and there are of course others doing similar things, I'm not the only one - is use empirically measured chromatic aberration information to adjust the pixel positions of the raw sensor. In other words, at the edge, green light may be focused a few pixels away from where it should be, whereas red light is at the right place."

Sounds really hokey, mate. Granted that you'd be better off buying third-party software for this, you're still forgetting the fact that Canon has been doing proprietary RAW encryption since 2000 (at least.) Where was your crazy ranting then?

And what does encryption of white balance do to your little experiment anyhow? What you are describing as far as bayer pattern has NOTHING to do with white balance.

[gopi @ Dec 19th 2005 12:13AM]

"Sounds really hokey, mate."

Hokey or not, what I described is what professional researchers in the field are working on. I've seen results, I've seen papers, I've talked to people working on this, and what I described is very plausible and can produce really great results.

I've been dealing with the problems of limited dynamic range in QuickTime VR movies since around 1996 when I got my first Kaidan head and started shooting QTVR movies. I fixed it then by bracketing and using Photoshop to merge over and under exposed images.

"Granted that you'd be better off buying third-party software for this,"

Third party software? I'd be implementing algorithms that are leading edge research - I've gone to various paper presentations on the stuff I want to do, and the advanced parts aren't anywhere near third-party software yet, they're still in the research domain.

You can call it "hokey" all you want, but I've seen the results and it works.

"you're still forgetting the fact that Canon has been doing proprietary RAW encryption since 2000 (at least.) Where was your crazy ranting then?"

Canon hasn't made it _illegal_ for me to reverse engineer, merely not told me how they do it. As for ranting about companies that work to restrict third party access to their files, I have ranted more than enough on the subject.

Do you understand the difference between "undocumented" and "illegal to decrypt?" If I figure out Canon's formats, I can do whatever I want with my software. I can give it to you or to anybody I want. If I admit to decrypting the white balance information in a Nikon file, I can be arrested. Do you _really_ think that reading the output from a Nikon camera without Nikon's express, written permission should be a criminal offense?

"And what does encryption of white balance do to your little experiment anyhow? What you are describing as far as bayer pattern has NOTHING to do with white balance."

You are so fundamentally misunderstanding the point here, I'm not sure why I'm bothering to respond.

1. I would generally expect to want to use the white balance information. Why wouldn't I? It's useful information.

2. I'm concerned about the precedent here - I believe that part of the reason for Nikon's encryption of only _part_ of the file - a very strange thing to do - is to see what the reaction is, and to gauge commercial response to it.

[ed @ Dec 19th 2005 12:13AM]

"If I admit to decrypting the white balance information in a Nikon file, I can be arrested. Do you _really_ think that reading the output from a Nikon camera without Nikon's express, written permission should be a criminal offense?"

Where sir did you get the information that decrypting the Nikon white balance would get you arrested? Where sir did you get the idea that such an act is a criminal offense? You are not quoting facts but basing your tirades on DMCA hyperbole.

Nikon has not made it illegal to decrypt anything. It is not illegal to decrypt anything. Welcome to the United States.

"You are so fundamentally misunderstanding the point here, I'm not sure why I'm bothering to respond."

Kisses to you too, blowhard.

[gopi @ Dec 19th 2005 12:13AM]

ed,
"Where sir did you get the information that decrypting the Nikon white balance would get you arrested? Where sir did you get the idea that such an act is a criminal offense? You are not quoting facts but basing your tirades on DMCA hyperbole."

I've read the DMCA. Have you? I've been to DC and watched hearings on the subject, and listened to people such as Jack Valenti speak on the subject. "A little demagoguery never heard anybody" is a direct quote of his, referring to the action they took against Sony over Betamax.

The DMCA makes it illegal to circumvent technical measures that control access to copyrighted works - regardless of who owns the copyright. Under the Berne Convention, all works are copyrighted when they are fixed in a tangible form. One could argue about whether the white balance information was enough of a component of the work to garner copyright protection, but it's a grey area and it is possible a court would rule that.

If I receive any financial benefit, or any commercial advantage, it's criminal.

http://www4.law.cornell.edu/uscode/17/usc_sec_17_00001204----000-.html

"Nikon has not made it illegal to decrypt anything."

You're right, the DMCA did that, if the encryption is an access control measure controlling access to a copyrighted work.

"It is not illegal to decrypt anything. Welcome to the United States."


http://www4.law.cornell.edu/uscode/17/1201.html
(A) No person shall circumvent a technological measure that effectively controls access to a work protected under this title. The prohibition contained in the preceding sentence shall take effect at the end of the 2-year period beginning on the date of the enactment of this chapter.

(3) As used in this subsection�
(A) to �circumvent a technological measure� means to descramble a scrambled work, to decrypt an encrypted work, or otherwise to avoid, bypass, remove, deactivate, or impair a technological measure, without the authority of the copyright owner;

Oh, what is it that we see here? That's right. Decryption. Mentioned by name. Now, ed, you say it isn't illegal to decrypt anything. I see here that the US Congress passed a law that makes it illegal to decrypt some things.

Just because people like to exaggerate the DMCA, doesn't mean that it's not a bad law.

I'm sorry if I come across as antagonistic, but I don't appreciate being described as "technical [sic] incompetent" for my opinions on the impacts of a law - opinions shared with people who know far more about this subject that either you or I. Just to remind you, people already _have_ been arrested over the DMCA. Dmitry Sklyarov, for example. Professor Ed Felten was threatened for publishing a paper exposing weaknesses in an access control measure.

[Brad Selfridge @ Dec 19th 2005 12:13AM]

Ya'll don't get it. Having written software for hardware devices, it is common to encrypt the results so that you can TRY to protect your hardware design from your competition. Would you publish, to your competition, how your widget works? I don't think so. That's why Nikon gives away the SDK to approved developers. I doubt that they give it or even sell it to Canon.

This has nothing to do with trying to abuse the public.

[Michael Edelman @ Dec 19th 2005 12:13AM]

I suppose I'm in the minority here, but I don't see the difficulty. Nikon wants to hang on to some of their proprietary technology. In the long run, the market, and not Nikon, will determine what they do. If the use of a digital image standard, like the one Adobe is pushing, leads to more sales, you can bet that Nikon will adopt it. The fact that they're not saying no, we won't go that way, certainly leaves open the possibility.

[Michael Czeiszperger @ Dec 19th 2005 12:13AM]

I'm surprised no one in this discussion pointed out that the white balance information in the Nikon D2X RAW output isn't even encrypted, the darn thing's "encoded", which doesn't mean a thing since most of the data in any binary format is "encoded". What Nikon did do is change the way it was encoded, which they have a perfect right to do.

[David @ Dec 19th 2005 12:13AM]

Everyone seems to be hung up on trying to determine the legal or technical basis for the 'white balance encryption' issue. All that really matters to me is if ACR or PhaseOne CaptureOne COMPLETELY support the NEF format for the camera I want to use (DX2). If they don't then that certainly matters enough to me to make me switch my entire Nikon film system to Canon when I make the digital switch! (I own Nikon Capture now for use with a coolpix 8800 I use for fun and it is unbelievably slow to use and unacceptable to me.)