Update 2: Apple has issued an official response.
Update: This story has generated followups from Thomas Ricker at Engadget, David Pogue, Andy Ihnatko and Dan Moren at Macworld. Researcher Alex Levinson points out that despite the Guardian's positioning of the location data as a shocking revelation, this data stash was well-known to forensics experts for some time (it's even mentioned in Levinson's book on iOS forensic analysis); as we noted below, analyst Christopher Vance had written about it in 2010 as well.
All concerned also note that while the location data is cached on your phone and on your computer, there is absolutely no evidence that Apple or your cell carrier are accessing that file to track you over time (although it's likely that AT&T or Verizon know your location anyway, since your phone registers itself with cell towers as you roam). That doesn't mean Apple is off the hook, by any means: US Senator Al Franken has written to Steve Jobs looking for answers about how this data is gathered, what it's used for and why it isn't encrypted.
The Guardian reports that independent researchers Pete Warden, a former Apple employee, and Alasdair Allan, a data visualisation scientist, announced today at Where 2.0 that they've explored a file hidden inside iOS backups which appears to track the location of the device going back as far as the installation of iOS4. [The first indications of location data in this file were actually brought up some time ago, but with substantially less fanfare. –Ed.]
The above map shows data taken from a test iPhone, and shows a detailed history of the phone's movements around the south of England. Warden and Allan have put up a page with a FAQ discussing their findings and an Open Source software tool you can run on your own backups to see what you iPhone knows about you. They've also published a video demonstrating the problem.
Particularly worrying is that this file is unencrypted on the device (and on your Mac or PC, unless you encrypt your iTunes backups) and so easily accessible by anyone with access to your computer's iTunes backups folder or with jailbreak access to your device. Because of the way iTunes handles backup-and-restore, the data will even automatically persist across devices if you replace your iPhone.