Gold Capped: The Undermine Journal may have to close down

Every week, WoW Insider brings you Gold Capped, in which Basil "Euripides" Berntsen aims to show you how to make money on the auction house, and Insider Trader, which is all about professions. For Gold Capped's inside line on crafting for disenchanting, transmutation, cross-faction arbitrage and more, check in here every Thursday, and email Basil with your comments, questions or hate mail!

The Undermine Journal is an invaluable site that a lot of auctioneers have been making great use of. I wrote about it when it was in alpha, as well as how to use its market alert to watch for cheap deals. Unfortunately, the days of having a convenient graph showing you the price history for your realm are numbered. There's a very real possibility that this service will be killed by Blizzard's new auction house interface, and we'd be back to each keeping our own spreadsheets.

What's going on?

The site works by logging into the auction house web interface (the pre-Cataclysm one) and scanning through every auction. In order to access a realm, Erorus, the owner of The Undermine Journal, has to create a level 1 character for each realm/faction. With 50 characters per account, this has meant a slow but steady increase of realm faction coverage, dictated by user votes. Each realm is crawled once an hour or so, but the crawlers were designed to minimize the impact on Blizzard's servers.

The old WoW Armory interface for Blizzard has started to time out on scans, and the new Battle.net interface doesn't allow for scans like this. Blizzard has never acknowledged that it's aware of the UJ, and since its change has (potentially unintentionally) hamstrung it, Erorus has stopped accepting donations and adding realms (which involves creating new accounts).

The value of The Undermine Journal

This site is essentially a hobby; Erorus described it as his "train set." It costs real money to keep running, but building and maintaining a site of this complexity is the kind of pastime that some people really enjoy. The core functionality of the site sounds simple: provide near-to-real-time data and a historical graph. However, reading through the developer blog archives, it's remarkable how complex these things get under the covers. There are a bunch of servers performing various parts of the job, a large and complex-sounding database, and one of the best login systems I've seen in our corner of the internet.

There had been all kinds of new features being added once the core engine was running smoothly: heat maps and sold item tracking, an XML feed (which provides the raw data Blizzard should be providing), syndicated editorials, the alerts I mentioned before, and of course, a ream of new realms added very frequently.

Personally, I make the most use out of the alerts and the historic graphs for commodities. When I found myself buying volatiles two days after launch, being able to see how far the market had dropped since launch really helped me. I also use it to keep tabs on my competition, which helps me plan when to schedule some playtime (ideally right after they historically log off). I'm not the only person who uses this site, and assuming it's left running by Blizzard, I suspect a lot of people would get value from it.

But is it legal?

Parsing the data from the auction house of hundreds of realms into an easily manipulated format is not only complex; some people wonder whether it's even allowed. The short answer is that yes, it's completely, 100 percent in accordance with the terms of service as they are now. However, this is Blizzard's game, and if it decides that this site is not something it wants to encourage, it always has the right to turn it off.

Intent matters, though. A player-funded hobby site that takes every possible measure to avoid causing unnecessary stress on the Blizzard servers and carefully follows the guidance for data usage on the wowarmory.com site is much more likely to be tolerated (or hopefully even encouraged) than a site that takes advantage and doesn't pay any attention to the resources it consumes.


How Blizzard can save the Undermine Journal

If Blizzard continues with its plan to phase out the WoW Armory in favor of Battle.net, the Undermine Journal will close down. I suspect it will not have been deliberate on Blizzard's part, because the few occasions when some player-generated innovation crossed a line that made the game less fun (like AVR, QA Poster, and the original Decursive), Blizzard was pretty up front about its desire for it to go away. The Undermine Journal seems to be under Blizzard's radar still, but I like to think that if it were looked at, there might be a compromise where the site could be allowed to continue to exist in some way. I suspect that Blizzard probably wouldn't mind any of the features but might be a little peeved about the necessity of crawlers.

If this is the case, that's a simple problem to fix, and there's precedent. Auctioneer's scan used to be a resource-wasting server hog. Instead of disabling or throttling Auctioneer, Blizzard elected to provide an efficient data dump that could be only run every 15 minutes. Assuming it notices the UJ fast enough to make a similar decision, the answer could be to create a raw XML feed of the auction house.

This would take us from a situation in which one person has to pay for enough accounts that he's able to execute an incredibly inefficient scan at great expense to both his bank account and the Blizzard servers to a situation in which a very efficient XML feed is provided. The UJ (along with any other innovators who have an idea that would make the auction house minigame more fun) could focus on manipulating, interpreting, and displaying that data.

Honestly, I just don't want to have to go back to my spreadsheet. This site is infinitely better than the dinky little sheet I had made for myself to track prices, and it takes (me, at least) a lot less work. So long as Erorus is willing to keep programming for us, I hope Blizzard lets him.

Maximize your profits with more advice from Gold Capped, plus the author's Call to Auction podcast. Do you have questions about selling, reselling and building your financial empire on the auction house? Basil is now taking questions for a special series, "Ask an auctioneer," at basil@wow.com.