Currently, the encounter is trivial according to just about everyone that has tested it. The 10-man version is a breeze, and the 25-man version can be done with 10 people. I think it's safe to say this encounter isn't complete yet, or at least it needs some major retuning. I'm not sure how I feel about what loot this encounter drops. If it's intended, it seems like a rather trivial way to award those things. Season 5 armor from a raid boss? I know around Patch 2.4 they put in the ability to trade armor tokens from dungeons for old season gear, but this boss is essentially awarding the cutting edge arena gear for a 10 minute jaunt into a dungeon. Granted, your faction needs to have control of Wintergrasp, but the point remains. It's really, really easy to get gear that should take a little effort to get. There's still the RNG to deal with, but I still think this boss should have a wholly different loot table.
Of course, that brings up the fact that this boss is supposed to be motivation to take part in the battle for Wintergrasp. If this boss didn't award hella good loot, nobody would care. It's incentive to not just farm HKs in Wintergrasp, but to win and rally your faction into winning. There's also the issue of difficulty. Should it be tuned for PvE players? PvP players? You can't tune a boss the same way for both. What a PvE geared tank can handle isn't the same as what a tank in Honor/Arena gear can handle, unless they're packing multiple sets of gear. The boss also needs to be easily done in a PUG, since you can't schedule a raid for this with the expectation that you'll have Wintergrasp on a Friday night. Or if you can, that's an issue in itself. It means your faction is clearly dominating the zone so much that you can set your calendar (and clock) according to it.
All in all, I'm interested in seeing what they do with this. It feels like it's going to be something that'll be really hard to tune and balance. It's about a step away from being so good it trivializes the arena, or so bad nobody cares that it even exists.