They aren't that crazy
First of all, there are some important points to digest about the Pandemonium Warden fight that don't seem to have made it out into the larger web. People know the fight exists, yes, but they think of it as an unapproachable nightmare that takes half of forever. This isn't true, and there are several things that those only aware of the fight's existence really should know. First of all, the fight has been beaten, and is certainly considered doable now. And it doesn't take 18+ hours to do so.
In fact, it can't take that long. Square-Enix took a look at the fight pretty much as soon as everyone knew about the attempt recorded here for all time, and the boss was changed to despawn after two hours. I would like to think that after they were told the fight had been going on for so needlessly long, at least one developer asked why the heck anyone would continue it for so much time.
And make no mistake, this was a case of people essentially throwing good time after bad. Jamie Madigan has an excellent article about sunk cost that should give you a fairly good picture of why this fight became so incredibly long. (He's talking about Duke Nukem Forever, but the same philosophy applies.) Someone should have said far earlier that they were going to be fighting this thing forever, and yet no one did.
Lastly, Square doesn't tell you how to beat bosses. There aren't hints or pointers, and there isn't the huge test server community that exists in World of Warcraft trying to suss out how to kill something immediately after it can be approached. And since most of these bosses are hard to approach for fighting, much less actually battle and defeat, people will cling to their attempts long after it becomes clear that a given strategy isn't working, or at least will not work within any sort of reasonable timeframe.
Now that we have a baseline of knowledge established about this fight, we can understand that the insanely long battle wasn't a rule but an aberration. It was something that spiraled out of control and isn't indicative of every single boss fight in the game. For some reason, it's gotten far more press than similar attempts for other bosses which have taken even longer -- and once proper strategies were developed, yeah, those fights were easier too. Most of the truly challenging fights in the game are expected to take less than an hour.
But they are crazy, yeah
Notice how I said that most of these bosses are hard to even approach?
That isn't a question of difficult terrain, trash a la raid instances in World of Warcraft, or anything else. The fact is that it takes a lot of time and work to force bosses to spawn. With bosses such as the Warden, that work often amounts to slaying several world spawns that you may or may not get a claim on. (The fact that several of these spawns have incredibly long respawn timers means that if you miss it on Monday, you're frequently unable to try again until Tuesday.) If you do claim the target, you hope that it drops something, and pool several drops to spawn another boss. That boss may or may not drop something, and if you pool several of those drops, you can summon the boss.
I hate coming back to raids, I really do, but that one-week lockout is more lenient than you realize. Especially when you consider that if you spawn Pandemonium Warden and die... well, you're back to square one. He will not wait around for you. If you're insanely lucky, you might get a second shot within his two-hour window of existence, but the odds of that are somewhere between slim and none.
Boss fights in any MMO are a slow process of figuring out the strategy. You start by looking at what you do know about the boss, frequently helped by data mining or previous encounters from other players. Then you form a strategy, go in, and frequently fail the first time around. You collect data on that failure and integrate it into the next attempt. In most modern games, the turnaround time isn't very long -- you can generally attempt the same boss several times in one night, trying out different tactics based on group makeup and patience.
But if you're lucky on some of FFXI's uberbosses, you might get an attempt every other week. More likely it's once every month or every other month. That makes practical testing of a strategy nigh impossible, since you can't make small adjustments over the course of a night and see how the changes play out. Each attempt has a very strong push to be the winning fight rather than a step along the path, because it takes a lot to just try the fight.
Less crazy after all these years
All this placed on the table, what does the future look like for FFXIV? Are we going to see these fights return?
The odds are decent for them to show up again, yes. And yes, they're still crazy. But we're putting this in context here -- namely, the fact that the developers have made it clear that their love of the insane boss fight is only sort of married to their love of making an endgame. Pretty much every game in the franchise has some crazy optional bosses, and the high-end notorious monsters are being fairly consistently phased into the realm of "optional."
The June Version Update contained an add-on and two new endgame systems with the Walk of Echoes and the Revitalization Team. All three contain bosses, but all three are reasonably-difficult bosses that offer rewards just for fighting them rather than purely drop-based rewards. Furthermore, Revitalization Team rewards you just for fighting your enemy, and the Abyssea areas don't explicitly require anyone to engage the bosses unless they feel they can handle it.
So, yes, the bosses will be there. A better question would be whether or not they will dominate the endgame, to which the answer is that it's not very likely. There, aren't you happy that all of this backstory led up to that?
Questions, comments, and various explanations about why the huge boss fights were the best/worst thing about FFXI can be sent to firstname.lastname@example.org or left in the comments. (I'm not necessarily arguing with either viewpoint.) Next week we're diving back into the community highlights, so be sure to send in your favorite threads of the past month or so.