That's not to say that it's super hard -- it's not, especially if you know not to stand in the fire and you've got the kind of gear on that lets you conquer the Heroic achievements. A lot of guilds have 25 of those people, and they're doing very well in Ulduar so far. But as Dueg says, Naxx is a casual instance, and Ulduar is not. In Naxx, you can get away with losing a few people, or having a few folks in greens along. In Ulduar, you can't.
It's not the apocalypse for guilds -- most guilds will go back to Naxx if they have trouble in Ulduar, grab a few more epics and tier pieces, and try again later. And some probably won't bother with Ulduar at all -- my casual guild is having fun just taking our time finishing Naxx wing by wing. But Ulduar seems to be where the rubber meets the road with casual raiding. If there's a guild out there who has a few lesser raiders carried along by a few high-level veterans, Ulduar's likely to cause some friction.