Amanna over at Adventures in Azeroth (who I have to wonder might be the first appliance-spec druid judging by the name) asks this question after having done this exact thing. Many times over the past couple of years I have been tempted to do this myself. There is always such a recoil when you leave a guild, even if you try to take the high road and make it as drama-free as possible. No matter how many people there may be that irritate you in any given guild, there are always two or three -- or twelve -- that really like you and don't want you to go. Thus, your planned drama-free departure is suddenly less drama-free.
How can you get out of a guild and make a clean break? Have you ever performed the stealth /gquit personally? Was it successful?
(And yes, I watched Mr. Deeds the other day, and it was foremost in my mind when I created the accompanying picture. Does it show?)