Guild banks were initially intended to go live with the Burning Crusade launch. Apparently Blizzard wasn't happy with them and postponed their implementation until "a later date." That date has been confirmed by a blue on the WoW Europe forums as Patch 2.3.
According to this preview article, guild banks are designed so that every member of the guild can deposit or withdraw cash or items. Limits on transactions can be set so a new recruit doesn't walk off with all the good stuff. Also, all activity will be viewable by the guild so the guild officers can't keep the best things for themselves without everyone else knowing.
My favorite feature of the new guild bank system is the ability to use guild funds when paying for repair costs. A new button will appear on all repair NPCs and repair-bots. This function can also be set by the guild leader to work for only certain ranks within the organization.
Soon we'll be saying goodbye to the guild mules and hello to an era where guild resources are viewable, if not available, to all.
Update: we have tons more patch 2.3 info!