Many people I've grouped with in game have complained of extra latency while the Downloader is running. If you start to notice this problem while playing tonight, you can always alt-tab out and suspend the download. Also, you can always access the options for the Downloader through the WoW Launcher which, in turn, can be enabled/disabled from your login screen. Additionally, if you'd like to start downloading without starting up WoW, you can use the Background Downloader icon in the WoW folder on your hard drive. For further questions, be sure to check out the Blizzard support page for more help on the subject.