It only takes a few seconds to get over the confusion and realign your remote, but it gets a little annoying when it happens multiple times in one play session. The problem is especially apparent when switching from gesture-based to pointer-based functions, such as transferring from sword-swinging to boomerang throwing in Zelda, or from racquet swinging to menu selection in Wii Sports: Tennis.
It doesn't seem to be solely a hardware problem either -- the sensor bar seems perfectly capable of detecting much of the off-screen pointing, but the software and system menus just don't acknowledge it. Even if the pointer is completely out of sensor range, some sort of on-screen indicator of its last known position would be nice. We urge any and all future Wii developers to consider this problem carefully when developing their pointer-based gameplay.