It's just a problem across any micro-USB device.
As long as the port on the device itself is not wiggling, should not be an issue - the plug itself probably wiggles a bit.
Mine does it too, but doesn't seem to be a problem.
Thanks for the responses, everyone! Glad to hear there isn't something wrong with my nook.