I can confirm the same behavior. The problem started from ubuntu 10.10 but after some updates it was fixed. Now that I upgraded again my system to 11.04 I am having the same problem. I have a latitude E6510 with a usb logitech m510 mouse.
Is there any way of restarting the daemons responsible for mouse and usb instead of rebooting again and again?