It seems like it is causing various problems. X is taking a lot of CPU when merely updating some progress bar seems to be happening:
chromium-browser keeps crashing (after upgrading from 11.04):

Do these happen with other drivers? I hate it, when my system becomes slow and unstable after upgrading.

I have this old AGP graphics card:
VGA compatible controller: NVIDIA Corporation NV17 [GeForce4 MX 420] (rev a3)