When you previously did the remove of the nvidia packages, did you use the purge switch (or apt-get purge nvidia...)? Without the purge, some config files are probably left around. If not, try the purge and install again).