In my case R was not built with X support, and I was also missing some openGL libraries. The info I needed was in the README inside the unzipped tarball for rgl. But rgl cleaned up after failing...