Yes - same problem. (Just fixed some bug in an update that removed my user from the audio group - had thought that something else was wrong).
I'm not sure that the "speaker-test" program uses PulseAudio at all, since it's directly using the ALSA C API to make its pink noise.
I have some free time today so I might try building alsa again from alsa-project.org's sources. I tried using module-assistant and alsa-src already in the repositories, but they wouldn't compile right.
After downloading alsa-driver-1.0.17 from the Alsa Project page, and following the steps described here for building the drivers from source, the 'make' script dies producing the following:
ANOTHER EDIT: Works fine after installing the latest development version of alsa-driver, alsa-lib and alsa-utils from the ALSA project web page.
/home/cian/src/alsa/alsa-driver-1.0.17/acore/info.c: In function ‘resize_info_buffer’:
/home/cian/src/alsa/alsa-driver-1.0.17/acore/info.c:90: error: implicit declaration of function ‘PAGE_ALIGN’
make: *** [/home/cian/src/alsa/alsa-driver-1.0.17/acore/info.o] Error 1
make: *** [/home/cian/src/alsa/alsa-driver-1.0.17/acore] Error 2
make: *** [_module_/home/cian/src/alsa/alsa-driver-1.0.17] Error 2
make: Leaving directory `/usr/src/linux-headers-2.6.27-4-generic'
make: *** [compile] Error 2