Sound disables after periods of inactivity

I'm having a very annoying problem. When I leave my computer alone for a long period of time and come back to it the later the sound won't work unless I do a restart. I always check and make sure the volume is up, I even try muting and then unmuting it. I've tried messing with the settings.. Nothing has worked except doing a full restart. The sound problem is system-wide. Flash videos, audio files, program noises are all completely gone. My computer doesn't suspend or hibernate or anything while I'm not using it either. Any ideas?