As far as I can tell, X does everything and nothing. It draws stuff, I guess. I doesn't actually do windows or rendering or anything like that... thost are handled elsewhere, and then X draws them or something.
But then it's all tied up with input and keyboard mapping and dbus and whatever. Your touchpad or touchscreen isn't going to be to be doing anything without X, that's for sure.
And it's, like, a dependency for everything except "ls" and whatever.
And when you try to configure stuff (espeically keyboard stuff, which I've done more of than I care to talk about) it's really complex and crufty. One of the current heads at X.org was talking about how it looks like it was designed by someone on drugs. X is like a some kind of insane micro-kernel unto itself.
So, what was X originally supposed to be, What is X today, and when do we get to dump this weird crufty system for Wayland or something that isn't terminally handycrapped by a legacy of feature creep?