You can remove it - just make sure it's empty before actually deleting the /media/sdb2 folder. The side panes are just shortcuts, so you may be able to remove the sdb2 folder by right clicking and deleting. If you can't, open nautilus as root, make sure there is nothing in the folder, then move up one level and remove/delete the 'sdb2' folder.
Originally Posted by mzimmers
mount # Check that sdb2 is not mounted on /media/sdb2 (or any other partition, for that matter)
gksu nautilus /media/sdb2
I don't use Windows so I'll defer to someone who does since I don't know how they are displayed.
On a related question, I do see a couple of partitions for my Windows boot in my sidebar; is there a way to remove these, or is this just the way it's going to be?