the rtorrent account has little to do with cifs. the account you should add to your vista machine is your ubuntu user because that's the account that has to negotiate the permissions for the share.
so, if your ubuntu user account has been added to your vista machine, and your current fstab line looks like this:
then about the only thing i have left to suggest to you is to look into your vista setup.
//NERDY/tor /home/maddox/tor cifs credentials=/root/.smbcredentials,nounix,rw,iocharset=utf8,file_mode=0777,dir_mode=0777 0 0