Can't switch focus to desktop when Focus Follows Mouse

[spencer-x]

I'm using Unity Tweak Tool to enable focus-follows-mouse, but there's no way for me to switch the focus back to the Ubuntu Desktop without doing Ctrl+Super+D, when I'd like to be able to switch the focus by just mousing over the desktop. How can I fix this?