OK, here is another workaround, although not very good.
Log in to an Ubuntu Classic session. Open the Compiz Configuration Settings Manager (CCSM) and select the Window Decoration plugin. Replace the "Command", which defaults to '/usr/bin/compiz-decorator' with '/usr/bin/metacity --replace'
Close the CCSM, log out then log in again to the Ubuntu Classic session. Vuescan 64bit (now at version 9.0.31) should start normally. This is now pretty much the same as the Maverick (Ubu 10.10) desktop.
I think the only way to actually be able to use Vuescan version 9.x.xx in the full blown Unity interface of Ubu 11.04 would be to put together a launch script that turned Compiz Window Decorations OFF, launched Vuescan, then turned Compiz Window Decorations ON again. This does work manually. The script would probably have to pass the appropriate messages to Compiz via the Dbus, but I don't know how to do this in a script.