Try using the official STA driver from Broadcom instead. To install it, run this command on a terminal:
Then, disable the b43 driver and activate the STA driver:
sudo apt-get install bcmwl-kernel-source
Now, see if it works better. If it does, permanently disable the b43 driver:
sudo modprobe -r b43
sudo modprobe wl
If it doesn't, switch back and permanently disable the STA driver:
sudo bash -c "echo blacklist b43 >> /etc/modprobe.d/blacklist.conf"
sudo modprobe -r wl
sudo modprobe b43
sudo bash -c "echo blacklist wl >> /etc/modprobe.d/blacklist.conf"