What is your default route when VPN is on?
Code:
route -n
If the server tells the client to use the VPN tunnel as default when connected, then ping will continue to work going out through the VPN tunnel.

Run a trace route to the same destination and see which path it takes.