If you use rmdir instead of rm -r it will only remove if empty

Code:
sudo rmdir /media/john/MediaExternalDrive1
Code:
NAME
       rmdir - remove empty directories

SYNOPSIS
       rmdir [OPTION]... DIRECTORY...

DESCRIPTION
       Remove the DIRECTORY(ies), if they are empty.