Re: What is this pem key thing
AWS will generate a .key and .pem file for the initial setup of an instance. This is used to guarantee a temporary access to the instance for you, and underlying access to some of the "other" AWS Services you can tie to your instance.
You should generate or add an existing key.pub contents to the instance's /root/.ssh/authorized_keys or the appropriate /home/$user/.ssh/authorized_keys since root login is disabled on new AWS instances... (you'll have to either enable it or use sudo from the ubuntu user account)
If you want things you've never had, you have to do things you've never done.