[Howto] Successfully use Mutt and Gmail on Gutsy Gibbon

[andrew.46]

*** This thread has been locked at the poster's request. The new version of this howto may be found here ***

================
Introduction
================

This page is a guide to using the email client Mutt to send, receive and read email under Ubuntu using a Gmail account as a relay. Despite the thread title, which speaks of Gutsy Gibbon, this guide should work under most versions of Ubuntu. I am using it myself under Intrepid Ibex with no problems.

There are a few steps involved but if followed carefully and in sequence you will soon be using Mutt successfully with your Gmail account.

In sequence we will:

1. Set up an SSL and some certificates
2. Set up the Mail Transport Agent msmtp
3. Set up the Mail Transport Agent Fetchmail
4. Set up the Mail Delivery Agent Procmail
5. Set up our mail program Mutt

Sections that require your details inserted are in bold and red to eliminate some confusion . Lets get started with SSL:

============
SSL Tools
============

Gmail uses the POP3-over-SSL protocol to protect the transmission of username and password over the Internet. You will need to install open ssl and a certificate pack:

$ sudo apt-get install openssl ca-certificates

Later you will need to add the necessary SSL instructions to Fetchmail, but the next step is to install the software to send mail from your computer to the server: ssmtp.

=========
msmtp
=========

msmtp is a great and wonderfully reliable way to move mail from your computer. Download it from the repository as follows:

$ sudo apt-get install msmtp

There is a single configuration file to be altered: $HOME/.msmtprc. It can be created and permissions set as follows:

$ touch $HOME/.msmtprc
$ touch $HOME/.msmtp.log
$ chmod 0600 $HOME/.msmtprc

Below is the required configuration for a Gmail server:

account default              
host smtp.gmail.com          
port 587                     
from full.gmail.address@gmail.com   
tls on                       
tls_starttls on              
tls_trust_file /etc/ssl/certs/ca-certificates.crt
auth on                     
user gmail.username        
password mypassword        
logfile ~/.msmtp.log

Now to the MTA Fetchmail:

============
Fetchmail
============

Next to download fetchmail from the Ubuntu repository:

$ sudo apt-get install fetchmail

Once again a single configuration file is required and so you will need to create ~/.fetchmailrc as follows:

 $ vim ~/.fetchmailrc

The following configuration allow Fetchmail to access and fetch email from Gmail, leave a copy of the mail on the Gmail server and pass the mail to Procmail on your local machine. The SSL configuration is included here as well:

poll pop.gmail.com                   # Tell fetchmail about server
with proto POP3                      # Use the POP-protocol       
user 'Gmail Username '               # Your Gmail Username            
there with password 'Gmail Password' # Your Gmail Password  
is 'username' here                   # Local Username                    
mda "/usr/bin/procmail -d %T"        # Tell fetchmail which MDA to use   
options                              # Options duh                                      
keep                                 # Keep the mail on server = safe
ssl                                  # Use ssl
sslcertck                            # Check the certificates
sslcertpath /etc/ssl/certs           # Path to the certificates

All done except as a final touch, since the username and password are openly in this file,you will need to make the file readable only by the file owner. If this is not done Fetchmail will not even run:

 $ chmod 600 ~/.fetchmailrc

Now would be a good to time to make sure you have POP forwarding enabled in your Gmail account. You will find this in: Settings - Forwarding and POP. Note also that Gmail has a little oddity in regard to the "keep" and "nokeep" command of Fetchmail. You cannot remove mail from Gmail servers via POP3 but you can choose to have your messages archived, kept or deleted once they have been downloaded via POP3. This is a Gmail setting hidden in Settings - Forwarding and POP: "When messages are accessed with POP...".

Now for the mail delivery program Procmail:

===========
Procmail
===========

Procmail can be easily downloaded from the repository:

$ sudo apt-get install procmail

So where will Procmail deliver to? Traditionally all mail goes to the location specified in the $MAIL environment variable, but in a default Ubuntu system this is often not set. Set the MAIL variable by opening ~/.bashrc as follows:

 $ vim ~/.bashrc

and adding the following, using your own username:

# Sets the Mail Environment Variable
MAIL=/var/spool/mail/username && export MAIL

A very simple configuration file must be created for procmail as follows:

$ vim ~/.procmailrc

and below is a very simple start to what can be quite a complex file:

# Environment variable assignments
PATH=/bin:/usr/bin:/usr/local/bin 
VERBOSE=off                   # Turn on for verbose log
MAILDIR=$HOME/Mail            # Where Procmail recipes deliver
LOGFILE=$HOME/.procmaillog    # Keep a log for troubleshooting.
# Recipes
:0:
* ^TOmutt-user
mutt

I include a very simple sorting recipe with the file: this one intercepts everything addressed to "mutt-user" and directs it to $HOME/Mail/mutt. This is the mutt-user mailing list which I would advise all new mutt users to join. And lets not forget to create the Mail folder:

$ mkdir $HOME/Mail

Now finally to the MUA: Mutt.

=======
Mutt
=======

The following command downloads mutt from the Ubuntu repository:

$ sudo apt-get install mutt

Mutt is driven by a configuration file that can be created as follows:

$ vim ~/.muttrc

I have spent some time building this file from scratch but for you, Gentle Reader, I include here a more basic version, similar to the one I started from:

#======================================================#
# Boring details
set realname = "Your realname"
set from = "Email address"
set use_from = yes
set envelope_from ="yes"
set sendmail="/usr/bin/msmtp"

# If not set in environment variables:
set spoolfile = /var/spool/mail/user-name

#======================================================#
# Folders
set folder="~/Mail"                # Mailboxes in here
set record="+sent"                 # where to store sent messages
set postponed="+postponed"         # where to store draft messages
set move=no                        # Don't move mail from the spool.

#======================================================#
# Watch these mailboxes for new mail:
mailboxes ! +Fetchmail +slrn +mutt
set sort_browser=alpha    # Sort mailboxes by alpha(bet)

#======================================================#
# Order of headers and what to show
hdr_order Date: From: User-Agent: X-Mailer \
          To: Cc: Reply-To: Subject:
ignore *
unignore Date: From: User-Agent: X-Mailer  \
         To: Cc: Reply-To: Subject:
               
#======================================================#
# which editor do you want to use? 
# vim of course!
set editor="vim -c 'set tw=70 et' '+/^$' " 
set edit_headers          # See the headers when editing

#======================================================#
# Aliases

set sort_alias=alias     # sort aliases in alpha order by alias name

#======================================================#
# Colours: This scheme is fairly basic and only
# really works if your Terminal background is white

color hdrdefault black        default   
color quoted     red          default   
color signature  brightblack  default   
color indicator  brightwhite  red
color attachment black        green
color error      red          default   
color message    blue         default   
color search     brightwhite  magenta
color status     brightyellow blue
color tree       red          default   
color normal     blue         default   
color tilde      green        default   
color bold       brightyellow default   
color markers    red          default  

#======================================================#
# Odds and ends
set markers          # mark wrapped lines of text in the pager with a +
set smart_wrap       # Don't wrap mid-word
set pager_context=5  # Retain 5 lines of previous page when scrolling.
set status_on_top    # Status bar on top.
push <show-version>  # Shows mutt version at startup

Note: Procmail will create your mailbox in the spool, and set the appropriate permissions, when it first receives mail from fetchmail so don't worry that mutt cannot initially find this mailbox. If you wish to create the mailbox yourself the following permissions and ownership are required (taken from my own system):

-rw-rw---- 1 andrew mail 0 2008-10-23 10:12 /var/spool/mail/andrew

And finally it is reward time as you open Mutt, type ! to open a shell prompt, type fetchmail -v and start reading your mail! My parting gift is a little macro that was written for me by a generous person on the mutt-user mailing list that will actually do this for you when you simply press the key "I". Place the following in your ~/.muttrc file:

macro index,pager I '<shell-escape> fetchmail -v<enter>'

And welcome to the world of mutt!

October 23rd, 2008
Andrew Strong

[wounded]

I'd like to report one success.

I had tried to set mutt up a long while ago and always ran into some sort of issue with something, which I imagine is easy when you need to configure three to four programs to set one up correctly, but I went through and followed your post and only had one error (Which was due to me not realizing I had left a space at the end of my gmail username - easily fixed!).

Also, I think you also have to turn on POP forwarding in the settings of your gmail account. Not a big deal, unless you don't know about the gmail setting.


I have some questions that maybe you can help me with (Or at least point in the right direction )

In my gmail account I have filters that sort through and apply labels to different emails I receive. Is there a way for mutt to use these labels to sort through? Or maybe that would be procmail that would do that?

Thanks for the post, it helped at least one poor soul :]

[andrew.46]

Hi,

Glad you had success with the walkthrough:

Originally Posted by wounded I'd like to report one success. [...] Also, I think you also have to turn on POP forwarding in the settings of your gmail account. Not a big deal, unless you don't know about the gmail setting. I have some questions that maybe you can help me with (Or at least point in the right direction ) In my gmail account I have filters that sort through and apply labels to different emails I receive. Is there a way for mutt to use these labels to sort through? Or maybe that would be procmail that would do that? Thanks for the post, it helped at least one poor soul :]

Good point about the Gmail POP forwarding, I have added it in. As for filtering and labelling mutt will not use gmail filters. You will have to set procmail up for that and trust me that syntax is not easy!

All the very best,

Andrew

[wounded]

Originally Posted by andrew.46 Hi, Glad you had success with the walkthrough: Good point about the Gmail POP forwarding, I have added it in. As for filtering and labelling mutt will not use gmail filters. You will have to set procmail up for that and trust me that syntax is not easy! All the very best, Andrew

Well, that's not a very big deal. Would you happen to have any particularly good procmail/mutt links you could share?

[andrew.46]

Hi,

Had a quick dig in my bookmarks:

Originally Posted by wounded Well, that's not a very big deal. Would you happen to have any particularly good procmail/mutt links you could share?

For procmail:

http://lipas.uwasa.fi/~ts/info/proctips.html
http://partmaps.org/era/procmail/mini-faq.html

For mutt:

http://therandymon.com/content/view/42/79/
(This is the Woodnots: highly reccomended!)

http://mutt.blackfish.org.uk/

Should get you off to a start

Andrew

[wounded]

And now... imap howto?

[andrew.46]

Hi,

Thanks for the suggestion:

Originally Posted by wounded And now... imap howto?

and it strikes me that this could be a project that you might rmbrace yourself I do not actually use Ubuntu any more and simply hang around the forums supporting a few of the walkthroughs that I have put together.

I guess a starting point would be to compile mutt with the appropriate option: --enable-imap ?

Andrew

[hugmenot]

IMAP with Gmail is real easy, and a lot shorter than what you put up.
Just put a few lines into your muttrc and it works. No compilation or installs neccesary.

set imap_user = 'hug@gmail.com'
set spoolfile = imaps://imap.gmail.com:993/INBOX
set folder = imaps://imap.gmail.com:993
set record="" #gmail does recording on its own
set postponed="imaps://imap.gmail.com/[Gmail]/Drafts"
set trash="imaps://imap.gmail.com/[Gmail]/Trash"
set realname="Hug M. Not"
set from="hug@gmail.com"

#maybe these too
set imap_check_subscribed="yes"
set imap_list_subscribed="yes"

set smtp_url="smtp://hug@smtp.gmail.com:587/"

set header_cache=~/.mutt/hcache
set message_cachedir="~/.mutt/msgcache/"
set certificate_file=~/.mutt/certs

#and maybe some useful bindings
macro index,pager y "<save-message>=[Gmail]/All Mail<enter><enter>" "Archive"
macro index,pager d "<save-message>=[Gmail]/Trash<enter><enter>" "Trash"
macro index,pager s "<save-message>=[Gmail]/Spam<enter><enter>" "Spam"
macro index gi "<change-folder>=INBOX<enter>" "Go to inbox"
macro index ga "<change-folder>=[Gmail]/All Mail<enter>" "Go to all mail"
macro index gs "<change-folder>=[Gmail]/Starred<enter>" "Go to starred messages"
macro index gd "<change-folder>=[Gmail]/Drafts<enter>" "Go to drafts"
macro index go "<change-folder>=[Gmail]/Sent Mail<enter>" "Go to sent Mail"
macro index gp "<change-folder>=[Gmail]/Spam<enter>" "Go to Spam"

[boob11]

thnx

[andrew.46]

Hi,

Read your reply to my mutt walkthrough:

Originally Posted by hugmenot IMAP with Gmail is real easy, and a lot shorter than what you put up. Just put a few lines into your muttrc and it works. No compilation or installs neccesary.

You are being a little harsh there As far as I know Gmail only started rolling out imap to its accounts in the end of October 2007. Certainly this walkthrough will start to be a little dated now but I think it is still a valid approach while Gmail imap support matures.

Once this support has matured certainly this walkthrough will be ready for the archives, although it can easily be modified for standard POP3 accounts.

Andrew