You're not talking about using the clients at a place like Yahoo to manage a mailbox on your server, right? You are talking about sending a message from Yahoo with an @yahoo.com From address to an address in a domain for which your server is the MX, yes?
With sendmail, you get log messages that look like this:
The message originated on web142602...yahoo.com and arrived on a server I manage using ESMTP over port 25.
Jul 3 13:17:24 xxxxx sendmail: r63HHLk7029689: from=<firstname.lastname@example.org>, size=267838, class=0, nrcpts=1, msgid=1372871757.78624.YahooMailNeo@web142602.mail.bf1.yahoo.com>, proto=ESMTP, daemon=MTA, relay=[10.1.1.12]
Any server advertised as the MX for a domain needs to accept mail on port 25.