POP before SMTP HOWTO document

by Stephen McHenry


This document is designed to give a step-by-step guide to installing and configuring a POP before SMTP style authentication mechanism for use with Postfix.

I was not the originator of this program. I found it years ago while searching for a better solution than what I had. I modified it to work with Postfix (which didn't take much) and cleaned up the commenting a little.

Theory of Operation

To prevent relaying by unauthorized hosts, most mail systems now prevent computers from relaying email through their SMTP deamons unless those computers reside on a limited set of network addresses - typically those within an organization. These hosts have IP addresses in known ranges and IP address is used as the basis for authentication. Other hosts with fixed IP addresses can be added to the authorized hosts list, allowing SMTP relaying from certain predefined locations.

The problem arises when a mobile user is assigned a temporary IP address by an ISP. Since there is no way to know a priori what address will be assigned, there is no way to add that address to the list of authorized addresses. Furthermore, it would be undesireable to add it permanently, as that address might be reassigned to another computer which would then have the ability to use the SMTP daemon as an open relay.

Using the POP before SMTP authentication, when a user successfully authenticates with the POP daemon to fetch mail, the user's IP address is added to a special authorization file that is used by the SMTP daemon to allow relaying. After a configurable time period (usually 20-30 min), the address is removed from the file and relaying is no longer allowed from that address. Users must fetch mail before attempting to send, so that their IP address can be added to the file.

Other POP before SMTP daemons have limitations in that they only process the POP log at fixed intervals (every 1-5 minutes) so the user must sometimes wait before sending is allowed. The approach described here utilizes a FIFO from syslog to the "popauth" program so that the addition of the IP address to the authorization file is virtually instantaneous.

Mechanics

After some initialization, the program waits for a new line to appear on the FIFO (which will be each line written to the maillog file by the POP daemon or Postfix) or the expiration of a time interval - whchever comes first.

If it was a new line in maillog, Popauth inspects it to see if it was a successful POP login and, if so, writes the IP address to a directory. Then, it takes all the file names in that directory (which are all the IP addresses that have successfully authenticated and not expired) and creates a new popauth file in the postfix directory. Then, it rebuilds the hash db, and loops (to wait again).

If it was the timer expiration, popauth looks for the now expired IP address, removes it form the directory and then rebuilds the popauth file as described above. Then, it loops and waits again.

Extensions

Because this program watches the maillog file, and that file is (usually) used by both Postfix and the POP daemon, this program could be easily extended to look for events from Postfix, and take action when it sees them. For example, it could look for IP addresses or email addresses that are acting badly (spammer-like) and add those to a list of addresses to be rejected outright by Postfix. I have thought about doing that on a number of occasions (mostly for UCE control) but have never actually implemented anything yet.

Limitations

The POP daemon must output the IP address in the log file entry that confirms successful authentication. Some POP daemons apparently output the IP address when it connects to the daemon, and a different line when the user authenticates. POPAUTH (as it is below) will not work with that kind of POP daemon. If you have that kind of POP daemon, the choices are:

  1. change POP daemons
  2. modify your existing POP daemon to output the IP address with the successful authentication
  3. modify POPAUTH (more than the RegExp) - must save the process id and IP addr when it sees the connect, and match it with the successful authentication later - probably a lot of work.

Installation Procedure with Postfix

  1. POP daemon is installed and operating properly, and logging authentications vis syslog.
  2. SMTP daemon (Postfix) is installed and working correctly.
  3. Create directory /var/spool/popauth

    cd /var/spool
    mkdir popauth
  4. Add FIFO /var/adm/popauther
    cd /var/adm
    mkfifo popauther
    chmod to pwr------
  5. Add entry to /etc/syslog.conf to output log messages to the FIFO (note: none of the existing entries should need to be changed) -
    mail.notice     |/var/adm/popauther
     
    Notes:
    1) requires syslogd version 1.3-3 or later to output messages to the FIFO.
    2) If your POP daemon outputs the authentications at a level other than "notice", you may need to change notice to "info" or even "*" to allow Popauth to see them.
  6. Restart syslogd
  7. Create files in /etc/postfix

  8. touch popauth
    touch popauth-old
    postmap popauth
  9. Modify POPAUTH program to correctly parse your POP daemon's log entries (can't help with this one - curl up with a good Perl book) - see the "if" statement in the "add_new" subroutine.
  10. Copy POPAUTH program to /usr/local/bin (or your choice of locations)
  11. Add startup command to rc.local to start deamon when the machien boots
    #
    # start the pop authorization script for SMTP e-mail
    #
    /usr/local/bin/popauth &

  12. ...or whereever you put it in the last step.
  13. Start the daemon by hand (for now) or reboot the machine

  14. /usr/local/bin/popauth &
  15. Modify main.cf file to check /etc/postfix/popauth before most other access chacks. Here is one way that works (assumes you have everything in black already):
    smtpd_recipient_restrictions =
    reject_non_fqdn_sender,
    reject_non_fqdn_recipient,
    reject_unknown_sender_domain,
    reject_unknown_recipient_domain,
    permit_mynetworks,
    check_client_access hash:/etc/postfix/popauthip,
    reject_unauth_destination,
    check_recipient_access hash:/etc/postfix/recipient_checks,
    check_sender_access hash:/etc/postfix/sender_checks,
    check_client_access hash:/etc/postfix/client_checks,
    reject_unauth_pipelining,
    reject_invalid_hostname,
    reject_non_fqdn_hostname,
    reject_maps_rbl,
    reject_unknown_client,
    permit

Perl POPAUTH program

#!/usr/bin/perl
# - Makes a Fifo called /var/adm/popauther
# - reads from that Fifo all POP sessions from Syslog.
# - Puts the IPs in /var/spool/popauth/
# - Removes "expired" IPs from /var/sopol/popauth
# - Automatically rebuilds POP-authorized IP list
#
# Original code from: William R. Thomas - wthomas@poweruser.com
# Prior version is from: Harlan Stenn - harlan@pfcs.com
# This version is from: Stephen McHenry - stm /at/ mchenry /dot/ net


$fifo = "/var/adm/popauther";                  # Name of FIFO where Log File entries come from
$popauthspool = "/var/spool/popauth/";         # Directory to contain IP addresses
$watcherlog = "/var/log/popauth.watcher.log";  # Log file
$popwatcherpidfile = "/var/run/popauth.watcher.pid";
$secondstoallow = 30 * 60;                     # 30 minutes before permission expires
$minwakeupin = 5 * 60;                         # 5 minute minimum wakeup time

{
  # Outer loop - in case it exits from the main inner loop
  while(1) {
    
    # Set up FIFO if it doesn't exist
    unless( -p $fifo)
      {
        unlink $fifo;
        system("mkfifo $fifo") && die "Can't mkfifo $fifo: $!";
        chmod 0600, $fifo;
      }
    open(FIFO, "< $fifo") || die "Can't open $fifo: $!";

    # Open the log file and specify "flush all"
    open(LOG,">>$watcherlog") || die("Can't open $watcherlog");
    select(LOG);
    $| = 1;

    print LOG "\n";
    print LOG &tstamp." Starting log for popauth.watcher at pid $$\n";

    # Open STDOUT and specify "flush all"
    select(STDOUT);
    $| = 1;

    # Set up interrupts
    $SIG{'INT'} = 'exithandler';
    $SIG{'QUIT'} = 'exithandler';
    $SIG{'KILL'} = 'exithandler';

    # Output the process ID to the PID file
    open(PID,">$popwatcherpidfile");
    print PID "$$\n";
    close(PID);

    # When we startup, force an immediate wakeup and process
    $nextwakeup = time;
    $rebuild = 1;
    
    # Main loop - forever
    while(1)
      {
        $rin = "";
        vec($rin, fileno(FIFO), 1) = 1;
        my $now = time;

        # Calculate how long before we should wake up
        my $wakeupat = ($nextwakeup <= $now) ? $now : $nextwakeup;
        my $wakeupin = $wakeupat - $now;
        
        # Sleep until there is input on the FIFO, or it's time to expire an IP address
        $nfound = select($rout=$rin, undef, undef, $wakeupin);

        # When we woke up, it was because the FIFO had input
        if ($nfound)
          {
            $rebuild += add_new();
          }

        # When we woke up, it was because it was time to expire an IP address
        if ($nextwakeup <= time)
          {
            ($nextwakeup, $changed) = scan_old();
            $rebuild += $changed;
          }

        # Rebuild the list of authorized addresses
        rebuild() if $rebuild;
        $rebuild = 0;
      }
    close(LOG);
  }
  exit(1);
}

#
# add_new 
#
# Takes a line from the FIFO and dissects it to see if it is a POP daemon authentication.
# If it is, writes the IP address to the directory where IP addresses are collected
# and returns an indication that the authorization file needs to be rebuilt.
#
sub add_new
  {
    my $rebuild = 0;
    my $good = 0;

    $_ = <FIFO>;
    chomp;

    # You must change the regexp in the following if statement to match what your POP daemon outputs to the log file
    #  The RegExp in the if statement below matches a logfile entry of the format:
    #
    #  Sep 27 12:15:28 myhostname pop[26887]: (v1.2.9) POP login by user "johnny" at (somehost.somedomain.com) 66.211.179.160
    #
    if(!$good && /^([A-Za-z]+\s+\d+\s\d+\:\d+\:\d+)\s\w+\spop\[\d+\]\:\s\(v\d+\.\d+\.\d+\)\sPOP\slogin\sby\suser\s\"([a-z0-9]{2,8})\"\sat\s\(.*\)\s(\d+\.\d+\.\d+\.\d+).*$/)
      {
        $tstamp = $1;
        $user = $2;
        $ip = $3;
        ++$good;
      }
    if ($good)
      # Found an authentication line in the FIFO
      {
        # Flag to tell caller to rebuild list of authorized IPs
        ++$rebuild;
        print LOG "$tstamp $user authenticating relaying for $ip\n" ;
        
        # Put the IP address into the directory as a file name
        my $file = ">".$popauthspool.$ip;
        open(TEMP,$file);
        close(TEMP);
      }
    add_new_exit:
      return $rebuild;
  }

#
# scan_old 
#
# Scans the directory containing IP addresses and removes the ones that have passed
# expiration. Also determines, as it scans, when it needs to wake up again to get the
# next oldest file.
#
sub scan_old
  {
    my $now = time;
    my $next = $now + $minwakeupin;
    my $changed = 0;

    # Get all the files (IP addresses) in the directory
    opendir(DIR2,$popauthspool);
    my @dir2 = grep !/^\.\.?$/, readdir(DIR2);
    closedir(DIR2);
    
    foreach $file (@dir2)
      {
        # Get the creation time of the file (which is an IP address)
        my $mtime = (stat($popauthspool.$file))[8];
        my $exptime = $mtime + $secondstoallow;
        if( $exptime <= $now )
          { 
            # File is past its expiration - remove it
            print LOG &tstamp." removing authentication for relay from $file\n";
            unlink($popauthspool.$file);
            ++$changed;
          }
        else
          {
            # Keep track of the next time we need to wake up - it's when the next one expires
            $next = $exptime if ($exptime < $next);
          }
      }
    return ( $next, $changed );
  }

#
# rebuild 
#
# Rebuilds the authorization file used by Postfix by getting all the filenames (which are
# IP addresses) from the directory and adding a line for each to the authorization file.
# Also, it backs up the old authorization file before it starts.
#
sub rebuild
  {
    # Backup old authorization file
    system("mv /etc/postfix/popauth /etc/postfix/popauth-old");

    # Get all filenames (IP addresses) to be added to authorization file
    opendir(DIR, $popauthspool);
    my @dir = grep !/^\.\.?$/, readdir(DIR);
    closedir(DIR);

    # Add each one to the file with "OK" as the rule
    open(POPAUTH, ">/etc/postfix/popauth");
    foreach $_ (@dir)
      {
        if(/^\d+\.\d+\.\d+\.\d+$/)
          {
            print POPAUTH "$_\tOK\n";
          }
        else
          {
            print LOG &tstamp." rebuild: Unrecognized file: <$popauthspool$_>\n";
          }
      }
    close POPAUTH;

    # Rebuild the hash file
    sleep 2;                  # Don't know why this is here - Stephen
    system("cat /etc/postfix/popauth | makemap hash /etc/postfix/popauthip.db");
  }

sub tstamp
  {
    use POSIX qw(strftime);

    return POSIX::strftime("%b %d %H:%M:%S", localtime(time));
  }

sub exithandler
  {
    local($sig) = @_;
    close(POPPER);
    close(LOG);
    exit(0);
  }