Sylvain Lemoine
Another dev blog in the nexus…

The "Hey ! Our app is sending emails on non-production server environment" case

· by Sylvain Lemoine · Read in about 8 min · (1621 Words)
mail linux postfix dovecot nginx roundcube system ansible

At work, a dedicated system team provides us virtual machines for multiple environments: development, Q&A, pre-production…

Our currently in-development application is deployed on an application server for each of these environments. It is a common workflow application which sends a lot of notification emails to users.

Our system team kindly provides us a localhost postfix (smtp server) configuration on each application server. The postfix acts as a relay to a bigger company smtp server which will eventually deliver the email to the destination.

The good thing is that the the running of our app on each environment: The javamail client opens a connection on localhost:25 and sends emails through it.

The (really) bad thing is when someone injects a «real life» dataset on the development server thus the apps sending emails to some company’s higher-ups…

This issue could have been avoided by starting the app with mocked or disabled email feature. By far not the best option as it would have made the Q&A team works a non-iso production app. Furthermore, it’s an application scope solution. So if we have 10 apps on the same server we should be 10 times cautious about this.

I chose to address this issue by installing and configuring a fully working local email environment on the non-prod application servers : postfix, dovecot and a webmail. The configuration should be done one time and remains the same for any new deployed application on the same environment.

Ansible inside

This article deals with manual configuration on the server. But only few should want to repeat the same things several times, I provide at the end of this post the ansible roles and playbooks I now use to deploy all the following installation process automatically.

What we’ll put in the box

  • postfix - smtp server application. Send email to the destination i.e no more relay to some external smtp server. The appserver will act as the bucket of each email sent by applications.
  • dovecot - pop3/imap server application. Allow users to access the mail delivered by postfix
  • roundcube / nginx: The webmail used by the Q&A team or/and developpers to read all the emails received through the dovecot connection behind the scene.

To sum up, we have a backend application deployed on a server. The application sends email through postfix. Postfix sends email to the destination (i.e the same server in our case). Q&A and developper staff check emails through the roundcube webmail interface. Roundcube uses dovecot behind the scene to access the emails.

Disclaimer

I’m not a sysadmin. Anyway I like making system & devops stuff for my team, so I always try to be rigorous with the operations I make on a server.

The following configuration process is not intended for «open on the internet» production installation of smtp / pop3/ imap server. This should remains stricly local a private network. So port 25 and 143 should not be reachable from the outside world.

In any case, if you find something insecure or something which should probably fixed in this post, please let me know in the comment section. Thank you !

Environment

I work on a Centos 7 OS with SELinux and firewalld disabled.

Create the mail user

sudo adduser -m mailuser -p mailuser

Postfix installation

install postfix and enable it at boot

sudo yum install postfix
sudo systemctl enable postfix
sudo systemctl start postfix

we will edit the postfix main configuration /etc/postfix/main.cf

myhostname = localhost
mydomain = localdomain
myorigin = $myhostname

and add the following:

virtual_alias_domains = mycompany.com gmail.com
virtual_alias_maps = hash:/etc/postfix/virtual

The above two lines enable the virtual alias configuration. We will enable our postfix to act as a final destination for the company domain (and gmail address too) through virtual alias mapping. Doing this allows us to keep a realistic dataset of business data users with their company email in our application database. We don’t have to replace their email adresses with fake ones to avoid spamming of company’s employees.

let’s configure the mapping by adding the following lines at the end of /etc/postfix/virtual :

@mycompany.com mailuser
@gmail.com mailuser

now all emails sent to the @mycompany.com domain will be delivered to mailuser@localhost.localdomain user.

Reload postfix configuration to make the changes effective:

sudo postfix reload
sudo postmap /etc/postfix/virtual

test it

echo "hello" | mail -s "postfix alias mapping test" bruce.wayne@mycompany.com
sudo tail /var/log/maillog

You should found the kind of following log line:

Jan 29 13:14:51 XXXX postfix/local[10663]: DEA8B108A405: to=<mailuser@localhost.localdomain>, orig_to=<bruce.wayne@mycompany.com>, relay=local, delay=0.04, delays=0.02/0.01/0/0
, dsn=2.0.0, status=sent (delivered to mailbox)

and email content could be read from:

sudo tail -20 /var/mail/mailuser

eg:

From root@localhost.localdomain  Wed Jan 31 21:52:28 2018
Return-Path: <root@localhost.localdomain>
X-Original-To: bruce.wayne@mycompany.com
Delivered-To: mailuser@localhost.localdomain
Received: by localhost (Postfix, from userid 0)
	id CFD6CDABE24; Wed, 31 Jan 2018 21:52:28 +0100 (CET)
Date: Wed, 31 Jan 2018 21:52:28 +0100
To: bruce.wayne@mycompany.com
Subject: postfix alias mapping test
User-Agent: Heirloom mailx 12.5 7/5/10
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Message-Id: <20180131205228.CFD6CDABE24@localhost>
From: root@localhost.localdomain (root)

hello

The postfix part is done !

Dovecot installation

sudo yum install dovecot
sudo systemctl enable dovecot
sudo systemctl start dovecot

Let’s make mailuser a user capable of log in through dovecot and access his emails.

[slemoine@myserver ~]$  sudo su -
[root@myserver ~]# su - mailuser
[mailuser@myserver ~]$ echo $(echo $USER):$(doveadm pw -s SHA512-CRYPT -p [PASSWORD]):$(id -u):$(id -g)::$(echo $HOME) > ~/users
[mailuser@myserver ~]$ logout
[root@myserver ~]# mv /home/mailuser/users /etc/dovecot/
[root@myserver ~]# chown dovecot:dovecot /etc/dovecot/users
[root@myserver ~]# chmod 640 /etc/dovecot/users

The above commands set the dovecot password and user information of mailuser in the /etc/dovecot/users file.

[PASSWORD] should be replaced by the password of your choice.

I use the SHA512-CRYPT method for password encryption but there is other (stronger) algorithms supported by the doveadm command. Please refer to the dovecot documentation.

In /etc/dovecot/conf.d/10-auth.conf

comment out

#!include auth-system.conf.ext

and uncomment:

!include auth-passwdfile.conf.ext

the /etc/dovecot/conf.d/auth-passwdfile.conf.ext should contains (by default):

passdb {
  driver = passwd-file
  args = scheme=CRYPT username_format=%u /etc/dovecot/users
}

userdb {
  driver = passwd-file
  args = username_format=%u /etc/dovecot/users

  # Default fields that can be overridden by passwd-file
  #default_fields = quota_rule=*:storage=1G

  # Override fields from passwd-file
  # override_fields = home=/home/virtual/%u
}

In /etc/dovecot/conf.d/10-mail.conf, look for the mail_location and mail_privileged_group properties and set

mail_location = mbox:~/mail:INBOX=/var/mail/%u
mail_privileged_group = mail

INBOX=/var/mail/%u is a reference to the mbox directory and mbox:~/mail is the directory for IMAP mailboxes.

In order to allow dovecot to copy emails from /var/mail/* to ~/mail without errors:

sudo chmod 0600 /var/mail/*

reload dovecot

sudo doveadm reload

test the dovecot Configuration (you need telnet package for this)

Open a telnet on localhost:143

[slemoine@myserver ~]$ telnet localhost 143
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
* OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE STARTTLS AUTH=PLAIN] Dovecot ready.

Check login:

a login mailuser [PASSWORD]
a OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHI
LDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS SPECIAL-USE BINARY MOVE] Logged in

Check Inbox access for logged user:

b select INBOX
* FLAGS (\Answered \Flagged \Deleted \Seen \Draft)
* OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted.
* 852 EXISTS
* 0 RECENT
* OK [UNSEEN 400] First unseen.
* OK [UIDVALIDITY 1513786247] UIDs valid
* OK [UIDNEXT 1048] Predicted next UID
b OK [READ-WRITE] Select completed (0.000 secs).

Dovecot configuration is done !

Webmail installation

Technically speaking, this part is optional. But you’ll probably need it when your Q&A staff will ask you to check emails sent by your app. I chose Roundcube as the webmail and deploy it on an nginx installation.

I’ll cover the nginx installation and roundcube installer installation here.

Install nginx

sudo yum install epel-release
sudo yum install nginx
sudo systemctl start nginx
sudo systemctl enable nginx

Roundcube is a php webmail. We configure php_fastcgi extension for nginx to allow nginx to correctly process php files.

sudo yum install php php-fpm
sudo systemctl start php-fpm
sudo systemctl enable php-fpm

php-fpm is apache ready by default. Let’s change it to nginx:

chown -R root:nginx /var/lib/php

I will use the default /etc/nginx/nginx.conf server block for the sake of this post brievety. I never use it in real life project.

Add the following lines into server block of /etc/nginx/nginx.conf to enable php fascgi processing.

index index.php index.html index.htm;

      location ~ \.php$ {
          try_files $uri =404;
          fastcgi_pass unix:/var/run/php-fpm/php-fpm.sock;
          fastcgi_index index.php;
          fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
          include fastcgi_params;
      }

Edit php configuration for nginx

vim /etc/php-fpm.d/www.conf

Set the user and group to nginx:

listen = /var/run/php-fpm/php-fpm.sock
listen.owner = nobody
listen.group = nobody
;listen.mode = 0666

; Unix user/group of processes
; Note: The user is mandatory. If the group is not set, the default user's group
;       will be used.
; RPM: apache Choosed to be able to access some dir as httpd
user = nginx
; RPM: Keep a group allowed to write in log dir.
group = nginx

You will need the following php extensions for roundcube:

sudo yum install php-xml php-pdo php-mbstring
sudo systemctl restart php-fpm
sudo systemctl restart nginx

download roundcube complete package: roundcubemail-1.3.4-complete

wget https://github.com/roundcube/roundcubemail/releases/download/1.3.4/roundcubemail-1.3.4-complete.tar.gz

Unzip it the directory served by nginx.

sudo tar -xzvf roundcubemail-1.3.4-complete.tar.gz -C /usr/share/nginx/html/

By default /usr/share/nginx/html/ directory should be ok.

I usually deploy it on the location webmail instead of roundcubemail-x.x.x, so

sudo mv /usr/share/nginx/html/roundcubemail-1.3.4/ /usr/share/nginx/html/webmail/

and then

sudo chown -R nginx:nginx /usr/share/nginx/html/webmail/

Run the installer

http://myserverhostname/webmail/installer/

Follow the installer process…

Hint: do not forget to set a valid path for the sqllite connection string (if you use sqlite):

$config['db_dsnw'] = 'sqlite:////usr/share/nginx/html/webmail/roundcubemail.db?mode=0646';

At the end you should be able to connect to the webmail with the mailuser account and the mailuser’s dovecot password at http://myserverhostname/webmail/

eg:

Ansible playbook and roles

The whole above installation process is fully compiled as an ansible playbook available on my github repo: https://github.com/slem1/mailconfig-ansible

Thank you for reading !

Comments