How to setup standalone firewall using shorewall

In this tutorial, i will show you how to setup firewall for standalone server (redhat, centos based)
1.Install shorewall
$yum install shorewall 
2.Setup & configuration
$vi /etc/shorewall/zones
###############################################################################
#ZONE   TYPE            OPTIONS         IN                      OUT
#                                       OPTIONS                 OPTIONS
fw      firewall
net     ipv4
#LAST LINE – ADD YOUR ENTRIES ABOVE THIS ONE – DO NOT REMOVE

$vi /etc/shorewall/interfaces
#ZONE   INTERFACE       BROADCAST       OPTIONS
 net    eth0            detect          tcpflags,dhcp,routefilter,nosmurfs,logmartians
#LAST LINE — ADD YOUR ENTRIES BEFORE THIS ONE — DO NOT REMOVE

$vi /etc/shorewall/policy
#SOURCE         DEST            POLICY          LOG             LIMIT:BURST
 fw             net             ACCEPT
 net            fw              DROP            info
 all            all             DROP            info
#LAST LINE — DO NOT REMOVE
 

$vi /etc/shorewall/rules
ACCEPT          fw              net
ACCEPT          net             fw              tcp     22
ACCEPT         net             fw              tcp     80
ACCEPT          net             fw              icmp
#LAST LINE — ADD YOUR ENTRIES BEFORE THIS ONE — DO NOT REMOVE 

$vi /etc/shorewall/shorewall.conf  (Must have this to start shorewall)
Change 

STARTUP_ENABLED=No into STARTUP_ENABLED=Yes
 
$/etc/init.d/shorewall restart 
Advertisements

Email with Postfix, Dovecot, and MySQL

Getting Started

Setting up a mail server is a big project. Before installing and configuring the necessary packages on your mail server, you should learn what everything does and understand how the components work together to send and receive email. For the purposes of this guide, we’ll assume that you’ll be using the following packages and operating system:

  • Postfix: This Mail Transfer Agent (MTA) handles relaying mail between different servers. It decides what to do with email from the outside world, and whether a particular user is allowed to send email using your server. It handles both incoming and outgoing SMTP. Postfix hands off local delivery (that is, the actual saving of the mail files on the server) to Dovecot’s Local Mail Transfer Protocol service (LMTP). Postfix also lets Dovecot take care of authentication before users are allowed to send email from the server. We’ll use version 2.9.6.
  • Dovecot: This IMAP/POP3 server handles requests from users who want to log in and check their email. Dovecot’s LMTP service functions as the Mail Delivery Agent (MDA) by saving mail files on the server. Dovecot also handles all authorization. It checks users’ email addresses and passwords in the MySQL database before allowing them to view or send email. We’ll use version 2.0.19.
  • MySQL: This database server stores lookup tables for domains, usernames and passwords, and aliases on the mail server. We’ll use version 14.14 Distrib 5.5.29.
  • Ubuntu 12.04 LTS: These instructions are designed to work with Ubuntu 12.04 LTS. Other distributions can also be made to work with Postfix, Dovecot, and MySQL, but those instructions are outside the scope of this guide.

If you encounter problems while using this guide, please double-check that you are using Ubuntu 12.04, and that the package versions match the ones listed above.

What’s Not Covered

For the sake of brevity, we’ve decided not to cover the following topics in this guide:

  • Spam and virus scanning for incoming messages to your users. You could use SpamAssassin to add that functionality later.
  • Webmail to allow users to access their email from a web browser.
  • GUIs for administration. You could use phpMyAdmin for MySQL or Postfix Admin for Postfix to add that functionality later.

Prerequisites

Before setting up your mail server, you’ll need to set up your Linode as specified in the Getting Started and Securing Your Server guides. You’ll also need to verify that you’ve completed the following steps:

Once you’ve verified these prerequisites, you’re ready to continue.

How It Works

Before we dive into the nitty-gritty of getting everything set up, let’s take a look at how we want everything to work together once it’s installed. The following process details what happens when an incoming message from the someone@somewhereelse.com email account makes its way to your Linode.

  1. someone@somewhereelse.com sends an email to me@mydomain.net.
  2. DNS is checked. The MX record for mydomain.net points to my Linode.
  3. The message reaches Postfix, the MTA.
  4. Postfix checks whether it is allowed to relay for mydomain.net by checking the virtual domains table in MySQL.
  5. MySQL returns a positive response for mydomain.net.
  6. Postfix relays the message using Dovecot’s LMTP socket.
  7. Dovecot saves the message to the me@mydomain.net mailbox on the server, which is located at /var/mail/mydomain.net/me/.

Incoming mail diagramThe email is now saved in the appropriate mailbox on the server. Next let’s see what happens when you check mail. The process starts when you decide you want to check your me@mydomain.net email from your local email client.

  1. Local Mail Client to Dovecot: Can I make a secure IMAP Connection?
  2. Dovecot to Local Mail Client: Sure. Here’s my SSL certificate. Now I need your username and password.
  3. Local Mail Client to Dovecot: Here’s my username and password.
  4. Dovecot to MySQL: MySQL, are this username and password in the users table?
  5. MySQL to Dovecot: Yes. This username and password are in the users table.
  6. Dovecot accesses the mailbox at /var/mail/mydomain.net/me/.
  7. Dovecot gets the mail files.
  8. Dovecot shows the messages to your local mail client using the IMAP protocol.

Checking mail diagramNow you can read your email using Outlook, Apple Mail, Thunderbird, etc. Finally, let’s see what happens when you send an email message from your account. Let’s say you want to send a reply from me@mydomain.net back to someone@somewhereelse.com. You compose a message in your local mail client and send it. What happens?

  1. Local Mail Client to Postfix: Can I make an SMTP connection?
  2. Postfix to Local Mail Client: Sure. You have to use encryption. Here’s my SSL certificate. Now I need your username and password.
  3. Local Mail Client to Postfix: Here’s my username and password.
  4. Postfix to Dovecot: Dovecot, check this username and password for me.
  5. Dovecot to MySQL: MySQL, are this username and password in the users table?
  6. MySQL to Dovecot: Yes. This username and password are in the users table.
  7. Dovecot to Postfix: Postfix, this user is authenticated.
  8. Postfix to Local Mail Client: You are allowed to send your message.
  9. Local Mail Client to Postfix: Here’s the message.

Sending mail diagramPostfix sends the email. This is known as relaying. The reason there are so many processes involved is for security – you don’t want just anyone to be able to send email through your server, otherwise they would quickly start sending lots of spam. The authentication process makes it safe for you and your authorized users to send email using this server while blocking everyone else.

Configuring DNS

Start thinking about the best time to switch your DNS records. Once you switch the MX records, you’ll start sending and receiving mail from your Linode. If you currently have live email accounts on another server, you shouldn’t change the DNS until you have everything set up and working. In the meantime, you can test your mail server setup with the default domain name Linode assigns to your server. And if you’re setting up a new domain, you might as well point the DNS records at your Linode now so you don’t have to change anything later.

Either way, you can lower the time to live (TTL) on your domain’s zone file now, in anticipation of the upcoming DNS change. This will help the DNS records propagate faster when you’re ready to switch them. You should do this whether you are planning to change your DNS right away or later.

When you’re ready to switch the DNS and start sending mail to the server, edit your domain’s MX record so it points to your Linode’s domain or IP address, similar to the example below:

example.com         MX      10      example.com
example.com         MX      10      12.34.56.78
mail.example.com    MX      10      12.34.56.78

Make sure you do this for all domains and subdomains that might receive email for your domain. If you use Linode’s DNS Manager, you will need to create an MX record that points to the desired domain or subdomain, and then create an A record for that domain or subdomain as well, that points to the correct IP address.

Installing an SSL Certificate

You should think about whether you need to purchase a valid SSL certificate or not. In this guide, you’ll use the default self-signed certificate that comes with Dovecot for free. This certificate encrypts your mail connections just like a purchased certificate, but your email users will receive warnings about the certificate when they attempt to set up their email accounts.

This can be confusing for users, and it may encourage bad security habits by forcing them to accept a self-signed certificate. If you’re going to set up all of your users’ mail clients yourself, or if you have a small number of tech-savvy users, this might not be a problem. You’ll need to use your best judgement to decide whether you need to purchase a signed SSL certificate or not. For information about SSL certificates, see these guides in the Linode Library.

Finding the Hostname

You’ll need your Linode’s hostname to configure Dovecot and Postfix. Before following these instructions, make sure you’ve set a hostname. Here’s how to find your Linode’s hostname:

  1. Open a terminal window and log in to your Linode via SSH.
  2. Find your server’s hostname by entering the following command, and then make a note of it:
    hostname
  3. Find your server’s fully-qualified domain name (FQDN) by entering the following command, and then make a note of it:
    hostname -f

Save these hostnames – you’ll need them later!

Installing Packages

Now that you understand how everything works and have finished preparing your Linode to act as a mail server, let’s configure your server for mail. We’ll start by installing all of the necessary packages. Here’s how:

  1. Log in as the root user by entering the following command:
    su
  2. Enter the password for the root user when prompted.
  3. Install the required packages by entering the following command. Here’s what you’ll install:
    apt-get install postfix postfix-mysql dovecot-core dovecot-imapd dovecot-pop3d dovecot-lmtpd dovecot-mysql mysql-server
  4. When prompted, type a new secure password for the root MySQL user, as shown below.

    Set your root MySQL password.

  5. Type the password again, as shown below. Make sure you remember what it is – you’ll need it later.

    Re-enter your root MySQL password.

  6. You’ll be prompted to select a Postfix configuration. Select Internet Site, as shown below.

    Choose "Internet Site" for Postfix.

  7. You’ll be prompted to enter a System mail name, as shown below. You can use your FQDN or any domain name that resolves to the server. This will become your server’s default domain for mail when none is specified.

    Set the system mail name for Postfix.

You just installed packages to support three applications: MySQL, Postfix, and Dovecot. Now it’s time to configure the individual applications to work together as a mail server.

MySQL

First, you’ll create a dedicated database in MySQL for your mail server. It will have three tables: one with domains, one with email addresses and encrypted passwords, and one with email aliases. You’ll also create a dedicated MySQL user for Postfix and Dovecot.

Note

Strictly speaking, you don’t have to use MySQL to store this information. You could, for example, just list it all in the Postfix and Dovecot config files. But that gets unwieldy pretty quickly when you have lots of domains and users. Having the information in a database makes it easier to access and update, and it should make the maintenance of your mail server easier in the long run.

Creating the Database

Here’s how to create the necessary database and tables in MySQL:

  1. Create a new database by entering the following command. We’ll call the database mailserver in this example.
    mysqladmin -p create mailserver
  2. Enter the MySQL root password.
  3. Log in to MySQL by entering the following command:
    mysql -p mailserver
  4. Enter the root MySQL password. You should see a command line prompt that looks like this:
    mysql>
  5. Create a new MySQL user (mailuser) by entering the following command. You’ll grant the user local, read-level access on the mailserver database, and you’ll also set the user’s password, which is mailuserpass in the example below. Please change this and make a note of the password for future use.
    GRANT SELECT ON mailserver.* TO 'mailuser'@'127.0.0.1' IDENTIFIED BY 'mailuserpass';
  6. Reload MySQL’s privileges to make sure the user has been added successfully:
    FLUSH PRIVILEGES;
  7. Enter the following command to create a table for the domains that will receive mail on your Linode. You can copy and paste the whole block of code at once – MySQL won’t execute it until you get to the semicolon (;). This will create a table called virtual_domains and give it two fields, an id field, and a name field for the domains.
    CREATE TABLE `virtual_domains` (
      `id` int(11) NOT NULL auto_increment,
      `name` varchar(50) NOT NULL,
      PRIMARY KEY (`id`)
    ) ENGINE=InnoDB DEFAULT CHARSET=utf8;
  8. Enter the following command to create a table for all of the email addresses and passwords. This command will create a table called virtual_users. It has adomain_id field to associate each entry with a domain, a password field to hold an encrypted version of each user’s password, and an email field to hold each user’s email address.
    CREATE TABLE `virtual_users` (
      `id` int(11) NOT NULL auto_increment,
      `domain_id` int(11) NOT NULL,
      `password` varchar(106) NOT NULL,
      `email` varchar(100) NOT NULL,
      PRIMARY KEY (`id`),
      UNIQUE KEY `email` (`email`),
      FOREIGN KEY (domain_id) REFERENCES virtual_domains(id) ON DELETE CASCADE
    ) ENGINE=InnoDB DEFAULT CHARSET=utf8;
  9. Enter the following command to create a table for your email aliases. This lets you forward mail from one email address to another. This command will create a table called virtual_aliases. It has an id field, a domain_id field which will associate each entry with a domain, a source field for the original email address, and adestination field for the target email address.
    CREATE TABLE `virtual_aliases` (
      `id` int(11) NOT NULL auto_increment,
      `domain_id` int(11) NOT NULL,
      `source` varchar(100) NOT NULL,
      `destination` varchar(100) NOT NULL,
      PRIMARY KEY (`id`),
      FOREIGN KEY (domain_id) REFERENCES virtual_domains(id) ON DELETE CASCADE
    ) ENGINE=InnoDB DEFAULT CHARSET=utf8;

Congratulations! You have successfully created the database and necessary tables in MySQL.

Adding Data

Now that you’ve created the database and tables, let’s add some data to MySQL. Here’s how:

  1. Add your domains to the virtual_domains table. You can add as many domains as you want in the VALUES section of the command below, but in this example you’ll add just the primary domain (example.com), your hostname (hostname), your FQDN (hostname.example.com), and localhost.example.com. (You’ll add localhost in a different file later). Be sure to replace example.com and hostname with your own domain name and hostname. You’ll need an id value and aname value for each entry. Separate each entry with a comma (,), and close the last one with a semicolon (;).
    INSERT INTO `mailserver`.`virtual_domains`
      (`id` ,`name`)
    VALUES
      ('1', 'example.com'),
      ('2', 'hostname.example.com'),
      ('3', 'hostname'),
      ('4', 'localhost.example.com');

Note

Make a note of which id goes with which domain – you’ll need for the next two steps.

  1. Add email addresses to the virtual_users table. In this example, you’ll add two new email addresses, email1@example.com and email2@example.com, with the passwords firstpassword and secondpassword, respectively. Be sure to replace the examples with your own information, but leave the password encryption functions intact. For each entry you’ll need to supply an id value, a domain_id, which should be the id number for the domain from Step 1 (in this case we’re choosing 1 for example.com), a password which will be in plain text in this command but which will get encrypted in the database, and an email, which is the full email address. Entries should be separated by a comma, and the final entry should be closed with a semicolon.
    INSERT INTO `mailserver`.`virtual_users`
      (`id`, `domain_id`, `password` , `email`)
    VALUES
      ('1', '1', ENCRYPT('firstpassword', CONCAT('$6$', SUBSTRING(SHA(RAND()), -16))), 'email1@example.com'),
      ('2', '1', ENCRYPT('secondpassword', CONCAT('$6$', SUBSTRING(SHA(RAND()), -16))), 'email2@example.com');
  2. If you want to set up an email alias, add it to the virtual_aliases table. Just like in the previous step, we’ll need an id value, and a domain_id value chosen from the virtual_domains list in Step 1. The source should be the email address you want to redirect. The destination should be the target email address, and can be any valid email address on your server or anywhere else.
    INSERT INTO `mailserver`.`virtual_aliases`
      (`id`, `domain_id`, `source`, `destination`)
    VALUES
      ('1', '1', 'alias@example.com', 'email1@example.com');

That’s it! Now you’re ready to verify that the data was successfully added to MySQL.

Testing

Now that you’ve entered all of the information into MySQL, you need to double check that it’s there. Here’s how:

  1. Check the contents of the virtual_domains table by entering the following command:
    SELECT * FROM mailserver.virtual_domains;
  2. Verify that you see the following output:
    +----+-----------------------+
    | id | name                  |
    +----+-----------------------+
    |  1 | example.com           |
    |  2 | hostname.example.com  |
    |  3 | hostname              |
    |  4 | localhost.example.com |
    +----+-----------------------+
    4 rows in set (0.00 sec)
  3. Check the virtual_users table by entering the following command:
    SELECT * FROM mailserver.virtual_users;
  4. Verify that you see the following output (the hashed passwords will be longer than they appear below):
    +----+-----------+-------------------------------------+--------------------+
    | id | domain_id | password                            | email              |
    +----+-----------+-------------------------------------+--------------------+
    |  1 |         1 | $6$574ef443973a5529c20616ab7c6828f7 | email1@example.com |
    |  2 |         1 | $6$030fa94bcfc6554023a9aad90a8c9ca1 | email2@example.com |
    +----+-----------+-------------------------------------+--------------------+
    2 rows in set (0.01 sec)
  5. Check the virtual_users table by entering the following command:
    SELECT * FROM mailserver.virtual_aliases;
  6. Verify that you see the following output:
    +----+-----------+-------------------+--------------------+
    | id | domain_id | source            | destination        |
    +----+-----------+-------------------+--------------------+
    |  1 |         1 | alias@example.com | email1@example.com |
    +----+-----------+-------------------+--------------------+
    1 row in set (0.00 sec)
  7. If everything looks good, you’re done with MySQL! Enter the following command to exit MySQL:
    exit

Now you’re ready to set up Postfix so your server can accept incoming messages for your domains.

Postfix

As the Mail Transfer Agent, Postfix decides where to relay messages that get directed to your server from anywhere else on the Internet. It also handles all SMTP connections and sends out messages for your users. In this section, you’ll modify some of these Postfix configuration options:

  • Virtual domains, aliases, and users, so you don’t have to make an actual UNIX user for everybody that needs an email address
  • MySQL access, so it can read the list of domains for which it should be handling mail
  • Hand-off for incoming email to Dovecot’s LMTP service so it can get saved on the server
  • STARTTLS encryption for all connections, for increased security
  • Hand-off for authentication to Dovecot

Here’s how to configure Postfix:

  1. Before doing anything else, enter the following command to make a copy of the default Postfix configuration file. This will come in handy if you mess up and need to revert to the default configuration.
    cp /etc/postfix/main.cf /etc/postfix/main.cf.orig
  2. Open the configuration file for editing by entering the following command:
    nano /etc/postfix/main.cf
  3. The default configuration file looks like this. The myhostname and mydestination lines are specific to your server, but everything else should be as it looks here:

    File excerpt:/etc/postfix/main.cf

    # See /usr/share/postfix/main.cf.dist for a commented, more complete version
    
    # Debian specific:  Specifying a file name will cause the first
    # line of that file to be used as the name.  The Debian default
    # is /etc/mailname.
    #myorigin = /etc/mailname
    
    smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu)
    biff = no
    
    # appending .domain is the MUA's job.
    append_dot_mydomain = no
    
    # Uncomment the next line to generate "delayed mail" warnings
    #delay_warning_time = 4h
    
    readme_directory = no
    
    # TLS parameters
    smtpd_tls_cert_file=/etc/ssl/certs/ssl-cert-snakeoil.pem
    smtpd_tls_key_file=/etc/ssl/private/ssl-cert-snakeoil.key
    smtpd_use_tls=yes
    smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache
    smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
    
    # See /usr/share/doc/postfix/TLS_README.gz in the postfix-doc package for
    # information on enabling SSL in the smtp client.
    
    myhostname = hostname.example.com
    alias_maps = hash:/etc/aliases
    alias_database = hash:/etc/aliases
    myorigin = /etc/mailname
    mydestination = example.com, hostname.example.com, localhost.example.com, localhost
    relayhost =
    mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128
    mailbox_size_limit = 0
    recipient_delimiter = +
    inet_interfaces = all
  4. Comment out all of the lines in the #TLS parameters section, and then paste in the four new lines shown below. Since we’re using Dovecot for authentication, we’re going to use Dovecot’s default certificate rather than Postfix’s default certificate. For increased security, we’re also going to force users to use TLS encryption.

    Note

    If you have purchased an SSL certificate for your mail server, you should use the path to that certificate and its corresponding key, not the default Dovecot certificate. Otherwise, you can just use the following values.

    Explanation of parameters:

    • smtpd_tls_cert_file: The location of your SSL certificate.
    • smtpd_tls_key_file: The location of your SSL certificate’s private key.
    • smtpd_use_tls: This tells connecting mail clients that STARTTLS encryption is available.
    • smtpd_tls_auth_only: This forces connecting mail clients to use STARTTLS before users are allowed to authenticate, ensuring that your users’ passwords are never sent in plain text.

    File excerpt:/etc/postfix/main.cf

    # TLS parameters
    #smtpd_tls_cert_file=/etc/ssl/certs/ssl-cert-snakeoil.pem
    #smtpd_tls_key_file=/etc/ssl/private/ssl-cert-snakeoil.key
    #smtpd_use_tls=yes
    #smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache
    #smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
    
    smtpd_tls_cert_file=/etc/ssl/certs/dovecot.pem
    smtpd_tls_key_file=/etc/ssl/private/dovecot.pem
    smtpd_use_tls=yes
    smtpd_tls_auth_only = yes
  5. Copy and paste the following values into the config file below the TLS settings. This will ease the restrictions and allow users to send email from their home or office. By default, only users who are logged into the server locally are able to send email. They will be required to log in with a password before being able to send email – this is very important, or anyone could start using your server to send spam! The smtpd_sasl_type and smtpd_sasl_path lines tell Postfix to use Dovecot for user authentication. Dovecot already authenticates users checking their email, so it makes sense to have it handle outgoing authentication too.

    Explanation of parameters:

    • smtpd_sasl_type: SASL (Simple Authentication and Security Layer) is the framework for authentication that Postfix uses. Authentication is needed so that only authorized users can use your server to send mail. In this case, we’re telling Postfix to use Dovecot’s authentication.
    • smtpd_sasl_path: This is the path to the authentication socket. The path used here is relative to /var/spool/postfix/. The socket is located at/var/spool/postfix/private/auth, or it will be when we create it with Dovecot.
    • smtpd_sasl_auth_enable: This tells Postfix to let people send email using this server if they’ve successfully authenticated. If this was turned off, Postfix would let people send email only if they were already on the server (e.g., they were logged in with SSH).
    • smtpd_recipient_restrictions: This tells Postfix which types of users are allowed to send email to other email addresses using the server. (Specifically, it applies to messages that have a RCPT TO component.) The first two parameters we added tell Postfix to allow sending for SASL-authenticated users and for users connecting from a network listed in the mynetworks parameter (in our case, just the server’s local network). The final parameter tells Postfix to reject sending email unless the recipient is for someone on this server.

    File excerpt:/etc/postfix/main.cf

    #Enabling SMTP for authenticated users, and handing off authentication to Dovecot
    smtpd_sasl_type = dovecot
    smtpd_sasl_path = private/auth
    smtpd_sasl_auth_enable = yes
    
    smtpd_recipient_restrictions =
            permit_sasl_authenticated,
            permit_mynetworks,
            reject_unauth_destination
  6. Comment out the existing mydestination line and replace it with one for localhost. This allows you to use the virtual domains listed in our MySQL table. It’s important that there is no overlap between the domains in the MySQL table and the domains in the mydestination line. Keeping the localhost entry inmydestination lets you keep things simple for mail sent within the server using localhost, which could be helpful if you’re ever having problems with your virtual domains.

    File excerpt:/etc/postfix/main.cf

    #mydestination = example.com, hostname.example.com, localhost.example.com, localhost
    mydestination = localhost
  7. Add a new line for local mail delivery (the service that actually saves the emails to individual user mailboxes). We’re telling Postfix not to use its own Local Delivery Agent (LDA) and instead use Dovecot’s LMTP (Local Mail Transfer Protocol) for local delivery. This applies to all virtual domains listed in the MySQL table.

    File excerpt:/etc/postfix/main.cf

    #Handing off local delivery to Dovecot's LMTP, and telling it where to store mail
    virtual_transport = lmtp:unix:private/dovecot-lmtp
  8. Add the following values to configure your virtual domains, users, and aliases. No changes are necessary.

    Explanation of parameters:

    • virtual_mailbox_domains: Here you tell Postfix that you’re using MySQL to store virtual domains, and then give it a path to another file where you’ll put all the MySQL connection details.
    • virtual_mailbox_maps: Same as above, but for email users.
    • virtual_alias_maps: Same as above, but for aliases.

    File excerpt:/etc/postfix/main.cf

    #Virtual domains, users, and aliases
    virtual_mailbox_domains = mysql:/etc/postfix/mysql-virtual-mailbox-domains.cf
    virtual_mailbox_maps = mysql:/etc/postfix/mysql-virtual-mailbox-maps.cf
    virtual_alias_maps = mysql:/etc/postfix/mysql-virtual-alias-maps.cf
  9. Compare your Postfix configuration file to our final configuration file shown below. If necessary, make changes to your file before proceeding.

    File excerpt:/etc/postfix/main.cf

    # See /usr/share/postfix/main.cf.dist for a commented, more complete version
    
    # Debian specific:  Specifying a file name will cause the first
    # line of that file to be used as the name.  The Debian default
    # is /etc/mailname.
    #myorigin = /etc/mailname
    
    smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu)
    biff = no
    
    # appending .domain is the MUA's job.
    append_dot_mydomain = no
    
    # Uncomment the next line to generate "delayed mail" warnings
    #delay_warning_time = 4h
    
    readme_directory = no
    
    # TLS parameters
    #smtpd_tls_cert_file=/etc/ssl/certs/ssl-cert-snakeoil.pem
    #smtpd_tls_key_file=/etc/ssl/private/ssl-cert-snakeoil.key
    #smtpd_use_tls=yes
    #smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache
    #smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
    
    smtpd_tls_cert_file=/etc/ssl/certs/dovecot.pem
    smtpd_tls_key_file=/etc/ssl/private/dovecot.pem
    smtpd_use_tls=yes
    smtpd_tls_auth_only = yes
    
    #Enabling SMTP for authenticated users, and handing off authentication to Dovecot
    smtpd_sasl_type = dovecot
    smtpd_sasl_path = private/auth
    smtpd_sasl_auth_enable = yes
    
    smtpd_recipient_restrictions =
            permit_sasl_authenticated,
            permit_mynetworks,
            reject_unauth_destination
    
    # See /usr/share/doc/postfix/TLS_README.gz in the postfix-doc package for
    # information on enabling SSL in the smtp client.
    
    myhostname = host.example.com
    alias_maps = hash:/etc/aliases
    alias_database = hash:/etc/aliases
    myorigin = /etc/mailname
    #mydestination = example.com, hostname.example.com, localhost.example.com, localhost
    mydestination = localhost
    relayhost =
    mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128
    mailbox_size_limit = 0
    recipient_delimiter = +
    inet_interfaces = all
    
    #Handing off local delivery to Dovecot's LMTP, and telling it where to store mail
    virtual_transport = lmtp:unix:private/dovecot-lmtp
    
    #Virtual domains, users, and aliases
    virtual_mailbox_domains = mysql:/etc/postfix/mysql-virtual-mailbox-domains.cf
    virtual_mailbox_maps = mysql:/etc/postfix/mysql-virtual-mailbox-maps.cf
    virtual_alias_maps = mysql:/etc/postfix/mysql-virtual-alias-maps.cf
  10. Save the changes you’ve made to the /etc/postfix/main.cf file.
  11. Create the three files you specified earlier. These files will tell Postfix how to connect to MySQL to read the lists of domains, email addresses, and aliases. Create the file for virtual domains by entering the following command:
    nano /etc/postfix/mysql-virtual-mailbox-domains.cf
  12. Enter the following values. At a minimum, you’ll need to change the password entry to the one you created for mailuser. If you used a different user, database name, or table name, customize those settings as well.

    File excerpt:/etc/postfix/mysql-virtual-mailbox-domains.cf

    user = mailuser
    password = mailuserpass
    hosts = 127.0.0.1
    dbname = mailserver
    query = SELECT 1 FROM virtual_domains WHERE name='%s'
  13. Save the changes you’ve made to the /etc/postfix/mysql-virtual-mailbox-domains.cf file.
  14. Restart Postfix by entering the following command:
    service postfix restart
  15. Enter the following command to ensure that Postfix can find your first domain. Be sure to replace example.com with your first virtual domain. The command should return 1 if it is successful; if nothing is returned, you have an issue.
    postmap -q example.com mysql:/etc/postfix/mysql-virtual-mailbox-domains.cf
  16. Create the connection file for your email addresses by entering the following command:
    nano /etc/postfix/mysql-virtual-mailbox-maps.cf
  17. Enter the following values. Make sure you use your own password, and make any other changes as needed.

    File excerpt:/etc/postfix/mysql-virtual-mailbox-maps.cf

    user = mailuser
    password = mailuserpass
    hosts = 127.0.0.1
    dbname = mailserver
    query = SELECT 1 FROM virtual_users WHERE email='%s'
  18. Save the changes you’ve made to the /etc/postfix/mysql-virtual-mailbox-maps.cf file.
  19. Restart Postfix by entering the following command:
    service postfix restart
  20. Test Postfix to verify that it can find the first email address in your MySQL table. Enter the following command, replacing email1@example.com with the first email address in your MySQL table. You should again receive 1 as the output:
    postmap -q email1@example.com mysql:/etc/postfix/mysql-virtual-mailbox-maps.cf
  21. Create the file that will allow Postfix to access the aliases in MySQL by entering the following command:
    nano /etc/postfix/mysql-virtual-alias-maps.cf
  22. Enter the following values. Again, make sure you use your own password, and make any other changes as necessary.

    File excerpt:/etc/postfix/mysql-virtual-alias-maps.cf

    user = mailuser
    password = mailuserpass
    hosts = 127.0.0.1
    dbname = mailserver
    query = SELECT destination FROM virtual_aliases WHERE source='%s'
  23. Save the changes you’ve made to the /etc/postfix/mysql-virtual-alias-maps.cf file.
  24. Restart Postfix by entering the following command:
    service postfix restart
  25. Test Postfix to verify that it can find your aliases by entering the following command. Be sure to replace alias@example.com with the actual alias you entered:
    postmap -q alias@example.com mysql:/etc/postfix/mysql-virtual-alias-maps.cf

    This should return the email address to which the alias forwards, which is email1@example.com in this example.

Congratulations! You have successfully configured Postfix.

Dovecot

Dovecot allows users to log in and check their email using POP3 and IMAP. In this section, you’ll configure Dovecot to force users to use SSL when they connect so that their passwords are never sent to the server in plain text. Users will have to connect using the standard SSL ports – 993 for IMAP and 995 for POP3 – and only those ports. Dovecot’s LMTP service will function as the MDA and store incoming messages in the proper locations on the server. Dovecot will also be handling all user authentication for mail.

Dovecot 2 uses a number of different configuration files. The primary configuration file contains a few directives, and then several inclusions of other configuration files. This helps to separate different configuration parameters logically so they’re not all grouped together in one file. This is a major change from Dovecot 1, where virtually everything was configured in the same file.

In this section, you’ll configure Dovecot to:

  • Set the IMAP, POP3, and LMTP protocols
  • Define the mail location
  • Use MySQL for username/password lookups for authentication
  • Configure needed sockets for authentication and LMTP
  • Require SSL encryption

You’ll modify a total of 7 Dovecot configuration files. Here’s the list:

  • /etc/dovecot/dovecot.conf: Dovecot’s main configuration file
  • /etc/dovecot/conf.d/10-mail.conf: Deals with the server’s file system
  • /etc/dovecot/conf.d/10-auth.conf: Defines how user authentication is handled
  • /etc/dovecot/conf.d/auth-sql.conf.ext: New authentication file for SQL-type authentication
  • /etc/dovecot/dovecot-sql.conf.ext: An included authentication file with the MySQL connection parameters
  • /etc/dovecot/conf.d/10-master.conf: Where sockets are configured
  • /etc/dovecot/conf.d/10-ssl.conf: Where SSL-related parameters are specified

Here’s how to configure Dovecot:

  1. Copy all of the configuration files so that you can easily revert back to them if needed. Enter the following commands, one by one:
    cp /etc/dovecot/dovecot.conf /etc/dovecot/dovecot.conf.orig
    cp /etc/dovecot/conf.d/10-mail.conf /etc/dovecot/conf.d/10-mail.conf.orig
    cp /etc/dovecot/conf.d/10-auth.conf /etc/dovecot/conf.d/10-auth.conf.orig
    cp /etc/dovecot/dovecot-sql.conf.ext /etc/dovecot/dovecot-sql.conf.ext.orig
    cp /etc/dovecot/conf.d/10-master.conf /etc/dovecot/conf.d/10-master.conf.orig
    cp /etc/dovecot/conf.d/10-ssl.conf /etc/dovecot/conf.d/10-ssl.conf.orig
  2. Enter the following command to open the main configuration file for editing:
    nano /etc/dovecot/dovecot.conf

    Note

    Click this link to see the final, complete version of our dovecot.conf example file.

  3. Verify that dovecot.conf is including all of the other configuration files. This option should be enabled by default:

    File excerpt:/etc/dovecot/dovecot.conf

    !include conf.d/*.conf
  4. Add the following line to /etc/dovecot/dovecot.conf so Dovecot knows to support IMAP, POP3, and LMTP. In this example, we have inserted it below the existing!include_try /usr/share/dovecot/protocols.d/*.protocol line:

    File excerpt:/etc/dovecot/dovecot.conf

    # Enable installed protocols
    !include_try /usr/share/dovecot/protocols.d/*.protocol
    protocols = imap pop3 lmtp
  5. Save your changes to the /etc/dovecot/dovecot.conf file.
  6. Open the /etc/dovecot/conf.d/10-mail.conf file for editing by entering the following command. This file allows us to control how Dovecot interacts with the server’s file system to store and retrieve messages.
    nano /etc/dovecot/conf.d/10-mail.conf

    Note

    Click this link to see the final, complete version of our 10-mail.conf example file. This is a long file, so you may need to use your editor’s search feature to find the values you need to edit.

  7. Find the mail_location variable, uncomment it, and then set it to the following value. This tells Dovecot where to look for mail. In this case, the mail will be stored in /var/mail/vhosts/example.com/user/, where example.com and user are variables that get pulled from the connecting email address. For example, if someone logs in to the server with the email address email1@example.com, Dovecot will use example.com for %d, and email1 for %n. You can change this path if you want, but you’ll have to change it everywhere else the mail storage path is referenced in this tutorial. It’s useful to keep this location in mind if you ever need to manually download the raw mail files from the server.

    File excerpt:/etc/dovecot/conf.d/10-mail.conf

    mail_location = maildir:/var/mail/vhosts/%d/%n
  8. Find the mail_privileged_group variable. Uncomment it, and then set it to the following value. This allows Dovecot to write to the /var/mail/ folder.

    File excerpt:/etc/dovecot/conf.d/10-mail.conf

    mail_privileged_group = mail
  9. Save your changes to the /etc/dovecot/conf.d/10-mail.conf file.
  10. Enter the following command to verify the permissions for /var/mail:
    ls -ld /var/mail
  11. Verify that the permissions for /var/mail are as follows:
    drwxrwsr-x 2 root mail 4096 Mar  6 15:08 /var/mail
  12. Create the /var/mail/vhosts/ folder and the folder(s) for each of your domains by entering the following command:
    mkdir -p /var/mail/vhosts/example.com
  13. Create the vmail user with a user and group id of 5000 by entering the following commands, one by one. This user will be in charge of reading mail from the server.
    groupadd -g 5000 vmail
    useradd -g vmail -u 5000 vmail -d /var/mail
  14. Change the owner of the /var/vmail/ folder and its contents to belong to vmail by entering the following command:
    chown -R vmail:vmail /var/mail
  15. Open the user authentication file for editing by entering the command below. You need to set up authentication so only authenticated users can read mail on the server. You also need to configure an authentication socket for outgoing mail, since we told Postfix that Dovecot was going to handle that. There are a few different files related to authentication that get included in each other.
    nano /etc/dovecot/conf.d/10-auth.conf

    Note

    Click the link to see the final, complete version of 10-auth.conf.

  16. Disable plain-text authentication by uncommenting this line:

    File excerpt:/etc/dovecot/conf.d/10-auth.conf

    disable_plaintext_auth = yes
  17. Set the auth_mechanisms by modifying the following line:

    File excerpt:/etc/dovecot/conf.d/10-auth.conf

    auth_mechanisms = plain login
  18. Add a hash tag (#) to comment out the system user login line:

    File excerpt:/etc/dovecot/conf.d/10-auth.conf

    #!include auth-system.conf.ext
  19. Enable MySQL authentication by uncommenting the auth-sql.conf.ext line. That section should look like this:

    File excerpt:/etc/dovecot/conf.d/10-auth.conf

    #!include auth-system.conf.ext
    !include auth-sql.conf.ext
    #!include auth-ldap.conf.ext
    #!include auth-passwdfile.conf.ext
    #!include auth-checkpassword.conf.ext
    #!include auth-vpopmail.conf.ext
    #!include auth-static.conf.ext
  20. Save your changes to the /etc/dovecot/conf.d/10-auth.conf file.
  21. Now you need to create the /etc/dovecot/conf.d/auth-sql.conf.ext file with your authentication information. Enter the following command to create the new file:
    nano /etc/dovecot/conf.d/auth-sql.conf.ext
  22. Paste the following lines into in the new file:

    File excerpt:/etc/dovecot/conf.d/auth-sql.conf.ext

    passdb {
      driver = sql
      args = /etc/dovecot/dovecot-sql.conf.ext
    }
    userdb {
      driver = static
      args = uid=vmail gid=vmail home=/var/mail/vhosts/%d/%n
    }

    Explanation of parameters:

    • passdb tells Dovecot how to look up users for authentication. We’re telling Dovecot to use MySQL. In the args line, we’re also specifying the file that contains the MySQL connection information.
    • userdb tells Dovecot where to look for users’ mail on the server. We’re using a static driver since the path will be in the same format for everyone.
  23. Save your changes to the /etc/dovecot/conf.d/auth-sql.conf.ext file.
  24. Update the /etc/dovecot/dovecot-sql.conf.ext file with our custom MySQL connection information. Open the file for editing by entering the following command:
    nano /etc/dovecot/dovecot-sql.conf.ext

    Note

    Click the link to see the final, complete version of dovecot-sql.conf.ext.

  25. Uncomment and set the driver line as shown below:

    File excerpt:/etc/dovecot/dovecot-sql.conf.ext

    driver = mysql
  26. Uncomment the connect line and set your MySQL connection information. Make sure you use your own password and any other custom settings:

    File excerpt:/etc/dovecot/dovecot-sql.conf.ext

    connect = host=127.0.0.1 dbname=mailserver user=mailuser password=mailuserpass
  27. Uncomment the default_pass_scheme line and set it to SHA512-CRYPT. This tells Dovecot to expect the passwords in an ecrypted format (which is how they are stored in the database).

    File excerpt:/etc/dovecot/dovecot-sql.conf.ext

    default_pass_scheme = SHA512-CRYPT
  28. Uncomment the password_query line and set it to the following. This is a MySQL query that Dovecot uses to retrieve the password from the database.

    File excerpt:/etc/dovecot/dovecot-sql.conf.ext

    password_query = SELECT email as user, password FROM virtual_users WHERE email='%u';

    Note

    This password query lets you use an email address listed in the virtual_users table as your username credential for an email account. The primary email address should still be used as the username, even if you have set up your email client for an alias. If you want to be able to use the alias as your username instead (listed in the virtual_aliases table), you should first add every primary email address to the virtual_aliases table (directing to themselves) and then use the following line in /etc/dovecot/dovecot-sql.conf.ext instead:

    password_query = SELECT email as user, password FROM virtual_users WHERE email=(SELECT destination FROM virtual_aliases WHERE source = '%u');
  29. Save your changes to the /etc/dovecot/dovecot-sql.conf.ext file.
  30. Change the owner and group of the /etc/dovecot/ directory to vmail and dovecot by entering the following command:
    chown -R vmail:dovecot /etc/dovecot
  31. Change the permissions on the /etc/dovecot/ directory by entering the following command:
    chmod -R o-rwx /etc/dovecot
  32. Open the sockets configuration file by entering the following command. You’ll change the settings in this file to set up the LMTP socket for local mail delivery, and the auth socket for authentication. Postfix uses these sockets to connect to Dovecot’s services.
    nano /etc/dovecot/conf.d/10-master.conf

    Note

    Click the link to see the final, complete version of 10-master.conf. There are many nested blocks of code in this file, so please pay very close attention to your brackets. It’s probably better if you edit line by line, rather than copying large chunks of code. If there’s a syntax error, Dovecot will crash silently, but you can check /var/log/upstart/dovecot.log to help you find the error.

  33. Disable unencrypted IMAP and POP3 by setting the protocols’ ports to 0, as shown below. This will force your users to use secure IMAP or secure POP on 993 or 995 when they configure their mail clients:

    File excerpt:/etc/dovecot/conf.d/10-master.conf

    service imap-login {
      inet_listener imap {
        port = 0
      }
    ...
    }
    
    service pop3-login {
      inet_listener pop3 {
        port = 0
      }
    ...
    }

    Note

    Make sure you leave the secure versions alone – imaps and pop3s – so their ports still work. The default settings for imaps and pop3s are fine. You can leave the port lines commented out, as the default ports are the standard 993 and 995.

  34. Find the service lmtp section and use the configuration shown below. You’ll need to add a few lines in the unix_listener block. This section makes the socket for LMTP in the place we told Postfix to look for it.

    File excerpt:/etc/dovecot/conf.d/10-master.conf

    service lmtp {
     unix_listener /var/spool/postfix/private/dovecot-lmtp {
       mode = 0600
       user = postfix
       group = postfix
      }
      # Create inet listener only if you can't use the above UNIX socket
      #inet_listener lmtp {
        # Avoid making LMTP visible for the entire internet
        #address =
        #port =
      #}
    }
  35. Locate the service auth section and use the configuration shown below. You’ll need to create a new unix_listener block, modify the existing one, and then uncomment and set the user. This section makes the authorization socket where we told Postfix to look for it:

    File excerpt:/etc/dovecot/conf.d/10-master.conf

    service auth {
      # auth_socket_path points to this userdb socket by default. It's typically
      # used by dovecot-lda, doveadm, possibly imap process, etc. Its default
      # permissions make it readable only by root, but you may need to relax these
      # permissions. Users that have access to this socket are able to get a list
      # of all usernames and get results of everyone's userdb lookups.
      unix_listener /var/spool/postfix/private/auth {
        mode = 0666
        user = postfix
        group = postfix
      }
    
      unix_listener auth-userdb {
        mode = 0600
        user = vmail
        #group =
      }
    
      # Postfix smtp-auth
      #unix_listener /var/spool/postfix/private/auth {
      #  mode = 0666
      #}
    
      # Auth process is run as this user.
      user = dovecot
    }
  36. In the service auth-worker section, uncomment the user line and set it to vmail, as shown below.

    File excerpt:/etc/dovecot/conf.d/10-master.conf

    service auth-worker {
      # Auth worker process is run as root by default, so that it can access
      # /etc/shadow. If this isn't necessary, the user should be changed to
      # $default_internal_user.
      user = vmail
    }
  37. Save your changes to the /etc/dovecot/conf.d/10-master.conf file.
  38. Verify that the default Dovecot SSL certificate and key exist by entering the following commands, one by one:
    ls /etc/ssl/certs/dovecot.pem
    ls /etc/ssl/private/dovecot.pem

    Note

    If you are using a different SSL certificate, you should upload the certificate to the server and make a note of its location and the key’s location.

  39. Open the SSL configuration file for editing by entering the following command. This is where we tell Dovecot where to find our SSL certificate and key, and any other SSL-related parameters.
    nano /etc/dovecot/conf.d/10-ssl.conf

    Note

    Click the link to see the final, complete version of 10-ssl.conf.

  40. Verify that the ssl_cert setting has the path to your certificate, and that the ssl_key setting has the path to your key. The default setting here uses Dovecot’s built-in certificate, so you can leave this as-is if you are using the Dovecot certificate. You should update the paths if you are using a different certificate and key.

    File excerpt:/etc/dovecot/conf.d/10-ssl.conf

    ssl_cert = </etc/ssl/certs/dovecot.pem
    ssl_key = </etc/ssl/private/dovecot.pem
  41. Force your clients to use SSL encryption for all connections. Set ssl to required:

    File excerpt:/etc/dovecot/conf.d/10-ssl.conf

    ssl = required
  42. Save your changes to the /etc/dovecot/conf.d/10-ssl.conf file. Dovecot has been configured!
  43. Restart Dovecot by entering the following command:
    service dovecot restart
  44. Set up a test account in an email client to make sure everything is working. You’ll need to use the following parameters:
    • Your full email address, including the @example.com part, is your username.
    • Your password should be the one you added to the MySQL table for this email address.
    • The incoming and outgoing server names must be a domain that resolves to your Linode.
    • Both the incoming and outgoing servers require authentication and SSL encryption.
    • You should use Port 993 for secure IMAP, Port 995 for secure POP3, and Port 25 with SSL for SMTP.
  45. Try sending an email to this account from an outside email account and then reply to it. If it works, you’re in business! You can check your mail log file in/var/log/mail.log, where you should see something like this (the first block is for an incoming message, and the second block for an outgoing message):

    File excerpt:/var/log/mail.log

    Mar 22 18:18:15 host postfix/smtpd[22574]: connect from mail1.linode.com[96.126.108.55]
    Mar 22 18:18:15 host postfix/smtpd[22574]: 2BD192839B: client=mail1.linode.com[96.126.108.55]
    Mar 22 18:18:15 host postfix/cleanup[22583]: 2BD192839B: message-id=<D4887A5E-DEAC-45CE-BDDF-3C89DEA84236@example.com>
    Mar 22 18:18:15 host postfix/qmgr[15878]: 2BD192839B: from=<support@linode.com>, size=1156, nrcpt=1 (queue active)
    Mar 22 18:18:15 host postfix/smtpd[22574]: disconnect from mail1.linode.com[96.126.108.55]
    Mar 22 18:18:15 host dovecot: lmtp(22587): Connect from local
    Mar 22 18:18:15 host dovecot: lmtp(22587, email1@example.com): 5GjrDafYTFE7WAAABf1gKA: msgid=<D4887A5E-DEAC-45CE-BDDF-3C89DEA84236@linode.com>: saved mail to INBOX
    Mar 22 18:18:15 host dovecot: lmtp(22587): Disconnect from local: Client quit (in reset)
    Mar 22 18:18:15 host postfix/lmtp[22586]: 2BD192839B: to=<email1@example.com>, relay=host.example.com[private/dovecot-lmtp], delay=0.09, delays=0.03/0.02/0.03/0.01, dsn=2.0.0, status=sent (250 2.0.0 <email1@example.com> 5GjrDafYTFE7WAAABf1gKA Saved)
    Mar 22 18:18:15 host postfix/qmgr[15878]: 2BD192839B: removed

    File excerpt:/var/log/mail.log

    Mar 22 18:20:29 host postfix/smtpd[22590]: connect from 173-161-199-49-Philadelphia.hfc.comcastbusiness.net[173.161.199.49]
    Mar 22 18:20:29 host dovecot: auth-worker: mysql(127.0.0.1): Connected to database mailserver
    Mar 22 18:20:29 host postfix/smtpd[22590]: AA10A2839B: client=173-161-199-49-Philadelphia.hfc.comcastbusiness.net[173.161.199.49], sasl_method=PLAIN, sasl_username=email1@example.com
    Mar 22 18:20:29 host postfix/cleanup[22599]: AA10A2839B: message-id=<FB6213FA-6F13-49A8-A5DD-F324A4FCF9E9@example.com>
    Mar 22 18:20:29 host postfix/qmgr[15878]: AA10A2839B: from=<email1@example.com>, size=920, nrcpt=1 (queue active)
    Mar 22 18:20:29 host postfix/smtp[22601]: AA10A2839B: to=<support@linode.com>, relay=mail1.linode.com[96.126.108.55]:25, delay=0.14, delays=0.08/0.01/0.05/0.01, dsn=2.0.0, status=sent (250 2.0.0 Ok: queued as C4232266C9)
    Mar 22 18:20:29 host postfix/qmgr[15878]: AA10A2839B: removed

Congratulations! You now have a functioning mail server that can securely send and receive email. At this point, you may want to consider adding spam and virus filtering and a webmail client. If you haven’t switched the DNS records for your mail server yet, you should be able to do so now. Once the DNS records have propagated, you will start receiving email for your domain on the server.

Adding New Domains, Email Addresses, and Aliases

Now your mail server is up and running, but eventually you’ll probably need to add new domains, email addresses, and aliases for your users. To do this, all you’ll have to do is add a new line to the appropriate MySQL table. These instructions are for command-line MySQL, but you can just as easily use phpMyAdmin to add new entries to your tables as well.

Domains

Here’s how to add a new domain to your Postfix and Dovecot setup:

  1. Open a terminal window and log in to your Linode via SSH.
  2. Log in to your MySQL server with an appropriately privileged user. In this example, we’ll use the root user:
    mysql -u root -p mailserver
  3. Enter your root MySQL password when prompted.
  4. You should always view the contents of the table before adding new entries. Enter the following command to view the current contents of any table, replacingvirtual_domains with your table:
    SELECT * FROM mailserver.virtual_domains;
  5. The output should resemble the following:
    +----+-----------------------+
    | id | name                  |
    +----+-----------------------+
    |  1 | example.com           |
    |  2 | hostname.example.com  |
    |  3 | hostname              |
    |  4 | localhost.example.com |
    +----+-----------------------+
  6. To add another domain, enter the following command, replacing newdomain.com with your domain name:
    INSERT INTO `mailserver`.`virtual_domains`
      (`name`)
    VALUES
      ('newdomain.com');
  7. Verify that the new domain has been added by entering the following command. You should see the new domain name in the output.
    SELECT * FROM mailserver.virtual_domains;
  8. To exit MySQL, enter the following command:
    quit

Congratulations! You have successfully added the new domain to your Postfix and Dovecot setup.

Email Addresses

Here’s how to add a new email address to your Postfix and Dovecot setup:

  1. Enter the following command in MySQL, replacing newpassword with the user’s password, and email3@newdomain.com with the user’s email address:
    INSERT INTO `mailserver`.`virtual_users`
      (`domain_id`, `password` , `email`)
    VALUES
      ('5', ENCRYPT('newpassword', CONCAT('$6$', SUBSTRING(SHA(RAND()), -16))) , 'email3@newdomain.com');

Note

Be sure to use the correct number for the domain_id. In this case, we are using 5, because we want to make an email address for newdomain.com, andnewdomain.com has an id of 5 in the virtual_domains table.

  1. Verify that the new email address has been added by entering the following command. You should see the new email address in the output.
    SELECT * FROM mailserver.virtual_users;
  2. To exit MySQL, enter the following command:
    quit

Congratulations! You have successfully added the new email address to your Postfix and Dovecot setup.

Aliases

Here’s how to add a new alias to your Postfix and Dovecot setup:

  1. Enter the following command in MySQL, replacing alias@newdomain.com with the address from which you want to forward email, and myemail@gmail.comwith the address that you want to forward the mail to. The alias@newdomain.com needs to be an email address that already exists on your server.
    INSERT INTO `mailserver`.`virtual_aliases`
      (`domain_id`, `source`, `destination`)
    VALUES
      ('5', 'alias@newdomain.com', 'myemail@gmail.com');

Note

You will need to use the correct number for the domain_id. You should use the id of the domain for this email address; see the explanation in the email users section above.

  1. Verify that the new alias has been added by entering the following command. You should see the new alias in the output.
    SELECT * FROM mailserver.virtual_aliases;
  2. To exit MySQL, enter the following command:
    quit

CentOS – Installing LiteSpeed

CentOS – Installing LiteSpeed


This tutorial will walk you through installing the LiteSpeed web-server on CentOS 5.2.

The tutorial assumes the following:

  • You are working on a clean server
  • You have completed the CentOS Setup Guide
  • You are familiar with basic Linux commands
  • You are running the commands as a normal user with ‘sudo’ permissions

Contents

Update Your System

The first thing we need to do is make sure your Cloud Server has all of it’s security patches. Connect to your Cloud Server with SSH (or the Console from the Control Panel) and login as your normal user. For the purposes of this walk-through our sample user is conveniently named ‘user’.

Execute the YUM package manager and use the ‘upgrade’ command to upgrade the system:

# sudo yum upgrade

A series of items will fly by and you will be prompted to install… press Y followed by Enter. This will take a few minutes.

It is a good idea to restart your Cloud Server to make sure any new packages that were downloaded are freshly loaded.

# sudo reboot

You can use the ping command to check when your server has come back online.

$ ping 67.23.13.230

PING 67.23.13.230 (67.23.13.230) 56(84) bytes of data.
64 bytes from 67.23.13.230: icmp_seq=33 ttl=56 time=19.1 ms
64 bytes from 67.23.13.230: icmp_seq=34 ttl=56 time=9.52 ms
64 bytes from 67.23.13.230: icmp_seq=35 ttl=56 time=10.4 ms
...

Once it is responding to ping requests go ahead and connect to your Cloud Server again.

Install developer tools

If you haven’t installed them already, you should install the CentOS standard developer tools to cover a requirement for glibc (and because the dev tools can be useful to have around).

sudo yum groupinstall 'Development Tools'

Download LiteSpeed

Now we need to download the LiteSpeed web server. This walk-through will cover installation of the free Standard Edition.

To download the software you will need to do the following:

  • Visit the LiteSpeed download page
  • Scroll down to LiteSpeed Web Server Standard Edition: Free
  • Download the file associated with Linux (x86)
  • Transfer the file to your Cloud Server (not covered in this tutorial)

Another option is to copy the direct link and download the file manually on your Cloud Server. For purposes of demonstration we will show you this option. The link below was valid at the time of writing and may have changed — please adjust accordingly.

# cd ~
# wget http://www.litespeedtech.com/packages/4.0/lsws-4.0.3-std-i386-linux.tar.gz

You will see the wget utility appear and it will download the file. Once it has finished you will be returned to the prompt.

Installing LiteSpeed

We need to unpack the LiteSpeed package that we downloaded. Type the following command to unpack: (note that your file name may be different)

# sudo tar -zxvf lsws-4.0.3-std-i386-linux.tar.gz

Now we need to enter the directory that we extracted the files into:

# cd lsws-4.0.3

We are now ready to start the installer. Type the following command to start the installer:

# sudo ./install.sh

You will be prompted with a license agreement. Read the agreement and press the space bar multiple times until you reach the end.

After you ‘space’ through the license you will be prompted with the following:

IMPORTANT: In order to continue installation you must agree with above 
           license terms by typing "Yes" with capital "Y"! 

Do you agree with above license?

Type Yes at this point and press Enter. Note that you *must* put a capital Y.

The next screen that appears will ask you what directory you would like to install LiteSpeed. The default directory is sufficient. Simple press Enter to accept the default.

Please specify the destination directory. You must have permissions to 
create and manage the directory. It is recommended to install the web server 
at /opt/lsws, /usr/local/lsws or in your home directory like '~/lsws'.

ATTENTION: The user 'nobody' must be able to access the destination
           directory.

Destination [/usr/local/lsws]:

The next prompt will ask you for the administrative login that you would like to use for the administrative console. Simply press Enter to accept the default.

Please specify the user name of the administrator.
This is the user name required to log into the administration web interface.

User name [admin]:

Enter the password that you’d like to use for administering your web server. Please make sure this is secure as it has the power to stop your server! Press Enter once you have entered it.

Please specify the administrator's password.
This is the password required to log into the administration web interface.

Password:

Retype the password again.

Enter an e-mail address for the server administrator. This will be displayed on error messages so the server administrator may be contacted in the event of server failure.

Please specify administrators' email addresses.
It is recommended to specify a real email address,
Multiple email addresses can be set by a comma 
delimited list of email addresses. Whenever something
abnormal happened, a notificiation will be sent to 
emails listed here.

Email addresses [root@localhost]:

Next you will be prompted for the user that the web server should run as. Leave it the default user of nobody and pressEnter.

As you are the root user, you must choose the user and group
whom the web server will be running as. For security reason, you should choose
a non-system user who does not have login shell and home directory such as
'nobody'.

User [nobody]:

You will be asked for the group next. Press Enter.

Please choose the group that the web server running as.

User 'nobody' is the member of following group(s):  nobody
Group [nobody]:

Next you will be prompted for the port that the web server should answer on. Default HTTP traffic is port 80. Because the default port that the server selects is not correct, type in 80 and press Enter.

Please specify the port for normal HTTP service.
Port 80 is the standard HTTP port, only 'root' user is allowed to use 
port 80, if you have another web server running on port 80, you need to
specify another port or stop the other web server before starting LiteSpeed
Web Server.
You can access the normal web page at http://<YOUR_HOST>:<HTTP_PORT>/

HTTP port [8088]: 80

You will be prompted for the port that the administrative control panel should answer on. We will select the default port of 7080. Simply press Enter.

Please specify the HTTP port for the administration web interface,
which can be accessed through http://<YOUR_HOST>:<ADMIN_PORT>/

Admin HTTP port [7080]:

You will be asked if you would like to install PHP support. For our demonstration we will turn on PHP support. Type Yand press Enter.

You can setup a global script handler for PHP with the pre-built PHP engine
shipped with this package now. The PHP engine runs as Fast CGI which  
outperforms Apache's mod_php. 
You can always replace the pre-built PHP engine with your customized PHP 
engine.

Setup up PHP [Y/n]: Y

You will then be asked for the default PHP extension. Select the default PHP and press Enter.

Suffix for PHP script(comma separated list) [php]:

Next you will be prompted to install AWStats, a web-traffic logger. For our demonstration we have chosen to not install it. Press N and then press Enter.

AWStats is a popular log analyzer that generates advanced web server 
statistics. LiteSpeed web server seamlessly integrates AWStats into 
its Web Admin Interface. AWStats configuration and statistics update
have been taken care of by LiteSpeed web server.

Note: If AWStats has been installed already, you do not need to
      install again unless a new version of AWStats is available.

Would you like to install AWStats Add-on module [y/N]? N

A large amount of text will pass (please read!) and then you will be prompted if you would like LiteSpeed to start automatically. Press Y and then Enter.

Would you like to have LiteSpeed Web Server started automatically
when the server restarts [Y/n]? Y

If that step completes successfully you will be asked if you would like to start LiteSpeed now. Press Y and then pressEnter.

[OK] The startup script has been successfully installed!
Would you like to start it right now [Y/n]? Y

If the server starts successfully you will be given an output that looks similar to the one below:

[OK] lshttpd: pid=4517.

LiteSpeed Web Server started successfully! Have fun!

Testing LiteSpeed

Open up your web-browser and point it to your Cloud Server’s IP address (or domain name if you have DNS setup). You should see something like the following:

litespeed_test.png

If you receive a similar screen then you have successfully installed LiteSpeed! Be sure to check out your administrative control panel at http://12.34.56.78:7080/. Change 12.34.56.78 to your IP address. If you changed the administrative port number you will have to change that as well.

Top 20 OpenSSH Server Best Security Practices

OpenSSH is the implementation of the SSH protocol. OpenSSH is recommended for remote login, making backups, remote file transfer via scp or sftp, and much more. SSH is perfect to keep confidentiality and integrity for data exchanged between two networks and systems. However, the main advantage is server authentication, through the use of public key cryptography. From time to time there are rumors about OpenSSH zero day exploit. Here are a few things you need to tweak in order to improve OpenSSH server security.

Default Config Files and SSH Port

  • /etc/ssh/sshd_config – OpenSSH server configuration file.
  • /etc/ssh/ssh_config – OpenSSH client configuration file.
  • ~/.ssh/ – Users ssh configuration directory.
  • ~/.ssh/authorized_keys or ~/.ssh/authorized_keys – Lists the public keys (RSA or DSA) that can be used to log into the user’s account
  • /etc/nologin – If this file exists, sshd refuses to let anyone except root log in.
  • /etc/hosts.allow and /etc/hosts.deny : Access controls lists that should be enforced by tcp-wrappers are defined here.
  • SSH default port : TCP 22

SSH Session in Action

SSH Session in Action

#1: Disable OpenSSH Server

Workstations and laptop can work without OpenSSH server. If you need not to provide the remote login and file transfer capabilities of SSH, disable and remove the SSHD server. CentOS / RHEL / Fedora Linux user can disable and remove openssh-server with yum command:
# chkconfig sshd off
# yum erase openssh-server

Debian / Ubuntu Linux user can disable and remove the same with apt-get command:
# apt-get remove openssh-server
You may need to update your iptables script to remove ssh exception rule. Under CentOS / RHEL / Fedora edit the files /etc/sysconfig/iptables and /etc/sysconfig/ip6tables. Once donerestart iptables service:
# service iptables restart
# service ip6tables restart

#2: Only Use SSH Protocol 2

SSH protocol version 1 (SSH-1) has man-in-the-middle attacks problems and security vulnerabilities. SSH-1 is obsolete and should be avoided at all cost. Open sshd_config file and make sure the following line exists:

Protocol 2

#3: Limit Users’ SSH Access

By default all systems user can login via SSH using their password or public key. Sometime you create UNIX / Linux user account for ftp or email purpose. However, those user can login to system using ssh. They will have full access to system tools including compilers and scripting languages such as Perl, Python which can open network ports and do many other fancy things. One of my client has really outdated php script and an attacker was able to create a new account on the system via a php script. However, attacker failed to get into box via ssh because it wasn’t in AllowUsers.

Only allow root, vivek and jerry user to use the system via SSH, add the following to sshd_config:

AllowUsers root vivek jerry

Alternatively, you can allow all users to login via SSH but deny only a few users, with the following line:

DenyUsers saroj anjali foo

You can also configure Linux PAM allows or deny login via the sshd server. You can allow list of group name to access or deny access to the ssh.

#4: Configure Idle Log Out Timeout Interval

User can login to server via ssh and you can set an idel timeout interval to avoid unattended ssh session. Open sshd_config and make sure following values are configured:

ClientAliveInterval 300
ClientAliveCountMax 0

You are setting an idle timeout interval in seconds (300 secs = 5 minutes). After this interval has passed, the idle user will be automatically kicked out (read as logged out). See how to automatically log BASH / TCSH / SSH users out after a period of inactivity for more details.

#5: Disable .rhosts Files

Don’t read the user’s ~/.rhosts and ~/.shosts files. Update sshd_config with the following settings:

IgnoreRhosts yes

SSH can emulate the behavior of the obsolete rsh command, just disable insecure access via RSH.

#6: Disable Host-Based Authentication

To disable host-based authentication, update sshd_config with the following option:

HostbasedAuthentication no

#7: Disable root Login via SSH

There is no need to login as root via ssh over a network. Normal users can use su or sudo (recommended) to gain root level access. This also make sure you get full auditing information about who ran privileged commands on the system via sudo. To disable root login via SSH, update sshd_config with the following line:

PermitRootLogin no

However, bob made excellent point:

Saying “don’t login as root” is h******t. It stems from the days when people sniffed the first packets of sessions so logging in as yourself and su-ing decreased the chance an attacker would see the root pw, and decreast the chance you got spoofed as to your telnet host target, You’d get your password spoofed but not root’s pw. Gimme a break. this is 2005 – We have ssh, used properly it’s secure. used improperly none of this 1989 will make a damn bit of difference. -Bob

#8: Enable a Warning Banner

Set a warning banner by updating sshd_config with the following line:

Banner /etc/issue

Sample /etc/issue file:

----------------------------------------------------------------------------------------------
You are accessing a XYZ Government (XYZG) Information System (IS) that is provided for authorized use only.
By using this IS (which includes any device attached to this IS), you consent to the following conditions:
+ The XYZG routinely intercepts and monitors communications on this IS for purposes including, but not limited to,
penetration testing, COMSEC monitoring, network operations and defense, personnel misconduct (PM),
law enforcement (LE), and counterintelligence (CI) investigations.
+ At any time, the XYZG may inspect and seize data stored on this IS.
+ Communications using, or data stored on, this IS are not private, are subject to routine monitoring,
interception, and search, and may be disclosed or used for any XYZG authorized purpose.
+ This IS includes security measures (e.g., authentication and access controls) to protect XYZG interests--not
for your personal benefit or privacy.
+ Notwithstanding the above, using this IS does not constitute consent to PM, LE or CI investigative searching
or monitoring of the content of privileged communications, or work product, related to personal representation
or services by attorneys, psychotherapists, or clergy, and their assistants. Such communications and work
product are private and confidential. See User Agreement for details.
----------------------------------------------------------------------------------------------

Above is standard sample, consult your legal team for exact user agreement and legal notice details.

#8: Firewall SSH Port # 22

You need to firewall ssh port # 22 by updating iptables or pf firewall configurations. Usually, OpenSSH server must only accept connections from your LAN or other remote WAN sites only.

Netfilter (Iptables) Configuration

Update /etc/sysconfig/iptables (Redhat and friends specific file) to accept connection only from 192.168.1.0/24 and 202.54.1.5/29, enter:

-A RH-Firewall-1-INPUT -s 192.168.1.0/24 -m state --state NEW -p tcp --dport 22 -j ACCEPT
-A RH-Firewall-1-INPUT -s 202.54.1.5/29 -m state --state NEW -p tcp --dport 22 -j ACCEPT

If you’ve dual stacked sshd with IPv6, edit /etc/sysconfig/ip6tables (Redhat and friends specific file), enter:

 -A RH-Firewall-1-INPUT -s ipv6network::/ipv6mask -m tcp -p tcp --dport 22 -j ACCEPT

Replace ipv6network::/ipv6mask with actual IPv6 ranges.

*BSD PF Firewall Configuration

If you are using PF firewall update /etc/pf.conf as follows:

pass in on $ext_if inet proto tcp from {192.168.1.0/24, 202.54.1.5/29} to $ssh_server_ip port ssh flags S/SA synproxy state

#9: Change SSH Port and Limit IP Binding

By default SSH listen to all available interfaces and IP address on the system. Limit ssh port binding and change ssh port (by default brute forcing scripts only try to connects to port # 22). To bind to 192.168.1.5 and 202.54.1.5 IPs and to port 300, add or correct the following line:

Port 300
ListenAddress 192.168.1.5
ListenAddress 202.54.1.5

A better approach to use proactive approaches scripts such as fail2ban or denyhosts (see below).

#10: Use Strong SSH Passwords and Passphrase

It cannot be stressed enough how important it is to use strong user passwords and passphrase for your keys. Brute force attack works because you use dictionary based passwords. You can force users to avoid passwords against a dictionary attack and use john the ripper tool to find out existing weak passwords. Here is a sample random password generator (put in your ~/.bashrc):

genpasswd() {
	local l=$1
       	[ "$l" == "" ] && l=20
      	tr -dc A-Za-z0-9_ < /dev/urandom | head -c ${l} | xargs
}

Run it:
genpasswd 16
Output:

uw8CnDVMwC6vOKgW

#11: Use Public Key Based Authentication

Use public/private key pair with password protection for the private key. See how to use RSAand DSA key based authentication. Never ever use passphrase free key (passphrase key less) login.

#12: Use Keychain Based Authentication

keychain is a special bash script designed to make key-based authentication incredibly convenient and flexible. It offers various security benefits over passphrase-free keys. See how to setup and use keychain software.

#13: Chroot SSHD (Lock Down Users To Their Home Directories)

By default users are allowed to browse the server directories such as /etc/, /bin and so on. You can protect ssh, using os based chroot or use special tools such as rssh. With the release of OpenSSH 4.8p1 or 4.9p1, you no longer have to rely on third-party hacks such as rssh or complicated chroot(1) setups to lock users to their home directories. See this blog post about new ChrootDirectory directive to lock down users to their home directories.

#14: Use TCP Wrappers

TCP Wrapper is a host-based Networking ACL system, used to filter network access to Internet. OpenSSH does supports TCP wrappers. Just update your /etc/hosts.allow file as follows to allow SSH only from 192.168.1.2 172.16.23.12 :

sshd : 192.168.1.2 172.16.23.12

See this FAQ about setting and using TCP wrappers under Linux / Mac OS X and UNIX like operating systems.

#15: Disable Empty Passwords

You need to explicitly disallow remote login from accounts with empty passwords, update sshd_config with the following line:

PermitEmptyPasswords no

#16: Thwart SSH Crackers (Brute Force Attack)

Brute force is a method of defeating a cryptographic scheme by trying a large number of possibilities using a single or distributed computer network. To prevents brute force attacks against SSH, use the following softwares:

  • DenyHosts is a Python based security tool for SSH servers. It is intended to prevent brute force attacks on SSH servers by monitoring invalid login attempts in the authentication log and blocking the originating IP addresses.
  • Explains how to setup DenyHosts under RHEL / Fedora and CentOS Linux.
  • Fail2ban is a similar program that prevents brute force attacks against SSH.
  • security/sshguard-pf protect hosts from brute force attacks against ssh and other services using pf.
  • security/sshguard-ipfw protect hosts from brute force attacks against ssh and other services using ipfw.
  • security/sshguard-ipfilter protect hosts from brute force attacks against ssh and other services using ipfilter.
  • security/sshblock block abusive SSH login attempts.
  • security/sshit checks for SSH/FTP bruteforce and blocks given IPs.
  • BlockHosts Automatic blocking of abusive IP hosts.
  • Blacklist Get rid of those bruteforce attempts.
  • Brute Force Detection A modular shell script for parsing application logs and checking for authentication failures. It does this using a rules system where application specific options are stored including regular expressions for each unique auth format.
  • IPQ BDB filter May be considered as a fail2ban lite.

#17: Rate-limit Incoming Port # 22 Connections

Both netfilter and pf provides rate-limit option to perform simple throttling on incoming connections on port # 22.

Iptables Example

The following example will drop incoming connections which make more than 5 connection attempts upon port 22 within 60 seconds:

#!/bin/bash
inet_if=eth1
ssh_port=22
$IPT -I INPUT -p tcp --dport ${ssh_port} -i ${inet_if} -m state --state NEW -m recent  --set
$IPT -I INPUT -p tcp --dport ${ssh_port} -i ${inet_if} -m state --state NEW -m recent  --update --seconds 60 --hitcount 5 -j DROP

Call above script from your iptables scripts. Another config option:

$IPT -A INPUT  -i ${inet_if} -p tcp --dport ${ssh_port} -m state --state NEW -m limit --limit 3/min --limit-burst 3 -j ACCEPT
$IPT -A INPUT  -i ${inet_if} -p tcp --dport ${ssh_port} -m state --state ESTABLISHED -j ACCEPT
$IPT -A OUTPUT -o ${inet_if} -p tcp --sport ${ssh_port} -m state --state ESTABLISHED -j ACCEPT
# another one line example
# $IPT -A INPUT -i ${inet_if} -m state --state NEW,ESTABLISHED,RELATED -p tcp --dport 22 -m limit --limit 5/minute --limit-burst 5-j ACCEPT

See iptables man page for more details.

*BSD PF Example

The following will limits the maximum number of connections per source to 20 and rate limit the number of connections to 15 in a 5 second span. If anyone breaks our rules add them to our abusive_ips table and block them for making any further connections. Finally, flush keyword kills all states created by the matching rule which originate from the host which exceeds these limits.

sshd_server_ip="202.54.1.5"
table <abusive_ips> persist
block in quick from <abusive_ips>
pass in on $ext_if proto tcp to $sshd_server_ip port ssh flags S/SA keep state (max-src-conn 20, max-src-conn-rate 15/5, overload <abusive_ips> flush)

#18: Use Port Knocking

Port knocking is a method of externally opening ports on a firewall by generating a connection attempt on a set of prespecified closed ports. Once a correct sequence of connection attempts is received, the firewall rules are dynamically modified to allow the host which sent the connection attempts to connect over specific port(s). A sample port Knocking example for ssh using iptables:

$IPT -N stage1
$IPT -A stage1 -m recent --remove --name knock
$IPT -A stage1 -p tcp --dport 3456 -m recent --set --name knock2

$IPT -N stage2
$IPT -A stage2 -m recent --remove --name knock2
$IPT -A stage2 -p tcp --dport 2345 -m recent --set --name heaven

$IPT -N door
$IPT -A door -m recent --rcheck --seconds 5 --name knock2 -j stage2
$IPT -A door -m recent --rcheck --seconds 5 --name knock -j stage1
$IPT -A door -p tcp --dport 1234 -m recent --set --name knock

$IPT -A INPUT -m --state ESTABLISHED,RELATED -j ACCEPT
$IPT -A INPUT -p tcp --dport 22 -m recent --rcheck --seconds 5 --name heaven -j ACCEPT
$IPT -A INPUT -p tcp --syn -j doo
  • fwknop is an implementation that combines port knocking and passive OS fingerprinting.
  • Multiple-port knocking Netfilter/IPtables only implementation.

#19: Use Log Analyzer

Read your logs using logwatch or logcheck. These tools make your log reading life easier. It will go through your logs for a given period of time and make a report in the areas that you wish with the detail that you wish. Make sure LogLevel is set to INFO or DEBUG in sshd_config:

LogLevel INFO

#20: Patch OpenSSH and Operating Systems

It is recommended that you use tools such as yumapt-getfreebsd-update and others to keep systems up to date with the latest security patches.

Other Options

To hide openssh version, you need to update source code and compile openssh again. Make sure following options are enabled in sshd_config:

#  Turn on privilege separation
UsePrivilegeSeparation yes
# Prevent the use of insecure home directory and key file permissions
StrictModes yes
# Turn on  reverse name checking
VerifyReverseMapping yes
# Do you need port forwarding?
AllowTcpForwarding no
X11Forwarding no
#  Specifies whether password authentication is allowed.  The default is yes.
PasswordAuthentication no

Verify your sshd_config file before restarting / reloading changes:
# /usr/sbin/sshd -t

Tighter SSH security with two-factor or three-factor (or more) authentication.

How To Set MaxClients in Apache/prefork

The Basics

  1. Determine how much RAM is available to Apache
  2. Determine how much RAM each Apache process uses
  3. Set MaxClients to: (RAM available to Apache) / (RAM per Apache process)
  4. Watch and adjust

How To Change the Value

The MaxClients value is located in the main Apache configuration file, typically /etc/apache2/apache2.conf or /etc/httpd/httpd.conf.

The relevant section looks like this:

# prefork MPM
# StartServers: number of server processes to start
# MinSpareServers: minimum number of server processes which are kept spare
# MaxSpareServers: maximum number of server processes which are kept spare
# MaxClients: maximum number of server processes allowed to start
# MaxRequestsPerChild: maximum number of requests a server process serves
<IfModule mpm_prefork_module>
    StartServers        5
    MinSpareServers     5
    MaxSpareServers     10
    MaxClients          150
    MaxRequestsPerChild 0
</IfModule>

After changing the value, save the file and restart Apache with apache2ctl graceful-stop && apache2ctl start.

Why It’s Important

One of the most common causes of a public Web server crash is that it’s consumed all available physical memory (RAM). When this happens, the operating system attempts to free RAM by moving some of it to disk. In Linux this is called the “swap” space; in Windows “page file.” This process defeats the purpose of RAM (which is to provide very fast access to data) and it requires resources (CPU power, disk I/O) to perform the “swapping” of data between the disk and RAM. The swap space can also become completely consumed, but the real problem is that in a very fast-paced environment like a Web server, the system can quickly become overwhelmed and effectively crash just by trying to manage its own memory. This is called thrashing.

The name of the game is to avoid thrashing and the only way to do so is to keep Apache from using more physical RAM than is available to it.

How To Determine How Much RAM is Available to Apache

The amount of RAM available to Apache is the total amount of RAM on the system minus the amount that will be used by all other processes. Unfortunately, it’s difficult to determine how much RAM is being used on the system.* It’s even harder to determine how muchmaximum possible RAM will be used by other processes when the server is under heavy load.

And it gets yet more difficult if you’re running MySQL on the same machine. Typically, each request handled by Apache will open at least one connection to MySQL, so as the Apache requests increase, so will MySQL’s RAM usage. In short, MySQL’s memory usage depends on how it’s being used by your application. You can use the mysqltuner script to get an absolute maximum value that MySQL will use, but once you have Apache tuned properly it will rarely hit this amount. Your best bet is to move MySQL to a separate server. Other than that, use the max value initially, watch the server under load, and make adjustments.

For all the other processes, use a value of 256 MiB. If you’re using a lot of other services, such as on a virtual hosting machine, you may want to use a higher value.

So, initially, for a typical system, reserve 256 MiB + max MySQL RAM usage.

How to Determine How Much RAM a Single Apache Process Uses

Again, due to shared memory (especially if you’re using APC), buffering, caching, etc. it’s very difficult to determine this. In our experience, the best tool for determining memory usage per process is ps_mem.py.

ps_mem.py will produce output like this:

$ sudo ./ps_mem.py
 Private  +   Shared  =  RAM used       Program 

  4.0 KiB +  22.5 KiB =  26.5 KiB       upstart-udev-bridge
  4.0 KiB +  61.0 KiB =  65.0 KiB       login
  0.0 KiB +  70.0 KiB =  70.0 KiB       udevd (3)
[...]
123.0 MiB + 221.0 KiB = 123.2 MiB       mysqld
  3.3 GiB +  27.9 MiB =   3.4 GiB       apache2 (123)
---------------------------------
                          3.6 GiB

Here you can see that there are 123 apache2 processes, consuming a total of 3.4 GiB, so each Apache process is using roughly 28 MiB of RAM. This system has about 8 GiB of RAM, and although mysqltuner.pl reports that MySQL’s maximum memory use is about 1 GiB, we can see that it’s using much less. To be safe though, I’ll reserve 1.5 GiB for all other processes and round up Apache’s average usage to 32 MiB:

(6.5 x 1024) / 32 = 208

To be extra safe, round this down to 200. It’s better to under-estimate.

Watch The Server

Keep an eye on the number of Apache processes, and the total RAM used. Here’s a command that wraps this into a single output that updates every second:

watch -n 1 "echo -n 'Apache Processes: ' && ps -C apache2 --no-headers | wc -l && free -m"

It produces output like this:

Every 1.0s: echo -n 'Apache Processes: ' && ps -C apache2 --no-headers | wc -l && free -m

Apache Processes: 27
            total       used        free        shared      buffers     cached
Mem:        8204        7445        758         0           385         4657
-/+ buffers/cache:      2402        5801
Swap:       16383       189         16194

Use the -/+ buffers/cache row. The values are in MiB. You can see that at this point in time, about 2.3 GiB is in use and 5.7 GiB is free. If your Apache processes get close to the MaxClients setting and there’s plenty of extra RAM available, adjust the value up in small steps. If for some reason the value in the free column gets small (say less than 500), reduce Apache’s MaxClients value and restart.

Various Usage of Shell Script

These have only been tested on linux, but most should work on any UNIX.
For development history see github.com/pixelb/scripts.

Name Description Type
l An enhanced and more portable `ls -l` shell
ps_mem.py list processes by memory usage python
crudini handle ini files from shell python
findrepo find a string in a repository working copy shell
newest list newest (modified) files in tree shell
timeout run a command with a timeout shell
truncate truncate a file to the specified size shell
split_to run a command with chunks of input shell
sw simple command line stopwatch script shell
gcccpuopt get optimum gcc options for your CPU shell
whatrequires list reverse dependencies on rpm or deb distros python
fix edit a file while not changing the modification time shell
idiff colour diff output for easy review shell
ansi2html.sh convert ANSI colours to HTML shell
errno show all errnos or meaning of specified errno shell
inpy enhanced python interactive introspection python
dutop better du python
add sum a column of numbers. See also funcpy shell
find_mm_yyyy list files last modified in mm yyyy shell
disk_grep find a string in a disk device shell
svndiff review svn repo changes with graphical diff viewer shell
xchat_warn popup message when referenced directly in xchat python
ximkeys show keys available in X to generate a character python
lomount.sh loopback mount partitions within a disk image file shell
tcpdump_not_me Allow running tcpdump on remote system over ssh shell
ccodes list country codes in locale database shell
ansi_colours.sh display all ANSI colours shell
funcpy functional command line toolkit python
bc invoke bc with some useful functions predefined bc
getpw Shows how to securely prompt for a password shell
command_prompt_here Open a terminal in current/selected directory in nautilus python
xfullmag Start xmag full screen with specified magnification shell
sysinfo show a quick summary of system hardware shell
human.py convert numbers for human consumption python
dosfiles find text files with any CRLF line terminators shell
day display the date for {this,next,last} {mon,tue,…} shell
datedir set date of directory based on contents shell
lsdate print date (of file) as ls would shell
lsmode print file permissions in octal (like chmod accepts) shell
bashfeed generate feed for newest files in website (example) shell
threadsafe search C/C++ files for functions that are not threadsafe shell
moz open specified file in mozilla shell
sematree lock files with semaphore interface shell
download Download list of URLs from file with history shell
quote reformat misquoted mail shell
gen_timeline generate page for newest files in website (example) shell
lowercase efficiently change any uppercase names in tree shell
rmws remove redundant whitespace from files shell
conv simple wrapper around units shell
self_modify.sh e.g. script to show self modifying shell
7seg e.g. script to show self referencing shell

Apache tuning

Overview

First check the MPM (Multi-Processing Module) used, most distributions come with a default prefork configuration where a minimum number of Apache processes is started (preforked) and more are added depending on incoming traffic. In general Apache optimization is about monitoring the number of Apache processes(also called workers) in combination with the total memory used by the system. Real time monitoring how many workers are in use and what they are doing is easy after installing and configuring the Apache mod_status module.

We advise you enable server-status monitoring, calculate the number of memory needed by your typical workload and optimize and constantly monitor processes/workers used by your Apache installation.

Step 1: Enable server-status module and check the server-status

Example status-page at Apache.org http://www.apache.org/server-status

If you see very few inactive workers (represented by “.” characters) you want to increase the appropriate configuration directive depending on Apache version and the MPM used.

Background:

Step 2: Check what mode Apache is running in

Apache can use a number of ways to handle requests. Apache calls this MPM (Multi-Processing Module) and there are several to choose from. By far the most widely used (at least on *nix platforms) are the three main ones: prefork, worker, and event. Essentially, they represent the evolution of the Apache web server, and the different ways that the server has been built to handle HTTP requests within the computing constraints of the time over its long (in software terms) history.

Prefork or mpm_prefork is compatible with everything including (simple) PHP and SSL and is easier to debug but not so kind on RAM usage if you have a lot of concurrent requests. For now we prefer to use mpm_prefork and enough free memory and keep a number of free workers to handle all http clients (both webacess/webapp and Z-push devices) at the same time.

sudo apache2 -V | grep -A 2 'MPM:'

or CentOS/RedHat and openSUSE/SLES will use httpd or httpd2

sudo httpd -V | grep -A 2 'MPM:'

Output should be like this:

Server MPM:     Prefork
  threaded:     no
    forked:     yes (variable process count)

Background:

Some installations can report the ITK MPM like:

sudo apache2 -V | grep -A 2 'MPM:'
Server MPM:     ITK
  threaded:     no
    forked:     yes (variable process count)

mpm_itk (see http://mpm-itk.sesse.net/) mpm-itk is based on the traditional prefork MPM, which means it’s non-threaded. This means you can run non-thread-aware code (like many PHP extensions) without problems. You lose out to any performance benefit you’d get with threads. You will also take an additional performance hit over prefork, since there’s an extra fork per request.

mpm_worker uses threading which is a big help for concurrency. A worker spins off some child processes, which in turn spin off child threads; some spare threads are kept ready if possible, to service incoming connections. This approach is kinder on RAM, since the thread count doesn’t have a direct bearing on memory use like the server count does in prefork. It also handles concurrency much more easily, since the connections just need to wait for a free thread (which is usually available) instead of a spare server in prefork. If you have the RAM available and start enough spare workers processes then the advantages of mpm_worker are limited.

Step 3: Measure and calculate maximum memory usage

Investigate the current memory usage of Apache using one liner scripts. Take these measurements for systems that have been running for a few days to get an accurate measurement for the distribution of web and z-push users. After measuring the individual average, calculate to total memory needed based on the number of devices and concurrent web sessions.

ps -ylC apache2 | awk '{x += $8;y += 1} END {print "Apache Memory Usage (MB): "x/1024; print "Average Proccess Size (MB): "x/((y-1)*1024)}'

or CentOS/RedHat and openSUSE/SLES will use httpd or httpd2

ps -ylC httpd | awk '{x += $8;y += 1} END {print "Apache Memory Usage (MB): "x/1024; print "Average Proccess Size (MB): "x/((y-1)*1024)}'

Alternative for a more detailed look on all memory consumed by any process if you intend to run zarafa-server and MySQL on the same host use this script:https://github.com/pixelb/scripts/blob/master/scripts/ps_mem.py

Step 4: Increase memory of the system and change settings

In this example we limit the total number of concurrent connection to 512 by adding the ServerLimit directive in the IfModle mpm_prefork setting. The ServerLimit in Apache version 2.2 is set by default on 256 if you need more then 256 concurrent workers you need to increase both MaxClients and ServerLimit but keep them equal if you go over 256 seehttp://httpd.apache.org/docs/2.2/mod/mpm_common.html#maxclients and http://httpd.apache.org/docs/2.2/mod/mpm_common.html#serverlimit.

NOTE 1 Setting the MinSpareServers and the MaxSpareServers parameter to a large number is almost always a bad idea because the added overhead of shutting down or starting spare(free) workers for background see http://httpd.apache.org/docs/2.2/mod/prefork.html#minspareservers and http://httpd.apache.org/docs/2.2/mod/prefork.html#maxspareservers

In the /etc/apache2/apache2.conf or /etc/httpd/httpd.conf update.

<IfModule mpm_prefork_module>
    StartServers          50
    MinSpareServers       25
    MaxSpareServers       50
    ServerLimit           512
    MaxClients            512
    MaxRequestsPerChild   10000
</IfModule>

NOTE 2 Do a full restart of your Apache server NOT a reload because not all changes can be done without completely shutting down the Apache mother process.

Step 5: Enable monitoring of the server-status page and plot some nice graphs

Because workloads change you need to continuously monitor the number of idle Apache processes and the memory/swap usage of your setup. Please use this Nagios plugin and complement your monitoring by using a project like Munin.

Monitoring and graphing for Nagios

Background:

Additional graphing using Munin

sudo apt-get  --no-install-recommends install munin-common munin-node munin-plugins-extra libcache-cache-perl

To be able to use the Munin MySQL measurement plugins, install the package libcache-cache-perl.

Check to see which plugins need additional settings like apache-status do a:

sudo /usr/sbin/munin-node-configure --suggest

Enable the new plugins and reload the munin node:

sudo /usr/sbin/munin-node-configure --shell | sudo sh -x
sudo /etc/init.d/munin-node reload

Background:

TIP: On the munin-node

hostname -f
munin-node1.zarafa.com

must result result in the same full name as used in the vonfig of the munin server.

/etc/munun/munin.conf

example

[munin-node1.zarafa.com]
        address 192.168.1.1

Optional further testing and/or benchmarking

apt-get install httperf
httperf --hog --server=www --num-conns=100 --rate=10 --timeout=5