Header Image - Learn Linux Online

Hi, developer!

by Nathaniel Morales

On this website, whether you are a junior developer, student or senior developer with great experience, everyone will find useful information about Linux.

  • For beginners, there is information that will give a good understanding of how it works and how to use it, which will give you a big boost and some baggage of knowledge for the possible start of your career as a developer.
  • For students, this resource is a great source of information for obtaining a large amount of knowledge in an accessible and understandable format or writing scientific papers and essays, although the last one takes a lot of time and it is better to use an essay writing service such as PeachyEssay – college essay writing service. This may significantly improve the quality and informativeness of your text and, as a nice bonus, it will save a lot of time for getting new knowledge on this site
  • For professionals, it will always be useful to check and refresh their knowledge and maybe get some new information that may be useful in work or can help you to optimize your workflow

Add Linux wireless drivers to Debian Install ISOs

TL;DR:

This HowTo shows you the process to find, download, and add linux wireless drivers, network drivers, or other drivers (missing firmware) to a Debian installation ISO, and then remaster it for install from a USB stick.

Background

I recently bought a Lenovo X1 Carbon ultrabook. It’s a slim, lightweight machine with no CD / DVD drive, and a couple of USB ports.  That means I needed to wipe the Windows install that was on there, and install Debian 8. Of course, getting the Debian ISOs was no problem. And creating a bootable USB is easy because they are hybridized. But, I was quickly greeted with a screen that complained it could not find my wifi – i.e., it couldn’t detect my wireless card – because it didn’t have the drivers.


Install or load firmware for missing network drivers on Debian

The problem stems from the way Debian is packaged. They do not include “non-free” drivers in the distribution ISOs. It’s a philosophical choice, and it’s not going to change, so there’s no point in tilting at that windmill.

Normally, you can select the option that says: “Install driers from a USB stick”, but, on an ultrabook, there’s a twist: since you don’t have a CDROM / DVDROM drive, adding those drivers from a secondary USB stick doesn’t work! The anaconda installer does not notice a secondary usb stick has been inserted, and flat does not scan it! So, even though you’ve put the needed drivers on that USB stick, it’s still useless.

The answer is to slipstream (Windows lingo) the drivers into the installation media. Debian offers some relatively simple ways to remaster an installation CD, which will cover in depth here in a moment. First, let’s talk about how to get the drivers in the first place.

Getting Debian Linux Wireless Drivers

In my case, the intel network card caused debian to complain that it was missing iwlwifi-6000g2a-6.ucode. A quick DuckDuckGo search reveals that this is part of the firmware-iwlwifi package in Debian. Specifically, 0.43 or higher.

So, if we scroll to the bottom of that page, we can download the .deb file.

Downloading the Debian Installable ISO

Next, we need to get an ISO that we’ll use to create the installable media from which we’ll install Debian. My preferred choice is to use BitTorrent to download the ISO files. It lessens the load on the servers hosting the ISOs, and helps the community give back to the rest of us by using a peer-to-peer connection. For this tutorial, we’re going to be dealing with the CD image, not the DVD image. So, download the CD image from here.

Setting up our working folder

We need a place to work. So, create the folder /home/youruser/custom, and copy the downloaded ISO there.

Prepare the helper scripts

Now, Debian’s Modify CD page has some technical information about this process, but this article is an attempt to make the instructions a little more palatable. That being said, we are going to be using the helper scripts at the bottom, so, copy / paste those scripts, and put them in the/home/youruser/custom directory as well.

Now, open each one, and do a find / replace to find sudo and replace with nothing (we’re removing the sudo commands because it’s easier to operate as root for the duration of this tutorial).

Become root

Because we need access to “root only” system utilities, su to root for the rest of this tutorial. (Alternatively, you could constantly use sudo for everything, but being root for the duration of this operation is probably easier).

The starting point

Right now, your custom directory should look like this:

Now, run this command:

./diunpk debian-8.2.0-amd64-CD-1.iso src dest

This will unpack the ISO file into a timestamped directory. Rename this directory to “CD” for ease of use, then copy ./dipk into that directory:

The script has created a src and dest directory under the CD directory. “Src” is a readonly version of the ISO image you downloaded, and dest is where we are going to be making changes to add our firmware.

Add the non-free firmware to the image.

Create the a directory named firmware-nonfree using this command:

mkdir ./pool/main/f/firmware-nonfree/

Now, copy our downloaded deb file to this directory:

cp /home/youruser/Downloads/firmware-iwlwifi_0.43_all.deb ./pool/main/f/firmware-nonfree/

Lastly, we need to make a symbolic link from the firmware directory just below the root of the ISO to this file so that that installer can find it during the natural installation process, so change directory to the firmware directory (/home/youruser/custom/dest/firmware), and then create the symbolic link:

cd firmware
ln -s ../pool/main/f/firmware-nonfree/firmware-iwlwifi_0.43_all.deb

Here’s what my whole process looked like:

  1. I used the find command to find existing firmware just to confirm it’s location. As I expected, it was in ./pool/main/f/firmware-free.
  2. I created ./pool/main/f/firmware-nonfree to store the non-free drivers.
  3. I used tab completion (in the red box) to create the correct command to copy the firmware from my Downloads directory to the nonfree directory I just created.
  4. I changed directory to the firmware directory that is just below dest/.
  5. I did a directory listing to ensure I was seeing what I was expecting to see, and to look at the symlink for the existing firmware.
  6. I created a symbolic link to the new, non-free firmware I just added.
  7. I did a directory listing to make sure it was done correctly.

Run dipk to generate the ISO.

Chagnge directories back to CD (the parent directory of dest), and run the following command:

./dipk ../custom.iso src dest

This will create a file called custom.iso in the custom directory  (the parent directory of the CD directory) using the src and dest files that we have been working with.

Preparing the ISO for use with a USB Drive.

In order to use a USB stick to boot this ISO file and install Linux, we need to use the utility isohybrid, so run the follwoing command from the custom directory:

isohybrid custom.iso

This changes the ISO file to allow modern BIOSes to recognize its structure as a bootable disk.

Prepare your USB Stick

Lastly, put your USB stick in the computer, and run dmesg to figure out which device it is. (It’s usually the last /dev/sdX to show up in dmesg). In my case, it was /dev/sdm. Now, just copy the ISO file to that device:

cp custom.iso /dev/sdX

(Note: be sure to replace the “X” with the proper letter that represents your USB drive!).

Enjoy.

OpenSSL Generate: Certificates, CSRs, and Self-Signed CA Certificates

If you’re a computer novice, start reading this at the top. If you know what you’re doing (even somewhat).

Introduction (Crash Course for Beginners)

SSL Certificates are all about trust. It’s how a website tells a user: “Hey, I’m trust worthy because other people trust me, so you should too.” The primary tool of that trust? OpenSSL. Depending on your needs, you might use OpenSSL to:

  1. Generate a self signed certificate
  2. Generage a CSR (Certificate Signing Request)
  3. Become your own CA (Certificate Authority), and if you do that:
  4. Sign CSRs that have been created by others.

This tutorial is going to teach you how to do each one of these tasks, but first, let’s discuss the public key infrastructure (PKI) and the  various degrees of trust it has. The PKI was established as a way to tell a complete stranger that another complete stranger is a good guy or a bad guy. When a webmaster decides they need a “secure” website (an httpswebsite), they seek out a certificate authority to give them a certificate. (“Give” is not really the appropriate word, “sell” might be better!). The webmaster is essentially paying the CA to vouch for them.

In order for the Certificate Authority (CA) to vouch for a webmaster and his site, they need (at least) a minimum amount of information about that webmaster and his site. This very basic information is:

  • Your Country
  • Your State / Province
  • Your Locality (City, Town, Village, etc…)
  • Your Organization Name
  • The Organization Unit Name (If you have one… could be “Sales” or “Corporate” or “IT DIvision”)
  • The common name (this is the full URL of your website)
  • Your email address.

This information, plus confirmation that you have control of that domain (via email or uploading a special file to the root of your website) is usually enough to get you a basic SSL certificate. These basic certificates can range anywhere between $5 – $80 depending on the vendor.

There are more expensive certificates out there, which go up in price from $200 – $10,000+. These are known as “EV” or “Extended Validation” certificates. In order to qualify for one of these certificates, you have to submit much more information about you and your company (as well as the basics above). EV Certificates give you that pretty green bar at the top of our browser. SSL certificate vendors sell this as a way to get the utmost trust from consumers, but the reality is that most consumers don’t know the difference. So, while it may underwhelm you to see that Bank of America is using an expensive, fancy EV certificate:

It may be very interesting to note that Amazon.com does not use an EV Certificate:

So, whether or not you choose to pony up the extra cash to get an EV certificate really boils down to answering two questions:

  1. Are my visitors / customers tech savvy enough to notice that I am using an EV certificate (and therefore trust me more)?
  2. Do I have the extra cash to buy an EV cert or will a standard certificate suffice?

In this tutorial, we are going to focus on standard SSL certificates and generating our own self-signed certs and self-signed Certificate Authority certificates because it’s completely free.

We’re not going worry about EV.

But, there is a VERY IMPORTANT distinction between purchasing an SSL cert from an established certificate authority versus generating your own (certificate or CA), and that’s called “the trusted root”.

Commercial Certificate Authorities like VeriSign, Digicert, GoDaddy, among others have their certificates in the “trusted root” of many operating systems. This means that when they sign your CSR and issue you a certificate where they vouch for you, the resulting certificate will be automatically accepted by FireFox, Chrome, Safari, and Internet Explorer without any further action. If you generate your own certificates, strangers that come to your site will get a certificate warning instead of a lock on their address bar.

There is a way around this problem if you become your own Certificate Authority and install certificates onall your computers (and the computers of all the users that will access your website or email system), but if you’re looking to allow strangers to be able to trust yoru site without installing software on their computers, servers, or mobile devices, you need to get a for-pay certificate from a commercial CA!

A Note on How Keys, Certificates and Cryptography Work

It’s important that you understand that modern cryptography works on the principle that two VERY large prime numbers form a key pair. These huge numbers are used because it is unfathomably difficult to factor the sum of two large prime numbers back into those two numbers. The public and private key are (essentailly) those two large numbers. If you have the private key, you can encrypt or decrypt information. If you have the public key you can only encrypt it. So, when you create a certificate, CSR, or get your key “signed” by a CA, you are essentially dealing with a pair of very large prime numbers and their sum. For more information, read Plain English: an RSA Encryption and Authentication Primer.

Common Certificate Generation Tasks in OpenSSL

Generating a CSR (Certificate Signing Request) with OpenSSL

The CSR (Certificate Signing Request) is a text file that we generate with OpenSSL, which contains both of the following items:

  1. The organization information (noted above as Country, Province, Locality, Organization, Unit, common name, and Email address)
  2. The public key that was generated as part of the key pair.

The information provided in #1 above must be cryptographically signed with the private key, or the CA will not be able to verify the information sent to them in the CSR. (When you sign information with a private key, you can forward that information along with a public key to a third party, and that third party can verify the signature using the public key – BUT they cannot duplicate the signature without the private key, which is always kepts secret, and never leaves your possesion).

Here’s the code to generate a private key (from which a public can will be derived) and a CSR that can be safely forwarded to a CA for signing:

openssl req -new -newkey rsa:2048 -nodes -keyout yourdomain.key -out yourdomain.csr

Here’s what’s going on with this command:

req tells OpenSSL that we are requesting something

-new makes it a new request for a…

-newkey, which takes the parameter: rsa:2048RSA is the type of key we are creating, and 2048 is the strength (length in bits, whcih is directly related to the complexity of th key).

-nodes is frequently confused as “nodes” when in reality it means “no DES” or, “do not produce a PKCS#12 compiant file. Most certificate authories cannot use this type of file; however, this is the type of file that IIS (Internet Information Services) on Windows Server systems uses to store their certificate files. Specifying -nodes means that the resulting file will not be encrypted, which is perfectly fine since you’re going to be transmitting a public key, which does not need protection as well as public information about yourself and your server.

-keyout tells OpenSSL to write the key to yourdomain.key. You can make this any output filename you want, but it makes the most sense to label it as yourdomain.key. Many people (myself included) use -yourdomain.tld.key, which is particularly useful if you own both the .net and .com of a particular domain name.

-out tells OpenSSL ‘write the certificate signing request to this file”. This is the file that you will transmit (in the clear) to the CA for signing.

Once you have completed this step, you’ll want to make sure that you keep your .key file under wraps. You don’t ever want to lose control of this file because then an evil actor (bad guys) could replicate your keys, decrypt your traffic, or even spoof your site’s online identity. The CSR, however, you can email to the CA for signing or use thier upload system to send it in to be signed.

Here’s what the execution of this command looks like:

You can see that the first thing it does is generate an RSA private key, then it writes that key to the specified output file, learnlinuxonline.com.key as I specified in the command on the first line. Next, you can see it asked me for information about myself, my site, and our location. Since learnlinuxonline.com is a free service of my company, High Powered Help, Inc., I entered in the company information. You should also notice that I left the challenge passwordblank. If you were to include a challenge password during the creation of these certificates, you would be required to enter the password into Apache every time you booted, restarted the web server or every time you restart or reload the Apache service. Ultimately, this inconvenience would cause you to end up stripping out the password anyway. So, don’t bother putting one in to begin with.

The resulting files that are generated are learnlinuxonline.com.key and learnlinuxonline.com.csr:

And, here’s what the contents of the CSR look like:

I’ll even show you what the private key looks like because this key was generated specifically for this tutorial, and discarded. So, try as you might, it isn’t going to help you spoof my site. But even still, I obfuscated a few lines anyway…

Generating a self-signed certificate with OpenSSL

self-signed certificate is nothing more than fulfilling a certificate signing request with a key you generated yourself. It’s a big like writing your own recommendation letter. Others will take it with a grain of salt. But, if you are dealing with a small number of users or just doing something in a testing environment, this can be an easy way to develop an application or website without having to go through the expense of buying a certificate from a certificate authority.

The self-signed certificate procedure is rather simple:

  1. Create a private key.
  2. Create a CSR for your server
  3. Sign the CSR with the private key

How to Create a Private Key for Self-Signed Certificates

As always, we’re going to execute an OpenSSL command:

<b>openssl genrsa -des3 -out server.key <b>2048</b>

Here’s how the command works:

  • genrsa tells OpenSSL to generate an RSA based private key.
  • des3 tells OpenSSL to use triple DES encryption
  • out tells OpenSSL where to write the resulting keys (into which file)
  • 2048 tells OpenSSL to use 2048-bit long modulus, which is the current standard (as of this writing).

The command execution looks like this:

You’ll notice that it asks for a pass phrase to protect the private key. You can make this a simple, easy to remember (and even weak) password for now because we are just going to remove it in later steps.

 Generate a CSR

Now that we have a key, you’ll need to generate a CSR as shown above (in great detail). Once you’ve done that, come back here, and proceed to the next step.

 Using OpenSSL to Self-Sign your CSR to create the Self-Signed Certificate

Now that we have a private key with which we can do the signing and a CSR that needs to be signed, we’re almost ready to generate the certificate. But, as noted above, we’re going to need to remove that pesky password that we added to the private key before we can get started. Why? Because if you retain the password, you’ll need to enter it every time you restart Apache or reboot the server. That can cause a lot of downtime. Without the password, Apache cannot read the certificate (since it was encrypted with 3DES using that password as the key).

To prevent this pandemonium, we’re just going to remove the password and rely on file system permissions and security to keep the key safe. So, first we’ll make a backup of the original key:?

cp server.key server.key.backup

Then we’ll remove the password with this command:?

openssl rsa -in server.key.backup -out server.key

This command tells OpenSSL to read -in the original key (that we’ve saved as a backup copy) and output a pure RSA key (with no encryption) to the -out file server.key.

Now that we have a private key without a password, we’re ready to sign the CSR with that key to create the self-signed certificate:

The OpenSSL command for this is:

openssl x509 -req -days 365 -in server.csr -signkey server.key -out server.crt

This command has the following paremeteres:

  • x509 is the certificate type, which has been standardized for the Public Key Infrastructure (PKI). This format is required for browsers to be able to interact with them properly.
  • -req tells open SSL we are dealing with a CSR (request).
  • -days tells OpenSSL we only want to vouch for this certificate for X number of days. In most cases, 365 days for a self-signed certificate is sufficient. (Later, when we start running our own CA, you’ll see that  value go up to 730 days – two years).
  • -in tells OpenSSL what file to use as the certificate signing request.
  • -signkey tells OpenSSL which private key to use for the signing
  • -out tells OpenSSL where to write the resulting certificate.

The whole process looks like this:

How to become your own CA (Certificate Authority)

Becoming your own CA (Certificate Authority) is nothing more than having a specific private key that you use to sign all CSRs. It is essentially the same process that we used above in the section about creating self signed certificates, but the difference is: instead of using a throw-away key, we re-use a single key AND we have to install that key on computers that we want to recognize and trust us.

But first, before we go through this effort, let’s discuss when and why we want to be our own CA:

  1. You are generating a number of self-signed certificates that a number of users will all need to know and trust. (And you’re looking to save some cash).
  2. You are running a VERY high security application where you want to maintain 100% control over the entire trust chain.
  3. You think it would be fun.

In any case, you’ll need to have the UTMOST security precautions surrounding your master CA private key. I highly recommend that you store it on an encrypted drive with very strict file permissions (600) so that ONLY your user can access and deal with the key. It should never be stored “in the cloud” and it should be backed up in multiple locations (also encrypted). If you choose to use a cloud backup service, the only one I would recommend would be Carbonite, but ONLY if you use their RSA encryption backup scheme.

Bascially, you have to treat this private key like the most improtant, confidential, and sensitive data possible because it is.

Setting Up your OpenSSL Config File and CA Directories

As a Certificate Authority, you need to setup basic information and configurations so that your entire system is consistent. We also need to have a nice place to store everything. So, let’s first create a directory to store everything in /root/ca. We have chosen the /root/ directory to hold this information because it should be the most guarded account on your system. So, first: become root using either the sudo or su command:

sudo su

Now, change directory to the root user’s home directory, and create a directory structure for the CA system:

cd ~/ mkdir ca cd ca mkdir private mkdir certs echo "101010" > serial echo -n > index.txt echo "101010" > crl_number

You should now be in /root/ca/, and have an additional sub-directories certs and private in that directory along with three files: serial, index.txt, and crl_number. The serial file holds the serial numbers that willbe assigned to certificates as they are issued. index.txt is the database that holds information about certificates. With this directory structure setup, we are ready to configure our openssl.cnf file for our CA.

Download our example-openssl.cnf, and customize it to your own settings. Be sure to fill in the blank areas, and rename the file to openssl.cnf from example-openssl.cnf.

Make sure you keep this in the same directory that your private key and other root-ca data will be kept!

Generating a CA Root Certificate with OpenSSL

First, we’ll need to generate our root-ca private key using this command:?

[openssl genrsa -des3 -out root-ca.key 4096/bash] Next, we'll need to create the root-ca certificate [bash]openssl req -new -x509 -days 3650 -key root-ca.key -out root-ca.crt -config openssl.cnf

Once these have been generated, we need to put them in the directories that are specified in the openssl.cnf file for the CA. If you used the defaults, that would be /root/ca/private.

Installing your CA certificate in the trusted certificates store

Installing the CA Certificate in Linux (Ubuntu / Debian)

  1.  Create a directory in /usr/loca/share/ca-certificates to hold your cert.
  2. Copy the cert to that directory
  3. Run update-ca-certificates.

For example, you can create the directory (as root):

mkdir /usr/local/share/ca-certificates/yourdomain.com

Now, copy the cert there, or download it from the web using wget.

cp ./mycert /usr/local/share/ca-certificates/yourdomain.com/

or

wget http://certs.yourdomain.com/yourcert.crt

Then, once that is there, run update-ca-certificates to automatically update the system to trust that new certificate.

When you do it correctly, you should see output similar to this:

Installing the CA Certificate in the Windows Trusted Root Store

In order for your CA Certificate to be put into the trusted root store of a Windows computer (or server), you must first convert it to p12 format. The command to perform this conversion is:

openssl pkcs12 -export -out server.p12 -inkey root-ca.key -in root-ca.crt

This will output a file, server.p12, which you can then transport to your windows server and install.

For a Windows Computers (Workstations or Servers, individually)

The certificate can be copied to the desktop, and then, just double click it:

Make sure you select the Local Machine, not Current User.

Next, browse to the location of the p12 file. Remember to keep this file secure because it contains the private key!

When prompted for the password for the private key, leave it blank since we did not set a password when we exported the key to p12 format.

When prompted, select Trusted Root Certification Authorities as the designated certificate store:

When prompted to select the certificate store, do not let it “automatically” select. This will place the certificate in the least invasive store possible. Usually a personal store. We want this to be trusted absolutely, so we’re going to purposefully place it in the Trusted Root Certification Authorities store:

Lastly, confirm the import:

To confirm that the certificate was installed properly, you can find it in the certificate manager in Windows. Check the Trusted Root Certificate Authorities store, and you should find your certificate there next to Godaddy, Equifax, DigiCert, and other famous CAs.

For a Windows Domain (Group Policy)

To roll out your trusted root certification network wide using group policy, see the Microsoft Knowledge-base Article Use Policy to Distribute Certificates.

 Installing the CA Certificate in the Trusted Root of Macs

This section will be written in the future. If you really want it, leave a comment below to ping me.

Signing CSRs with your CA Certificate

By the time you reach this section of this tutorial, you should know that a CSR is generated by a client / third party who wants you to vouch for them. In general, when you sign a certificate signing request you are saying: “I know this person / organization, and their resources are legit.” So, you wouldn’t want to sign CSRs without doing some sort of confirmation that they are who they say they are. Typically, this includes ensuring that they have control over the domain that the SSL certificate will be used (domain validation) or even going as far as doing a background check on them (Extended Validation style).

If you’re running your own CA because you have a need for a lot of certificates that your own users will all be using, then you can probably vouch for yourself!

Bottom line, when you receive a CSR from a third party, figure out who they are before you sign it. If you’re generating the CSR requests yourself, then there is no one to check up on.

To sign a CSR with your root-ca certificate, execute the following command:

openssl ca -config openssl.cnf -out client.crt -infiles client.req

This, of course, requires that you copy client.req into the /root/ca/ directory so that it can be accessed. Additionally, it requires that you are root as well.

In our example, we are going to generate a CSR as if we were a third party. (This is the same procedure described above, but shortened because we are using it as an example):

This example creates client.key in our /root/ca/ directory, but since we would normally never have this file when filling a CSR request, let’s delete it. Thus, at this point, we have essentially “copied” the CSR into the ca directory to fulfill the signing request as if it had come in via email. So, our directory currently looks similar to this:

Notice that the only “new” file here is client.csr.

Now we are ready to sign the CSR (assuming you’re ready to vouch for the requestor!). To sign the CSR:

openssl ca -config openssl.cnf -out client.crt -infiles client.csr

This process will require that you:

  1. Enter the password for the CA certificate.
  2. Verify the information for the cert is as you expected it
  3. Confirm the certification commit

The resulting file, client.crt, is the certificate the requester can use on the mail, web, or other servers.

Here’s What Else Just Happened:

When you executed the signing of the certificate, a few other things happened as well, which are part of the record keeping process.

  1. A copy of the signed certificate was stored in the certs/ directory as {serial}.pem.
  2. The serial number in the serial file was incremented by 1, and the original was saved as serial.old
  3. The database (index.txt) was managed to keep any relevant information.

Because these things happened, it is wise to keep backups of your CA directory.

Setting Up a VPN with Tinc VPN Software

Setting up a VPN can seem like a daunting task. There are some high-priced options out there, but most of those are closed source, and therefore not to be trusted. Instead, the internet has a variety of open source and free VPN server solutions. OpenVPN is a good one, but does not offer a distrbuted style “mesh network”. It requires a dedicated server, which may or may not be in your budget or scope. But, tinc-vpn offers “point to point” VPN as well as server style and everything in between.

As stated on the project homepage , tinc is a Virtual Private Network (VPN) daemon that uses tunnelling and encryption to create a secure private network between hosts on the Internet. It’s is an excellent way to create connections to servers and workstations over the internet. In particular, I came across it while looking for free alternatives to logmein.com. While tinc does not provide any sort of interface, it can be used to solve the problem of “permanent” remote access.

There are not a lot of great tutorials on the internet for tinc. So, we are going to take an in-depth look at exactly how to set-up a “server” and a client.

For starters, let’s build out our network on paper before we attempt to get it working in real life. In order to get around firewalls on remote networks, we have “punch through” by initiating a connection from within those remote networks. Routers assume that connections made by computers on the “inside” of their networks have permission to do so, and let those connections be made. Additionally, those same routers allow the return traffic through the firewall because: “what good is a request of we don’t allow the response back through to the requester?”

Thus, we need a centralized VPN “server” that all our remote nodes can connect to. Once we have that set-up, we can connect other computers (like a management workstation and a remote file server that we need to manage) to the VPN via the central node.

Setting up a VPN: the tinc VPN Basics

Up until now, I have been referring to a “server” for the central node. tinc does not have a “server”mode because all nodes are configured the same. For our purposes, the “server” we are configuring is a regular node that doesn’t connect to anyone else (pro-actively). Instead, it’s the place where “everyone else” connects to the network.

With that piece of information out of the way, let’s get back to the basics.

Order of Operations: How tinc reads configuration files (and in what order).

When tincd is started, the first thing it does is check /etc/tinc/nets.boot. Each line in that configuration file can contain the name of a network to join. For this tutorial, we are just going to use a single network, but you could set-up as many networks for your server as you wanted to.

After reading /etc/tinc/nets.boot, tinc goes to each network folder looking for a tinc.conf file. So, if your nets.boot file has a single entry: myvpn, tincd will look in /etc/tinc/myvpn/ for a tinc.conf file. (It will look for /etc/tinc/myvpn/tinc.conf).

Once it finds the tinc.conf file for this network, it will read it, and access information under the hosts directory for that network. So, if we use the “ConnectTo” directive to tell tincd that it should connect to a given server, it will look in /etc/tinc/myvpn/hosts/ for the value of that directive.

So, for example, if we are configuring a client (node) to ConnectTo a server that we have nicknamed “MainServer”, we’d use a directive like this:

ConnectTo=MainServer

Once tincd sees the value of that ConnectTo directive is “MainServer” it will immediately look in /etc/tinc/myvpn/hosts for the MainServer file. (It will want /etc/tinc/myvpn/hosts/MainServer). Once it finds that file, it will load configuration information about that host (node) from that file.

But, it also looks in the hosts directory to find information about itself! So, to get two computers to connect to one another, you must (at minimum) have two files in the hosts/ directory for your connections. One for the “server” and one for the “client”

But, tinc also uses the files in the hosts/ directory to get information about itself! As you’ll see in our example, we will create a file in hosts/ for machines each node will connect to as well as the node itself.

Authentication: How Each Node Knows It’s OK To Grant Access to the VPN Network.

tinc uses RSA authentication to authenticate nodes. If you’re not sure how RSA works, take a look at our RSA authentication primer. But for now, let’s suffice it to say that this is a key exchange. Each end of the node has to provide a key to be accepted by the other nodes

This means that each node has to have its own public / private RSA key pair. But don’t let that intimidate you, we’ll show you had to make these without having a Ph.D. in mathematics.

Preparing Your Computer

Whether you are running a server or a workstation, you have to make sure you have fulfileld the system requirements for tinc. They are:

  • A kernel that supports tun/tap. If you are using a modern version of any Debian based system, you’re already set here. (This includes Debian, Ubuntu, Mint, and other flavors). Most other modern Linux systems support this right out of the box. Except those distros that require you compile the kernel in order to install… but then again, if you’re using one of those experts-only distros, you won’t need much help installing this feature.
  • Make sure you have the following packages installed and available:
  1. OpenSSL
  2. zlib
  3. lzo

How to Install tinc-VPN

In Debian based systems, this is easy. Just use

apt-get install tinc

Do this for all the machines that will be participating in the VPN.

Setting up the Network

Do it on Paper First! Write down:

  1. Public IP of the “server”.
  2. VPN IP of the “server”.
  3. VPN IP of the “client(s)”

In our case, we are going to assume:

  • Public IP of the “server”: 12.345.67.89.
  • VPN IP of the “server”: 192.168.100.1
  • VPN IP of the client: 192.168.100.2

How To: Setting up the tincd “Server”

Setting up the “server” simply means we need to setup the files tinc uses in the order that it uses them:

  1. nets.boot
  2. the network directory
  3. tinc.conf in the network directory
  4. the hosts directory for the network
  5. the individual host files for hosts on the network.
  6. tinc-up
  7. tinc-down

First, change to the /etc/tinc/ directory, and edit the nets.boot file. We are going to add a single line to the end of the file: myvpn.

echo 'myvpn' >> nets.boot

Next, create the myvpn folder under /etc/tinc because tincd is going to look in that folder after it discovers that nets.boot has “myvpn” as one of the entries.

mkdir /etc/tinc/myvpncd /etc/tinc/myvpn

Next, create the tinc.conf file, which tells tincd how to setup its own node. Using your favorite Linux text editor (vim!), create tinc.conf with these two lines in the file:

Name=vpnserverDevice=/dev/net/tun

Save the file.

This, of course, assumes you are running a Debian based system that has a device at /dev/net/tun. Most do. But, if your distro doesn’t, you can change this to something like /dev/tap0 of you have tunctl installed. But for our example, we’re going to leave it like  this.

Now, if you remember from our order of operations above, as soon as tincd finds this file, it’s going to look for mycpn in the hosts directory. So, we need to create that directory so we can create the file.

mkdir /etc/tinc/myvpn/hosts

But we don’t create the myvpn file that goes in hosts/! That will be created for us when we generate the RSA keys for this system. So, let’s generate the keys:

tincd -n myvpn -K

If you haven’t set-up the tinc.conf file correctly (or yet), This command will generate two keys: rsa_key.priv and rsa_key.pub. Both of these will appear in your /etc/tinc/myvpn/ directory (because we specified -n myvpn in the command above). These two keys represent the keys to THIS system. So, we’ll keep the private key where  it is, but let’s move the public key into hosts:

mv /etc/tinc/myvpn/rsa_key.pub /etc/tinc/myvpn/hosts/vpnserver

If you DID correctly set-up your tinc.conf file, tincd will read that file, and find the name of this computer and use it to put the public key in the correct spot. So, instead of creating the public key as /etc/tinc/myvpn/rsa_key.pub, it should read Name=vpnserver, and put the public key in /etc/tinc/myvpn/hosts/vpnserver.

Right now, that file (/etc/tinc/myvpn/hosts/vpnserver) only contains the RSA public key, but we need to add some more information about this node to that file. So, let’s edit that file:

vim /etc/tinc/myvpn/hosts/vpnserver

and add some lines to it to identify the node both to itself, and later, to the other nodes on the network. We’ll add two lines: Subnet and Address.

Subnet identifies the IP address this node will have on the VPN network. Address identifies (to other nodes) what IP address they should use to connect to this server.

Subnet=192.168.100.1Address=12.345.67.89-----BEGIN RSA PUBLIC KEY-----MIIBCgKCAQEAoEgxyY5DANAlKSP3pkHJvX5Co1uihxcCwFGW7G3bXUsKTkg6TE1Pqs7Fae9HQIYPzu0uHhjR0jFNP0rAEWl3VoQnpe3E6uIBs+8PWlIpB6OvLXjaYbo7FhCje3OYTQMDwbhGaeZ/TdoOvAhHlu8giHZFc4SZ/Bd4z58UmLC5ShAtHKhMJr6KdYsZjBWnz341Q/dY+NRW5RXpH8akt8yW7xw/9So8CM3Lyf9Vvtn1RyY0IJcIq1kVUXYOmx/j5Ef48GrbziF5DhEhYCqVSYzqfeIS0PKesNyTWvqr0/n2owSH3q5a2mNIb+DuppRFSWxzkymrvrGfxVRuhi1Hj5lQPwIDAQAC-----END RSA PUBLIC KEY-----

Lastly, we need to set-up the tinc-up and tinc-down scripts. These scripts execute automatically when the VPN network comes up and when the VPN network goes down (respectively).

Essentially, we need these scripts to create the interface that the computer will use to talk to the VPN network when the VPN comes up, and then remove it when the VPN service stops.

Create a file called /etc/tinc/myvpn/tinc-up, and put the following lines of code in it:

#!/bin/bashifconfig $INTERFACE 192.168.100.1 netmask 255.255.255.0

Because tincd runs in router mode by default, we do not need to put any other configurations (or routes) in this file because tincd will build and maintain routing tables for us.

Now for tinc-down, which will execute when we shut down tinc and disconnect from the VPN. Create a file: /etc/tinc/myvpn/tinc-down, and add this content.

#!/bin/bashifconfig $INTERFACE down

This removes the vpn adapter from the system when the vpn service is shut down.

Once you have created those two files, you have to grant them permission to execute:

chmod +x /etc/init.d/myvpn/tinc-*

Now, you’re ready to start the tinc vpn server:

/etc/init.d/tinc start

If all goes well, you should see the following in syslog:

tincd 1.0.19 (Apr 22 2013 21:45:36) starting, debug level 9/dev/net/tun is a Linux tun/tap device (tun mode)Executing script tinc-upListening on 0.0.0.0 port 655Listening on :: port 655Ready

If you see any errors, google them, fix them, and then restart tinc until the errors resolve.

Setting up the VPN Clients

Because a “server” and a “client” have just ONE difference (the Server doesn’t use a ConnectTo directive in tinc.conf), The basic setup is exactly the same. So, follow the steps above to setup your client(s), but note these differences in /etc/tinc/myvpn/tinc.conf:

  • Clients will use a “ConnectTo” directive.
  • Clients need to be uniquely named. The server uses Name=vpnserver, but the clients should have their own individual (unique) network names. Name=Client01 would work, so would Name=George.

There is two major differences for clients in the public rsa key, which is stored in hosts/ for the network:

  • The Name directive must be unique
  • The subnet must be a different IP for the client than the server has.

Here is an example tinc.conf for a client:

Device=/dev/net/tunConnectTo=webservicesName=George01

Notice the Name is unique (this is for “George’s computer”) and it also uses the ConnectTo directive to tell this client node to connect to the main vpn server node.

Here is an example /etc/tinc/myvpn/hosts/George01 file:

Name=George01Subnet=192.168.100.2-----BEGIN RSA PUBLIC KEY-----MIIBCgKCAQEAzkkYpNIWSrm1kNX49PXYZR4SALGUtDY/iKHVGF2oqvsoKhl5hENiiNw9QqUtKUSDuJxP8w2AbeHBYaqr9kVyw3c/2Vzp1oGIxpbtMRcSEDJUcgJlpNeJ8iEvjEPUiliLNrfnpu7dEk8gT6Fu+b94R1n/5JnLueny3i0p8+qbA5/z4KUqVQCHnQqcDQ+8DY2Otrljae6YwEMgtShtUNA6nkUfJ61Y/2UITL6RQP7rAXbn3kJYozm/gjJPQ4W0oUlTSFwM2qziGIj68KrUXBj6V3VjInuVdAgFii6B2aXI+qUst705B/Bw+BZIsQxiKNruU+gi/+aQx2mtP2YPiTYk1QIDAQAB-----END RSA PUBLIC KEY-----

Notice:

  • the file name matches the Name directive in the /etc/tinc/myvpn/tinc.conf file.
  • The Subnet is different than the server. In this case, it is the next available address, but it doesn’t half to be. It could be any address on the 192.168.100.0/24 network.

Now, your client is setup! Start the tincd service and watch /var/log/syslog for errors that need correcting:

/etc/init.d/tinc start

The Last Step: Exchanging Keys

As mentioned in the introduction as well as in our RSA Authentication Primer, both sides of the VPN need to exchange public keys so that the “conversation” can actually take place.

So, let’s use secure copy to copy the vpnserver public key to the client. From the client:

cd /etc/tinc/myvpn/hostsscp root@12.345.67.89:/etc/tinc/myvpn/hosts/vpnserver .

If (when) prompted to enter your password, enter the password for the VPN server’s root account.

Now, let’s send the client public key to the vpn server:

cd /etc/tinc/myvpn/hostsscp George01 root@12.345.67.89:/etc/tinc/myvpn/hosts

Testing Your Connection

If all has gone well, you’ll now be able to ping the machines over the vpn network. From the “client” node:

ping -c4 192.168.100.1

should give you:

PING 192.168.100.1 (192.168.100.1) 56(84) bytes of data.64 bytes from 192.168.100.1: icmp_req=1 ttl=64 time=1.15 ms64 bytes from 192.168.100.1: icmp_req=2 ttl=64 time=0.905 ms64 bytes from 192.168.100.1: icmp_req=3 ttl=64 time=0.979 ms64 bytes from 192.168.100.1: icmp_req=4 ttl=64 time=0.943 ms

Form the server node, you should be able to execute:

ping -c4 192.168.100.2

and get:

PING 192.168.100.2 (192.168.100.2) 56(84) bytes of data.64 bytes from 192.168.100.2: icmp_req=1 ttl=64 time=1.15 ms64 bytes from 192.168.100.2: icmp_req=2 ttl=64 time=0.905 ms64 bytes from 192.168.100.2: icmp_req=3 ttl=64 time=0.979 ms64 bytes from 192.168.100.2: icmp_req=4 ttl=64 time=0.943 ms

Special Thanks

Special thanks to Guus Sliepen, who not only developed tinc-vpn, but also generously provides support via his tinc-vpn mailing list

RSA Encryption and Authentication Primer

RSA Encryption is a cryptographic system that uses public and private keys to exchange messages. It was first published in 1977 by Ron Rivest, Adi Shamir, and Leonard Alderman. RSA stands for Rivest, Shamir, and Alderman.

The system works by generating two RSA encryption keys: one public and one private. The key generation itself requires a good random number generator and the ability to resolve very large prime numbers. (Wikipedia has a more precise explanation of how the keys are generated, but this is sufficient for our purposes).

The public key can:

  • Encrypt messages destined for the person who has the corresponding private key.

The private key can:

  • Encrypt messages for anyone who has a the public key (not used often)
  • Decrypt messages encrypted with the private key (99.9% of all usage)
  • Sign messages to authenticate “I am who I say I am”

The public key can be used to encrypt messages, but only the private key can decrypt messages.

Because a public key can encrypt, but only the corresponding private key can decrypt, this means:

  • I have to keep my private key a secret. I can’t share it, email it, or let anyone see / touch it.
  • I can give my public key to anyone without regard. I can publish it on the internet if I like.
  • This public / private key set can only be used to send me messages. If I want to send someone else a message, my keys won’t work.

This last point is very crucial. In order to have a conversation, you and your friend both have to have a public and private key set. You will keep your private key a secret, and your friend will keep theirs a secret. But, you have to email your public key to your friend and your friend has to email you their public key.

Once the exchange of public keys is complete, you can have a conversation.

How a Conversation Works

There are two parties to the conversation: You and your friend (who we will call “Friend” for simplicity’s sake).

  1. You send friend your public key.
  2. Friend sends you their public key.
  3. You use Friend’s public key to encrypt a message to them.
  4. You send the encrypted message to Friend.
  5. Friend receives the message.
  6. Friend uses his private key to decrypt the message.
  7. Fried reads the message.
  8. Friend uses your public key to encrypt his response.
  9. Friend sends you the message encrypted with your public key.
  10. You receive the message from your friend.
  11. You decrypt the message using your private key.
  12. You read the message.

At this point, if you want to respond, you’ll return to step 3 and repeat steps 3-12 until the conversation ends.

How RSA Authentication Works

RSA Authentication works similarly to how a conversation between two friends works above. The Both sides have to have private keys, and both sides must exchange their public keys so the other side has it. Then, a system known as challenge-response is used to prove to both sides that each is the entity it claims to be.

Typically, a server administrator will setup an application, system service, or daemon to use RSA keys for authentication. When they do this, they:

  1. Generate a public / private key pair for server A
  2. Generate a public / private key pari for server B.
  3. Give server A server B’s public key so that Server A can validate server B when it tries to sign in.
  4. Give server B server A’s public key so that it can validate Server A when it tries to sign in.
  5. Start the relevant system services or daemons.

Now, when server A attempts to authenticate with server B, the conversation goes something like this:

Server A: “Hello. I am Server A. I would like access please.”

Server B looks at the request, and doesn’t trust this “computer” who claims to be server A. So, Server B looks up the public key for Server A, and generates a random “challenge”, which it encrypts with server A’s public key. It then sends the encrypted challenge to the computer claiming to be server A.

Server A receives the encrypted challenge. Since Server A really is who it claims to be, this server has the private key needed to decrypt the challenge from Server B. So, it uses the private key to decrypt the message and finds out that the challenge phrase is “I love world peace and fluffy bunnies”. But, at this point, Server A still needs to convince Server B that they really are the computer they say they are. At the same time, we can’t send the challenge message back to server B in plain text. So, Server A uses Server B’s public key to re-encrypt the message, and then sends it back to server B. This is called the response.

Server B receives the response, decrypts it using its private key, and then checks to make sure the response message is identical to the challenge message. If it is, Server B will grant access. If it’s not, no access will be granted.

Note: In reality, the challenge string will not be human readable. It is much more likely to be a series of randomly generated values like: sxCz3yQ0jj94TNBHCavAK43jiIkOVOTBkSA4oQGClXw=

How to Fix the vsftpd GnuTLS Error 15 with FileZilla on Ubuntu 12

by Nathaniel Morales

Like many of you around the interwebs, you have been upgrading your Ubuntu 10.04 LTS boxes to 12 because 10 is getting end-of-life’d. And, of course, it is not without headache. You upgrade your server, re-install vsftpd from apt, copy over your config files,  and it starts up just fine. Then you try to connect to vsftpd with your FileZilla FTP client, and the vsftpd GnuTLS Error 15 shows up in the FileZilla log. It worked before, but now it doesn’t. Here how I fixed it.

Here’s what I saw in my logs that prompted me to start this voyage:

Status:    Resolving address of yourserver.com
Status:    Connecting to 123.456.78.90:990...
Status:    Connection established, initializing TLS...
Status:    Verifying certificate...
Status:    TLS/SSL connection established, waiting for welcome message...
Response:    220 (vsFTPd 2.3.5)
Command:    USER michael
Response:    331 Please specify the password.
Command:    PASS ********
Error:    GnuTLS error -15: An unexpected TLS packet was received.
Error:    Could not connect to server

Well… that’s annoying.

What’s the cause of the vsftpd GnuTLS Error 15

When you upgrade from Ubuntu 10.04 to 12.04, it also upgrades vsftpd from version 2.2.2 to 2.3.5. Somewhere between 2.2.2 and 2.3.5, vsftpd became more secure, and it exposed a bug in FileZilla where FileZilla doesn’t present the client certificate properly or allow session reuse after certificate negotiation. (Source: https://security.appspot.com/vsftpd/Changelog.txt). The FileZilla people know about it because a bug is listed with the FileZilla, but “there is no general solution yet.”

Here’s how to fix it.

As an overview, here’s what we have to do to fix this problem:

  1. Revert to vsftpd 2.2.2 by removing the repo version and compiling from source.
  2. Fix the  vsf_findlibs.sh file, because it is looking for resources with hard-coded paths.
  3. Configure vsftpd to compile with ssl support
  4. Compile.
  5. Re-build init scripts / upstart jobs

Removing the Repo Version of vsftpd

It should be as easy as apt-get remove vsftpd. However, you may want to run the command below to see if there are any “straggler” files that might interfere.

 locate vsftpd | grep -v src

It’s ok if this shows results. We just want to make sure there are no results that contain a bin or sbin like:

/usr/bin/vsftpd

Getting the Proper “Working” Version of vsftpd

Download this version of vsftpd into /usr/src/, but changing directories to the /usr/src, and using wget to download it:

sudo su
cd /usr/src/
wget -r ftp://vsftpd.beasts.org/users/cevans/untar/vsftpd-2.2.2/

Fix  vsf_findlibs.sh

The file vsf_findlibs.sh in version 2.2.2 no longer works under Ubuntu 12 x64 because the file is (by its own admission in the comments) is very hackish. It looks to see if a file exists in a certain place, and if it does, it returns a configuration directive (like: -ssl -lpam, etc…).

The problem is: if it doesn’t ind those files, it doesn’t return the right configurations even though your system as all these resources. So, you have to fix it.

which you’re welcome to try. But, in case it doesn’t, here’s how to fix yours for yourself:

Throughout the file, you’ll see that there are directives / switches that are related to .so files. Each one of those directives / switches only needs to find those library files, to return that switch for the Makefile. So, all you have to do is use the locate command to find out where the file is in your installation, and add a line.

In the picture below, I have highlighted the switch “-lpam,” from the original file with red arrows. The line in the orange box shows the line I added to fix the file. All I did was execute the command below to find the library file, and added the line with the correct path.


vsf_findlibs.sh

Just repeat this process throughout the file. But there is one exception (shown below):

Because the results of this part of the script actually echo back a file location, you have to create symlinks. So, lets assume you find libcap.so.1 in /usr/lib/x86_x64-linux-gnu/libcap.so.1. You would create a symlink like this:

ln -s /usr/lib/x86_x64-linux-gnu/libcap.so.1 /lib/libcap.so.1

And then, repeat for the libcap.so.2 file as well. In my case, only libcap.so.2 was present on my system.

When you’re done, exit the editor, and run vsf_findlibs.sh manually. It should run with no errors, and return a series of configuration directives, as shown below:

 Configure SSL Support

Open the builddefs.h file, and modify it as shown below, or download the one I did here: builddefs.zip

Compile and Test vsftpd

First, you have to prepare the system by installing required dependencies:

apt-get build-dep vsftpd

Now, that we have that setup and a correctly modified source set, we can compile the source using:

make
make install

Now, use the which command to see where it has been installed:

which vsftpd

which will probably return this path: /usr/local/sbin/vsftpd.

Now run vsftpd -v, to double check to make sure the correct version (2.2.2) is displayed:

root@michael-desktop:/usr/src/vsftpd-2.2.2# vsftpd -v
vsftpd: version 2.2.2
root@michael-desktop:/usr/src/vsftpd-2.2.2#

Now, run vsftpd with the following command, which will start it stand-alone mode:

/usr/local/sbin/vsftpd&amp;

Now, try to connect with your favorite FileZilla client. If it works, move to the next step. If it doesn’t, check your configs, re-read the instructions above, and try again.

Configuring the Upstart Job (Init Script)

It’s easiest to start with a working init script and upstart job, so you can download  Download vsftpd Upstart and Init Scripts. Download this file into your root directory, and then execute the following command to restore the two files to their respective positions:

tar -xvf vsftpdupstart.tar

At this point, most likely, it’s going to fail because the upstart job has the wrong exec path, so here’s how to fix that:

  1. Execute which vsftpd to double check the path. Mine is now in /usr/local/sbin/vsftpd
  2. Edit /etc/init/vsftpd, and find the last line where it says: exec /usr/sbin/vsftpd, and change it to reflect your binary location you found in step 1.
  3. Save the file, and exit

Now, that you have correctly setup the init script and upstart jobs, run start vsftpd, and you should get something like this:

root@michael-desktop:/# start vsftpd
vsftpd start/running, process 14189
root@michael-desktop:/#

If it’s not running, check all your paths. Otherwise, you should now have a working installation! Yay Downgrades!

Creating a Network Lab for Server Pre-Deployment

by Nathaniel Morales

Wouldn’t it be wonderful if you could prepare a network lab that would allow you to build a server and fully test it (including the production network configuration) before deploying it to a client’s office? As an IT consultant, you will reap many benefits by fully testing a server or a system in a network lab before it ever hits the client’s office. For starters, if something breaks, is mis-configured, doesn’t work, or is just plain wrong, you can fix it at your leisure and without the client ever knowing you made a mistake. If you try to build out servers and systems at the client’s office, however, they are going to see every hiccup and every mistake, which damages your credibility with them.

In this article, I will teach you how to build a network lab that you can use to completely re-create your client’s network inside your own lab, so that not only can you build the server before you give it to the client, but you can fully test it (including it’s network configuration) before deploying it.

Understanding the Concept

Essentially, we are going to create two, concentric networks inside your existing LAN. (This makes a grand total of three concentric networks with an internet connection.)

In order to create each of these concentric networks, we’ll need to configure a router to route traffic appropriately between them. For the sake of simplicity, just know that with every router, a new circle is created. And, the router doesn’t care that the circle you are creating is a private or public network. It just routes traffic.

By definition, a router is a device that joins two networks together. The secret is: it doesn’t care what the networks are. While IP address calculation and subnetting are outside the scope of this article, I will tell you that you can create a small network inside your network that will mimic the public internet at your client’s place.

Terms Used in this Article

PseudoWAN: The emulated WAN or public internet connection we create inside your LAN, which serves to mimic your client’s internet connection.

PseudoLAN: The emulated local area network that is inside your LAN, which serves to emulate your client’s LAN.

AlphaRouter: The router that connects your LAN to the PseudoWAN (and by virtue of its existence and configuration, creates the PseudoWAN inside your LAN.)

AlphaSwitch: The switch that lives in the PseudoWAN.

BetaRouter: The router that connects the PseudoLAN to the PseudoWAN (and by virtue of its existence and configuration, creates the PseudoLAN).

BetaSwitch: The swith that lives inside the PseudoLAN.

Network Universe: The entirety of the three concentric networks, which include your LAN, the PseudoWAN, and PseudoLAN

What Hardware You’ll Need

Routers

To do this, we are going to need two routers in addition to the one you are already using, some cables, and the computers you are going to hook up to test in configuration.For the purposes of this article, we are going to use a Linksys WRT54G, but most any router that allows you to fully configure both the WAN address and the LAN address will do. (If you’re lucky enough to have a WRT54GL and can put either the Tomato firmware or DD-WRT on it, that’s even better!).

Your Client’s Network Topography

You’ll need to know what his IP addresses are. Smaller offices use a 192.168.x.x/24 network. You’ll need to figure out what they are using so you can properly configure your pseudo network.

Your Client’s ISP Configuration

You’ll need to get your client’s IP address block information so you can successfully create the pseudo public internet. For the purposes of this tutorial, we are going to use a 6 host block, which gives us 5 usable IP addresses. The IP addresses we have available are:

  • 1.2.3.1
  • 1.2.3.2
  • 1.2.3.3
  • 1.2.3.4
  • 1.2.3.5

Our gateway is 1.2.3.6, and the network is 1.2.3.0/29 (this is the same as 1.2.3.0 with a subnet mask of 255.255.255.248).

So, our IP configuration for our first usable IP is going to be:

Address: 1.2.3.1
Netmask: 255.255.255.248
Gateway: 1.2.3.6

We will make this the firewall address.

Our second usable address will be configured to be a server. It’s network configuration will be:

Address: 1.2.3.2
Netmask: 255.255.255.248
Gateway: 1.2.3.6

Map Out Your Network Lab

I highly advise that you draw out what you’re about to make and put in the relevent IP addresses for all the equipment. It will make it a lot easier to understand and a lot more efficient to build. Below, you will find the network map of the example Network Universe we are building. You can see that it flows from top to bottom, from the internet, to the LAN, to the pseudo public network for the client, to the client’s pseudo network. Be sure to come back to this diagram and reference it as you read the rest of this tutorial and even as you build your own network universe.

Mimicking  the Public Internet

To create our own, personal, version of our client’s internet connection, we’ll need to configure the first router to create the same network conditions that exist at our client’s office. Thus, we need to create a gateway for their block of public IP addresses. So, let’s grab our Linksys router, and configure the WAN and LAN sides of the router so it will act like our client’s ISP’s gateway.

WAN is LAN and LAN is WAN!

The WAN port of the Linksys router will need to be configured as an address on your local LAN. This allows the router to pass information into your LAN from our pseudo networks. So, let’s configure it to be 192.168.1.15/24.

Now, let’s configure the LAN side to be the gateway of the WAN for the client. Enter your client’s public gateway IP address in for the LAN IP address of the router, configure the gateway, and disable the DHCP server, then click save.

Now, plug the WAN port from this router into the switch on your network, and plug the LAN port into a switch. Your public network is now operational.

Installing Your Public Server in the PseudoWAN

  1. Connect your server’s LAN card to the switch, which is connected to the router we just setup.
  2. Configure the network card of your server with the following:
Address: 1.2.3.2
Netmask: 255.255.255.248
Gateway: 1.2.3.6

Restart the networking services if necessary, and use ping to verify you can ping the gateway 1.2.3.6. Next, use ping to verify that traffic from the PseudoWAN is traversing the router properly and entering your LAN: ping your gateway (192.168.1.1). Lastly, confirm that network traffic can get from your PseudoWAN, through your network, and out on to the public internet by pinging Google’s DNS Servers: 8.8.8.8.

Assuming that all three of these ping tests receive replies, you are clear to move to the next step. If you get packet losses at any of the steps above, go back, and reconfirm your settings.

Setting Up the PseudoLAN

This process is done with a second router. You can use the same make and model router as you used before. They do not have to be different. Again, we are using a WRT54G in the examples.

1. Plug the WAN port of the BetaRouter into the AlphaSwitch.

2. Configure the WAN port of the BetaRouter with the following:

Address: 1.2.3.1
Netmask: 255.255.255.248
Gateway: 1.2.3.6

3. Configure the LAN

First, you’ll need to change the LAN IP address, and click Save / Updatebecause until you do that, the DHCP server range will be inaccurate. Once you have done that, you can configure the DHCP server range, and click save / update again. Configure the LAN IP of this router to be the gateway of the PseudoLAN, which mimics the LAN in your client’s office.

Add Client Computers to the PseudoLAN

At this point, you can add a computer to the PseudoLAN, and give it an IP address that is identical to what it would have while in production at your client’s office. Once you have a computer setup, use ping to verify connectivity from that computer to the PseudoLAN gateway (192.168.10.1), the PseudoWAN gateway (1.2.3.6), your LAN gateway (192.168.1.1), and finally the public internet (Google’s DNS at 8.8.8.8). When all those tests come back positive with good replies, you have successfully built your network universe to parrot your client’s network so you can build and install!

How to Setup a Linux Mail Server on Debian / Ubuntu

by Nathaniel Morales

A Linux mail server consists of software packages that can send and receive mail using the SMTP protocol and retreive mail using the standard protocols POP3 and / or IMAP. Since we are about to setup your new Linux mail server, it is important to understand the differences between the various protocols and their purposes. SMTP is typically used to deliver mail from a client to a server or pass messages from one server to another. POP3 (Post-Office Protocol) is an uni-directional protocol, which retrieves mail from the mail server and delivers it to a mail client like Mozilla’s Thunderbird. POP3 differs from IMAP with one significant respect: IMAP synchronizes messages between different d whereas POP3 simply downloads copies of messages.

The default configuration for the mail server software we are going to use to build our server is setup for a single domain. However, it is more useful to create a server that can not only accomodate multiple email addresses, but also multiple domains. The most convenient way to manage these multple domains and email addresses is to use a MySQL database as a backend.

Preparing Your Linux Mail Server

First, let’s prepare the system by installing the required packages:

apt-get install vim postfix postfix-mysql postfix-doc mysql-client mysql-server courier-authdaemon courier-authlib-mysql courier-pop courier-pop-ssl courier-imap courier-imap-ssl libsasl2-2 libsasl2-modules libsasl2-modules-sql sasl2-bin libpam-mysql openssl phpmyadmin apache2 libapache2-mod-php5 php5 php5-mysql libpam-smbpass php5-imap

During installation, you’ll be asked the following questions:

Question 1: What type of server site do you want? Select Internet Site

Question 2: Enter the FQDN (Fully Qualified Domain Name)

This is the full domain name that you will put into in the MX Records. Make sure this has already been entered into the DNS at your registrar / name servers. The example below shows mail.domain.com, but you will want to change it to your legitimate mail server address (or backup MX record).

Question 3: Enter the administrative “root” user password.

Make sure this is a strong password, and make sure you write it down! You’ll need it later…Once you enter in the password, you will (of course) be asked to confirm it.

Question 4: Create directories for web-based administration?

Enter: No.

Question 5: Is not really a question.

It just tells you that you are going to need an SSL certificate. Just clickOK.

Question 6: For the Samba server, leave the workgroup name “WORKGROUP”

Question 7: Next, phpMyAdmin will ask you to choose the web server it will use. Select Apache.

Question 8: Configure the database for phpMyAdmin with dbconfig-common? Select Yes.

Question 9: Enter the password for the database administrative user.

Question 10: Next, it will ask you for a password to register with the MySQL server. Enter in the root password you put in for question 3 above.

The installation of the software packages will continue and return you to the prompt. Next, we’ll need to configure the database system so we can support multiple domains and multiple users. The easiest way to do this is to install postfixadmin, and let postfixadmin create the database tables we will use to manage the system.

Setting Up the Database

Making Sure the MySQL Database Server is “listening”

Edit the /etc/mysql/my.cnf file to ensure that bind-address on line 47 is configured to be 127.0.0.1. If it was not, change the directive to the following:

bind-address = 127.0.0.1

and then restart the MySQL service with this command:

/etc/init.d/mysql restart

Setup a User for the Database

Log into the MySQL client:

msyql -uroot -p

Enter the password you created for root above. (See Question 3 above!)

CREATE DATABASE postfix;GRANT ALL ON postfix.* to 'postfixadmin'@'localhost' IDENTIFIED BY 'changethispassword';

Change to the /var/www/ directory (Apache’s document root)

cd /var/www/

Download the current version of postfixadmin:

wget -O postfixadmin.tar.gz http://sourceforge.net/projects/postfixadmin/files/latest/download

Untar the package:

tar -zxvf postfixadmin.tar.gz

Postfixadmin will likely untar into a directory like postfixadmin-2.3.5, which is not very convenient for us. So, let’s move it to a more memorable, easier directory:

mv -v postfixadmin-2.3.5/ postfixadmin/

Configure PostfixAdmin

Edit config.inc.php, and change line 27 to set $CONF[‘configured’] to true:

//$CONF['configured'] = false;$CONF['configured'] = true;

Now, configure the database permissions on line 52-54 (approximately). Change the username to postfixadmin and the password to the password you configured above.

Once you have completed the configuration of PostfixAdmin, you can now navigate to the setup script in the browser by going to the IP address of the server /postfix/admin.php.

http://192.168.1.100/postfixadmin/setup.php

The script will check your setup and alert you to any errors. If you have followed this tutorial closely (and correctly) there should be no errors. If there are, however, double check the process, and correct any errors that are being shown.

Once you correct any errors, it will ask you to enter in a setup password. Enter in the password, and click the Generate Password Hash button. Once you do so, it will give you a line configuration that we will enter into config.inc.php.

Copy and paste that configuration line to the end of the config.inc.php, and then refresh the page.

Once the page is refreshed, enter in the setup password, and create a master email account that will serve as the superadmin account. Once you have completed this process, double check to make sure that the setup script has notified you of two events:

Everything seems fine... attempting to create/update database structureDatabase is up to date

Once you see these messages, the database and table structures are created, and we are ready to move on to the next part of the process.

Setup phpMyAdmin

Change directory to /var/www/

cd /var/www/

Download the current version of phpMyAdmin:

wget -O phpmyadmin.tar.gz http://downloads.sourceforge.net/project/phpmyadmin/phpMyAdmin/3.5.2/phpMyAdmin-3.5.2-english.tar.gz?r=http%3A%2F%2Fwww.phpmyadmin.net%2Fhome_page%2Fdownloads.php&ts=1342151305&use_mirror=iweb

Untar it

tar -zxvf phpmyadmin.tar.gz

Move the directory to something more usable:

mv -v phpMyAdmin-3.5.2-english/ phpMyAdmin/

Navigate to http://192.168.1.100/phpMyAdmin/ and log in using postfixadmin and the password you set for the postfixadmin user to double check to make sure that user permissions are correct. Upon logging in, you should see the postfix table listed on the left hand side, and then the following list of tables will be shown:

Configuring Postfix to Use MySQL

There are six files that we are going to configure to work with our configuration as we have set it up thus far, which are all located in /etc/postfix/:

  • mysql-virtual_domains.cf
  • mysql-virtual_forwardings.cf
  • mysql-virtual_mailboxes.cf
  • mysql-virtual_email2email.cf
  • mysql-virtual_transports.cf
  • mysql-virtual_mailbox_limit_maps.cf

mysql-virtual_domains.cf

This file tells postfix what domains the mail server will be receiving mail for. Here are the contents:?

user = postfixadminpassword = thepostfixadminpassworddbname = postfixquery = SELECT domain AS virtual FROM domain WHERE domain='%s'hosts = 127.0.0.1

mysql-virtual_forwardings.cf

This table sets up email forwarding and aliases. Its content should be:

user = postfixadminpassword = thepostfixadminpassworddbname = postfixquery = SELECT goto FROM alias WHERE address='%s'hosts = 127.0.0.1

mysql-virtual_mailboxes.cf

This file tells Postfix where in your local mail server’s file system the mail will be kept. The SQL statement parses the email address and turns it into a relative path from the mail root. It’s file contents are:

user = postfixadminpassword = thepostfixadminpassworddbname = postfixquery = SELECT CONCAT(SUBSTRING_INDEX(username,'@',-1),'/',SUBSTRING_INDEX(username,'@',1),'/') FROM mailbox WHERE username='%s'hosts = 127.0.0.1

mysql-virtual_email2email.cf

This tells Postfix how to find and indentify aliases. It’s contents should consist of:

user = postfixadminpassword = thepostfixadminpassworddbname = postfixquery = SELECT username FROM mailbox WHERE username='%s'hosts = 127.0.0.1

mysql-virtual_transports.cf

Postfix needs to be told how to transport emails from the point of receipt to teh final destination. For our purposes (and mostly because we are configuring this server with PostfixAdmin), we are going to use “virtual” as the transport because this server receives mail for virtual hosts. The file contents and configuration for this consist of:

user = postfixadminpassword = thepostfixadminpassworddbname = postfixquery = SELECT transport FROM domain WHERE domain='%s'hosts = 127.0.0.1

mysql-virtual_mailbox_limit_maps.cf

This last configuration file tells Postfix how to limit email accounts based on a quota. Its configuration file should be:

user = postfixadminpassword = thepostfixadminpassworddbname = postfixquery = SELECT current FROM quota WHERE username='%s'hosts = 127.0.0.1

Now, we have to change the ownership and group of the files to make Postfix happy:

chmod o= /etc/postfix/mysql-virtual_*.cfchgrp postfix /etc/postfix/mysql-virtual_*.cf

Now, we have to create a user and group that will own and operate the virtual mail stores (along with the home directory where all the mail will be stored)

groupadd -g 5000 vmailuseradd -g vmail -u 5000 vmail -d /home/vmail -m

Next, use the commands below to configure Postfix to use the MySQL tables and configuration files we have set up to this point. Be sure to change changeme.example.com to your FQDN or you’ll break Postfix!

postconf -e 'myhostname = changeme.example.com'postconf -e 'mydestination = changeme.example.com, localhost, localhost.localdomain'postconf -e 'mynetworks = 127.0.0.0/8'postconf -e 'virtual_alias_domains ='postconf -e 'virtual_alias_maps = proxy:mysql:/etc/postfix/mysql-virtual_forwardings.cf, mysql:/etc/postfix/mysql-virtual_email2email.cf'postconf -e 'virtual_mailbox_domains = proxy:mysql:/etc/postfix/mysql-virtual_domains.cf'postconf -e 'virtual_mailbox_maps = proxy:mysql:/etc/postfix/mysql-virtual_mailboxes.cf'postconf -e 'virtual_mailbox_base = /home/vmail'postconf -e 'virtual_uid_maps = static:5000'postconf -e 'virtual_gid_maps = static:5000'postconf -e 'smtpd_sasl_auth_enable = yes'postconf -e 'broken_sasl_auth_clients = yes'postconf -e 'smtpd_sasl_authenticated_header = yes'postconf -e 'smtpd_recipient_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unauth_destination'postconf -e 'smtpd_use_tls = yes'postconf -e 'smtpd_tls_cert_file = /etc/postfix/smtpd.cert'postconf -e 'smtpd_tls_key_file = /etc/postfix/smtpd.key'postconf -e 'transport_maps = proxy:mysql:/etc/postfix/mysql-virtual_transports.cf'postconf -e 'virtual_create_maildirsize = yes'postconf -e 'virtual_maildir_extended = yes'postconf -e 'virtual_mailbox_limit_maps = proxy:mysql:/etc/postfix/mysql-virtual_mailbox_limit_maps.cf'postconf -e 'virtual_mailbox_limit_override = yes'postconf -e 'virtual_maildir_limit_message = "The user you are trying to reach is over quota."'postconf -e 'virtual_overquota_bounce = yes'postconf -e 'proxy_read_maps = $local_recipient_maps $mydestination $virtual_alias_maps $virtual_alias_domains $virtual_mailbox_maps $virtual_mailbox_domains $relay_recipient_maps $relay_domains $canonical_maps $sender_canonical_maps $recipient_canonical_maps $relocated_maps $transport_maps $mynetworks $virtual_mailbox_limit_maps'

Create SSL Certificates for Secure Connections

Using the commands below, create a certificate for your server to use to send and receive secure, encrypted mail. (You may, alternatively, purchase one from a certificate authority)

cd /etc/postfixopenssl req -new -outform PEM -out smtpd.cert -newkey rsa:2048 -nodes -keyout smtpd.key -keyform PEM -days 365 -x509

Openssl will now generate SSL Certificates for you, but will need to encode some information about you and your organization to do so. (If you are a personal user, don’t worry… you can fill in anything you want here. Here are some suggested values:

Country Name (2 letter code) [AU]:USState or Province Name (full name) [Some-State]:GALocality Name (eg, city) []:AtlantaOrganization Name (eg, company) [Internet Widgits Pty Ltd]:High Society for Schrödinger's Cat LoversOrganizational Unit Name (eg, section) []:Cardboard Box DivisionCommon Name (eg, YOUR name) []:yourfqdn.yourdomain.comEmail Address []:ssl@yourdomain.com

As you can see, the only value here that really matters is the common name which must match the fully qualified domain name that points to the server. This is how the SSL certificate checks to ensure that there are no man in the middle attacks being perpetrated. The rest of the information is arbitrary.

Lastly, button up the permissions of our cryptographically generated key:

chmod o= /etc/postfix/smtpd.key

How to Configure Saslauthd for our Linux Mail Server

The Simple Authentication and Security Layer authentication daemon (Saslauthd) is a system service that handles plaintext authentication requests securely by inserting a security layer between the protocol and the connection. In layman’s terms, this provides security for connections that exchange usernames and passwords in plain text such as POP3 and IMAP.

First, we have to give it a place in which to operate. As with most Linux systems, we’re going to use the spool directory under /var/, and create a directory just for use with Postfix, then create another set of directories for saslauthd:

mkdir -p /var/spool/postfix/var/run/saslauthd

Next, we have to tell the saslauthd daemon to use our newly created directory. So, we’ll edit /etc/default/saslauthd, and navigate to the bottom line, and change this line:

OPTIONS="-c -m /var/run/saslauthd"

to this:

OPTIONS="-c -m /var/spool/postfix/var/run/saslauthd -r"

Next, we need to tell pam.d to use MySQL for authentication during SMTP transactions. (This requires that SMTP senders have a valid account before being able to send email, and is a vital step in protecting your email server from spammers. Create the file /etc/pam.d/smtp, and put the following lines in it:

auth required pam_mysql.so user=postfixadmin passwd=thepostfixadminpassword host=127.0.0.1 db=postfix table=mailbox usercolumn=username passwdcolumn=password crypt=1account sufficient pam_mysql.so user=postfixadmin passwd=thepostfixadminpassword host=127.0.0.1 db=postfix table=mailbox usercolumn=username passwdcolumn=password crypt=1

WARNING: Newer versions of Debian (and maybe Ubuntu) have some include statements that will break our SMTP authentication. If you have any uncommented lines in your /etc/pam.d/smtp file other than the ones above, it will break your setup!


This tells the saslauthd daemon to use MySQL for authentication, defines the username, password, server, database, table, and column where the credentials are kept, and lastly, tells it what type of cryptography to use when attempting to match passwords. Now, we have to tell Postfix to use saslauthd for these transactions.
We’re almost done with the file at this point, we just need to tell saslauthd to start automatically. Find the START directive in /etc/default/saslauthd, and change it from no to yes.

START=yes

Edit /etc/postfix/sasl/smtpd.conf, and put the following lines in it:

pwcheck_method: saslauthdmech_list: PLAIN LOGINallow_plaintext: trueauxprop_plugin: sqlsql_hostnames: 127.0.0.1sql_user: postfixadminsql_passwd: thepostfixadminpasswordsql_database: postfixsql_select: select password from mailbox where username = '%u@%r'

The first line tells Postfix to use saslauthd for authentication, the next line tells it that plaintext authentication is allowed (because saslauthd will encrypt it for us!), the rest of the entries tell Postfix / saslauthd how to ask MySQL for information to confirm the credentials.

Now, we’ll add the user postfix to the system, and make it a member of the sasl group so it has permissions to use sasl:

adduser postfix sasl

Now, let’s restart both of the services so our new settings will take effect:

/etc/init.d/postfix restart/etc/init.d/saslauthd
restart

Enabling IMAP and POP3 Access with Courier

Courier is a separate MTA, which is frequently used in conjuction with Postfix to provide access to mailboxes via a client. Simply put: Postfix sends and receives mail to the server, but Courier provides access so the mail can be downloaded to your email client. Courier provides a number of services such as ESMTP, IMAP, POP3, LDAP, SSL, and HTTP mail connectivity.

Enabling IPv4 Only SSL Connectivity

IPv6 is the future, but it is a non-implemented future. Courier has already embraced the IPv6 realm and decided that, by default, if you are going to connect to it via SSL, you’ll have to do it via IPv6. For now, that is not very useful for us. So, we have to force it to use the IPV4 address everyone is used to.

To Force Courier to use IPv4 for IMAP, edit /etc/courier/imapd-ssl, and change the SSLADDRESS directive to the IPv4 IP address of the network adapter.

SSLADDRESS=192.168.1.24

Repeat this process for the pop3d-ssl config file:

SSLADDRESS=192.168.1.24

Enabling SMTP SSL Send Connectivity

By default, Postfix recieves email on port 25. We also want to enable it to receive mail using SSL on port 465 (the default SMTPS Port). To do so, we just need to uncomment the following lines (17-22) from /etc/postfix/master.cf:

smtps inet n - - - - smtpd-o smtpd_tls_wrappermode=yes-o smtpd_sasl_auth_enable=yes-o smtpd_client_restrictions=permit_sasl_authenticated,reject-o milter_macro_daemon_name=ORIGINATING

Now, just reload and restart Postfix so the settings will be applied:

postfix reload/etc/init.d/postfix
restart

Configure MySQL

Like all the other services we have configured thus far, we need to tell Courier to use MySQL fr authentication. To do this, edi the following file:/etc/courier/authdaemonrc, and find the authmodulelist directive (on or about line 27). The default is authpam, but we need to change it to authmysql:

authmodulelist="authmysql"

Save the file.

Now, let’s configure the MySQL authentication module by editing /etc/courier/authmysqlrc. Delete the existing contents of the file so that we are starting anew. (If you had a working configuration before, consider backing up the file before making this change). The entirety of the file’s contents should be:

MYSQL_SERVER localhostMYSQL_USERNAME postfixadminMYSQL_PASSWORD thepostfixadminpasswordMYSQL_PORT 0MYSQL_DATABASE postfixMYSQL_USER_TABLE mailboxMYSQL_CRYPT_PWFIELD password#MYSQL_CLEAR_PWFIELD passwordMYSQL_UID_FIELD 5000MYSQL_GID_FIELD 5000MYSQL_LOGIN_FIELD usernameMYSQL_HOME_FIELD "/home/vmail"MYSQL_MAILDIR_FIELD CONCAT(SUBSTRING_INDEX(username,'@',-1),'/',SUBSTRING_INDEX(username,'@',1),'/')#MYSQL_NAME_FIELDMYSQL_QUOTA_FIELD quota

Setting Up Correct SSL Certificates

When you installed Courier, it generated some self-signed SSL certificates for localhost, but that will cause problems with mail delivery on most clients. So, we need to change that server name in the certificates to the fully qualified domain name for this mail server.

Change the IMAP Server Name

  1. Edit /etc/courier/imapd.cnf
  2. Find the Canonical Name Entry (CN=) and insert your FQDN for the mail server.
  3. Save the file.

Change the POP3 Server Name

  1. Edit /etc/courier/pop3d.cnf
  2. Find the Canonical Name Entry (CN=) and insert your FQDN for the mail server.
  3. Save the file

Now, we’ll need to delete and re-create the certificates for the IMAP and POP3 servers, and then restart Courier:

cd /etc/courierrm -f /etc/courier/imapd.pemrm -f /etc/courier/pop3d.pemmkimapdcertmkpop3dcert/etc/init.d/courier-authdaemon restart/etc/init.d/courier-imap restart/etc/init.d/courier-imap-ssl restart/etc/init.d/courier-pop restart/etc/init.d/courier-pop-ssl restart

As a side note, I create a script in /usr/local/sbin called restart-courier, which executes all the restarts above in the correct order to facilitate making changes to the system. You can download a copy of restart-courier here.

Setup the Postmaster Address

To be RFC Compliant, (and to ensure we get important system messages), we’ll need to modify the /etc/aliases file to route root and postmaster mail properly.

Edit the /etc/asliases file and ensure that the postmaster: root entry exists. This routes all “postmaster” mail to the “root” user.

Now, let’s route all the “root” mail to your email address (or the system administration address) by modifying the entry for root:

root: username

should be changed to:

root: youremail@domain.tld

Now, save the file, and exist the editor. Now, let’s tell Postfix to re-analyze the /etc/aliases file by running: newaliases, and then restart Postfix:

newaliases/etc/init.d/postfix
restart

Testing Your Mail Server.

Testing Your Ability to Add Domains and Users to the system.

  1. Log into Postfixadmin, and create a new domain for the system. (example.com)
  2. Create a new mailbox for that domain. (user@example.com)
  3. Once the user is created, you should see that example.com has been created under /home/vmail/. If this fails, make sure that Postfix is running as the correct user and that the servics have been restarted appropriately to make all changes to the configs take effect.

In the example below, you can see the mailbox structure:

  1. The username
  2. The domain name to which that username belongs
  3. The mailboxes (current messages, new (unread) and tmp (draft))

Testing SMTP Connectivity

SMTP has to be working in order to receive mail. Let’s test to make sure everything is in order:

  1. Go to MXToolbox.com
  2. In the testbox, enter smtp:[Server IP Address]

If you notice, this configuration does give us a warning that the reverse DNS does not match the SMTP banner. There are two resolutions to this. If you are running a dedicated mail server, and this is the primary mail server, call your ISP and ask them to put a pointer record on your account so the rDNS does, indeed, match up with your domain name. Alternatively, (or if you are running a secondary mail server with multiple hosts), you can just modify main.cf to put your rDNS in the banner:

  1. Edit /etc/postfix/main.cf
  2. Change the smtpd_banner directive to show the same rDNS that MXToolbox shows. In many cases, it will be an IP address separated by hyphens, followed by the FQDN of the node of your provider. 172-165-185-100-yourcity-abbrev-yourisp.tld.

After you make this change, you can restart postfix (/etc/init.d/postfix restart) and re-run the test, and it should pass with flying colors.

Testing Mailbox Deliverability

Your server may be up and running, but we need to confirm that mail can be delivered to a mailbox. We already added a domain and a mailbox to the system above, so now, let’s test and make sure mail can be directly delivered to that mailbox. To do this, we’ll use smtptest (a command line, Windows Program). Once you download and extract the .exe file, use the following syntax to sent a test email directly to your new mail server:

smtptest from=you@email.com to=them@server.net server=mail.server.com

Example:

Notice that we got a queued as serial number. This indicates that the message was successfully recieved by the system, the mailbox was found, validated, and available, and the message has been queued for delivery.

Testing Mail Retrievability

By now we know:

  1. The server is online and listening.
  2. Mailboxes are created in the system properly via Postfixadmin.
  3. We can deliver mail to the system and it is queue for delivery to a user / mailbox.

Now, we need to answer the question: Can I retrieve mail from the server?

To test this functionality, we are going to use www.mail2web.com.

  1. Navigate to www.mail2web.com.
  2. Click on Advanced Login.
  3. Enter in the IP Address of the server, the email address of the test account, and its password.

If you have configured everything correctly, it should log in and you’ll see your welcome email and the test email we sent with smtptest. However, if your are getting a failure here, it is easy to debug.

Debugging Loggin Failures

SASL Failures

Before you do anything else, in order to see what is happening, you need to:

  1. Add the entry log_level:10 to /etc/postfix/sasl/smtpd.conf. This enables you to see the SQL that goes on during authentication. This shows up in /var/log/auth.log.
  2. Add -v to the smtpd application command line in master.cf. (See: Making Postfix more Verbose)

Common failures that I see a lot: SASL failures and MySQL failures.

“SASL LOGIN authentication failed: no mechanism available”.

This happens when Postfix cannot properly communicate with saslauthd. Most likely, you-ve misconfigured (or failed to configure) /etc/courier/authdaemonrc for MySQL. Review the section above that deals with this, and check your settings.

 “authentication failed: authentication failure”.

This latter happens when saslauthd cannot properly communicate with MySQL. Check the configs /etc/postfix/smtpd.conf. Most likely you’ll see errors in /var/log/auth.log or /var/log/syslog that point to an incorrect database, incorrect password, etc…

warning: SASL authentication failure: cannot connect to saslauthd server: No such file or directory

This error happens when you skipped the steps included in the section above titled: How to Configure Saslauthd for our Linux Mail Server, above. Most probably, you have not done the first couple of steps (creating the directory and setting the options in /etc/default/saslauthd. Review all the steps in this section one-by-one, and make sure you did them correctly.

do_auth…auth failure… reason=PAM auth error

You probably didn’t configure PAM to use mysql, or you have erroneous configurations in your /etc/pam.d/smtp file. Search this page for “pam.d”, and re-check that section.

Debugging Other Login Failures

In the shell for the mail server, type:

tail -f /var/log/syslog

This will show you a continuous display of the log for the mail server. Once this is running, try to log in again via mail2web. If there are any errors, they will display here.

Example:

In the example above, you can see that pop3d and imapd both have an “authentication error: Input/output” error. This means that both of those services are having a hard time talking to MySQL to get information about a user credential. So, we’ll double check our configs for that part of the setup.

  1. Verify /etc/courier/authdaemonrc has authmodulelist=”authmysql” enabled.
  2. Verify /etc/courier/authmysqlrc has the correct username and password for MySQL

In this case, both were correct. So, we’ll need to turn on debugging. In /etc/courier/authdaemonrc, find the DEBUG directive, and set it to “3”

Now, we’ll restart Courier using our restart-courier script, and then re-check the syslog while we log in:

Now, you can see the error is with the SQL query we are using to authenticate users. This is contained in the /etc/courier/authmysqlrc file. Upon closer inspection, we can see that the copy / pasted SQL query contains special characters for single and double quotes. So, we’ll re-type the single and double quotes so that they are the correct, ASCII versions of the character, not a font version of the character.

Once we’ve done that, we’ll save the file and restart courier using our restart-courier script. Now, when we test again, we can log straight in!. Before we complete this part of the debugging process, don’t forget to turn DEBUG_LOGIN back to 0 in the /etc/courier/authdaemonrc file, and restart-courier once more.

Testing Email Client Connectivity

Now, let’s configure Thunderbird to make sure that we can send and receive mail (before we tell users and / or clients that they can start setting up their mail accounts!)

1.Start Thunderbird Email Client.

2. Edit > Account Settings

3. Click Account Actions > Add New Account

4. Enter Your name, email address, and password, then click Continue. The program will begin to search for your mail settings. Once this process begins, click Manual Config.

5. You will be presented with an SSL Certificate “Security Exemption” screen. This screen is shown to you because we are using a free, self-signed certificate. The SSL cert is just as valid for safety and encryption as a for-pay certificate; however, because there is no Certificate Authority (CA) to validate it as “true and real” you have to do it manually. Click the Confirm Security Exception Button.

6.

7. At this point, you should see some emails starting to flow in. Try to create a new email, and send it to an external address (like a gmail address). When that email arrives at your gmail address, it will prove that your send capabilities are completely working.

8. You may get an error telling you that you cannot send. If that is the case, close the test message you just composed, and search your open windows for another SSL Certificate Security exemption prompt like you saw in step 5 above. Sending via SMTPS uses SSL, and since that certificate is both different from the previous one you accepted and self-signed, you have to confirm you want to use it to send. Once you have confirmed it, create a new message, resend, and it should go out just fine.

Further Debugging

If you are finding that you have other errors than the common issues covered in this article, be sure to sign in using your Facebook account, and leave a comment asking for help. But, before you do, make sure that you have used the tail -f /var/log/syslog command to capture the logging output of the mail server because that’s the first thing I’m going to ask for. If you’re feeling especially saucy, you could always Google it before you ask.:Virtual Users And Domains With Postfix, Courier, MySQL And SquirrelMail (Debian Squeeze)

How to Setup a Backup Mail Server

by Nathaniel Morales

Introduction: Creating Backup Mail Servers

If you run your own email server, like Microsoft Exchange, chances are, you need to have a backup mail server just in case your primary server goes down. Fortunately, you can setup an alternate / failover email server for free if you have access to a secondary internet connection. If you don’t, it is usually pretty easy to barter with a friend or even a client to offer the service for them in exchange for their hosting the service for you.

This article covers how to setup your email, DNS records, and servers to use a failover topology in order to ensure that the flow of email and information never stops even if your server goes down.

Setting Up the Redundant Email Server

First, you’ll want to configure a minimum of two (2) email servers, which are on different Class C networks. If you’re wondering how to ensure that you have two different networks, here’s the easy way to all but guarantee you did it right: make the alternate server at a different location with a separate internet connection at least 1 mile away. (Ideally, it would be a different ISP as well, but that’s not always available).

Both of the mail server locations will require public, static IP addresses. The mail server may be behind a firewall or directly on the public, static IP address. Since email uses TCP, it will traverse routers, NAT, and firewalls very well.

Follow our guide on creating an open source email server to setup a mail server at your alternate location. Once you have created the server and fully tested it, return to this guide to configure your DNS to utilize the new servers.

Configuring DNS

DNS is responsible for telling remote mail server where to deliver mail for your domain. The DNS system uses an MX record (mail exchange) record to determine where an email should be sent. MX records are listed in order of priority from highest to lowest (ironically, the highest priority server has the lowest priority number. So, 10 is a higher priority than 20, and 20 is a higher priority than 30, and so on).

Create an A record in your DNS that points to the primary mail server. In this tutorial, we’ll call this mail.yourdomain.com. Now, create a secondary A record called backupmx.yourdomain.com.

Now that the two A records for the systems have been created, we need to tell the DNS system that 1) they are mail servers, and 2) which order in which they should be accessed. We’ll do this with MX records.

Let’s create two MX record entries: one for our primary mail server, and one for the secondary server. (Godaddy entries shown below):

So in case of hardware failure, users can still have access to all their mails. Is that possible?

When server A goes offline, mail gets directed to Server B because it is the secondary MX record. In regards to your request, there are several ways to accomplish this. MS Exchange does it nicely with the replication service, but that’s outside the scope of this site. If you are using maildir as your backend (See: /servers/how-setup-linux-mail-server-debian-ubuntu), then you can use rsync to sync the mail directories. This is tricky to setup, mind you.

What I do, however, is simply allow users to access their email via SquirrelMail on the secondary (backup) server so they can operate while the main server is put back online, then use fetchmail to get mail from the secondary server and suck them back down to the primary server so even though the email was sent to Server A while it was down, and stored by server B, all the mail is once again available on server A.

vim: Your Linux Editor of Choice

The most important thing you need in any Linux system is a text editor. All configurations are controlled by text files in Linux, and so without a text editor, you cannot get anything to work.

While there is a lot of opinion out there on what editor you should choose, I recommend vim. While many people say that vim is only for hard-core administrators, I recommend it because it is universally available on all systems. Its predecessor (vi) even comes installed with all versions of Linux.

Many people object to vim because it is difficult to learn. There are no menus or gui hints, and all the commands are hotkey based. For many people (like me) that makes it incredibly attractive, but for new users or people migrating from the hand-holding world of Windows, this can be a challenge. Fortunately, you only have to know a couple of commands to put vim (or vi) to work for you.

Installing vim

By default, only vi comes with your Linux operating system. So, you’ll need to install vim with the following command:

sudo apt-get install vim

Starting vim

To start vim, you need only type vim followed by the file you want to edit. In this example, we are going to create a new file called helloworld.txt:

The Interface

The interface has several key parts, which are shown below. One important difference between vim and other editors is the tilde ( ~ ) that shows you where the file ends. In our example, this file is empty, and so you see a series of ~ characters beneath the cursor, which visually indicate where the file ends. Also, the file’s various status indicators are shown at the bottom of the window. They are (from left to right) the file name, a new file indicator, the cursor position (row, column) and the percentage of the file that is currently viewable.

File Editing Basics

There are really only three commands you need to edit files in vim: i, w, and q.

The ‘i’ command sets the editor into “Insert Mode,” which allows you to insert text into the file. Use this command to change files or add information to files. Note: it will only insert text, and will not overwrite text. So, to delete text while in insert mode, you’ll need to use the backspace or delete key.

Once you are done adding / editing text, press the escape key (ESC) to exit insert mode.

Saving the File and Exiting Vim

File operations are done in the command console, which you can access by typing a colon ( : ). Saving a file (writing a file to disk) is accomplished by typing :w and pressing enter. This opens the file console, and issues the write command.

Quitting the file is done much the same way. Use the :q command to quit the program and exit to the shell.

As a shortcut to write changes and exit vim, you can combine the commands into :wq.

Editor Configurations

Vim is unbelievably powerful. Most users only use a fraction of what it is capable of doing. It has a built in macro recording feature, code and syntax highlighting among other powerful features.

There are two features that I highly recommend you use when working with vim, and one that should be turned off.

Turn on:

  • Color scheme. Changing the color scheme makes files easy to see and work with.
  • Auto Indent. This automatically indents files to make them easier to format and read. For example, if you make one line indented for formatting reasons, autoindent will automatically set the next line at the same indent depth, which relieves you from having to tab over for each line.
  • Syntax highlighting. This changes the color of reserved words for various programming languages and system files so that you can easily see what words the system recognizes.

You can manually turn these on using the following commands:

:colorscheme blue
:set autoindent
:syntax on

Turn Off:

  • Word Wrap.

Configuration files frequently get longer than the screen can display. When dealing with human readable text, like a letter, word wrap moves the line of text down to the next line and neatly formats it. However, in the computer world (especially when dealing with programming, scripting, or configuration files) word wrap can make those files difficult to read and decypher. Subsequently, I recommend you turn word wrap off to leave commands in-tact and on a single line when they are supposed to be on a single line. Use this command to turn word wrap off:

:set nowrap

Setting up Auto Configuration with the .vimrc File.

No one wants to type 5 commands to setup vim every time they start the program. Fortunately, there is a simple way that you can configure vim to set itself up every time you start the program. The .vimrc file is like the autoexec.bat file in Windows, except instead of running whenever the system is started, it is run whenever the vim program is started. Simply create this file in your home directory, and put the commands we listed above in it, save the file, and vim will configure itself just the way you want it whenever it is started up:

  1. At the command prompt, type: vim ~/.vimrc, and press enter to edit the .vimrc file in your home directory.
  2. Add in the commands we listed above.
  3. Save the file, and exit vim (use the :wq command).

Here is what my .vimrc file looks like: