Shicks! - Manual
This manual describes how to install and use the Shicks! POP3/SMTP server, version 2.0
The first section gives you some background on what Shicks! is, and what it can do for you.
What is POP3?
POP3 is a standardized internet protocol for retrieving mails from a mailbox. POP3 knows about users and mailboxes and how to read mail, but it doesn't know anything about how to send mail. This is why in most mail programs, you have to specify both a POP3 and a SMTP server.
Technical Note: The standard is defined in RFC 1725, which can be found online an http://www.ietf.org/rfc/rfc1725.txt. TCP/IP reserves Port 110 for POP3 servers.
What is SMTP?
SMTP is a standardized internet protocol for sending mails to a local or remote mailbox. SMTP knows how to send mail, but it doesn't know anything about how to read mail, or even what a mailbox is.
Technical Note: The standard is defined in RFC 821, which can be found online an http://www.ietf.org/rfc/rfc821.txt. TCP/IP reserves Port 25 for POP3 servers.
What is Shicks!?
Shicks! is a combined POP3/SMTP server. Actually, it is two distinct servers for each protocol, plus a MySQL database that is the central storage for both servers. So, the POP3 server can read the messages stored locally by the SMTP server in the MySQL database.
The MySQL database holds the mailboxes defined on your Server. For each mailbox, you have a username and a password. When you logon to the POP3 server, the username and password are checked against the database. Here are the most common operations:
- When you send a message to another local mailbox, the SMTP server does the following: 1) A file containing a copy of the message is created on the server. 2.) An entry in the database is made that says that this file belongs to the receiving mailbox.
- When you look for messages, the POP3 server does the following: 1) Look in the database if a message has been received, 2) Read the data from the harddisk.
- When you send a message to a remote user, the mail is deferred using a "relay SMTP server" (a remote SMTP server), which usually is your ISP's SMTP server.
- When a remote user sends you a message, the first and most important step is for the remote user to target the local SMTP server (See below). Then, everything proceeds as if you send a local message.
To be able to receive mails from remote users (users on the internet) their SMTP server has to find your SMTP server. This is done by resolving the name of the e-mail address. This is done by looking at the part of your e-mail address after the '@'. For example, an e-mail address "email@example.com" will be resolved by looking up DNS entry for "foo.com" and connecting the SMTP server there. Actually, most of the time your ISP's SMTP server will do the name lookup for you.
So, in order to receive mails from remote users, your server must have a known DNS entry on the internet
Note that in Shicks!, in contrast to other mail servers, local addresses can be anything. If the SMTP server recognizes a local address, it sends the mail to the local recipient - no matter what. For example, even if your DNS entry is "foo.com", you can receive messages for "firstname.lastname@example.org" as long as the messages are sent from internal users to internal users. Remote users will not be able to access "email@example.com", because most likely "bar.com" will point to a different SMTP server.
But my machine is behind a firewall!
Well, this probably means you cannot receive remote mail. But Shicks! might still be usefull for you, if only to be able to better control mail sent inside your organisation without going to your ISP for every message. For example, if you share a DSL connection with two or three household members, you can easily use Shicks! to setup a local mail system. Using "full blown" mail servers would be way out of scope for your needs, probably.
Before you start, you need to check that your system meets the following requirements:
- Python 2.0 or higher.
- A MySQL database running.
- The MySQLdb module for python must be installed.
- A webserver. I use Apache, but other common webservers should work fine, too, provided they can run Python cgi-scripts.
- The user must have the rights to access TCP/IP ports 25 (SMTP) and 110 (POP3). Also, no other application must be using these ports! On Linux, you'll get a warning if the port is already in use - on Windows, it will "succeed", but you'll not get much mail.
- The operating system must be Linux, or Windows. For Linux you need root access. Other OS might work too, provided that the above mentioned conditions are met. Feedback welcome!
- The number of users/mails per day should be < 100. This is BETA software and has not been used for larger sites yet. (Although you're welcome to try it and give me lots of kudos if it works :)
Creating the database
The first step is to create the database. You can do this at the shell prompt using
mysqladmin create maildb
Here, "maildb" is the name of the database. You can choose a different name if you want, but then you'll have to edit the file "maildb.py". If the database admin account has a password, you can use the "-p" command line option to specify it. You would say:
mysqladmin -p create maildb
and then be asked for a password.
The next step is to create the structure of the database. For this, you need to connect to mysql.
- From a command line, start "mysql".
- Connect to the newly created database.
- Execute the SQL script "maildb.sql", which should be part of your Shicks! distribution.
NOTE: Before overwriting the database, please do check if a database named "maildb" already exists and is in possible use by another application.
Here is a sample session on a Win32 machine:
D:\scripts\shicks> c:\mysql\bin\mysql.exe -p Enter password: ************** Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 1 to server version: 3.23.43-nt Type 'help;' or '\h' for help. Type '\c' to clear the buffer. mysql> connect maildb; Connection id: 2 Current database: maildb mysql> source maildb.sql; Query OK, 0 rows affected (0.05 sec) Query OK, 0 rows affected (0.01 sec) Query OK, 0 rows affected (0.00 sec) Query OK, 0 rows affected (0.02 sec) Query OK, 0 rows affected (0.01 sec) Query OK, 0 rows affected (0.00 sec) Query OK, 0 rows affected (0.03 sec) Query OK, 0 rows affected (0.03 sec) Query OK, 0 rows affected (0.00 sec) Query OK, 0 rows affected (0.02 sec) Query OK, 0 rows affected (0.05 sec) Query OK, 0 rows affected (0.02 sec) mysql>
You can omit the -p option if your database doesn't need name and password. If everything went OK, you now have the data structure set up completely.
Accessing the database from python
IF you have protected your MySQL database with name and password (and you really should), then you need to edit the file maildb.py. Have no fear, its a very small and simple file that holds your name and password for the database at the top. You need to edit this, otherwise python won't be able to connect to the database. More instructions can be found in the sourcecode.
If something goes wrong
If you had problems in any of the steps in this section, you should verify:
- That the MySQL process is up and running. If possible, try to connect to the database with some tool like phpMyAdmin and see if it works.
- Check if mysql and mysqladmin are on your executable path. (%PATH% for Windows, $PATH for Linux)
- Check if your database is protected by name and password.
Installing the Shicks! Administrator Tool
First of all, open the file "shicks-admin.py" in your favourite editor. You need to change the line that reads
#! c:\\python21\\pythonw.exe -u
to point to the executable of python. (The linux distribution already uses /usr/bin/python as default, but just in case you have installed python to a different directory you should still check if that line is ok).
You need to copy the files
to your servers' cgi-bin directory. The sourcecode currently assumes that the webserver path will always be
If your webserver has different settings (hint: IIS probably does), you need to either change the source or provide a name mapping between /cgi-bin/ and the directory you install shicks-admin.py to.
Once installed, open a webbrowser and go to http://localhost/cgi-bin/shicks-admin.py . You should see a welcome screen saying "Shicks! Administration".
Check the settings. The path has some defaults, you might want to change these to fit your installation. Note that the path will be created if it doesn't yet exist.
Add at least one user, having one email to the system, so you can later check if it works.
TODO: Add documentation for the admin tool, even though its pretty self-explanatory. Volunteers welcome :)
If something goes wrong
If you had problems in any of the steps in this section, you could take one of the following actions:
- Verify that the webserver is up and running
- Consult the logfiles of the webserver!
- Check if you've changed the path to python in step #1
Starting the servers
Well, this is the easy bit: Just run POP3Server.py and SMTPServer.py. On Windows, if you have the ActivePython distribution installed, you can simply doubleclick the files. (You'll be presented with two console windows saying "POP3Server/SMTPServer is up & running"). On Linux, you should start the apps from a xterm.
Linux only: Determining the installation path for python
The python scripts come with the header line "#!/usr/bin/python". If your copy of python 2.0 resides in a different directory (or is named differently, e.g. "python2.1"), you must modify the script headers. An alternative would be to manually start the files by using a syntax like this:
firstname.lastname@example.org $ /usr/local/bin/python2.1 SMTPServer.py etc.
Installing the Shicks! cgi-Administrator
Copy the file "shicks-admin.py" to your httpd/cgi-bin directory and mark it as executable (the last step is only required on linux). Then, point your browser to http://localhost/cgi-bin/shicks-admin.py and off you go.
Editing the first-time configuration.
When you first started shicks!, it created some default configuration entries. You should now take the time to edit them to your personal needs. Go to the Shicks! Administrator and click on Configuration. The following settings are available:
- Logfile directory: This is the directory where the shicks! logfiles are kept. It defaults to /usr/share/shicks/log on Linux, and c:\shicks\log on Windows.
- Relay SMTP server: This is the address of the relay server you want to use. Normally, this is the SMTP server of your provider.
- Directory for mails: This is the directory where the mails are kept. It defaults to /usr/share/shicks/mail on Linux, and c:\shicks\mail on Windows.
- Admin account: This is an email address, where administrative messages will be sent to. Example: email@example.com
- Local domain: This is the name of this domain. Example: acme.com
- Unknown account: This is an email address for messages sent to the local domain but without a valid recipient. Example: firstname.lastname@example.org
- Local IP-Range: This is an (optional) list of local IP addresses. Local IP addresses are not scanned for spam when sending, and are allowed to send to any reciever. Example: 126.96.36.199-188.8.131.52
- Strict relay test: If set to 1, either the sender or ALL receivers must be local addresses. If set to 0, either the sender or AT LEAST ONE OF the receivers must be local addresses. Defaults to 1.
shicks! tries to automate spam detection by analyzing incoming subject lines. The keywords recognized are in the file "spam_data.py". You can edit this file if you have new subject lines.
Checking if the mail works
First of, configure your email client. Email client configuration varies widely, so I cannot give a general manual on that; but all Email clients should have configuration entries for POP3 server and SMTP server, respectively. As server name, choose the name of the machine you let both servers run (for testing, this can be localhost [127.0.0.1]).
Then, you should check, if:
- You can send yourself a plaintext message
- You can send yourself a mail with attachment(s)
- You can send yourself a HTML e-mail
- You can send mail to a mailing list (and have it received, e.g. if you send it to a mailing list you yourself are a member of)
- At this point, also look in to the Shicks! Administrator again, to see if the mail statistics are correctly.
Provided that you've set up a relay SMTP server, you can try the following:
- You can send e-mail to someone outside your organisation.
- You can receive e-mail from someone outside your organisation.
If everything works fine, you've got yourself a shiny new Mailserver :)