Filtering Scarab e-mail with postfix filters

Intro

Scarab is a pretty good open source tool for issue management (bug-tracking, or as they like to call it, artifact management) unfortunately its e-mail handling leaves a lot to be desired. With this in mind, I wrote this entire mess of Perl to assure we have e-mail delivered where we want it and not anywhere else.

This will be useful to you if:

You use Scarab.
You use Postfix -- I suppose in theory this can be done with Sendmail as well but I wouldn't even know where to start so not going there.
You know some system administration and some Perl. My instructions probably leave some unanswered questions and blank stares.
You don't care for the lack of flexibility in Scarab's handling of e-mail.

Why?

Because I can! Well, not, not really. I found there are two ways to get e-mail out of Scarab

(a) Everyone in the universe who has touched the issue in question in some way.
(b) A mailing list that by definition includes everyone in the universe that can possibly be interested in an issue from the given module.

This is meant to give us flexibility in how Scarab e-mail is directed, as an example, if the issue was set "toDev" we only want developers associated with the particular module to see an e-mail about it. This eliminates unnecessary e-mail to a QA person that already knows he's directing the issue and saves him/her from assigning a developer to it just to produce an e-mail.

Looking through the scarab development mailing list archives, you may find that this issue has been brought up many times before but there has been no real resolution. This isn't a real resolution, it's just a hack and it was easier than attempting to modify Scarab code to behave properly.

What you will need.

Part I

First things first, creating a bunch of mailing lists that divide our users according to their role in Scarab. This can be done with some simple mailing-list software, but then it requires manual configuration to add and remove users. I wrote a script that rebuilds the lists from Scarab's database on the fly. In short, the script creates flat files containing a list of users for each role/module combination.

Things to do:

Download three files:
Use your favorite editor to enter the correct username, password, database, etc, information in login.sub
Check scarab_mailinglists.pl to verify the settings for two options, $working_dir and $file_ext.
Run scarab_mailinglists.pl and examine the generated lists, file names should be modulecode-Role.lst and contain just one e-mail address per line.
Add aliases to postfix and point them at the generated files. Alias names are important, since this is how the filtering script directs its email.
Example (in /etc/aliases):
DFT-Developer: :include:/home/scarab/files/DFT-Developer.lst
Where DFT is scarab module code and Developer is the scarab role. The included file is obviously the one we generated in the step above.
Optionally, schedule the mailing-list generating script to run at fixed intervals. I run it once a day and manually if it's really necessary (impatient co-workers).

First part done, you should now have some working mailing lists that are based on user's roles in each module.

Part II

Configuring postfix to use the filter. There is only one thing that is common in all e-mails coming out of scarab, the Reply-To header. This is what we filter on using header-checks.

Things to do:

Download mail_filter.pl, you already should have the other two required files from the step above.
Open mail_filter.pl in your favorite editor and modify a bunch of settings in the script, they're cleanly documented.
Create the required table in MySQL scarab database, the sql for it is included in the script, or just copy-paste from here:
```
CREATE TABLE scarab_email (
   id           int(11)      NOT NULL AUTO_INCREMENT,
   module_name  varchar(255) NOT NULL, 
   scarab_event varchar(99)  NOT NULL,
   role_name    varchar(99)  NOT NULL,
   PRIMARY KEY (id)
) TYPE=MyISAM;
```
The comment section of the script also contains sample data for 'default' module (data that will be used by default) which can be inserted into the table.
If there is no per-module or default data in the table, the filter will send e-mail as Scarab originally intended, so it's just wasted CPU cycles.
Configure postfix to know about the filter.
In /etc/postfix/master.cf:
scarabfilter unix - n n - - pipe flags=Rq user=mail argv=/home/scarab/mail_filter.pl $recipient
Make sure the location of the script is correct and use the user your postfix runs under (for me it's mail)!
Configure header_checks to call the filter when it sees e-mail coming out of scarab. Note, this works for both outgoing and incoming email. Double check that the e-mail address is the same as configured in Scarab.
/^Reply-To:.*/ FILTER scarabfilter:dummy
Don't forget postfix reload!

Finito

In theory, this should all now work, of course, as anyone who has done system administration knows, that never happens. Turn up debug in the script, check /var/log/maillog and check the maillog generated by the script (it's configurable in the script as to where it should live)

This script is most useful if you can customize it to your particular setup (scarab modules, roles and events). It's all documented pretty clearly in the script itself, so please, make sure to read the comment section if you're going to use it.

Questions, comments, mockery? E-mail me at kasia@ this domain.

Last updated Sun Feb 8 21:47:54 EST 2004