For others that are interested in the Sun ONE software

This file is intended primarily for people outside Rutgers who are interested in how we did certain things. It isn't full documentation on what we've done: there's lots of work on administrative tools using PHP. Other sites are welcome to a copy of that, but it might take you more time to figure out everything I've done than to do appropriate tools yourself. The PHP code uses the mail and calendar command line tools, and hacks on the LDAP database. PHP has excellent support for LDAP, so doing this kind of tool is fairly easy.

Use of the system over NFS

Our system is based on a Network Appliances file server. We use this as our standard storage management, and didn't want to do something different for this project. We particularly like the Netapp snapshot feature. Snapshots are our primary backup medium. We think the Netapp's have enough redundancy (we implement them in pairs, which theoretically have no single point of failure) that we only do tape backups once a week. We have a feature in the administrative tool to let people restore calendar files from snapshots.

Sun doesn't support this software over NFS. There seems to be some reason for this, although it wouldn't be hard to fix.

First, you shouldn't even try to access the same file system from more than one machine. We've put our installation on NFS, but at any given time only one system uses it. The Sun code does various kinds of locking. It assumes that one system is controlling the locking. Things are likely to get very confused indeed if you access the files from more than one system. (It should be safe to do mail on one system and calendar on the other. They use different sets of files. It should also be safe for system administrators to look at configuration and log files from another system.)

Now, we have run into several issues that you should know about before trying this:

1. The mail software gets into loops when trying to delete a folder. This is easy to duplicate yourself. With a file system that's on NFS

mkdir foo
cat >foo/bar &
rm -rf foo

The rm -rf will go into a loop. If you truss it, you'll find that it is unlinking files named .nfsXXX. Here's what's happening:

bar is open. On a local Unix filesystem the rm -rf would remove the directory entry for bar, but the inode would continue to exist until the file is closed. On NFS, the protocol requires a directory entry somewhere. So when you unlink a file that's open, the client system renames it to .nfsXXX, where XXX is a random thing to make it unique. The .nfsXXX file is in the same directory that the original file was. Typical code to delete all files loops through the directory unlinking everything. But the .nfsXXX entry goes at the end of the directory, so the loop picks it up. It unlinks it, creating yet another .nfsXXX, which is picked up the next time around the loop. ...

Anyway, the mail system will trigger this problem, in imap, web mail, and tools such as mboxutil. There is a workaround: interpose code around unlink so that if someone tries to unlink a .nfsXXX file, you move it to another directory (I put a directory "deleted" at the top of the file system) and then unlink it. I claim that's what the OS should have done in the first place.

To do this, you need two things: (1) a sharable library that interposes code around unlink, and (2) environment variables set when you run any mailsystem code.

Here is a link to the source directory for the interposition code. You'll need to modify the code to use the appropriate locations for your system (or fix it to find the appropriate location automatically).

Here is what you need to do in order to use the .so files:

In start-msg, insert the following near the beginning:

LD_PRELOAD_32=/army/sunone/lib/unlinkx32.so
LD_PRELOAD_64=/army/sunone/lib/unlinkx64.so
export LD_PRELOAD_32 LD_PRELOAD_64

Of course you're likely to put the .so files in a different location.

You'll also need to make sure that anyone who uses mboxutil to delete folders has these environment variables set.

2. There are various minor annoyances in doing installations. For some reason the creat system call doesn't seem to work for files that are setuid. This means that Sun's patch installation scripts get failures for all setuid files, and a few other things don't work. You have to be alert for error messages and be prepared to copy a few files manually. This doesn't seem to be a problem for any of the software you use in normal operation.

3. We had a file system failure (the first Netapp problem in 3 years -- it took the file system offline when a second disk failed while it was still recovering from a disk failure). The system wouldn't come up automatically afterwards. The startup processes for each major component went into loops. Doing truss, they were all writing repeatedly to Berkeley db log files. We don't actually know that this has anything to do with NFS. I tried copying the system to a local disk, and the same problem occurred. But it's still possible. Anyway, for the moment we don't have any better solution than to remove the log files when restarting after a failure. See my document on recovering from failures.

Other than this, we haven't run into any major problems that seem to be related to NFS.

SSL for the Calendar

The Sun calendar system doesn't support SSL. We use stunnel in front of it. Here's an example stunnel configuration file:

cert = /usr/local/etc/stunnel/stunnel.pem
chroot = /usr/local/var/run/stunnel/
# PID is created inside chroot jail
pid = /stunnel.pid
setuid = nobody
setgid = nogroup

[calserv]
accept = 1025
connect = 81
TIMEOUTclose = 0

You really want the TIMEOUTclose line for any stunnel service that goes in front of a web server.

The .pem file comes from whatever process you use to generate your certificates. It must have both the private and public keys:

-----BEGIN RSA PRIVATE KEY-----
MII...
...
...=
-----END RSA PRIVATE KEY-----
-----BEGIN CERTIFICATE-----
MIIC7zCCAligAwIBAgIDCeOvMA0GCSqGSIb3DQEBBAUAMIHEMQswCQYDVQQGEwJa
...
fdkigPO5JNDmIHqXvf4hgknyZgy9NqSiVLqE4awW1Qla/UA=
-----END CERTIFICATE-----

You must tell the calendar server that it's being accessed through a frontend, so that it adjusts the URLs it gives users.

!
! Base Server Addres, eg "https://proxyserver/"
ui.base.url = "https://rulink.rutgers.edu:1025"

Note that we also use stunnel in front of web mail as well. There are two reasons:

1. We use single signon. Single signon only works if all access comes from the same IP address. The calendar server sees its users all coming from 127.0.0.1, since they go through stunnel to get to it. So web mail must go through stunnel as well.

2. Several browsers complained that web mail's SSL used low grade encryption. This was not an issue with stunnel. It isn't a certificate problem, because I was using the same certificate with both.

Outlook free/busy

I've had several requests for the PHP script we use to let Outlook access free/busy information. This is based on a script from Sun, but it's been rewritten in PHP. There were both security and functional problems with Sun's original shell script.

Here's getbusy.php. The script is fairly well commented, so there isn't much else to say. Look at our documentation for Outlook users. There are a number of caveats about using this.

Macintosh ical access

I've done a quick script to let Apple's iCal application subscribe to calendars here. See the user documentation on iCal access for details.

Here's ical.php. Note that the script checks to see what port you're coming from. It asks for a password if you're coming from the port we use for SSL. That lets users access public calendars using normal port 80 without authenticating, and asks them to login for SSL.

iCal's handling of SSL is really, really buggy. I don't know how many people will find it usable. I do.

The reason we require that the user type CALID.ics, and strip the ics, is that Apple's iCal doesn't seem to recognize a web page as a calendar unless it ends in ics. I guess they're into PC emulation. [Windows tends to ignore MIME types, using the extension instead. I'm surprised to find Apple doing this.]

My thanks to Chris Adams' web page for information that was critical in making the SSL support work. [This web page isn't viewable with Netscape 4.7.]

How to remove microsoft executables from email

One of our departments is moving from a system where any attachment with a file ending in .com, .exe, etc, is removed. We use anti-virus software (on a separate appliance), but they're worried that they will get viruses during the brief period (up to a week at times) before the av software gets protection for a new virus. Most people think they're paranoid. I think they're probably right.

Anyway, I figured out how to make the mail systems remove these attachments, but only for people with a certain LDAP attribute. Figuring it out was difficult enough that I thought I should tell others.

1) imta.cnf, create a separate channel for mail that needs checking. I called the channel ims-msa. It's basically a clone of ims-ms. job_controller.cnf uses wildcards, so any channel starting with ims-ms will be processed the same way.

! ims-msa
.ims-msa-daemon $U%$H.ims-msa-daemon@ims-msa-daemon
!
! here's the code that actually routes mail to ims-msa, based on an ldap attribute
ims-ms-daemon $U@${USUB2,$U}
ims-ms-daemon $U@${USUB1,$U}

! this goes down in the section with other channel blocks
! for Direct LDAP mode remove "filter ssrd:$A" from the keywords below
! the next two lines are actually one long line
ims-msa defragment subdirs 20 notices 1 7 14 21 28 backoff 
  "pt5m" "pt10m" "pt30m" "pt1h" "pt2h" "pt4h" maxjobs 1 pool IMS_POOL fileinto $U+$S@$D
ims-msa-daemon

2) In mappings, add code to check for the ldap attribute. This works with the rules above to move messages into the ims-msa channel.

USUB2
  *+*$%* $C$0%$1
  *$%*  $Cok:$]ldap:///o=$1,o=rutgers.edu?uid?sub?(&(uid=$0)(rutest=*))[
  ok:*  $Yims-msa-daemon
  *     $N

USUB1
  *+*   $C$0
  *     $Cok:$]ldap:///o=rutgers.edu,o=rutgers.edu?uid?sub?(&(uid=$0)(rutest=*))[
  ok:*  $Yims-msa-daemon
  *     $N

I'm looking for the rutest attribute. Obviously you can use whatever attribute you like. There are separate rules to handle mail for our main domain and the hosted domains. An address user@rutgers.edu is the main domain. By this point it looks like user or user+folder. An address user@dept.rutgers.edu is for a department hosted on our system. By this point it looks like user%dept or user+folder%dept. The exact setup of these rules will depend upon your directory structure.

3) You need an entry in mappings to route things for the ims-msa channel into the conversion channel. Here's our version:

CONVERSION

  IN-CHAN=tcp_av;OUT-CHAN=ims-msa;CONVERT Yes
  IN-CHAN=l;OUT-CHAN=ims-msa;CONVERT Yes

tcp_av is a channel for mail coming from our av box. In many cases you'll be safer with the following

CONVERSION

  IN-CHAN=*;OUT-CHAN=ims-msa;CONVERT Yes

That might even be best for us.

4) You need the actual conversions file. (All these files are in imta/config.)

in-channel=*; out-channel=*;
  in-type=*; in-subtype=*; 
  in-parameter-name-0=name; in-parameter-value-0=*.bat;
  out-type=text; out-subtype=plain; out-mode=text; out-disposition=inline;
  command="/army/sunone/servers/mailserv/msg-rulink-mail/imta/programs/convtest"

in-channel=*; out-channel=*;
  in-type=*; in-subtype=*; 
  in-parameter-name-0=name; in-parameter-value-0=*.cmd;
  out-type=text; out-subtype=plain; out-mode=text; out-disposition=inline;
  command="/army/sunone/servers/mailserv/msg-rulink-mail/imta/programs/convtest"

etc. for whatever extensions you want to trap.

5) The command generates the text that is put in place of the attachment you removed:

#!/bin/sh
cat >$OUTPUT_FILE <<EOF
An attachment has been removed from this message because it is a Microsoft
executable file. Such files normally contain viruses and other dangerous
content. In case someone else is trying to send you a legitimate 
executable file, please ask them to rename the file so that its name
doesn't end in any of the following extensions, and then send it again.

   bat ceo chm cmd com exe js jse mdb mtx pif reg scr shs vb vbe vbs
   vbx wsf wsh

EOF

6) Set the ldap attribute, in our case rutest. That's done as part of our PHP administrative tools. A departmental admin can set it for all their users.