Help on Mail to Web Setup

For help on Mail-to-Web usage, check out http://www.slb.com/admi/util/help/help_on_using_mail_to_web.html

Here are the notes required to do a setup of mail-to-web on your server. This setup assumes the worst case we support of a multiple web server on a single machine with a setup that follows our directory setup exactly. In cases where the structure does not match, it is likely possible to get it to work, but you will need assistance from Corporate Houston Web Information Group (likely).

  1. Setup Instructions
  2. Description of the Various Processes
  3. Control Files
  4. HTML Data Can Be Included
  5. Attachments Can Be Used
  6. Mailkey Can Be URL
  7. Area Indexformat.html Controls are Editable
  8. Default Mailkey for each Area
  9. Node List Can Skip Area if areatitle is Blank

Setup

Automatic Setup

The following approach uses the big hammer approach. If you already have a full setup with some special modifications, this may overwrite it. But, by design, it is supposed to add the needed files and no overwrite those files that have changed since these files were last created. But, FOR SURE, there could be problems. On the "good" side, it often works!.

This must be done as root. 

#cd /ftp/web/build/webmail
#cd bin
#./buildsite xxxx xxxx must the name of your web user - srv1 in example
There should be no errors reported, but if you have some special files, it may note that it was unable to overwrite newer files on your server. If you want the full build exactly as designed, you will need to delete those shown as not being replaced and then run buildsite again.

Once this has been done successfully, you may need to edit the local configuration files in /ftp/db/conf/xxxx (srv1 in this example) to have the right defaults for your new server. If you are doing a re-install, then you may need to compare /ftp/db/conf/srv1/back/*.html and /ftp/db/conf/srv1/*.html to see what needs changing.

Manual Details

This section can be ignored if the automatic buildsite worked without errors. But, it may help to step through it to understand the setup built by buildsite.

Description of Various Processes

Mail arrives to be put on server

Omnes Excite Search Setup

Structure

Our version of Excite (substantially modified) requires the following setup (again, xxxx is the server WebKey)..

Setup

Untar the omnebuild.tar file in "/ftp/web/build/webmail/".

Copy the given slbp.conf to srv1.conf and slbp.cf to srv1.cf for your srv1 server setup (in /ftp/web/excite). Copy do_slbp.pl to "do_srv1.pl" and ensure execute privileges are set.

When initially installing a server, create the areas listed above and set the ownership listed. All files in /ftp/web/excite are owned by web except for srv1.conf and srv1.cf which must be owned by the web owner (srv1) (I believe it will work if they are owned by web as long as srv1.cf does not need to be rewritten by the system - should not...).

Copy admi/util/sear area, data, subdirectories, etc into your server at /ftp/web/srv1/admi/util/sear (owned by web). No writing will take place in this area by the web server.

Setup the /ftp/db/excite/srv1 and /ftp/db/excite/srv1/collections area (owned by srv1).

Change to user srv1 and cd /ftp/web/excite.and then execute do_srv1.pl . It should build your indexes if you have data at /ftp/web/srv1 .

The files to be found in /ftp/web/excite are (serv1 is your WebKey):

No files are needed under /ftp/db/excite/srv1 but the directory needs to be there for each webserver (plus /ftp/db/excite/serv1/collections) and need to be owned by the web area owner.

Copy at_search.pl above to /ftp/web/cgi-bin area for use by web forms.

IF you hare using a standard setup, then you can ignore do_XXXX.pl and just use do_index.pl. Copy it (or check that some previous web build has already done so) to /ftp/web/bin/do_index.pl and ensure that it is executable (775). This shared version will build an index named for your webserver when you run it as your web server's user (XXXX).

IFF you have modified do_XXXX.pl to be specific to your needs and it is no longer the generic version that indexes all your webserver, then you must copy do_XXXX.pl to /ftp/web/bin/XXXX/do_XXXX.pl (replace XXXX with your webkey). Ensure that it is set to be executable (775).

Use "crontab -e" (when running as XXXX user) to add the line (depending on whether you can use the generic version of you must use the version specific to your webserver): 

50 00 * * * /ftp/web/bin/do_index.pl 
or
50 00 * * * /ftp/web/bin/XXXX/do_XXXX.pl

Excite Bugs - How to Survive Indexing

The excite indexer has some severe bugs. All (except 1 to date) can be worked around iwth knowledge. Here is our best knowledge on how to deal with them.

To date, we have made indexing work on up to 200000 files.
When over roughly 50,000 files, the "Search by Example" buttons fail to work due to some internal limits being exceeded in the binaries.

Even small collections can fail during indexing due to "empty" files. Note that "empty" needs description - files of 0 length (left over junk) are "empty" and should be deleted in any case as they are not userful on a web. But, note that any file that contains no text other than in HTML constructs is "empty". All that is needed is one \  somewhere in the file.

Debugging for files of this type requires either a special script that searches for such files (usually found in frame implementations) or multiple trial indexing passes. There is a log file (in /ftp/db/excite/$webkey/*.log) after any excite indexing has run or failed. The last line shows the file that caused the crash during indexing. If you edit this file, you will find that it contains no text other than the HTML constructs (often a table with only graphic buttons). Just add an \&nbsp; before the ending </html> (or before the last </body> if used as it should be). Then rerun the index (/ftp/web/bin/do_index.pl on most Corporate machines). When it fails again, repeat until it completes successfully.

Control Files

WebMail has some serious control file areas (for each xxxx server).

Mailkey can now be the URL path - 10-Jun-96

As detailed above, the mailkey can be replaced with the actual URL path (without the "http://IP.Address.Part"). In the example above, "/serv/gen/" (note the starting and ending "/") can be used in place of servgen. If the keyword has not yet been defined in webtable_$webkey.html, but the area is setup with the right permissions and indexfiles, then the URL path can be used intead.

Each Area's Indexformat.html Controls are Editable - 01-Oct-96

From each list of files (indexed list), there is a button with a Vulcan ship or a sideway's "3" that will allow you to enter a password for the area (the mailkey for the area is already filled in) which will then show the settings in the indexformat.html file and allow each of them to edited (for most of them - some are not set to be editable as yet).

Default mailkey for each area - 30-Nov-96

Each area automatically has a mail that is the full URL path without the unix directory slashes. In other words, serv/proj/repo would get an automatic mailkey (beyond any specified in the indexformat.html file) of servprojrepo. This is only true for areas that have at least one legal mailkey specified in the indexformat.html file.

If areatitle is blank, the Node list will skip the area 14-Apr-97

No longer true? IWF
If you do not fill in some non-blank data for the areatitle parameter in the area's indexformat.html file, then the node list above that area will skip that area in the list. This can be used to advantage to create areas that can be found by setting the URL directly but cannot be seen on the automatic list above. For example, one can set up dum1 and dum2 directories that are invisible until the indexformat.html file for the area is updated with a non-blank areatitle entry.
http://www.choral.org/admi/util/help/.html


o o o -> -> o
fetterley@houston.sns.slb.com