This section contains technical information about the news mechanisms of concern to newsgroup moderators, including standard news headers, dealing with submissions, and generating special purpose headers to better serve your newsgroup.
A moderated newsgroup is marked as such in the news transport software (most often with a trailing "m" flag in the active file of news systems). What this means is that articles must be approved before they are accepted in the group. When an article is posted to a moderated group, the news transport software will mail the article to the moderator [See Section 5.1 Mailpath Usage] for approval and actual injection into the news system.
Once the moderator has received a submitted article in the incoming mailbox, (and if necessary has edited the article's content) the moderator needs to process the article's headers a bit before actually posting it to the group. The descriptions below explain usage of the various headers.
Technically, all that the moderator has to do is add an "Approved" header, and repost the article. The only thing that differentiates an "approved" from a "non-approved" posting is the existence of the "Approved:" header. The news transport will then accept it and transfer it to other machines.
New moderators, especially those not familiar with news mechanisms, may need to refer back to the list of standard news headers (Section 5.2) often as they familiarize themselves with the moderation process.
When a net.citizen posts a message to a moderated newsgroup, the news software looks up the moderator's submission address. The software then mails the unapproved article to the moderator for approval and injection into the news system.
To make this work the mailpaths file is used on B News or C News systems. INN uses the moderators file and the inn.conf file to provide the same functionality.
The list of moderator addresses can change almost daily and trying to keep up with it can be a job in itself at times. For that reason the mailpaths file can be configured to send all unapproved submissions to moderated newsgroups to a site which has volunteered and been approved as a mail forwarder for Usenet moderators. In almost all cases it is best to configure news software to forward unapproved articles to one of the established mailpath forwarders.
David Lawrence <tale@isc.org.net>; maintains the periodic posting
"How to Construct the Mailpaths File"
It describes the syntax of the contents of the file and how to construct it for your B News or C News system. It also lists the sites that are maintaining a current listing of moderator addresses and are acting as mail forwarders for the Usenet moderator infrastructure.The article is periodically posted to the newsgroups news.lists, news.admin.misc and news.answers.
Articles posted to moderated newsgroups, like all other news articles, must conform to the article specifications of the Usenet news system, as described in RFC 1036[1]. The list below explains the standard news headers as they pertain to moderating Usenet newsgroups, though if there is any doubt about the specifications of a particular header RFC 1036 should be consulted. [See Section 5.10 for more information on obtaining RFC 1036 and other NetNews related RFCs and documents.]
There is no preferred order of headers. Compliant software should accept the articles with the following headers in any order.
Any article posted to a moderated newsgroup must contain an Approved: line. Always sign the approved line with your electronic address. The software won't care what is here, but in case something goes wrong, the community will know who approved the article.
Some moderators sign the Approved: line with the moderator's submission address, so that any comments-to-the-moderator tend to get routed into the moderation mailbox.
A sample Approved: line:
Approved: kent@landfield.com (comp.sources.misc)
If an article has been approved by the moderators of different moderated groups, the moderator with final approval should try to put the other moderators on the Approved: line as a way of documenting that it was approved to appear in multiple groups.
A sample Approved: line marking approval in more than one group:
Approved: kent@landfield.com, tale@isc.org
While showing multiple approvals is not required, it is informative to the readership and common courtesy to the other moderator(s) to do so.NOTE: Beware of approving cross-posted articles. Refer to "Section 5.2.8 Newsgroups: Line", "Section 8.5 Dealing with cross-posted articles" and "Section 15.6. Cross-posting to other moderated groups" for a discussion of the problems.
Strip the Date: header from submitted articles, or change it into something like X-Original-Date:. Do not include an X-Original-Date: header without a good reason. For example, an article might refer to "today's New York Times", or might mention software "uploaded to an FTP site today. Proving your timeliness isn't a good reason unless, for some reason, it has been in question.
The problem with keeping the original Date: header is that it might badly confuse the news posting software, or some latency could cause the article to be unnecessarily rejected at sites, especially when the date was completely wrong.
The Distribution: header should be stripped from any submitted article. You should try not to post things of a definite local nature to world-wide groups with the current state of network news propagation.
Unfortunately, using the Distribution: header rarely produces the intended or desired results. An article posted with a restrictive Distribution: header is almost certain to be propagated far beyond the intended area, and will be equally likely to miss some sites that would be interested in that region's news, and might even be physically located in the intended target zone.
In addition, many articles are posted with "na" (North America) or "usa" (U.S.A.) distributions because of poorly-thought-out software defaults, rather than any conscious decision by the poster. Many non-North-American readers are annoyed by this needless limitation on what news reaches them.
In theory distributions work as intended, but in practice, due to lack of verification by posting agents, misconfigured news transport agents, wide-area sites which pick up all news regardless of distribution, and inadequate controls on the names of the distributions, they are relatively useless.
Moderators should consider adding an Expires: header if the information being posted has a limited period of usefulness. For example, a Call For Papers (CFP) posted to the group news.announce.conferences might be valid only until a certain date. The Expires: header can then be set to expire the article the day after the deadline specified in the CFP.
Many sites with limited news retention times keep articles with explicit Expires: headers online longer than the default time period, so an Expires: header can help keep periodically posted information readily available to readers at all times.
Your use of the Expires: line should be documented in your group's periodic policy posting.
If you have a policy of directing all followups to the article submitter, or if the submitter requests it, use the header line
Followup-To: poster
The news reader software will then email followups to the address listed in the Reply-To:, and if non-existent, to the From: address.In some cases it might be appropriate to place the name of an unmoderated discussion group in this header.
For example: Comp.sys.amiga.announce does not carry any discussions. Articles there contain the line
Followup-To: comp.sys.amiga.misc
With this header, when a reader with compliant news software tries to followup to an article appearing in the group, their article is actually redirected to the unmoderated discussion group comp.sys.amiga.misc.The appropriate use and content of this header are very dependent on the community of readers that the newsgroup is serving.
NOTE: It is never correct to put an actual email address in the Followup-To: line.
Postings to newsgroups should have a From: line that refers to the submitter, unless the posting is a digest, in which case the From: line should be that of the compiler of the digest.
Since most news readers display From: line information, it is appropriate to accurately depict who the article's content is "From", when possible.
Keywords: lines should be included as received in the posted article. Some moderators may want to add a Keywords: line if it doesn't already exist. Some moderators have added "SPOILERS" to the Keywords: line in articles posted to movie or book discussion groups if the article gives away the ending.
Some moderators have a list of all of the keywords used in the group and adjust the Keywords: line as needed. Limiting the set of keywords makes keyword searching a lot easier and avoids problems with synonyms and variant spellings.
Keywords: should augment rather than replace keyword usage on the Subject: line because, unfortunately, some news reader programs cannot use Keywords: to auto-select articles.
If the moderator receives a request to cross-post an article to multiple groups, and the moderator has a policy of honoring cross- posting requests, the moderator should try to comply with the poster's specification if the other groups make sense and are not moderated.
If the submitter requests cross-posting to newsgroups that the moderator cannot post to, the submitter should be so notified, unless there is a clear policy statement covering this inability. For example, users at many sites cannot post or cross-post articles to any alt groups.
If one or more of the other requested groups are moderated, the moderator can either inform the submitter that the article is being cross-posted to only unmoderated newsgroups or coordinate with the moderator(s) of the other group(s). Leave the final decision of what is relevant on other newsgroups with moderators for those newsgroups.
Due to the nature of existing news software, an article cross-posted by a moderator to multiple moderated newsgroups appears in all the specified moderated groups without requiring the further approval of the other moderator(s). A posting of this type will probably surprise and may even anger the other moderator(s) if the article posted violates the charter of the other moderated newsgroup(s).
The original Path: line should be removed and the news system should be allowed to generate a new one. The purpose of the Path: line is to show the path that the article took since being injected into the news system. Since the moderator is the one that actually injects an article into the news system, any previous Path: line should be discarded before the moderator posts the article.
Also insure your news transport software generates a non-replyable Path: line. For example:
Path: host!not-for-mail
This allows it to be propagated back to the site it came from. It also assures that mail from seriously broken news sites is not returned to you.New moderators shouldn't need to worry about this. If there is not a Path: line in an article, most news transport software generate one similar to that shown above.
The References: header is used by some threaded newsreaders to chain a set of articles together. It allows a discussion thread or multi- part posting to be dealt with as a unit. The second and and subsequent articles in a set should include a References: header.
News reader software needs to be able to reconstruct the article tree even if (a) the root article is missing, such as the article has expired, (b) the immediate predecessor is missing as in a canceled article. The software must do this based solely on information from the References: headers of existing articles.
The References: headers are used in different ways today depending on the article flow in the moderated group. If articles are posted so that all linked articles are posted in sequence and within a short period, such as is done in sources groups, then References: headers can be constructed with a minimalist method. Otherwise, groups where referenced articles are not in sequence or are posted days apart should use the standard References: header usage.
The standard and recommended usage of the References: headers is to include the Message-ID of both the first and one to three immediately prior article(s), in chronological order. The reason for this strategy is to keep news reader programs with thread-specific kill files happy after some articles have expired.
With the minimalist method used by source or binary moderated newsgroups, the References: header contains the Message-ID of the first part of the series (or package). The References: header only lists the Message-ID of the first part posted and not all the intermediate parts.
By using this header, threaded news readers present each set of postings as a single item to the user making it much easier for them to read.
NOTE: Another way of linking articles is to list the Message-ID of every part. This is not recommended as it just increases the size of the articles without adding much additional information or utility.
The Reply-To: line should be preserved if it existed in the submission. This allows the news reader software to email replies back to the article's submitter at their preferred address.
Standardizing your use of the Subject: line somewhat can really help readers choose which articles to read and construct accurate kill files. A leading or trailing keyword system can help immensely, for example. Moderators of source and binary newsgroups use the Subject: line in a de-facto way to make it easier for the readership to see what an article is. For example:
v43i001: ecuman - Manual for ECU comm package v3.30, Part01/06
The leading volume-issue and the trailing Part number information are helpful in giving the readership quick clues to an article.
Assure that your use of the Subject: line is documented in the newgroup's policy posting so that the readership knows it is occurring and can take advantage of it.
There are additional headers that a submitter may supply from time to time. Informational headers such as Summary: and Organization: lines should be included as received in the posted article.
Submitted articles may arrive in your mailbox with one or more
headers that should be removed before posting. Automated scripts can
do this for you, or, for a low-volume group, you might prefer to
remove them by hand, or write your own pre-processing tools.
Status:
Lines:
Received:
Apparently-To:
X-*
Cc:
Message-ID:
Sender:
In-Reply-To:
X-VM-v5-Data:
Originator:
Some moderators allow all postings to go out with the original signature block as received. Others trim excessive signature blocks off, or remove all but a few lines. In other cases, the moderator will append a standard newsgroup signature to the bottom of the posting, typically containing a line or two describing how to submit articles to the newsgroup, how to retrieve the FAQ, or other highly condensed information.
Moderators need to be aware that news software may be appending the moderator's own personal signature file to the end of postings. This may not be desired and can cause confusion with the original submitter's signature. The moderator should decide what is the most appropriate way to deal with their personal signature.
A moderator may find that their newsgroup is better served with the addition of non-standard informational header lines to the individual postings. This can be done with user-definable "X-" headers placed in the RFC 1036 header portion of the article or by creating auxiliary headers as the comp.sources.* groups have done.
Source newsgroup moderators have established additional headers whose sole purpose is to support the posting of source code, automatic archiving and index creation.
Auxiliary headers do not appear in the RFC 1036 "News" header section of an article. Instead they are the first lines of the article text separated from the news headers by a single line containing a newline. The actual article text is then separated from the auxiliary headers by another single line containing a newline.
In either case, the moderator should inform the newsgroup of the purpose and use of the new headers. This should be done in the periodic policy posting.
Submissions are received by the moderator as mail. Although it is possible to use a personal mailbox, it is not advisable. The moderator mailbox should be either a separate account or an alias that points to the moderator's personal account. There are various reasons for doing this, among them:
Submissions to a moderated group should be automatically acknowledged when received. This can be accomplished using the deliver or procmail mail processing packages. The UCB Vacation program can also be used to generate acknowledgements.
Deliver is available from comp.sources.reviewed archives in volume1. Procmail is available from comp.sources.misc archives in volume43. Other mail filtering programs may be used as such as 'mh' and the 'filter' program that comes as part of Elm.
There are a couple ways that a reader can submit an article to be posted to a moderated newsgroup. The reader can post the article to the moderated newsgroup as if the group was not moderated. If the news software is properly configured then it will forward the article to the appropriate moderator for approval. Unfortunately, it is not unusual for a posting to be lost in a misconfigured news system. Readers then send mail to the moderator wondering where their article went to. The moderator has not seen it and has no idea what the submitter is talking about.
Other problems with encouraging direct posting to newsgroups is that the article might be cross-posted or might have been sent without knowing the group's moderation status. Another problem that makes directly posted articles harder to deal with is the duplicate headers problem described in Section 16.7.
Moderators should consider encouraging submitters to mail articles to the submission address directly instead of direct postings to the group. The benefits are, users are usually alerted to mail problems faster than news problems, duplicate headers are not a problem and articles received via email are guaranteed not to be cross-posted. All in all, emailed submissions tend to cause moderators less grief then do directly posted submissions.
Comments from the moderator, if necessary, should be added in a way that clearly differentiates the comments from the submitted article. This is usually done by including comments enclosed in brackets [ such as this ]. Whether the comments are included at the beginning or appended to the end of the article does not really matter. It has been suggested that placing the comments at the end of a posting is better since it does not interrupt the flow of the author's train of thought.
It is also sometimes appropriate to interject comments into the middle of a posted article; for example, if a post gives a vague reference to an FTP site, the moderator may wish to add a line with a specific reference immediately below that paragraph to avoid confusion.
Also moderators should sign the comments, either with their name or some way to identify the moderator (e.g., -mod, -editor or the moderator's initials).
No matter what method is chosen, the moderator should be consistent so that the group's readership can easily locate and recognize the moderator's comments.
The software and process a moderator uses to post to a newsgroup can be as simple as piping an article through a script from within the moderator's mailer which posts it. It can be as full blown as a program that creates Auxiliary headers for a source submission and checks for all sorts of potential name conflict problems and common posting errors.
The moderator should determine what is needed to make these tasks easier. Taking the time to try to figure out actual posting procedures can potentially save time every day.
Posting software is available on the moderator tools archive. From the simplest "submit" script to the complication of "postit", the archive has a wide range of posting tools that are there for others to grab and modify for their purposes. [See Section 15 for the location of the archive.]
From time to time you may need to cancel an article. It may be that you need to cancel an article with forged approval or an article that was posted in error. Whatever the reason, know how to cancel an article so that when the need arises you are prepared to cancel it quickly and correctly. To cancel an article, create a cancel message and post it the very same way the article was originally posted. The From: and Sender: headers need to be the same as they were in the original article. Take the Message-ID: of the article being canceled and make it the cancel header by "Control: cancel <message-id>".
You should use the same Newsgroups: line as the original, and you must have an Approved: line, otherwise it'll get submitted to the moderator for approval.
You might choose to make the Subject: contents the same, but it is not necessary. Finally, if you provide your own Message-IDs for your articles make sure that you give the cancel message a new Message-ID. For example, to cancel this message:
From: I-made-a-mistake@erroneous.com (I. Goofed)
Sender: usenet@erroneous.com
Message-ID: <12345abcde@erroneous.com>
Subject: How to shoot yourself in the foot
Approved: <your usual Approved: line>
Post this cancel message:
From: I-made-a-mistake@erroneous.com (I. Goofed)
Sender: usenet@erroneous.com
Control: cancel <12345abcde@erroneous.com>
Subject: Cancelling erroneous article
Message-ID: <something.other.than.12345abcde@erroneous.com>
Approved: <your usual Approved: line>
Also put a note in the body of the cancellation message explaining why you cancelled the article. This is normally just a one-line comment.
There are scripts in the moderator tools archive to assist in canceling articles.
News programs communicate with each other according to standard protocols, some of which are described by RFCs. RFC stands for Request For Comment, but for many of the RFC's it might be better described as Requirements For Compliance. Many of the RFCs describe de-facto standards in the Internet Community. They are a form of a published software standard. Copies of RFCs are often posted to the net in the group comp.doc and obtainable from archive sites such as ds.internic.net.
Current news-related RFCs include the following:
Henry Spencer has a draft of a successor to RFC 1036 that attempts to document and explain all subsequent enhancements and existing practice as implemented in the newer news systems. This draft (often called son-of-1036) can be obtained by anonymous ftp from ftp.zoo.toronto.edu. Son-of-1036 is intended to be stand-alone reading and does not require that one also read RFCs 2822 or 1123.
Kent Landfield is currently developing an FYI describing Sources group moderation.
Chris Lewis maintains an FAQ that suggests a format for an FAQ.
FAQs: A Suggested Minimal Digest Format"
It is periodically posted to news.admin.misc and to the groups news.software.readers and *.answers.
It is also probably a wise thing to re-read the documents that are posted from time to time in news.announce.newusers so that you are aware of what the rest of the community is seeing. It may have been a long time since you last read those articles and they have changed over the years.
For additional background information on Usenet and moderation checkout
and