This page is a sneak preview of the forthcoming announcement of the Internet Utilities version 1.0.0, to give you a taster for what they will comprise. Pre-release testers are encouraged to also read the notes accompanying the pre-release distribution archives.

Announcing

The Internet Utilities for OS/2

version 1.0.0
by
Jonathan de Boyne Pollard

The Internet Utilities for OS/2 is a suite of native OS/2 utilities for using and providing various Internet-related services, including DNS, NNTP, HTTP, and SMTP.

• The Utilities
• Features
• Where to find IU
• About the Author

THE UTILITIES

The Internet Utilities comprise

• inetd replacement

  • tcpsrun is a replacement for inetd that allows TCP servers to be managed individually as separate services (under the aegis of runsvc from the Command Line Utilities, for example). Changing the state of one server thus does not affect the operation of any other servers. It operates using access control rules set by tcprule.

• DNS servers

  • Proxy DNS server dæmons

    • dnsrcpd is a resolving caching proxy server.
    • dnsfcpd is a forwarding caching proxy server.

  • Content DNS server dæmons

    • dnsgsd provides simple general-purpose content service over UDP out of a database. The database is compiled by the dnszcomp from source. The source is usually hand-generated, but a starter database can be constructed from an %ETC%\HOSTS file using the dnshconv utility.
    • dnszsd supports zone transfers and general-purpose content service over TCP out of a database.
    • dnsfsd provides specialised fixed-data content service.
    • dnstsd provides specialised TXT record content service.

• DNS clients

  • DNS command-line lookup utilities

    • dnsqry is a diagnostic tool that sends queries to a single server, printing out the response that is received in a compact yet human-readable form.
    • dnsqual converts partial domain names into fully-qualified form at the command line, using a set of configurable rewriting rules.
    • dnsgeta obtains and lists all of the IP addresses for a name. (IBM's host command lists only one address.)
    • dnsgetd obtains and lists all of the names for an IP address.
    • dnsgetns obtains and lists the IP addresses of the content server(s) for a domain.
    • dnsgetmx obtains and lists the IP addresses of the mail exchanger(s) for a domain.
    • dnsgetsrv obtains and lists the IP addresses of the servers of a named service for a domain.
    • dnsgetz requests a "zone transfer" and outputs the results in source form.
    • host performs both forward and reverse DNS lookups.
    • hostname displays the current configured hostname. (IBM's hostname displays the name associated with the IP address of the first network interface, which is not actually the same as the "host name" set in the TCP/IP configuration notebook.)

  • DNS client library

    • tcp32dll replaces the DNS client library supplied by IBM. It comprises a lightweight DNS client that uses only the DNS for looking up information, and no other sources.

• NICNAME server and client

  • whois is a NICNAME client. It is a modern NICNAME client that intelligently determines, from SRV information published in the public DNS database, where to find the appropriate NICNAME servers.
  • nicnamed is a simple NICNAME server that enables one to provide one's own NICNAME service.

• NTP clients

  • ntpqry is a diagnostic tool that sends queries to a single server, printing out the response that is received in a compact yet human-readable form.
  • ntpsync is a utility that synchronizes the operating system's clock to the time obtained from an NTP server. Scheduled under the control of runwhen from the Command Line Utilities, it can comprise a simple NTP synchronization service.

• Mail and news dæmons

  • Mail server dæmons

    • smtprd is an SMTP Relay server for accepting "inbound" mail. It provides strictly inbound service only.
    • smtpsd is an SMTP Submission server for accepting "outbound" mail. It adds missing necessary header fields and cleans up malformed header fields.
    • nntpd is a simple general purpose NNTP server. nnfix is a tool for repairing its database after hard disc crashes and the like.
    • imapd and pop3d are simple IMAP and POP3 servers that serve up a multiple mailboxes in multiple domains. They both operate upon the same mail directories, and can be run side by side to serve up the same inboxes via two protocols.

  • Mail processing dæmons

    • smtppd is a dæmon that manages the routing of queued mail to local or remote delivery.
    • smtpld is a dæmon that manages the local delivery of queued mail.
    • smtprcd is a dæmon that manages remote delivery of queued mail. It invokes smtprc to perform individual deliveries.
    • smtpcd is a dæmon that manages the removal of delivered mail from the queue.

• Mail utilities

  • sendmail is shim for compatibility with UCB sendmail, that transmits mail to an SMTP Submission server. It supports the -t option employed by Changi and other lazily written softwares. The mimeenc utility can be used to MIME encode a file into a mail message suitable for injection via sendmail.
  • mailto is a utility for directly injecting mail into the mail queue.
  • mbmake makes a "mboxrd" format mail file from a mail directory.
  • mbsplit takes the contents of an "mboxrd" format mail file and appends it to mail directory (as used by the mail server daemons). It understands the Mozilla message flags, and will mark messages converted to mail directory format with the equivalent IMAP message flags. (So, for example, a deleted but not expunged message in the mail file becomes a deleted but not expunged message in the mail directory and will appear appropriately to IMAP clients.)
  • etrn generates the SMTP commands for prodding store-and-forward mail servers into action.

• Message and file transfer utilities

  • httpget and httpgetn are tools for retrieving files from an HTTP server. The latter supports retrieving a sequence of files with numbers in their filenames. They preserve last modification dates.
  • nnfeedn feeds articles from one NNTP server to another using the NEWNEWS command and a timestamp file to track which articles to feed.
  • nnfeedg feeds articles from one NNTP server to another using the GROUP command and per-group lists of transferred article numbers to track which articles to feed.
  • pop3get retrieves the contents of a POP3 mailbox and writes it into a local message store directory, which can be served up live by the POP3 and IMAP daemons.
  • makernb makes a "rnews batch" format mail files.

• HTTP server dæmons

  • httpd is a simple secure content HTTP server that supports virtual hosts and persistent connections, and that derives HTML content types from the conventional OS/2 extended attributes used to store file types.
  • httppd is a simple secure proxy HTTP server that supports and persistent connections, and that understands SRV resource record lookup (emabling content server owners to specify load balancing and controlled failover).

• Other server dæmons

  • syslogd provides a translation service from logging over UDP/IP to logging to standard error.
  • identd provides simple IDENT protocol service with fixed information.

For full information, see the on-line documentation.

FEATURES

The Internet Utilities have several features:

• Pure 32-bit code. The utilities contain no 16-bit code whatsoever, and use no 16-bit OS/2 subsystems. They are designed to always use the native paradigms of 32-bit OS/2 whereever possible, and to operate in a way that takes best advantage of the way that 32-bit OS/2 works and 32-bit OS/2 features.

• Modularity and simplicity. Different executables perform different tasks. For examples: One doesn't need to learn about "zone files" and "boot files" simply in order to run a caching proxy DNS server, as is the case with certain other DNS servers. One doesn't need to configure local mailbox files, domain names, and user databases if one only wants to be able to send mail from one's machine.

• Full support for provider-specified server load balancing and failover. All of the various service clients are fully capable of using SRV resource records for locating services. They will respect the priority and weight information given by the service provider therein, and attempt to distribute client load across servers, and fail-over from primary to secondary servers, in the ways that the service-provider specifies.

  The HTTP client in the HTTP proxy server, httppd, does the same; thus effectively augmenting WWW browsers that do not directly, themselves, support load balancing across HTTP content servers.

• Clear mail service separation. The SMTP Relay server intentionally does not support the Old-fashioned Mail Injection Protocol. SMTP Relay can only be used to transfer "inbound" mail. Similarly, the SMTP Submission server intentionally requires authorisation, rendering it only suitable for transferring "outbound" mail.

• No delays in processing queued mail. Other MTS softwares, including the Sendmail port supplied by IBM, employ a model whereby a "queue runner" dæmon runs at regular intervals. Of course, the natural consequences of this are

  • The granularity of mail transport processing is only as fine as the queue run interval. In other words: If the queue run interval is 2 hours, mail transport will always be retried in multiples of 2 hours.

  • An attempt to transfer a message is not made as soon as it enters the queue. One is forced to choose between fast delivery that happens immediately but that doesn't queue the message (what Sendmail calls "immediate/background modes") and delivery that queues the message but that doesn't immediately happen. (what Sendmail calls "queued mode").

  The Internet Utilities has one option: "fast and queued". The mail submission agents in the Internet Utilities, mailto and smtpsd, trigger (via a semaphore) the smtppd dæmon when new mail is injected into the queue, causing it to run immediately. In its turn smtppd triggers smtpld and smtprcd whenever new local or remote mail has been routed, causing them to run immediately.

• SMTP Relay client connection re-use. smtprcd attempts to transfer all mail to a given domain in a single batch, using a single SMTP Relay session.

• Rejection of attempted DNS poisoning. dnsrcpd and dnsgetz both avoid DNS poisoning by discarding responses for names that are outside of the content DNS server's bailiwick. And yes, dnsrcpd and dnsfcpd are two proxy DNS server softwares, in use (by their author and pre-release testers at least) for some years now, that were using randomized IDs and source ports ab initio, but that Dan Bernstein didn't know about.

• Privacy walls. dnsfsd can be used to prevent attackers from learning the names of hosts within an organization from their IP addresses, or even from learning the number of hosts within that organization from DNS data.

• Handling dial-up IP blocks is simple. dnsfsd can be used by ISPs to automatically assign machine-generated names to the addresses in dial-up IP blocks, without needing large configuration files or lengthy databases listing all of the addresses explicitly.

• Bounded memory use in the caching proxy DNS servers. dnsrcpd and dnsfcpd have a fixed size cache, that never grows. (BIND, by contrast, has no fetters on its cache growth. As a consequence, many authorities recommend that BIND be killed and restarted every few days lest it use up all available memory. The caching proxy servers in the Internet Utilities are designed to run indefinitely.)

• EDNS0 capability. All of the DNS clients and servers support EDNS0.

  The various DNS clients use EDNS0 to advertise large DNS/UDP datagram support for datagram sizes up to 32KiB, eliminating the need for DNS/TCP in most cases. (The sole reason that they don't advertise the maximum 64KiB, which they are fully capable of handling, is that doing so turns out to trigger a daft 16-bit sign extension bug in OS/2's supposedly 32-bit sockets library.)

• "Split horizon" DNS service is simple to configure and to maintain. By modifying a single text file, any part of the DNS tree can be pruned away from the public DNS and redirected to an arbitrary set of alternative servers, that provide a different view of that particular portion of the namespace.

• Several choices of "." content DNS servers available "out of the box". dnsrcpd can be configured to use any arbitrary set of root content servers. It is supplied "in the box" with a set of "drop in" configuration files allowing one to choose among the root content servers run by

  • the U.S. government (ICANN),
  • OpenNIC,
  • the Open Root Server Confederation, or
  • the Public Root.

  The non-government organisations allow one to have access to names in many domains, including "arts.", "hotel.", "sport.", "food.", and "news." (e.g. www.bbc.news.).

• Migration path from %ETC%\HOSTS. dnshconv allows easy migration from having a separate %ETC%\HOSTS file on every machine, with all of the hassles involved in keeping them identical, to a fully-fledged proper DNS setup (which the DNS Utilities provides all of the tools for) with a central DNS database. It allows one to easily convert %ETC%\HOSTS to the DNS Utilities database format.

• "Relay block list" DNS service is simple to configure and to maintain. dnstsd serves up "TXT" records for names within an arbitrary domain, in the style of the services run by organisations such as MAPS and ORBS for barring the receipt of electronic mail. It can be used to serve up one's own private blacklists and whitelists, or even to provide a competitor to MAPS and ORBS if one wants to. The database format that it uses comprises ordinary text files.

• Easy DNS database replication. The general-purpose content DNS servers in the Internet Utilities merely require that one copy an ordinary file, by whatever means one chooses, to keep their databases identical. (Replication of the database with other DNS servers running other softwares using the "zone transfer" database replication mechanism can be performed using dnszsd and dnszget, of course.)

• IP version 6 capability. Although the underlying operating system may not support it, all of the tools support IP version 6. The DNS lookup tools can handle IP version 6 address information. The various service client tools can attempt to connect to IP version 6 servers. The various server tools can attempt to listen on IP version 6 addresses.