OpenIMAPD

Tree

Tree:: 680f8bbc86649ac1a650a71994ef3321260bee29
Date:: Sun Sep 27 23:40:47 2026 UTC
Message:: SEARCH BODY and TEXT, parse cache, RFC 5322 headers, and literals Ten changes, made one at a time and committed together. Each follows in the order it was made, under its own subject line. Together: the account worker keeps the ENVELOPE and BODYSTRUCTURE text it has had parsed; SEARCH answers BODY and TEXT, so it now answers every key of RFC 9051 section 6.4.4; ENVELOPE and SEARCH read addresses, groups and comments as RFC 5322 writes them; a string may be sent as a literal (RFC 9051 section 4.3) wherever imapd reads one, while a refused non-synchronizing literal's command now ends where RFC 7888 says it does; a header field is found when white space comes before its colon (RFC 5322 section 4.5); a bare CR in a quoted Content-Type parameter no longer loses that parameter and those after it; and SEARCH takes its CHARSET quoted, as RFC 9051 allows. Change 1 of 10: Cache ENVELOPE and BODYSTRUCTURE in the account worker Every ENVELOPE or BODYSTRUCTURE FETCH item cost a message open, a descriptor pass, a parse in the parser-worker and a wait for its reply, however often the same message had been asked for. On a mailbox of 10,000 messages, FETCH 1:* (ENVELOPE) took 4.4 times as long as FETCH 1:* (FLAGS), which needs no parser. The account worker now keeps the checked text of both items, in the shape of smtpd's envelope cache: a fixed bound with no directive, an entry moved to the front on each use, and the oldest evicted first. The bound is 4 MiB of text and bookkeeping per account worker. Only successes are kept; a message the parser could not answer is asked for again, and its strikes are counted as before. With the cache, a second FETCH 1:* (ENVELOPE) of the same 10,000 messages took about half as long as the first. The key is RFC 9051 section 2.3.1.1's: mailbox name, UIDVALIDITY and UID, which "must refer to a single, immutable (or expunged) message on that server forever". A hit must also match the message's file name in the index, so an index rewritten by another program, or a UIDVALIDITY issued twice, is a miss rather than another message's text. A hit still needs the message file on disk, as the uncached path does, so a FETCH answers as it did before for a message whose file has gone. EXPUNGE, CLOSE and MOVE drop the UIDs they end, DELETE and RENAME drop every entry of the name they end, and a FETCH drops its mailbox's entries when the first of them has another UIDVALIDITY. The key alone would make each of these cost memory rather than a wrong answer if missed. A message whose requested items need no parse, or are all in the cache, no longer waits for a parser-worker, so a warm FETCH of ENVELOPE or BODYSTRUCTURE does not start one. When an account worker that looked anything up exits, it logs one line with its hits, misses, additions, evictions, drops, stale entries found, and its peak entries and bytes, so the bound can be sized from what the cache holds. A worker that never fetched ENVELOPE or BODYSTRUCTURE logs nothing, so short sessions add no lines. Change 2 of 10: Answer SEARCH BODY and TEXT RFC 9051 section 6.4.4's BODY and TEXT were parsed and refused NO. They are now answered in the parser-worker, beside the header keys, from one request per candidate message as before; a request with either key reads the whole message, up to "attachment max", instead of the header alone. The message is walked as MIME. Only TEXT and MESSAGE parts are searched, as section 6.4.4 permits; base64 and quoted-printable are decoded first, as it requires. An unrecognized transfer encoding makes a part application/octet-stream (RFC 2045 section 6.4), so it is not searched. A missing or invalid Content-Type is text/plain (section 5.2), and so is a multipart with no boundary. An unrecognized multipart subtype is walked as mixed (RFC 2046 section 5.1.3), and a digest part with no Content-Type is MESSAGE/RFC822 (section 5.1.5). Preambles and epilogues are not searched. BODY matches no header field of any level: not the message's, not a MIME part's, not a forwarded message's. TEXT matches all of them, each field unfolded and its RFC 2047 words decoded, and the content BODY sees. A forwarded MESSAGE/RFC822 or MESSAGE/GLOBAL is walked into and its own parts decoded. HTML is searched as it is, markup included, and a phrase is matched only within one line of the decoded text. Message charsets are not converted; an ASCII word still matches beside other octets. A multipart has no limit on its parts; the message's size bounds them. An entity nested deeper than BODYSTRUCTURE's limit of 10 cannot be checked, so the SEARCH answers NO, as for any message the parser cannot check. Matching folds ASCII case and runs in time linear in the text (Knuth-Morris-Pratt), so what a crafted body and needle can cost grows with the text's length, not with the product of the two lengths. The message is read into a buffer that doubles rather than grows by 64K. Change 3 of 10: Parse RFC 5322 addresses for ENVELOPE and SEARCH as the RFC writes them ENVELOPE and SEARCH FROM, TO, CC and BCC share one address parse, and it did not know RFC 5322 section 3.4's groups or section 3.2.2's comments, and toggled on every double quote, escaped or not. So "team: a@b, c@d;" came out as a mailbox "team: a" and a host "d;"; an escaped quote in a display name dropped that address and every one after it; "jane@x.org (Doe, Jane)" split inside its comment; and a comment stayed in the host or the name, so SEARCH matched it. These were wrong answers with nothing to show for it. A group is now sent as RFC 9051 section 7.5.2 defines it: a marker holding the group's name before its members and an empty one after, an empty group included, so "undisclosed-recipients:;" is two markers where it was NIL. A group missing its ";" is closed at the end of the field. SEARCH still matches the members, not a group's name. Comments are removed before the list is split, each becoming one space, nested and with quoted-pairs, as mail(1)'s skin() drops them; a comment is never taken as a display name. A backslash escapes the next character in a quoted string. The display name loses its quoting and keeps single spaces between words; the local part loses its quoting; an address loses white space outside quotes, so the obsolete "jdoe@test . example" is "test.example". An obsolete route goes in the at-domain-list, not the mailbox, and SEARCH does not match it. An address that does not parse is still left out and the rest of the field kept. An address longer than 1024 octets is no longer dropped: only the whole ENVELOPE has a limit, as before. Change 4 of 10: End a refused non-synchronizing literal's command where RFC 7888 does A command whose non-synchronizing literal ("{n+}", RFC 9051 section 4.3) was refused, or answered before its end, had only the literal's octets discarded. The rest of the command, which RFC 9051 section 7.6 puts after the octets, was then read as a new command line, so "a SEARCH TEXT {5+}" followed by "hello UNSEEN" drew a second reply, "UNSEEN BAD Missing command", tagged with the client's own word. RFC 7888 section 3 requires the octets and the following line to be treated as part of the same command. The line after the octets is now discarded too, and if it ends in another "{n+}" those octets are discarded in turn. A non-synchronizing literal over 4096 octets still closes the connection, one of the two choices RFC 7888 section 4 allows, but the reply is now "* BYE [TOOBIG]" as that choice and section 5 describe, where it was "* BAD". APPEND's own refusal of a non-synchronizing literal over 4096 octets is removed. The listener closes the connection on such a line before any command sees it, so the check could not be reached. Change 5 of 10: Read literals as mailbox names and SEARCH strings RFC 9051 section 4.3 makes a literal, "{n}" or "{n+}" followed by n octets, one of the two forms of a string, and a client may send one wherever a mailbox name or a SEARCH string may go; the RFC's own SEARCH examples in section 6.4.4 do. imapd took a literal only as APPEND's message and answered BAD to every other, so a client that chose the literal form could not select, create or search. A parser that meets a literal ending the text it was given now asks for it instead of refusing it. The listener then gathers the command: it sends "+" for a synchronizing literal, reads the n octets and the line after them, and runs the whole command again from the start, as ldapd does with a request whose BER element is not all there yet. The parsers read a literal already in the gathered text by its count, so its octets may hold a space, a quote or a CRLF. Each command parses all of its arguments before it acts, so asking for a literal changes nothing; SEARCH now parses before it resets the session's last result. A command and its literals are held to 8191 octets. Over that, it is answered NO [LIMIT], before the "+" when the count is known in time. A NUL in a literal, which CHAR8 excludes, is answered BAD. In either case the rest of the command is discarded as RFC 7888 section 3 asks. This covers every mailbox argument, LIST's reference and pattern, SEARCH's strings and HEADER's field name. FETCH's HEADER.FIELDS names and ID's parameters still do not take a literal. Change 6 of 10: Take FETCH's header field names quoted or as literals, and gather ID's A HEADER.FIELDS name is an astring (RFC 9051 section 9, header-fld-name), but FETCH took it as an atom only: a quoted name was BAD, and a literal never reached the parser. Names are now read quoted, with RFC 9051's two escapes, or as literals, which FETCH turns into quoted strings before it parses, since a field name cannot hold the CR or LF only a literal could carry. The response echoes the list as the client sent it, a literal shown as the quoted string it became. A name that would hold a space is BAD, as the store takes the list space-separated, and so is an empty list, which header-list does not allow. The FETCH tokenizer no longer splits inside a quoted string. ID answered OK without reading its parameters, so a client that sent one as a synchronizing literal got the reply before its "+". It now reads just far enough to gather its literals (RFC 2971 section 3.1) and still answers NIL. Change 7 of 10: Accept a non-synchronizing literal on a pipelined command A command that arrived while another was in flight, and that carried a non-synchronizing literal, was refused with an untagged BAD, so its tag was never answered; RFC 9051 section 2.2.2 tags every completion, and section 4.3 lets a client send "{n+}" anywhere a literal may go. Its octets and the lines after them are now gathered into the queued command, held to the same 8191 octets as any other, and it runs in its turn like any pipelined command (section 5.5). A queued command also no longer waits for the store's next reply when the gathered command ahead of it finished without needing the store. Change 8 of 10: Find a header field whose name is followed by white space RFC 5322 section 4.5 lets any field put white space between its name and its colon, "From :" for "From:", and section 4 requires a receiver to accept that. imapd took a field's name to be everything before the colon, white space included, so such a field was never found: ENVELOPE showed NIL, SEARCH did not match it, HEADER.FIELDS left it out, and a "Content-Type :" multipart message was described and searched as one text/plain part. The name now ends before any SP or HTAB that precedes the colon, in the one function every header reader uses. HEADER.FIELDS still returns the field as the message holds it. Change 9 of 10: Keep a Content-Type parameter that holds a bare CR A CR, LF or NUL inside a quoted Content-Type parameter value made the quoted-string fail, and the parameter loop stopped there, so that parameter and every one after it were lost. text/plain; charset="us<CR>ascii" was shown with NIL parameters, and a multipart whose boundary came after such a parameter, or held the CR itself, could not be split: its BODYSTRUCTURE ended NO, its BODY[n] was empty, and SEARCH read the whole body as text, delimiter lines included. RFC 5322 section 4 says malformed data is not to be "irretrievably lost". The value is now kept, CR and LF included, so a boundary still matches delimiter lines that carry the same CR. A NUL becomes a space, since the value is held as a C string. Every parameter value already goes out through the nstring writer the other fields use, which sends a CR, LF or NUL as a space, since a quoted string cannot hold one (RFC 9051 section 9). A message holding a NUL is still refused for BODYSTRUCTURE, BODY[] and BODY[n], so a NUL in a parameter now matters only to SEARCH. Change 10 of 10: Read SEARCH's CHARSET as an atom or a quoted string RFC 9051 section 9 writes SEARCH's charset as "atom / quoted", but imapd took every octet up to the next space. So CHARSET "UTF-8" was compared with its quotes and refused NO [BADCHARSET], and a name the grammar does not allow, such as UTF(8 or a quoted string with no closing quote, was answered NO where its arguments are invalid, which section 6.4.4 answers BAD. The charset is now read as the grammar writes it: a quoted string, whose only escapes are \" and \\, or an atom, which refuses the atom-specials, controls and 8-bit octets; a space or the end of the command must follow it. A malformed charset is BAD. A well-formed one other than US-ASCII or UTF-8 is still NO [BADCHARSET], as section 6.4.4 requires.

README.md	commits \| blame
contrib/
src/

README.md

# OpenIMAPD

A from-scratch IMAP4rev2 ([RFC 9051](https://www.rfc-editor.org/rfc/rfc9051)) server for OpenBSD, written in C in the privilege-separated tradition of `smtpd(8)`, `httpd(8)`, and `ntpd(8)`. No third-party IMAP library.

**Status:** 0.1.7 Pre-release, actively developed. Not a port. See [Getting the source](#getting-the-source) below for the repository.

## What it is

- **Privilege-separated**, `smtpd`-style, across six processes: a root *parent* reads configuration and binds the listening sockets; unprivileged *listener* and *auth* children handle the network and credential checks; *keymgr* holds the TLS private key and performs every private-key operation on request, so the process terminating TLS never has the key in its address space; a *parser* child, spun up by a *store* child on demand and retired after a spell idle, does every parse of attacker-reachable content — `SEARCH`'s grammar and `FETCH`'s `ENVELOPE`/`BODYSTRUCTURE`/header-field parsing alike — confined to `pledge(2)` `"stdio recvfd"`: it opens nothing itself, reading each message over a descriptor the store child passes it already open; and a *store* child is forked per authenticated account, shared by all of that account's sessions, chroots into the mail spool, and drops privileges to that account's own user before ever touching a message. `pledge(2)`, `unveil(2)`, and `chroot(2)` enforce these boundaries, not just convention: *parent* is the only process that can pass a file descriptor at all.
- **Storage**: stock maildir format (`tmp/`/`new/`/`cur/`, atomic delivery via `rename(2)`), readable with `ls` and `grep`, and natively understood by `smtpd(8)`'s own `maildir` delivery action. IMAP's extra bookkeeping (UIDs, UIDVALIDITY, per-message mod-sequences, keywords) lives in a small, `flock(2)`-guarded, line-oriented index file per mailbox, plain colon-delimited text, not a database.
- **Transport**: STARTTLS on port 143 and implicit TLS on port 993 ([RFC 8314](https://www.rfc-editor.org/rfc/rfc8314)), via `libtls`. `AUTH=PLAIN` only, refused before TLS is established.

## Protocol coverage

`CAPABILITY`, `STARTTLS`, `AUTHENTICATE`, `ID`, `ENABLE`, `SELECT`, `EXAMINE`, `CREATE`, `DELETE`, `RENAME`, `LIST`, `LSUB`, `NAMESPACE`, `STATUS`, `FETCH` (including `ENVELOPE`, `BODYSTRUCTURE`, and MIME-part-addressed `BODY[<part>]`/`BODY.PEEK[<part>]`), `STORE`, `SEARCH`, `APPEND`, `COPY`, `MOVE`, `EXPUNGE`, `UNSELECT`, `CLOSE`, `SUBSCRIBE`, `UNSUBSCRIBE` (including LIST's [RFC 9051](https://www.rfc-editor.org/rfc/rfc9051) section 6.3.9.1 `SUBSCRIBED` selection option), the `UID`-prefixed form of every command that supports it, `IDLE` (with the cross-session caveat noted below), and the [RFC 7162](https://www.rfc-editor.org/rfc/rfc7162) `CONDSTORE`/`QRESYNC` extensions.

ACL/shared-mailbox support is deliberately left out.

**`IDLE` is a poll, not a kernel-driven push.** An `IDLE`ing session rechecks its selected mailbox every `idle poll` seconds (default 5, see `imapd.conf`), so new mail, whether delivered by an external MTA or by another IMAP session, is reported within one interval rather than instantly. Most polls are two `stat(2)` calls and no lock: the store child only re-reads the index and streams UIDs when the mailbox directory or `new/` has actually been touched. Setting `idle poll 0` disables polling entirely, which restores the earlier behaviour where an `IDLE`ing session saw nothing until it sent `DONE`.

## Requirements

OpenBSD only. This depends on `<imsg.h>`, `pledge(2)`, `unveil(2)`, and libutil's `imsgbuf_*` API, none of which exist outside OpenBSD. Links against libevent, libtls/libssl/libcrypto, and libutil, all base-system libraries (see `src/Makefile`).

**imapd requires OpenBSD -current, and will not build on 7.9.**. Building on the most recent stable release is a goal for 1.0.

`keymgr`, the process that isolates the TLS private key from `listener`, additionally depends on `tls_config_use_fake_private_key()` and undocumented ex_data-tagging behavior inside `tls_keypair_load()`, both unexported libtls/LibreSSL internals with no compatibility promise, and neither declared in libtls's public, installed `tls.h`.

## Building and installing

```
cd src
make
doas make install
```

Installs the daemon to `/usr/local/sbin/imapd`, man pages to `/usr/local/man/man8`, the `imapduser` account-provisioning tool alongside the daemon, and a sample config to `/usr/local/share/examples/imapd/imapd.conf`.

The `rc.d(8)` script is not installed automatically. `install(1)`, not `cp(1)`, so the installed copy is executable regardless of the source tree's own permission bits:

```
doas install -o root -g wheel -m 555 src/rc.d/imapd /etc/rc.d/imapd
```

## Configuring

Copy the sample config into place with restrictive permissions. imapd refuses to start against a config that's group- or world-writable, *or* world-readable:

```
doas install -o root -g wheel -m 600 \
    /usr/local/share/examples/imapd/imapd.conf /etc/imapd.conf
```

Every directive is documented inline in the sample file; the full reference is in `imapd(8)`'s FILES section.

## Creating an account

imapd's users aren't real system accounts, `imapduser(8)` manages a bespoke credentials file (`username:passwordhash:uid:gid:maildir`, bcrypt via `crypt_checkpass(3)`) and the matching maildir ownership together, since no combination of `useradd(8)`/`userdel(8)` can safely keep both in sync:

```
doas imapduser -a someuser
```

See `imapduser(8)` for `-d` (revoke login without touching mail) and the `-c`/`-s`/`-u`/`-g` overrides.

## Running

```
doas rcctl enable imapd
doas rcctl start imapd
```

## Known limitations

Beyond the deliberate protocol-scope decisions covered in `imapd(8)`'s CAVEATS:

- If the listener or auth process exits unexpectedly after startup, it is not automatically restarted. Recovery is `rcctl restart imapd`. See `imapd(8)`.

`SIGHUP` reloads `spool`, `append max`, `attachment max`, `account sessions`, `connections max`, `idle poll`, `lock timeout`, `login grace`, `startups`, and the TLS certificate/key without dropping connected sessions, `listen on` and `credentials` changes still require a restart. See `imapd(8)`.

IPv6 is supported (`listen on ::` or `listen on *` for dual-stack) but not the default, see `imapd(8)`'s `listen on` directive.

## Getting the source

The repository is hosted with [Game of Trees](https://gameoftrees.org/) (`got`), read-only anonymous access over SSH:

```
got clone ssh://anonymous@got.openimapd.dev/imapd
```

The repository is also git-compatible; a plain `git clone` against the same URL works too:

```
git clone ssh://anonymous@got.openimapd.dev/imapd
```

## Security

Report security issues to security@openimapd.dev. General questions or feedback: feedback@openimapd.dev.

## License

ISC. See the copyright header in each source file.

## More

`imapd(8)` and `imapduser(8)` are the authoritative technical reference.