maildir indexing; login answers; compile flags; voice; architecture Change 1 of 3: Give UIDs to message files found in cur/ imapd gave a message a UID only when it found it in new/ or wrote it itself with APPEND, COPY or MOVE. Nothing read cur/, and the new/ scan skipped any name holding ':'. So the mail in a maildir brought from another server was invisible, and every message in a mailbox whose imapd.index was lost disappeared with it, while SELECT answered OK. On the test host, a mailbox with four files in cur/ and no index showed 0 EXISTS, and one whose index had been removed showed 0 EXISTS under a new UIDVALIDITY. Each refresh of the index (SELECT, EXAMINE, STATUS, and an IDLE refresh with something pending) now also reads cur/, looks each base name up in a hash table of the index's names, and gives the files the index does not hold UIDs in name order, keeping the flags in their names. Like mail from new/, each is first rewritten with CRLF line endings, before it has a UID (RFC 9051 section 2.3.1.1). A file with no maildir info gets ":2," added, so it can be found. A file in new/ whose name carries ":2," info is moved to cur/ and indexed there, instead of being skipped. On a 100,000-message mailbox the scan measured 0.13 s warm and 0.19 s straight after a reboot, against an EXAMINE of about 0.40 s. EXPUNGE saves the index before it unlinks the files, so a file it failed to unlink, or one left by a crash in between, would come back as a new message. In a mailbox that already has an index, an unindexed cur/ file flagged T (\Deleted) is therefore left out and logged. A mailbox with no index takes every file. A file put in cur/ while a session is in IDLE is noticed at the next refresh that runs for another reason; imapd.8 says so. Change 2 of 3: Answer login OK only once its mailbox store has started An account's store child is started after its password is accepted, and only then enters the account's maildir. The parent handed the listener its channel to a new store at once, and the listener answered the login OK on receiving it, before the store had started. If the maildir was missing, as when its filesystem was not mounted, the store exited, and the client got OK and then a closed connection, with no BYE and no reason. On the test host every login to an account with no maildir was answered OK and then dropped. The parent now holds back the store channel from every session waiting on a store until the store reports that it is set up. A login is answered OK only once its store is in the maildir, and a store that cannot start refuses each waiting login with NO [UNAVAILABLE], the code RFC 9051 section 7.1 gives for a login whose backend is down. Change 3 of 3: Miscellaneous cold code review items A refused login, for that reason or because the account is at its "account sessions" limit, now gets an untagged BYE with the same response code before its tagged NO, as RFC 9051 section 3.4 asks of a server about to close the connection. The store's fatal message for a missing maildir now names the session. Warning flags: -Wextra and -Wcast-qual leave CFLAGS, and with them the 60 "(void)param;" casts that existed to quiet them. The six libc: strtonum(3) replaces the errno/strtoul/end-pointer dance at 15 sites, keeping the leading-digit guard since strtonum(3) accepts a sign; nitems() replaces sizeof(a)/sizeof(a[0]); getline(3) replaces three fgets-and-peek loops; a new opendirat() replaces nine inline openat/fdopendir/close sequences and, unlike them, preserves errno. Declarations: the anonymous { } scoping blocks are gone, as are declarations initialised from a call; blocks are sorted and aligned. The return conventions are stated once, in listener.h and store_internal.h: the parsers return 0, -1 (BAD) or -2 (NO), the handlers 1 while the session lives and 0 once torn down. Structure: parse_fetch_atts() takes a struct fetch_atts in place of fourteen out-parameters; lock_copy_move_mailboxes()'s fourteen parameters are copy_move_lock()'s four, and parse_content_type()'s eleven are three. parse_search_key_inner() is a keyword table and a switch. session_dispatch_client() and fetch_walk_step() are split into one function per state and per item; store_dispatch() and session_store_dispatch() keep their switches, as the OpenBSD daemons do, but each case is now the receive sequence once, through recv_fixed(), recv_header() and a union of the request types. Voice: "SS" in comments is "section", as the vendored sources write it. RFC citations are out of the text clients see. Log lines take the "%s: what" shape; the operational advice they carried is in imapd(8) and imapd.conf(5), where it already mostly was. Branches that said "can't happen" are either gone, where a same-size copy cannot truncate, or fatalx(), where a logic invariant would have to fail. Four fatalx() calls are new. Reachable conditions that had been labelled impossible keep their branches with honest messages. The listener is one process, one session: a static struct session, no session list, no session_find(). The licence headers of the six files that carried lineage essays are the copyright lines and the ISC text, with the essays moved verbatim to CREDITS. imapd gains -n, -v is cumulative, and the default "listen on" address is "*", so a configuration without the directive binds both address families; imapd.conf.example says so.

Free-space floor; SELECT/STATUS error codes; fail APPEND on fsync Change 1 of 3: Refuse APPEND, COPY, MOVE and CREATE below 5% free space On a full filesystem a user cannot free space: STORE \Deleted and EXPUNGE both write a new index before the old one is removed, so they fail with ENOSPC, and SELECT fails too when new mail is waiting. On the test host, with the spool full, every change to the mailbox failed and SELECT answered [NONEXISTENT]. Keep a floor, as smtpd(8) does for its queue (usr.sbin/smtpd/ queue_fs.c): the commands that add to the mail store, APPEND, COPY, MOVE to another mailbox and CREATE, are refused when less than 5% of the filesystem's space or inodes is free to non-root users. Index saves are not refused, so flags can still be changed and mail expunged below the floor. Unlike smtpd's check, a filesystem reporting no free blocks or inodes at all is refused, not accepted. A refusal answers NO [OVERQUOTA] (RFC 5530 section 3), as do APPEND and CREATE when a write, open or mkdir fails with ENOSPC or EDQUOT. Each refusal is logged, and imapd.conf.5 describes the floor under "spool". The check is adapted from smtpd's fsqueue_check_space(), and mbox_manage.c carries its copyright. smtpd still delivers below the floor, so a disk filled by delivery can still reach zero; then the commands that free space fail until the operator frees some. Change 2 of 3: SELECT and STATUS say [NONEXISTENT] only when it is true SELECT, EXAMINE and STATUS answered NO [NONEXISTENT] no such mailbox for every failure but a busy lock: the store reported every failure the same way. On the test host a full disk, and a mailbox whose directory could not be read, were both reported as missing, so a client could tell its user the mailbox was gone. The store now tells the listener why. A name that cannot exist, or a mailbox directory that openat(2) finds missing (ENOENT, ENOTDIR), still answers NO [NONEXISTENT] (RFC 5530 section 3). A full disk or quota while saving the index answers NO [OVERQUOTA], as APPEND does. Any other failure answers NO with no response code, "SELECT failed", "EXAMINE failed" or "STATUS failed" (RFC 9051 section 7.1.2), and is logged. index_save() keeps the errno of the call that failed across its cleanup so the caller can tell a full disk from other errors. Change 3 of 3: APPEND and COPY fail when fsync(2) or close(2) fails APPEND logged a failed fsync(2) of the new message as "(continuing)", did not check close(2), and went on to index the message and answer OK. COPY, when it copies a message rather than linking it, did the same. A message whose data the kernel could not write to disk was then listed as stored. Both now fail the command instead, remove the temporary file, and leave the index untouched, as index_save() and smtpd's mail.maildir already do. APPEND answers NO [OVERQUOTA] when the error is ENOSPC or EDQUOT, and NO APPEND failed otherwise.

smtpd delivery docs; oversize refusal; CRLF mail; child and login races Change 1 of 6: Document how smtpd(8) delivers into an imapd account imapd only reads maildirs. smtpd(8) writes them, and nothing told an operator how to configure it. imapd accounts are not system users, so smtpd needs a userbase table giving each account the uid, gid and maildir of its credentials line. Without one, smtpd delivers as the getpwnam(3) user of the same name, if there is one. On the test host that user's uid was not the account's, the maildir is mode 0700 for the account's uid, and every delivery failed with "mail.maildir: Permission denied" and stayed queued. imapd.conf.5: a new Mail delivery subsection. It gives the userbase line and says its uid and gid must match the credentials line, since smtpd writes each message mode 0600 as that user. It shows an example table, action and match rule using maildir "%{user.directory}", and explains why a bare maildir action is wrong: it delivers to ~/Maildir, which imapd shows as a mailbox. It covers rule order, that a listener declared with "auth" refuses mail from other hosts, smtpctl update table, the .Junk mailbox, and subaddress mail, which mail.maildir files in a mailbox named with a leading dot only if that mailbox exists. SEE ALSO gains table(5), smtpctl(8) and smtpd(8). imapduser: -a prints the account's userbase line on standard output, built from the same values as its credentials line, so "imapduser -a joe >> table" works. -d reminds the operator to remove that line. imapduser.8 says both. Change 2 of 6: Refuse a message over "attachment max" without reading it read_body_from_fd() (BODYSTRUCTURE, BODY[<part>]) and SEARCH's read_header() for BODY and TEXT refused a message over "attachment max" only after reading attachment max + 1 octets of it, 40 MiB by default, on every FETCH or SEARCH that reached it, while the account's other sessions waited on the parser. Both now refuse on the size fstat(2) reports, before reading anything. A message at the cap, and SEARCH's header-only read, are read as before. What a client sees is unchanged, and so are the log lines. imapd.conf.5: "attachment max" says what a FETCH answers for a larger message; that mail delivered by smtpd(8) is not bound by "append max"; that smtpd's max-message-size should stay below it, with room for the headers smtpd adds and does not count; and how to list the larger messages a maildir brought from elsewhere already holds. Change 3 of 6: Store delivered mail with CRLF line endings smtpd's mail.maildir ends each line with LF alone, and imapd served those bytes as they were, so BODY[] was not the RFC 5322 form of the message that RFC 9051 SS6.4.5 defines, and RFC822.SIZE was not its RFC 5322 size (SS2.3.4). When the store first indexes a file in new/, new_to_crlf() now copies it through tmp/ with each bare LF made CRLF, fsync(2)s it and renames it over the original, before the message has a UID; new/ is fsync(2)ed once after the scan. RFC 9051 SS2.3.1.1 lets no UID's text change, so messages already indexed are left as they are. A lone CR, NUL and every other byte are kept. A file with no bare LF is not rewritten. If the copy fails, a warning is logged and the message is indexed unconverted, so delivered mail is never hidden. A file that is not regular, or a symlink, is not opened for reading. APPEND still stores what the client sends. imapd.8 says delivered mail is rewritten with CRLF line endings. imapd.conf.5's advice on smtpd's max-message-size now leaves room for the added carriage returns. Change 4 of 6: A broken channel to one child no longer ends imapd The parent fatal()ed whenever a write to, or a read from, a listener, auth worker, parser or keymgr failed. Those children exit on their own schedule: an auth worker exits once its listener closes, so a client that logged in and out at once could close the listener before the parent's queued IMSG_AUTH_EXIT went out. The write then failed with EPIPE and imapd exited, ending every session. On the test host six clients logging in at once took imapd down this way. A message from a child with a bad length did the same, so a compromised listener or parser could stop the service. parent_dispatch_child() now drops that child instead, as the store's channel already did and as smtpd's mproc.c does for a closed pipe: it stops watching the channel and sends SIGKILL, and reap_child() frees the child on SIGCHLD. A failed write is logged at debug level, naming the child's process type and pid; a failed read is logged as a warning. keymgr is treated the same way, so losing it is reported as reap_child() already reported it. Change 5 of 6: A login answered OK no longer refuses what follows A granted login reaches the listener as two messages on two channels: the auth worker's IMSG_AUTH_RESULT, and the parent's IMSG_SETUP_PEER once the account's store is wired, which marks the session authenticated and sends the OK. Nothing orders the two. When the store's message came first, the late result put the session back in the state before the store was wired, and every command after the OK was answered "BAD Command not permitted in this state". On the test host five clients logging in at once saw it on 154 of 1000 logins. The listener now ignores a granted result that arrives after the store, and logs it at debug level. The auth worker died with "fatal: imsgbuf_write: Broken pipe" when its listener had already gone: after a session that had finished before the result went out, or a client that hung up while its password was checked. It now takes EPIPE on that channel as the listener closing, as httpd's and relayd's proc.c do, and exits quietly; other write errors stay fatal. Change 6 of 6: Trim imapd.8 to the shape of smtpd.8 imapd.8 had grown to 478 lines, against 167 for smtpd.8. It now has smtpd.8's sections and a short CAVEATS, in 178 lines. The credentials file's format and rules move to imapd.conf.5, as a Credentials file subsection beside the Mail delivery one; the credentials keyword points to it, and imapd.conf.5 lists the default credentials file under FILES. imapd.8 and imapduser.8 point to imapd.conf.5 for both. imapd.8 no longer lists the commands, describes the processes, or lists the protocol details a client meets (LIST options, FETCH BINARY, RENAME INBOX, subscriptions, mailbox name rules); CAVEATS says in one sentence that what is not supported is refused with BAD or NO, and keeps the notes an operator needs. STANDARDS keeps RFC 9051 and RFC 7162.

Add SEARCH SAVE; fix FETCH, queue, login timing, slow clients; man pages Change 1 of 6: Find where FETCH's modifiers begin by parsing, not by counting RFC 9051 section 9 lets a single message data item be sent without parentheses, and a HEADER.FIELDS item holds a space inside its brackets. RFC 4466 section 3 puts the optional fetch-modifiers after the item or list, separated by one SP. imapd looked for that boundary in split_trailing_modifiers(), which cut an unparenthesized item at its first space and counted parentheses in a list without skipping quoted strings. So "FETCH 1 BODY.PEEK[HEADER.FIELDS (Subject)]" was answered BAD, and so was any list naming a field such as "a)b", which RFC 5322 section 3.6.8 allows in a field name. fetch_dispatch() now takes the first token from fetch_att_tok(), the scanner parse_fetch_atts() already uses, which skips quoted strings and counts brackets and parentheses together. Whatever follows is the modifier list. split_trailing_modifiers() is gone. Change 2 of 6: Run queued commands after a refused literal, and keep both literals A command that arrives while another is in flight is queued, and imapd runs the queue in order, as RFC 9051 section 5.5 requires when commands could affect each other. A queued command whose non-synchronizing literal was still being gathered could be refused there: for a NUL, which RFC 9051 section 9's CHAR8 excludes, or for passing the 8191-octet command cap, which a correct client can reach with two 4096-octet literals. The listener then discarded the rest of that command, as RFC 7888 section 3 requires, but did not run the queue. Commands queued before the refused one waited for the next store reply, and a command sent after it ran first. If no later command used the store, they were never answered. The loop now runs the queue once a refused command's last line has been skipped, as it already did after a gathered command's last line. A queued command with two non-synchronizing literals also lost the text between them: the line after the first literal was copied into the command buffer without moving its length, so the second literal overwrote it, and "SEARCH TEXT {5+} ... TEXT {5+} ..." pipelined behind another command was answered BAD. The length now moves past the line. Change 3 of 6: Support SEARCH RETURN (SAVE) and the "$" marker RFC 9051 folds SEARCHRES (RFC 5182) into IMAP4rev2, so a client may send SEARCH RETURN (SAVE) and then use "$" wherever a sequence set goes, without asking for a capability. imapd refused SAVE with NO and "$" as an invalid sequence set. The store now keeps "$" for each session, as UID ranges. A SAVE search records its matches as it walks the index, one range per run of adjacent index lines, so the ranges stay exact: new messages get higher UIDs and an expunge only removes, as RFC 9051 section 6.4.4.1 requires. MIN and MAX without ALL or COUNT save only those messages (Table 4). SAVE alone sends no ESEARCH (section 6.4.4). A result of more than 500 runs is refused with NO [NOTSAVED], and "$" is emptied (section 6.4.4.3), as it is after any SAVE answered NO, including the listener's own NO for an unsupported CHARSET and a SAVE that gives up waiting for the index lock. A successful SELECT or EXAMINE empties it. "$" is accepted as the last element of a sequence set, as the grammar allows, so "1,$" works. FETCH, STORE, COPY, MOVE and UID EXPUNGE append the saved UIDs to the rest of the set, and answer NO [LIMIT] when the two together pass 500 ranges. SEARCH matches "$" by UID, so it costs one node however large it is. In a SELECT's QRESYNC known-uids, "$" is the value from before the SELECT. A "$" naming a message another session expunged is not refused as RFC 2180 section 4.1.2 allows for message numbers: "$" is kept by UID. Change 4 of 6: Make a failed login cost the same whether or not the account exists A failed AUTHENTICATE for a name in the credentials file paid for crypt_checkpass(3) at that entry's bcrypt cost. For a name with no entry, auth.c passed a NULL hash, which libc fakes at cost 8, while "encrypt -b a", as imapduser uses it, picks a cost for the machine; it picked 9 on the test host. A failure for a missing name was answered in about half the time (35.8 against 65.8 ms measured), which told a client which names exist. RFC 9051 section 11.7 asks that a failing login not say whether the user name is the invalid part. cred_lookup() now reads the whole credentials file on every lookup, keeps the first valid match, and reports the highest bcrypt cost among valid entries. A name with no valid entry is checked against a well-formed dummy hash at that cost. An entry hashed at a lower cost than the file's highest can still be told apart by timing. An over-long line in the credentials file now refuses every login, not only logins for names that come after it. Change 5 of 6: Let a slow client be slow, without stalling its account A client whose socket stayed full for 5 seconds was cut off, logged in or not: session_write() gave up after one poll(2) of 5 seconds without progress. While the listener slept in that poll it read nothing from its store channel, which blocked at the store's end, so the account's store child slept in sendmsg(2) once a FETCH had queued more than the socket held, and every other session of the account waited with it. On the test host a second session's NOOP took 5.05 s while one session stopped reading a FETCH. The timeout is now 5 seconds before login and 30 minutes after, the floor RFC 9051 section 5.4 sets for an inactivity autologout. Before login a longer sleep would outlast the login grace timer, which cannot fire while the listener sleeps. The store sets O_NONBLOCK on each listener channel as it attaches one, as keymgr already does, so a listener that stops reading fills only its own queue. A session whose write fails is torn down at once, not when the client next sends. Change 6 of 6: Split imapd.conf(5) out of imapd(8), and correct the man pages imapd.8 had grown to 992 lines, about half of them the configuration grammar under FILES. Every base daemon it was compared with keeps that in a section 5 page: smtpd.conf(5), httpd.conf(5), ntpd.conf(5), ldapd.conf(5), relayd.conf(5), sshd_config(5). The directives now live in imapd.conf.5, laid out as httpd.conf.5 is, with an EXAMPLES section. imapd.8 is cut to 295 lines on the shape of ldapd.8, with the credentials file in an AUTHENTICATION section. The rc.d install steps, the Makefile's relink check and the reasoning behind each limit are gone from the pages; the install steps were already in README.md. The rewrite also corrects what the old page said against the code. A missing /etc/imapd.conf is fatal, not a default configuration (parse.y pushfile(), main.c config_load()). "listen on" requires "port". The configuration file is refused when group-executable or accessible by others, not only when writable (check_file_secrecy()). The parent accepts connections and starts listener and auth workers for each one, and the process that is not restarted when it exits is keymgr, not the listener or auth process (parent.c reap_child()). The Makefile installs both pages, and imapd-teardown removes imapd.conf.5. imapduser.8 cross-references the new page. A second pass checked every remaining claim in imapd.8, imapd.conf.5 and imapduser.8 against the code and the RFC texts. Wrong, and now corrected: - The credentials file rule. The page asked for root, group _imapauth, mode 0640 or stricter, which allowed modes the auth process cannot read (0600, 0400) and forbade files it accepts. It now states what cred_file_secure() checks: owned by root or _imapauth, not group writable or executable, not accessible by others, and readable by _imapauth. 0640 is what imapduser sets. - A kept subscription was cited to RFC 9051 section 6.3.8 as a requirement. It is a SHOULD NOT in section 6.3.7; renaming is in 6.3.6. - The command list omitted NOOP and LOGOUT, and did not say that LOGIN is always refused. - SIGHUP: sessions already connected keep their settings; new ones get the reloaded values, since each listener worker is given them once. - attachment max also bounds SEARCH, so a BODY or TEXT search that reaches a larger message fails with NO. - imapduser rewrites the credentials file's ownership only when it creates the file or after -d, not on every write; -s is not ignored by -d; re-running imapduser -a does not fix the group. - smtpd.conf(5)'s directive is "smtp max-message-size". Added: the credentials file's line rules (comments, skipped lines, bcrypt only, uid and gid not 0, maildir relative without "." or "..", first valid line wins); the on-disk layout, with imapd.index and the lock and temporary files beside it; that a TLS key failing its check is not loaded; imapduser's username characters, id selection from 2000, and maildir mode. CAVEATS now lists RENAME INBOX refused, LIST return options and multiple patterns refused, FETCH BINARY items answered BAD, part-number HEADER, TEXT and MIME sections and the RFC822 items not returned, the response code each command gives an invalid UTF-8 name, and the reserved mailbox names. STANDARDS adds RFC 4616 and RFC 8314. imapduser.8's credentials line rendered with a space after each colon and now uses Ql.

Unstall accounts, find renamed mail, quieter STATUS, join address fields Change 1 of 6: Find a message renamed while a FETCH or SEARCH walk was paused The store reads cur/ once per command into a sorted snapshot and keeps it while a FETCH or SEARCH walk is paused. A walk pauses so that the account's other sessions can run, and if one of them changed a message's flags meanwhile, the file was renamed and the snapshot still named the old one. The lookup failed, the walk logged "indexed but missing on disk" and left the message out: a SEARCH ALL answered one short, a FETCH skipped a message the client asked for. RFC 9051 section 6.4.4 defines ALL as every message in the mailbox, and RFC 2180 section 4.3 allows a server to omit only messages that have been expunged. locate_message_file() and open_message_file() now treat a snapshot entry whose file has gone as stale: they read cur/ again, once, and look again. A message that really has gone still fails as before. Change 2 of 6: Plan STORE once, and stat only what is read STORE planned every message twice, once to check the whole command could be done before any rename, and again while renaming. It now plans once into the array it already kept, then renames from it; a rename that fails still undoes the ones before it. Each plan looked the file up with two fstatat(2) calls, one for new/ that almost always failed and one for a size that STORE and EXPUNGE never read. locate_message_file() now tries cur/'s snapshot first and new/ only on a miss, and a caller that passes no size gets no stat of the cur/ file. STORE and EXPUNGE read cur/ while holding the exclusive index lock, so nothing in imapd can rename behind them. Change 3 of 6: Let a SEARCH give way to the account's other sessions One process serves every session of an account and runs one command at a time. A SEARCH on keys the index answers, such as flags, never paused, so on a 100,000 message mailbox it held every other session of the account for its whole walk, about 6 seconds. It now yields every 1,000 messages, as it already did every 64 parser requests. Change 1 is what makes the pauses safe. Change 4 of 6: imapd.8: say that a long command delays the account's other sessions STORE, EXPUNGE, COPY and MOVE still run to completion. Before this change a STORE 1:* over 100,000 messages held the account for about 47 seconds, 34 of them in the renames and the index save that keeping flags in the filename requires. Change 5 of 6: Have STATUS write the index only when mail has arrived STATUS registered mail in new/ with its own copy of the scan, then rewrote the whole index, with an fsync(2) and a rename into the mailbox directory, on every call. Clients poll folders with STATUS, so every poll wrote the index, and the rename changed the directory the IDLE refresh watches, so each idling session on that mailbox read the whole index again. The copied scan also lacked the check that skips a new/ name containing ':' or a newline. index_append() refuses such a name, so one oddly named file made STATUS answer NO for a mailbox that SELECT opens; RFC 9051 section 6.3.11 keeps NO for "no status for that name". STATUS now calls refresh_index(), as SELECT does, which saves only when a message was added or the index is new. An error reading new/ now fails STATUS as it fails SELECT. On a 100,000 message mailbox a STATUS took 0.24 seconds before this change and 0.18 after. Change 6 of 6: Join repeated address fields in ENVELOPE, as SEARCH already read them SEARCH FROM, TO, CC and BCC matched an address in any occurrence of the field, but ENVELOPE showed only the first, and RFC 9051 section 6.4.4 defines those keys against the envelope. RFC 5322 section 4.5.3 says repeated To:, Cc: and Bcc: fields SHOULD be read as one list. ENVELOPE now joins every occurrence of each of its six address fields, each occurrence closing its own group, so an empty first Cc: no longer hides a second. RFC 5322 leaves repeated From:, Sender: and Reply-To: unspecified; they are joined too, so no address field has a rule of its own. SEARCH SUBJECT now matches only the first Subject:, the one ENVELOPE shows, as the SENT keys already use only the first Date:.

NOOP reports changes; describe forwards; check accounts; bound keymgr; set \Seen Change 1 of 6: Report other agents' changes at NOOP, and to IDLE from SELECT on NOOP answered OK and nothing else, so a client that polls rather than idles never learned of new mail, flag changes or expunges made by another session or by smtpd. RFC 9051 section 5.2 requires a mailbox size update whenever a command observes one, and section 6.1.2 names NOOP as the poll. IDLE did report such changes, but it took the mailbox as it found it at IDLE as its starting point, so a change made between SELECT and IDLE, or between DONE and the next IDLE, was never reported at all. The account worker now keeps what it last told each session from SELECT on, rather than from IDLE on. NOOP in the selected state asks it for the same comparison IDLE makes, and completes once the answer is written: EXISTS for arrivals, EXPUNGE for removals, and a FETCH carrying UID and FLAGS for each flag change, with MODSEQ once CONDSTORE is enabled. IDLE compares against the same record instead of starting afresh. The account worker answers every such request, even one it refuses, since NOOP now waits for the answer. A message the session expunges or moves out itself leaves that record as its EXPUNGE is sent, so it is never reported twice. One function now sends that EXPUNGE for EXPUNGE, UID EXPUNGE and both kinds of MOVE, so none can skip the step. A message the session appends to its selected mailbox joins the record. Its own flag changes do not, so a later NOOP or IDLE may report them again. A refresh still in flight when DONE arrives is now answered before IDLE's tagged OK. Its EXPUNGE, FETCH and EXISTS lines used to be dropped, though the account worker counted them as told, so they were never reported. Two errors in IDLE's reporting are fixed on the way. A removal and an arrival seen in one refresh left the count unchanged, so no EXISTS was sent and the client was one message short; EXISTS now follows any EXPUNGE a refresh sends, as RFC 9051 section 6.3.13's example does. And a session that had enabled QRESYNC was sent EXPUNGE for another session's removals, where RFC 7162 section 3.2.10.2 requires VANISHED. Change 2 of 6: Read sequence numbers as the session was told them A session's sequence numbers were read against the mailbox as it stood, not as the session had been told it stood. After another session expunged a message, FETCH 2 returned the message the client called 3, STORE 2 flagged it, and an EXPUNGE that followed removed it. RFC 9051 section 7.5.1 forbids EXPUNGE responses during FETCH, STORE and SEARCH so that the numbers stay in step; imapd kept the rule and lost the step. APPEND's EXISTS could also lower the count, which section 5.2 forbids. The account worker now reads every sequence set through the session's view, the record the first change keeps: number n is the nth message the session was told of, found by its UID, and every response is numbered the same way. Each command that reads the index first tells the session what changed: EXISTS and flag FETCHes always, EXPUNGE only where section 7.5.1 allows it, so UID FETCH and UID STORE may carry one and FETCH, STORE and SEARCH may not. A message expunged elsewhere and not yet reported stays in the view until it is. A command naming such a message answers as RFC 2180 section 4 describes. FETCH returns the others and a tagged NO [EXPUNGEISSUED] (section 4.1.2), and so does STORE without .SILENT (sections 4.2.2 and 4.2.3); STORE .SILENT stores the others and answers OK (4.2.1); SEARCH never matches it (4.3); COPY and MOVE copy the others, send the pending EXPUNGEs and answer OK (4.4.2), as RFC 9051 section 6.4.9 already asks of a UID COPY naming a UID that is gone. APPEND to the selected mailbox takes its EXISTS from the view, after any pending EXPUNGEs. A session's own flag changes are no longer reported back to it: STORE holds the index lock from its report to its save, so the mod-sequence it assigns is its own. Change 3 of 6: Describe and fetch forwarded messages; refuse what FETCH cannot return BODYSTRUCTURE refused any message holding a MESSAGE/RFC822 or MESSAGE/GLOBAL part, at any depth, so fetching the structure of a message with a forwarded message attached ended NO. RFC 9051 section 7.5.2 describes such a part by the envelope, body structure and line count of the message it holds, and its grammar allows no other description. The builder now writes them. The envelope comes from the code that answers ENVELOPE, now split from the read of the header so that both can use it. A forwarded message counts against the depth and part limits as one level, as SEARCH already counts it. Part numbers now reach through such a part, as section 6.4.5.1 describes: BODY[2] of a forward is the whole forwarded message, and BODY[2.1] its first part. HEADER, TEXT and MIME after a part number are still not supported. A FETCH naming such an item beside one imapd supports, or plain BODY[...] or RFC822.TEXT, returned the rest and ended OK, as though the item had been sent. Section 6.4.5 gives NO for data that cannot be fetched, and imapd already answers NO when a supported item cannot be produced. It now does the same here, after sending what it could. A FETCH naming only such items was already refused. Change 4 of 6: Refuse to start without the daemon accounts; document creating them imapd drops privileges to three accounts, _imapd, _imapauth and _imapkey, but nothing created them and README.md never named them, so a fresh install that followed it failed. A missing _imapkey stopped the daemon at startup; a missing _imapd or _imapauth let it start and then failed every connection. Each lookup's message promised an install script that does not exist. The parent now looks up all three before reading its configuration and exits with "unknown user" if one is missing, as bgpd, ldapd, ospfd, ripd, dvmrpd and rad do. The children keep their own lookups, now with the same message, and the three names are defined once in imapd.h. README.md gains the three useradd(8) lines, placed before the first mailbox account since imapduser(8) needs the _imapauth group. Change 5 of 6: Bound the listener's wait on keymgr; keep keymgr from blocking on one A listener asked keymgr for each private-key operation and then waited for the answer with no bound. A keymgr that stopped answering left every new TLS handshake waiting for ever: the login grace timer could not fire, and each stuck connection held an unauthenticated slot until new connections were refused on every port. Its answer is now awaited with poll(2) for at most 10 seconds over the whole exchange, as the store already waits for the parser and as relayd bounds its wait on its ca; after that the operation fails and the handshake ends. A failed RSA operation returned 0. RSA_private_encrypt(3) returns -1 on error, and under TLS 1.3's PSS padding libcrypto took the 0 as a signature of length 0 and sent it. Every failure now returns -1, so the handshake fails at the server with an alert. keymgr's end of each listener channel was a blocking socket, so a listener that sent requests and never read the replies could stop keymgr for everyone. keymgr now sets it non-blocking, as smtpd and relayd make their channels. The two wait loops are now one function. Change 6 of 6: Answer a plain BODY[...] and set \Seen as RFC 9051 requires Only BODY.PEEK[...] was answered. A plain BODY[...], the form RFC 9051 section 6.4.5 defines as setting \Seen and the form its own example uses, was dropped when the FETCH was parsed, so the FETCH ended NO and the message stayed unread. A plain BODY[...] is now answered as its BODY.PEEK form is, for every section that form supports, and sets \Seen. The account worker sets it before the walk, under the exclusive index lock, with the code STORE uses, which now lives in one function both call: one rename per message, one index write per FETCH, no mod-sequence change for a message already \Seen (RFC 7162 section 3.1.11), and the session's own change is not reported back to it. A message whose \Seen the FETCH set is answered with FLAGS, and once CONDSTORE is enabled with UID and MODSEQ, as RFC 7162 section 3.2.4 requires. In a mailbox opened by EXAMINE nothing is set (RFC 9051 section 6.3.3). RFC822, RFC822.HEADER and RFC822.TEXT, which RFC 9051 removed from its grammar, still end NO.

SEARCH BODY and TEXT, parse cache, RFC 5322 headers, and literals Ten changes, made one at a time and committed together. Each follows in the order it was made, under its own subject line. Together: the account worker keeps the ENVELOPE and BODYSTRUCTURE text it has had parsed; SEARCH answers BODY and TEXT, so it now answers every key of RFC 9051 section 6.4.4; ENVELOPE and SEARCH read addresses, groups and comments as RFC 5322 writes them; a string may be sent as a literal (RFC 9051 section 4.3) wherever imapd reads one, while a refused non-synchronizing literal's command now ends where RFC 7888 says it does; a header field is found when white space comes before its colon (RFC 5322 section 4.5); a bare CR in a quoted Content-Type parameter no longer loses that parameter and those after it; and SEARCH takes its CHARSET quoted, as RFC 9051 allows. Change 1 of 10: Cache ENVELOPE and BODYSTRUCTURE in the account worker Every ENVELOPE or BODYSTRUCTURE FETCH item cost a message open, a descriptor pass, a parse in the parser-worker and a wait for its reply, however often the same message had been asked for. On a mailbox of 10,000 messages, FETCH 1:* (ENVELOPE) took 4.4 times as long as FETCH 1:* (FLAGS), which needs no parser. The account worker now keeps the checked text of both items, in the shape of smtpd's envelope cache: a fixed bound with no directive, an entry moved to the front on each use, and the oldest evicted first. The bound is 4 MiB of text and bookkeeping per account worker. Only successes are kept; a message the parser could not answer is asked for again, and its strikes are counted as before. With the cache, a second FETCH 1:* (ENVELOPE) of the same 10,000 messages took about half as long as the first. The key is RFC 9051 section 2.3.1.1's: mailbox name, UIDVALIDITY and UID, which "must refer to a single, immutable (or expunged) message on that server forever". A hit must also match the message's file name in the index, so an index rewritten by another program, or a UIDVALIDITY issued twice, is a miss rather than another message's text. A hit still needs the message file on disk, as the uncached path does, so a FETCH answers as it did before for a message whose file has gone. EXPUNGE, CLOSE and MOVE drop the UIDs they end, DELETE and RENAME drop every entry of the name they end, and a FETCH drops its mailbox's entries when the first of them has another UIDVALIDITY. The key alone would make each of these cost memory rather than a wrong answer if missed. A message whose requested items need no parse, or are all in the cache, no longer waits for a parser-worker, so a warm FETCH of ENVELOPE or BODYSTRUCTURE does not start one. When an account worker that looked anything up exits, it logs one line with its hits, misses, additions, evictions, drops, stale entries found, and its peak entries and bytes, so the bound can be sized from what the cache holds. A worker that never fetched ENVELOPE or BODYSTRUCTURE logs nothing, so short sessions add no lines. Change 2 of 10: Answer SEARCH BODY and TEXT RFC 9051 section 6.4.4's BODY and TEXT were parsed and refused NO. They are now answered in the parser-worker, beside the header keys, from one request per candidate message as before; a request with either key reads the whole message, up to "attachment max", instead of the header alone. The message is walked as MIME. Only TEXT and MESSAGE parts are searched, as section 6.4.4 permits; base64 and quoted-printable are decoded first, as it requires. An unrecognized transfer encoding makes a part application/octet-stream (RFC 2045 section 6.4), so it is not searched. A missing or invalid Content-Type is text/plain (section 5.2), and so is a multipart with no boundary. An unrecognized multipart subtype is walked as mixed (RFC 2046 section 5.1.3), and a digest part with no Content-Type is MESSAGE/RFC822 (section 5.1.5). Preambles and epilogues are not searched. BODY matches no header field of any level: not the message's, not a MIME part's, not a forwarded message's. TEXT matches all of them, each field unfolded and its RFC 2047 words decoded, and the content BODY sees. A forwarded MESSAGE/RFC822 or MESSAGE/GLOBAL is walked into and its own parts decoded. HTML is searched as it is, markup included, and a phrase is matched only within one line of the decoded text. Message charsets are not converted; an ASCII word still matches beside other octets. A multipart has no limit on its parts; the message's size bounds them. An entity nested deeper than BODYSTRUCTURE's limit of 10 cannot be checked, so the SEARCH answers NO, as for any message the parser cannot check. Matching folds ASCII case and runs in time linear in the text (Knuth-Morris-Pratt), so what a crafted body and needle can cost grows with the text's length, not with the product of the two lengths. The message is read into a buffer that doubles rather than grows by 64K. Change 3 of 10: Parse RFC 5322 addresses for ENVELOPE and SEARCH as the RFC writes them ENVELOPE and SEARCH FROM, TO, CC and BCC share one address parse, and it did not know RFC 5322 section 3.4's groups or section 3.2.2's comments, and toggled on every double quote, escaped or not. So "team: a@b, c@d;" came out as a mailbox "team: a" and a host "d;"; an escaped quote in a display name dropped that address and every one after it; "jane@x.org (Doe, Jane)" split inside its comment; and a comment stayed in the host or the name, so SEARCH matched it. These were wrong answers with nothing to show for it. A group is now sent as RFC 9051 section 7.5.2 defines it: a marker holding the group's name before its members and an empty one after, an empty group included, so "undisclosed-recipients:;" is two markers where it was NIL. A group missing its ";" is closed at the end of the field. SEARCH still matches the members, not a group's name. Comments are removed before the list is split, each becoming one space, nested and with quoted-pairs, as mail(1)'s skin() drops them; a comment is never taken as a display name. A backslash escapes the next character in a quoted string. The display name loses its quoting and keeps single spaces between words; the local part loses its quoting; an address loses white space outside quotes, so the obsolete "jdoe@test . example" is "test.example". An obsolete route goes in the at-domain-list, not the mailbox, and SEARCH does not match it. An address that does not parse is still left out and the rest of the field kept. An address longer than 1024 octets is no longer dropped: only the whole ENVELOPE has a limit, as before. Change 4 of 10: End a refused non-synchronizing literal's command where RFC 7888 does A command whose non-synchronizing literal ("{n+}", RFC 9051 section 4.3) was refused, or answered before its end, had only the literal's octets discarded. The rest of the command, which RFC 9051 section 7.6 puts after the octets, was then read as a new command line, so "a SEARCH TEXT {5+}" followed by "hello UNSEEN" drew a second reply, "UNSEEN BAD Missing command", tagged with the client's own word. RFC 7888 section 3 requires the octets and the following line to be treated as part of the same command. The line after the octets is now discarded too, and if it ends in another "{n+}" those octets are discarded in turn. A non-synchronizing literal over 4096 octets still closes the connection, one of the two choices RFC 7888 section 4 allows, but the reply is now "* BYE [TOOBIG]" as that choice and section 5 describe, where it was "* BAD". APPEND's own refusal of a non-synchronizing literal over 4096 octets is removed. The listener closes the connection on such a line before any command sees it, so the check could not be reached. Change 5 of 10: Read literals as mailbox names and SEARCH strings RFC 9051 section 4.3 makes a literal, "{n}" or "{n+}" followed by n octets, one of the two forms of a string, and a client may send one wherever a mailbox name or a SEARCH string may go; the RFC's own SEARCH examples in section 6.4.4 do. imapd took a literal only as APPEND's message and answered BAD to every other, so a client that chose the literal form could not select, create or search. A parser that meets a literal ending the text it was given now asks for it instead of refusing it. The listener then gathers the command: it sends "+" for a synchronizing literal, reads the n octets and the line after them, and runs the whole command again from the start, as ldapd does with a request whose BER element is not all there yet. The parsers read a literal already in the gathered text by its count, so its octets may hold a space, a quote or a CRLF. Each command parses all of its arguments before it acts, so asking for a literal changes nothing; SEARCH now parses before it resets the session's last result. A command and its literals are held to 8191 octets. Over that, it is answered NO [LIMIT], before the "+" when the count is known in time. A NUL in a literal, which CHAR8 excludes, is answered BAD. In either case the rest of the command is discarded as RFC 7888 section 3 asks. This covers every mailbox argument, LIST's reference and pattern, SEARCH's strings and HEADER's field name. FETCH's HEADER.FIELDS names and ID's parameters still do not take a literal. Change 6 of 10: Take FETCH's header field names quoted or as literals, and gather ID's A HEADER.FIELDS name is an astring (RFC 9051 section 9, header-fld-name), but FETCH took it as an atom only: a quoted name was BAD, and a literal never reached the parser. Names are now read quoted, with RFC 9051's two escapes, or as literals, which FETCH turns into quoted strings before it parses, since a field name cannot hold the CR or LF only a literal could carry. The response echoes the list as the client sent it, a literal shown as the quoted string it became. A name that would hold a space is BAD, as the store takes the list space-separated, and so is an empty list, which header-list does not allow. The FETCH tokenizer no longer splits inside a quoted string. ID answered OK without reading its parameters, so a client that sent one as a synchronizing literal got the reply before its "+". It now reads just far enough to gather its literals (RFC 2971 section 3.1) and still answers NIL. Change 7 of 10: Accept a non-synchronizing literal on a pipelined command A command that arrived while another was in flight, and that carried a non-synchronizing literal, was refused with an untagged BAD, so its tag was never answered; RFC 9051 section 2.2.2 tags every completion, and section 4.3 lets a client send "{n+}" anywhere a literal may go. Its octets and the lines after them are now gathered into the queued command, held to the same 8191 octets as any other, and it runs in its turn like any pipelined command (section 5.5). A queued command also no longer waits for the store's next reply when the gathered command ahead of it finished without needing the store. Change 8 of 10: Find a header field whose name is followed by white space RFC 5322 section 4.5 lets any field put white space between its name and its colon, "From :" for "From:", and section 4 requires a receiver to accept that. imapd took a field's name to be everything before the colon, white space included, so such a field was never found: ENVELOPE showed NIL, SEARCH did not match it, HEADER.FIELDS left it out, and a "Content-Type :" multipart message was described and searched as one text/plain part. The name now ends before any SP or HTAB that precedes the colon, in the one function every header reader uses. HEADER.FIELDS still returns the field as the message holds it. Change 9 of 10: Keep a Content-Type parameter that holds a bare CR A CR, LF or NUL inside a quoted Content-Type parameter value made the quoted-string fail, and the parameter loop stopped there, so that parameter and every one after it were lost. text/plain; charset="us<CR>ascii" was shown with NIL parameters, and a multipart whose boundary came after such a parameter, or held the CR itself, could not be split: its BODYSTRUCTURE ended NO, its BODY[n] was empty, and SEARCH read the whole body as text, delimiter lines included. RFC 5322 section 4 says malformed data is not to be "irretrievably lost". The value is now kept, CR and LF included, so a boundary still matches delimiter lines that carry the same CR. A NUL becomes a space, since the value is held as a C string. Every parameter value already goes out through the nstring writer the other fields use, which sends a CR, LF or NUL as a space, since a quoted string cannot hold one (RFC 9051 section 9). A message holding a NUL is still refused for BODYSTRUCTURE, BODY[] and BODY[n], so a NUL in a parameter now matters only to SEARCH. Change 10 of 10: Read SEARCH's CHARSET as an atom or a quoted string RFC 9051 section 9 writes SEARCH's charset as "atom / quoted", but imapd took every octet up to the next space. So CHARSET "UTF-8" was compared with its quotes and refused NO [BADCHARSET], and a name the grammar does not allow, such as UTF(8 or a quoted string with no closing quote, was answered NO where its arguments are invalid, which section 6.4.4 answers BAD. The charset is now read as the grammar writes it: a quoted string, whose only escapes are \" and \\, or an atom, which refuses the atom-specials, controls and 8-bit octets; a space or the end of the command must follow it. A malformed charset is BAD. A well-formed one other than US-ASCII or UTF-8 is still NO [BADCHARSET], as section 6.4.4 requires.

re-architecture: parser process, account worker, admission control Twenty-five changes, made and checked on the test machine one at a time, committed together. Each follows in the order it was made, under its own subject line, with its own account of how it was checked. Together: COPY and MOVE link messages instead of reading them into memory; SEARCH parsing moves out of its own process and message parsing into a parser-worker; each logged-in account gets one store child shared by its sessions, with a parser only while one is needed; the auth-worker exits after its grant; three admission limits bound what one account, one address and the whole daemon may hold; TCP keepalive ends the sessions of clients that vanished; and SEARCH answers SUBJECT, HEADER, SENTBEFORE, SENTON, SENTSINCE, FROM, TO, CC and BCC in the parser-worker. Change 1 of 25: COPY and MOVE link messages instead of reading them into memory COPY and cross-mailbox MOVE used to read every message in the range whole into the store child's memory before writing anything, bounded at 64 MiB per message and 512 MiB per command. With 64 store children that is a large amount of memory reachable by ordinary clients, and the per-message bound refused to copy messages the server had accepted: "append max" goes up to 1 GiB, and mail from an MTA is not bound by it. A maildir message is never rewritten once delivered; its flags live in its name. So the copy is now the same file under a new name: pass 1 records each source path and its new name and reads nothing, and pass 2 linkat(2)s each one straight into the destination's cur/. If the link fails for any reason other than EEXIST (another filesystem, the link count at LINK_MAX, a file flag), the message is streamed through a 64 KiB buffer into tmp/, fsynced and renamed, as before but without holding it whole. EEXIST fails the COPY instead of falling back, because the fallback's rename(2) would replace the file already there. AT_SYMLINK_FOLLOW keeps the old behaviour for a symlinked message, whose target was what openat(2) used to read. pledge(2) and unveil(2) already allow this: SYS_linkat is PLEDGE_CPATH, which the store holds, and its "rwc" unveil gives the read and create permissions dolinkat() checks. The existing rollback is unchanged and covers both paths. Cross-mailbox MOVE still commits the destination before unlinking the source, so it now moves no data. COPY_STAGE_MSG_MAX and COPY_STAGE_TOTAL_MAX are gone, as is the banner comment that said commit_copy_messages() had no rollback. It has one, and a reviewer had believed the comment over the code. stage_copy_messages() lost a directory descriptor parameter that disagreed with the global it also used. A new wire test APPENDs a message one octet over the old cap and checks COPY to another mailbox, COPY into the selected one, and MOVE, each by size and by octets at the start, middle and end. Against the old code it failed all three for the right reason, the cap, as maillog showed; after, 24 of 24. It needs "append max" and "attachment max" raised, so it is run by hand. The copy test's optional low-cap section is removed with the cap. Not exercised: the fallback copy. Nothing a client can do makes a link fail. Change 2 of 25: remove the per-connection SEARCH-parsing process Every connection forked a third process whose only job was to parse the SEARCH grammar and send the parsed program back over imsg. The parse now happens in the listener-worker, which already parses every other command in that same process, holds no private key, no password hashes and no mail, runs as _imapd in /var/empty and is pledged "stdio recvfd". A grammar bug there reaches one session, because every connection has its own listener-worker. The isolation that remains is the one that faces a stranger's data: message content is parsed by the store child, and moving that parse into a process without write access to mail is separate work. A connection before login now costs two processes rather than three. On a 2013 Celeron J1900 with 8 GB, each process costs about 1.07 MiB of machine memory whatever it does, measured by free page count across 0, 10, 20 and 40 connections. Gone with it: PROC_SEARCH, IMSG_SETUP_SEARCH_PEER, IMSG_SEARCH_PARSE_REQUEST and IMSG_SEARCH_PARSE_RESULT, SESSION_SEARCH_PARSING, iev_search, listener_dispatch_search(), the parent's per-connection fork, its reaping and its search_pid bookkeeping, and the _imapsearch account the role dropped to. An installation that created that account can remove it. search_dispatch() now parses and calls search_dispatch_finish() directly; both, and the parser, are static to search_cmd.c. The parser is search_program_parse(), named for what it does rather than for the process that used to call it. struct imsg_search_parse_result is struct search_parse_result, and the two constants lose their ORACLE infix, since neither crosses an imsg any more. Clients see no change: the same BAD and NO replies from the same parser, and the same 8192-octet bound on criteria, which is smaller than the command line it is sliced from. The two "[UNAVAILABLE] search temporarily unavailable" replies are gone; both existed only for a missing or dead oracle channel. The build and the test suite are clean on OpenBSD 8.0-current, and the lock-timeout test of SEARCH passes 5 of 5 with "lock timeout 5". Change 3 of 25: accept a MIME body part that declares no headers A multipart body part may carry no header fields at all. RFC 2046 section 5.1.1 says the boundary delimiter is terminated "by either another CRLF and the header fields for the next part, or by two CRLFs, in which case there are no header fields for the next part", and that a part with no Content-Type field is text/plain. find_header_body_split() looked only for two consecutive line breaks, which such a part does not contain: it begins with the second CRLF and then the body. The function returned -1 and both callers read that as malformed. build_body_structure() failed the whole structure, so FETCH BODYSTRUCTURE answered "* n FETCH ()" with a tagged NO; find_mime_part() found no part, so FETCH BODY[n] answered an empty string with a tagged OK. Such a message arrives through APPEND, so producing one needs no access to the server. The function now treats a buffer that begins with a line break as having an empty header section, before the scan for a blank line. A part that does declare headers is unaffected: the scan it relies on is unchanged and still runs. Checked on the test machine against a twelve-message corpus covering nested multiparts, a message/rfc822 part, a base64 attachment, body text that nearly matches the boundary, an empty body, a folded header and quoted Content-Type parameters. Of every FETCH result that corpus records, exactly two moved. The part with no headers now reports ("TEXT" "PLAIN" ("CHARSET" "US-ASCII") NIL NIL "7BIT" 26 0) inside its MIXED parent, and BODY[1] returns its 26 octets. Every other line was byte-identical, including the two BODYSTRUCTURE refusals that are deliberate: message/rfc822 is still scoped out, and a part specifier naming a part that is itself multipart still returns nothing. Change 4 of 25: add a parser-worker process beside each store child The store child parses hostile message content in the same process that holds the account's maildir. A bug in the MIME or RFC 5322 parsing of a message reaches the mail it is parsing. This adds the process that separates the two, and nothing else: it answers no requests yet, so the tree's behaviour is unchanged. parser.c is the new role. It drains its init message and its peer descriptor on fd 3, refuses to run as uid or gid 0, chroots to /var/empty, drops to the account's own uid, and pledges "stdio recvfd". No "rpath", so openat(2) is denied (kern_pledge.c) and the confinement is a property of the process rather than of the file's care. It opens nothing: every byte it will parse arrives on a descriptor its peer passes with the request. The parent forks one beside each store child, because the store child cannot fork its own: its pledge has no "proc exec". The parser is told the account's uid and given that store child's channel, and nothing else; it is told no paths because it opens no files. Its lifetime needs no bookkeeping. The store channel is its only one, so it reads EOF and exits whenever that child does, however the child dies. The one window between the fork and the pairing, where it would have no store to outlive, is closed by hand. A store child now has two peers, so setup_recv_one_peer() grew a variant that reports the imsg id. The listener's IMSG_SETUP_PEER carries the session number and the parser's carries 0, which is safe as a discriminator because next_session_id starts at 1 and wraps to 1, so no session ever carries it. The store sorts its peers by that id rather than by arrival order, and refuses a duplicate or a missing one. Checked on the test machine. The build and the test suite are clean, and the twelve-message FETCH corpus is byte-identical, as it must be when no request type exists. Over six live sessions, ps shows one parser per store child, each at its own store's uid and none at root. Their state flags read Ipc against the store's IpUc: pledged and chrooted, with no unveil at all, which is the intended shape showing up in the kernel's own accounting. The store carries a locked unveil because it opens the maildir; the parser has none to lock because it opens nothing. Change 5 of 25: build ENVELOPE in the parser-worker, and check what comes back This moves the first attribute behind the parser-worker added in the previous change. build_envelope() now runs there, on a descriptor the store opens and passes, so the RFC 5322 header parsing and address-list decomposition of a hostile message no longer runs in the process that holds the maildir. read_message_header() keeps its signature and becomes an open, a call to a new descriptor-taking core, and a close. The core is what the parser can run: it has no "rpath" and cannot open a message itself. build_envelope() takes the message descriptor in place of the mailbox directory descriptor, which is the same argument shape, so no declaration changed. The request and its reply share one imsg type, as the keymgr forwarders in listener.c do, with a per-request serial on the imsg id field and the reply checked on both type and id so a stale one is discarded rather than answered. The store's side of the channel is a bare imsgbuf rather than an imsgev: every exchange is synchronous, so a libevent dispatcher would only race the reply the child is already blocked waiting for. The wait is bounded, which the comparable wait on keymgr is not. The peer here is the one process in this design assumed to be attackable, so a parser looping on crafted MIME would otherwise pin its store child for the session's life, and store children are a fixed, machine-wide resource. relayd bounds the same shape of wait at one second (RELAY_TLS_PRIV_TIMEOUT, usr.sbin/relayd/relayd.h); this is longer because a parse is not one private-key operation. sshd's privsep client blocks on its monitor with no bound at all (usr.bin/ssh/monitor_wrap.c), but there the process waited on is the privileged one. A parser that misses the deadline once is not asked again, since retrying would let a stuck one charge the full timeout per message. What comes back is checked before it is used. ENVELOPE text is spliced into an untagged FETCH response as raw bytes, which was safe while the store built it and was trusting its own builder. It is not safe when another process supplies it: a CR or LF inside the reply ends the untagged line early and what follows is read as a new server response. RFC 9051 section 4.3 excludes NUL, CR and LF from a quoted string, and parser_reply_safe() refuses a reply carrying any of them, or one that does not close its own parenthesised list and would swallow what follows it on the line. A confined process whose output is trusted is not confined. A refused or missing reply is reported the way an unreadable message already is, so no wire shape and no listener code changed. A fuzzing harness drives the check, built two ways on purpose: without the check, as the tree stood before it existed, it must report violations. It found 243988 of them over 308466 cases, and none with the check in place, with 8531 replies still accepted as safe so the check is not passing by refusing everything. Writing its cases found a fault in the check before any of it shipped: the first draft walked the text once and skipped a backslash-escaped byte without examining it, so a quoted backslash followed by CR would have passed. The scan for line breaks is now a separate unconditional pass over every byte. Checked on the test machine. The build and the test suite are clean and the twelve-message FETCH corpus is byte-identical, which is the result that matters here: every one of those envelopes is now built in another process, on a descriptor rather than a directory, marshalled back and validated, and not a byte moved. Change 6 of 25: strip NUL, CR and LF from an address display name envbuf_append_nstring() has always substituted a space for NUL, CR and LF, citing RFC 9051 section 4.3, which excludes them from a quoted string, and substituting rather than rejecting so one bad byte does not drop the field. envbuf_append_one_address() open-codes a second quoted string emitter for the display name, because it also has to unescape the source's backslash escapes, and that copy was missing the substitution. Two emitters in one file, one of which sanitised. The mailbox and host parts go through the sanitising one and were never affected; only the display name was. So a From header reading From: "quoted<CR>name" <a@b> produced an ENVELOPE carrying that CR inside a quoted string, which went into an untagged FETCH response as raw bytes. Such a message needs no access to the server: APPEND literal octets reach the store unfiltered, so a client can store one itself. Only a bare CR reaches this far. A full CRLF does not: hdr_next_field() ends the field there, and a folding continuation does not carry it through. So the effect is a corrupted response line rather than a forged one, and since the previous change it is neither, because parser_reply_safe() refuses the text. That is the wrong outcome too: the message loses its ENVELOPE rather than its CR. The substitution now runs after the unescape, so a quoted backslash followed by CR is caught as well. vis(3) is how base makes untrusted text safe to emit, and mda.c wraps it in quotes exactly as a quoted string is wrapped (usr.sbin/smtpd/mda.c). It does not fit here. VIS_SAFE deliberately treats CR as visible and would have left this alone (lib/libc/gen/vis.c), and without VIS_SAFE every byte above 0x7f is escaped, which would mangle the UTF-8 that RFC 9051 admits in QUOTED-CHAR. The exclusion set is not a judgement call: TEXT-CHAR is any character except CR and LF, so those two and NUL are the whole of it and every other octet is preserved. A fuzzing harness over the ENVELOPE builder found this. Its property is not crash-freedom, which the sanitisers already cover, but that every envelope the builder can produce is one parser_reply_safe() will send. A gap between those two sets is not an attack, it is a message quietly losing its ENVELOPE. It reported three violations over 402003 cases, all this one cause, and none after the fix. This is the path the parser-worker exists to contain, and it had no harness at all before this one, which is where the headerless body part bug lived until a characterisation run found it. Checked on the test machine: the build and the test suite are clean and the twelve-message FETCH corpus is byte-identical, which is what it should be, since none of those messages carries a bare CR and the substitution must not fire on ordinary mail. Change 7 of 25: build BODYSTRUCTURE in the parser-worker too This moves the attribute the split was built for. build_body_structure() is the recursive RFC 2045/2046 walk bounded by MIME_MAX_DEPTH and MIME_MAX_PARTS, and it is where hostile MIME does its real work. It now runs in the parser, on a descriptor the store opens and passes. What MIME parsing the store still does, for BODY[<section-part>], moves in the next change, with HEADER.FIELDS. read_message_body() splits the way read_message_header() did in the previous change: it keeps its signature and becomes an open, a call to a new descriptor-taking core, and a close. BODY.PEEK[<section-part>] still uses the whole-message read from the store and is unaffected; it moves in its own change. build_bodystructure() takes the message descriptor in place of the mailbox directory descriptor. The parser reads whole messages now, so it needs the same ceiling the store has: bodystructure_read_max rides on IMSG_PARSER_INIT, as it already rides on IMSG_STORE_INIT. The buffer is still sized to the message rather than to the ceiling, so the cap costs nothing on ordinary mail, and the parser frees it before answering, holding no state between requests. The channel is now written once rather than per attribute. The request and reply structs lose their attribute-specific names, and parser_envelope() becomes parser_request(type, label, dfd, basename, maxlen, ...) called directly from each of the two sites, with no wrapper in between. That is sshd's shape: mm_request_send() and mm_request_receive_expect() take the message type as an argument and sixteen typed callers use them directly, with no intermediate layer (usr.bin/ssh/monitor_wrap.c). The label names the attribute for the log, as read_message_body()'s already does. The alternative was a second copy of the serial, the poll(2) deadline, the id check and the reply validation; this tree already has one duplicated request loop, keymgr_forward_rsa() and keymgr_forward_ecdsa() in listener.c, and the timeout missing from it is missing from both copies. BODYSTRUCTURE needs no new check on the way back. It is spliced into the untagged FETCH response as raw bytes two lines below ENVELOPE, so parser_reply_safe() already covered it. The ENVELOPE harness grew to drive the recursive builder before this change was made, so that a fault would surface on the old code rather than after the move: build_body_structure, parse_content_type, split_multipart, mime_read_token_or_qstring, mime_str_upper and mime_is_tspecial. It found nothing. Unlike the address path, every quoted field here goes through envbuf_append_nstring(), which has always substituted a space for NUL, CR and LF, so the fault fixed in the previous change never existed here. That the result is not vacuous was checked by hand: a bare CR in a Content-ID, a Content-Description or a Content-Transfer-Encoding does reach the builder and does come back substituted. Checked on the test machine. The build and the test suite are clean and the twelve-message FETCH corpus is byte-identical, which is the whole claim: build_envelope() and build_bodystructure() now have exactly one caller each, both in the parser, so every envelope and every body structure in that corpus was produced in another process, read from a descriptor, marshalled back and validated, and not a byte moved. The two deliberate refusals are intact: message/rfc822 is still scoped out, and a part specifier naming a multipart part still returns nothing. Over seven live sessions ps shows one parser per store child at its own store's uid, none at root, each pledged and chrooted with no unveil, and their resident sizes unchanged from before they read whole messages. Change 8 of 25: move BODY[<part>] and HEADER.FIELDS into the parser-worker These were the last two FETCH items the store still parsed. A section-part ran the whole MIME walk, parse_content_type() and split_multipart() included, and HEADER.FIELDS split RFC 5322 fields and their folds, both in the process that holds the maildir. Both now run in the parser on a descriptor the store passes. The store still looks for the blank line that ends a header, for BODY[HEADER], BODY[TEXT] and BODY[], as smtpd's queue does when it bounces a message with headers only (usr.sbin/smtpd/bounce.c), and does nothing else with message content. Neither item is spliced into the response line the way ENVELOPE is. Both go out as literals, whose length is framed, so a CR or LF from a hostile parser is only an octet. What such a parser could still do is put a NUL on the wire, which RFC 9051 section 9 allows only in literal8, or name a range the listener cannot read, which it answers by shutting the connection because the literal's length has already gone out. So each reply has its own check in the store, chosen by type in parser_request(): parser_literal_safe() header octets: no NUL, within FETCH_HEADER_MAX parser_extent_safe() a part's offset and length inside the file, with no overflow extent_has_nul() no NUL in that range, read by the store The part's octets never cross the channel. The parser returns where the part lies, and the store checks that against fstat(2) on its own descriptor, never against a size the parser reported. It is the same descriptor the listener then reads: the caller opens the message once and parser_request() sends a dup(2) of it, so the extent is checked against the file the client is actually sent. ENVELOPE and BODYSTRUCTURE open at the call site too, so all four callers use one shape. read_message_header_fields() splits into filter_header_fields(), which works on a header already in memory, and a descriptor-taking caller. extract_mime_part() takes the message descriptor in place of the mailbox directory descriptor. read_message_body() lost its last caller and is removed. A section-part that names a multipart or message/rfc822 part still comes back empty, exactly as a missing part does. That is a deliberate divergence, and this change keeps it. The harnesses and tests landed before the change: - The parser baseline fetches HEADER.FIELDS, HEADER.FIELDS.NOT, a partial part and a missing part. Before this, its reference had never seen HEADER.FIELDS at all. - The ENVELOPE harness drives find_mime_part() and locate_mime_part(). It pins which parts are found, including the divergence above, and checks that every part found lies inside its body. Both directed-case checks were mutation-tested: flipping one expectation makes the harness fail. After the split it also drives filter_header_fields(). A selection and its .NOT must together make up the whole header plus one blank line, the filter must be idempotent, and any header the store would have read must pass parser_literal_safe(). - The reply-check harness has directed and swept cases for the three new checks. Built without them it reports NULs in literals, an oversized literal and short reads; built with them it reports none, with 20288 header literals and 473 part extents still accepted. - The header-walk test's generator had gone stale: its stub no longer matched read_message_header_fields()'s signature, so it could not build against the tree. It is repaired and now splices filter_header_fields(). Its 24 rows are byte-identical to the committed test's, both before and after the split. Checked on the test machine. The build is clean with no compiler warnings, and the test suite passes, 16 scripts of 16. Before this change was installed, the extended FETCH corpus was recorded against the old build: its earlier items were byte-identical to the previous reference, and all 48 new ones answered OK. After installing, the corpus is byte-identical to that record, all 144 fetches. read_message_header_fields() and extract_mime_part() now have exactly one caller each, both in the parser, so every header selection and every part in that corpus was found in another process and checked by the store, and not a byte moved. The two deliberate refusals are intact: message/rfc822 BODYSTRUCTURE is still scoped out, and a part specifier naming a multipart part still comes back empty. ps shows the parser beside its store child at the store's uid, pledged and chrooted with no unveil, at the same resident size as before, and maillog has no refused reply and no missed deadline. Change 9 of 25: gather a session's FETCH walk, IDLE and APPEND state into one structure The store keeps everything about its one session in file scope. That is right while a store child serves exactly one session, and it is what stands between this tree and a process that serves several sessions of one account. This is the first step of undoing it, and it changes no behaviour. struct store_session (store_internal.h) now holds the listener's channel and four pieces of per-session state that were statics until now: the paced FETCH walk and its batch counters mbox_fetch.c the IDLE change probe index.c the IDLE baseline, the UIDs last reported index.c the one APPEND in flight mbox_manage.c store.c keeps the one instance and registers it as the listener channel's callback argument, so store_dispatch() receives it. It reaches the handlers that use this state as an explicit argument: handle_mbox_select(), handle_mbox_fetch(), handle_mbox_idle_refresh() and the three APPEND handlers, together with the fetch_walk_*(), idle_*_reset() and append_abort() helpers. The lock-wait machinery records the session rather than the channel, so a deferred command is replayed against the same state it was deferred with. No handler reaches the structure by name, so nothing in it can be read on behalf of the wrong session once a process holds more than one. Still in file scope, and moved by the next change: the selected mailbox and its descriptor, the cur/ snapshot, and the lock-wait record itself. session_id stays a process global; it labels log lines. A wire test was added first. It fetches forty bodies in one command, which makes the walk pause twice at FETCH_FD_MAX and resume from the store's EV_WRITE handler. Nothing in the suite had fetched more than sixteen bodies in one command, so the pause and the resume had never run under it. It is now in the suite. Checked on the test machine. The new test passed against the build before this change, so it pins existing behaviour, and passes against this one. The build is clean with no compiler warnings, and the test suite passes, 17 scripts of 17. The FETCH corpus is byte-identical to the reference, all 144 fetches. With a five-second lock timeout, a FETCH, SELECT and APPEND held behind another session's STORE each wait, are refused NO [INUSE] within the bound, and leave the session usable, and an IDLE whose seed waited reports the holder's change once the lock is free. With seven sessions over three accounts logged in, ps shows one parser beside each store child at the store's uid, and maillog has no refused reply and no missed deadline. Change 10 of 25: drop the cur/ listing of a command that finished after a lock wait A command reads cur/ once and keeps the listing for as long as it runs (cur_snap, mime.c), and store_dispatch() drops it after every request. A command that could not take the index lock is not finished there: it is re-run from a timer by deferred_retry(), and nothing dropped the listing that run built. A STORE renames the files whose flags it changes, so after a STORE that had waited, the listing named files that were gone. The session's next command then found the old name, failed to stat it, and skipped the message as "indexed but missing on disk". A FETCH left the message out of its answer. A second STORE on it answered OK and changed nothing. A name absent from the listing falls through to a real scan of cur/; a stale name present in it did not. deferred_retry() now drops the listing when the command it re-ran is done, under the same rule as store_dispatch(): not while a FETCH is paused mid-walk, since that command has not finished. A wire test was added first. It makes a second session's STORE wait behind a STORE 1:* over a large mailbox, then checks that the next FETCH answers for the message with its new flags, and that a following STORE takes effect. Like the lock-timeout test it needs a large mailbox, so it is run by hand. Checked on the test machine. Against the build before this change the new test failed 4 of its 12 checks: in both rounds the FETCH after the waited command gave no response for the message, and maillog logged it as missing on disk. In the second round maillog shows only the FETCH skipping it, not the STORE between (a STORE plans in two passes and would log twice), which fits that STORE having waited too and left a stale listing of its own. With this change the build is clean with no compiler warnings, the new test passes 12 of 12, and the suite passes, 17 scripts of 17. Change 11 of 25: move the selection, the cur/ listing and the lock wait into the session The second step of gathering a store child's per-session state into struct store_session, after the FETCH walk, IDLE and APPEND. It changes no behaviour. What moved, from file scope into the session: mailbox_selected, the store's own gate store.c selected_mailbox, the selection's name store.c mailbox_dir_fd, the selection's directory store.c cur_snap, one command's listing of cur/ mime.c deferred, the command waiting for a lock store.c The handlers that reach any of it now take the session in place of the listener's channel: STORE, EXPUNGE, SEARCH, STATUS, COPY, MOVE, DELETE and RENAME, with the helpers under them. SELECT, FETCH and the IDLE refresh already did. CREATE, LIST, SUBSCRIBE and UNSUBSCRIBE touch none of it and keep the channel. The cur/ listing functions in mime.c take the listing they work on, struct cur_snapshot, rather than a static, so locate_message_file(), open_message_file(), read_message_header() and message_body_range() gain it as their first argument. The parser links mime.c and calls none of them. The lock-wait record keeps its one slot, now per session. Its retry timer carries the session as the callback argument, as the listener channel already does, so deferred_retry() and the functions around it take the session instead of reaching a static. What is left in file scope is per process: session_id, which labels log lines, the maildir root, the configured limits, append_counter and the parser channel. The delete-selected wire test was extended first. After a RENAME of the selected mailbox, a COPY and a MOVE to the new name must take the same-mailbox path. They choose it by comparing the destination with the selection's name, which is one of the things that moved. Checked on the test machine. The build is clean with no compiler warnings, and the test suite passes, 17 scripts of 17, the extended delete-selected test 18 of 18. The FETCH corpus is byte-identical to the reference, all 144 fetches. With a five-second lock timeout, each of the eleven commands that can wait for the lock (STORE, EXPUNGE, FETCH, SEARCH, CLOSE, SELECT, STATUS, COPY, MOVE, APPEND and IDLE) waits behind another session's STORE and is bounded, refused NO [INUSE] and followed by a working session, or for IDLE reports the holder's change. The lock-wait snapshot test passes 12 of 12. With seven sessions over three accounts logged in, ps shows one parser beside each store child at the store's uid. maillog has no refused reply and no missed deadline, and nothing has been logged as missing on disk since the lock-wait fix went in. Change 12 of 25: attach a store child's session at runtime; the parent says when it exits A store child used to receive its one listener channel during setup, serve it, and exit when the session ended. It now takes its sessions at runtime over its parent channel, as keymgr takes listener peers, and exits only when the parent tells it to. The parent still attaches exactly one session to each child, so nothing a client can see changes; every login now goes through the attach path, and every logout through the detach path, before any child carries a second session. In the store child (store.c): setup brings only the parser peer, then IMSG_SETUP_DONE store_parent_dispatch() keeps fd 3 on an event and takes IMSG_SETUP_PEER (attach a session, its number in the id) and IMSG_STORE_EXIT (detach everything and exit) store_attach() allocates the session and opens its directory; a duplicate id, or id 0, is refused store_detach(), on IMSG_STORE_SHUTDOWN or the listener's channel closing, frees one session and leaves the child running the sessions are a list, not a static pledge gains recvfd, the set smtpd's queue pledges The handshake can read the first attach along with IMSG_SETUP_DONE, so the dispatcher is run once by hand before the event loop starts; without that the session would sit in the buffer until the parent next wrote. If the parent goes away, the child serves the sessions it has and exits with the last of them, which is what it did before. In the parent (parent.c), the listener's channel is sent after IMSG_SETUP_DONE rather than before the parser's, and reap_child() sends IMSG_STORE_EXIT to a session's store child when it reaps that session's listener. IMSG_STORE_EXIT is new in imapd.h. A wire test was added first. It opens and ends 80 sessions one after another, half with LOGOUT and half by dropping the connection, and checks that every one logs in. A store child that outlived its session would count against STORE_CHILD_MAX (64) until logins were refused. It is in the suite, now eighteen scripts. Also, a comment in store_dispatch() that pointed at mime.c for the cur/ listing now points at struct cur_snapshot, where that description moved. Checked on the test machine. The new test passed against the build before this change, and passes against this one. The build is clean with no compiler warnings, the suite passes, 18 scripts of 18, and the FETCH corpus is byte-identical to the reference, all 144 fetches. The lock-wait snapshot test passes 12 of 12. ps lists the same processes before and after the store-exit test's 81 sessions, and with a mail client logged in to three accounts it shows one parser beside each store child at the store's uid. maillog has no refused reply, no missed deadline and no refused session. Change 13 of 25: one store child per account: later logins join it The parent now keeps one store child per account, keyed by uid, gid and maildir, and attaches each later login of that account to it over the parent channel, the way the first one is attached. The child serves the account's sessions side by side, each with its own selection, FETCH walk, IDLE state and lock wait, as the last three changes arranged. Twelve connections of one account used to cost twelve store children and twelve parsers; they now cost one of each. In the parent (parent.c): struct store_child records the account it serves, how many of its sessions' listeners are alive, and whether it has been told to exit; open_session records its store child parent_handle_store_fork() attaches to a live child of the same account, or forks one if there is none store_child_session_gone() sends IMSG_STORE_EXIT when the last listener goes, and a child told to exit takes no new login store_child_teardown() fails every session attached to a child that dies during setup, not only the first STORE_CHILD_MAX now counts accounts In the store child (store.c), a read, write or framing failure on one session's channel detaches that session rather than calling fatal(), since the account's other sessions are served by the same process. session_id, which labels log lines, is set to the session being served on every entry from the event loop. The store child's title is now "store uid N" and the parser's "parser uid N", since neither serves one session any more. One behaviour does change. A long command in one session, such as a STORE over a large mailbox, now delays every other session of the same account until it finishes, because one process serves them all. Before, only sessions that needed the same mailbox's lock waited, and a wait was bounded by "lock timeout" and answered NO [INUSE]. Between sessions of one account the lock wait no longer happens at all; it still guards against a second process. The parser is still one per store child, so it is now shared by the account's sessions, and a parser that misses its deadline or dies stops message parsing for the whole account until its sessions end. Starting it on demand and bounding its CPU is the next change. A wire test was added first. Two sessions of one account each see the other's APPEND and STORE, including while idling; one pages a 40-message FETCH through its pause while the other runs a command; one EXAMINEs INBOX without moving the other; and each goes on working after the other logs out or drops its connection. It is in the suite, now nineteen scripts. Checked on the test machine. The new test passed 14 of 14 against the build before this change and passes 14 of 14 against this one. The build is clean with no compiler warnings, the suite passes, 19 scripts of 19, and the FETCH corpus is byte-identical to the reference, all 144 fetches. With a five-second lock timeout, the lock-timeout test now fails its "bounded" and "NO [INUSE]" checks, as intended: the waiter is served after the holder's 47-second STORE, with OK, because both sessions are in one process. It passed before. ps shows two sessions of one account behind a single store child and a single parser, and six Mail.app sessions over three accounts behind three of each, where there were six. maillog has no refused reply, no missed deadline and no refused session. Change 14 of 25: parser on demand, with a CPU bound and three strikes per message With one store child per account, the parser became one per account too, and a parser that missed its deadline or died stopped message parsing for every session of the account until the last one ended. The parser is now started when a request needs one, let go when it is idle or used up, and replaced when it fails. In the store child (store.c, mbox_fetch.c), there is no parser at setup. A FETCH walk that needs one and has none asks the parent (IMSG_PARSER_WANT) and waits at that message, as it already waits for its queue to drain. It goes on when IMSG_SETUP_PEER with id 0 brings a parser, or without one on IMSG_PARSER_NONE or after PARSER_REPLY_TIMEOUT_SEC. Before each message the walk checks the parser's channel without blocking, so a parser that has died since its last request is replaced rather than blamed. A deadline missed, or the channel lost once a request has been sent, lets the parser go and counts against that message. At PARSER_STRIKES (3) the message is not sent to a parser again while the child lives; the count is kept in the child, never in the parser. A parser unused for PARSER_IDLE_SEC (60) is let go by closing its channel. The parser-channel failures that called fatal() now let the parser go instead, since the far end of that channel is the process assumed to be attackable. In the parent (parent.c), IMSG_PARSER_WANT forks a parser for that store child, ending any earlier one it still has, so there is at most one per account. RLIMIT_CPU is set between fork and exec, 8 seconds soft and 10 hard, because the parser cannot set it under pledge. In the parser (parser.c), a reply sent once it has used PARSER_CPU_RETIRE_SEC (4 s, half the soft limit) of CPU says so, and the parser exits after sending it; the store lets it go on reading that. Honest work therefore never reaches the limit. The worst honest request measured 1.7 s of CPU on the test machine. A wire test was added first. It fetches ENVELOPE and BODYSTRUCTURE, waits while the account's parser is killed on the server, and fetches them again. It needs a person on the server, so it is run by hand. Checked on the test machine. The new test fails 3 of 8 against the build before this change: once the parser is killed, FETCH answers neither ENVELOPE nor BODYSTRUCTURE, and says NO. It passes 8 of 8 against this one, with nothing new in maillog. The build is clean with no compiler warnings, the suite passes, 19 scripts of 19, and the FETCH corpus is byte-identical to the reference, all 144 fetches. ps, sampled every five seconds through one session, shows no parser after login, one from the first FETCH ENVELOPE, none again about a minute later with the session still open, and the store child gone at LOGOUT. maillog has no refused reply, no missed deadline, no refused session, no parser that failed to arrive and no message struck out. Change 15 of 25: hold a mailbox's index lock from outside imapd, for the lock tests One store process now serves every session of an account and runs one command at a time, so two sessions of one account never contend for an index lock: the second session's command is served after the first and never meets the lock. The two lock tests made their holder a second session of the same account, so they no longer reached the code that waits for a busy lock, which is kept for a second process. A small program added here is that second process. Run on the server as root and given an account's maildir and a mailbox in it, it takes flock(2) LOCK_EX on the mailbox's imapd.index.lock while a mailbox named lockhold exists at the top of the same maildir, and releases it when that goes. The tests CREATE and DELETE lockhold from a third session, so the test decides when the lock is held, and nothing on the two machines has to be started at the same moment. Taking one maildir for both means the lock and the signal cannot name different accounts. A hold ends after -t seconds (default 60) whatever the signal says, so a test that dies cannot leave the mailbox locked. It only reads: the lock file is opened O_RDONLY without O_CREAT, flock(2) takes LOCK_EX on a read-only descriptor, and it pledges "stdio rpath flock". The lock-timeout test can use either holder, the external one by default. The imap holder is kept: against the shared process it shows that two sessions of an account are served one after the other. With the external holder it makes 4 checks rather than 5, since the hold cannot be seen from the client; "not early" and "refused" show it together. IDLE is not tested with the external holder, which changes nothing: "+ idling" is sent before the seed, so a seed that met the lock would leave nothing on the wire to check. The lock-wait snapshot test uses only the external holder, since the imap one would now pass without reaching the lock-wait code. The hold is 2 seconds, so it runs under the same short "lock timeout" as the other test, on any mailbox holding the message it works on. It makes 10 checks rather than 12: the holder's own STORE is gone, and the check that the waiter really waited now times it against the release. Its docstring also named the cur/ listing by its place before it moved into struct cur_snapshot. No daemon code changes. Checked on the test machine, against the installed account-worker build with "lock timeout 5" and a mailbox of 100,000 messages. The holder builds with no compiler warnings. With it holding the mailbox's lock from outside imapd, the lock-timeout test passed 4 of 4 for each of the ten commands it sends (store, expunge, fetch, search, close, select, status, copy, move, append): each waited, was refused NO [INUSE] within the bound and not early, and left the session working. The holder logged ten holds of 6.3 to 6.5 seconds, none of which had to wait for imapd. That run used an earlier build of the holder that took the lock file and the signal as two paths; the rest used this one. The snapshot test passed 10 of 10. With the holder stopped, the timeout test failed 2 of 4, the two checks that need a held lock, so those checks depend on it. maillog has no "missing on disk", "cannot be sent", "did not answer within" or "refusing a session" line. Change 16 of 25: end the auth-worker once the parent holds its grant Each connection has its own auth-worker. Once it has granted a login it has nothing left to do: it refuses a second grant for its session, and IMAP has no way to authenticate again after login. It stayed alive only until the listener's channel closed at the end of the session, so every logged-in connection kept one process that did nothing. The parent now sends IMSG_AUTH_EXIT once it has accepted the worker's IMSG_AUTH_CRED, and the worker flushes its answer to the listener and exits. The parent says when, rather than the worker exiting as soon as it has answered, because reap_child() closes a child's channel without reading it: a worker that exited on its own could take with it a grant the parent had not yet read. The worker's end matches sshd's, whose pre-authentication child exits once authentication is done, and the parent deciding matches IMSG_STORE_EXIT. The listener now expects its auth channel to close after a login and logs that at debug level. Before a login it is still a warning. A connection that never logs in is unchanged: its auth-worker takes every attempt, and exits when the listener goes. One path changes as a consequence. After a login whose store spawn failed, a retried AUTHENTICATE reached a worker that refuses an already granted session without answering, so the client waited until the login grace ended the session, or for ever with "login grace 0". It is now answered NO [UNAVAILABLE], or, if the listener has not yet seen the worker go, the listener's write to it fails and the session ends. A wire test was added to the suite first. It checks that a failed attempt leaves the next one able to succeed on the same connection, that the session works once the worker is gone, that AUTHENTICATE and LOGIN after login are refused BAD, and that twenty logins in a row each succeed. A client cannot see the change, so it passes before and after; ps(1) on the server shows the difference. Checked on the test machine. Against the build before this change, the new test passed 8 of 8, and ps(1), sampled every 5 seconds, showed the held session's auth-worker alive for the whole two minutes it was logged in. With this change the build is clean with no compiler warnings, the suite passes, 20 scripts of 20, with the new test at 8 of 8, and the parser baseline is byte-identical to the reference. ps(1) over four minutes showed no auth-worker behind any logged-in session, the held one or the four others logged in at the time. maillog has no "auth-worker closed channel", "refusing IMSG_AUTH_CRED", "IMSG_AUTH_EXIT", "imsgbuf_flush IMSG_AUTH_RESULT", "cannot be sent", "did not answer within" or "refusing a session" line. Change 17 of 25: imapd.8: one store child per account, and what lock timeout bounds Three statements in the manual had gone stale. The description said a store child is forked per authenticated session. One store child now serves each logged-in account, shared by all of that account's sessions. "login grace" counted three processes per accepted connection, including the search-oracle, which is gone. A connection costs two until it logs in: its listener-worker and its auth-worker. "lock timeout" said a command waits for another session of the same user. That user's sessions share one store child, which runs one command at a time, so they never wait for each other's lock; a long command delays the account's other sessions until it is done, and the directive does not bound that. What it bounds is a wait on a lock held by another process: a store child whose sessions have all ended, still finishing a command, while a new login for the same account starts another. Checked on the test machine. mandoc -Tlint reports only the two STYLE notes it reported before: the missing RCS id, and imapduser(8) not found. The build it was checked on also carries the "account sessions" change. Change 18 of 25: limit the sessions one account may hold: "account sessions" Nothing bounded how many sessions one account could open. Each costs a listener process, all of them share the account's store child, and a client configured to open many connections could have as many as it asked for. The new directive "account sessions", default 8, caps them. An account is the store child's key, a uid, gid and maildir from the credentials file, and the parent already counts the sessions attached to each. When a login would pass the cap, the parent refuses to attach it, and the listener answers NO [LIMIT] (RFC 9051 section 7.1) and closes the connection. It does not keep the connection for a retry: the connection's auth-worker has spent its one grant, as sshd's pre-authentication child does, and sshd likewise disconnects when a limit is reached during authentication. No BYE is sent, since RFC 9051 section 7.1.5 lists four conditions for one and a refused login is not among them. The client may log in again on a new connection once one of the account's sessions has ended. The other failures on that path, a store child that could not be started or did not finish its setup, and a second grant for a session already authenticated, now close the connection the same way, after the same NO as before. Left open, such a connection could not log in again, and it was no longer counted by the startups throttle, which counts only connections not yet authenticated, so it lasted until the login grace ended it, or for ever with "login grace 0". The directive is reloaded on SIGHUP and documented in imapd.8 and imapd.conf.example. A wire test came first. It logs one account in up to the cap, and checks that one more login is refused NO [LIMIT] and closed without a BYE, that the sessions already open still work, and that once one logs out the next login succeeds and the one after it is refused again. It needs an account no other client is logged in as, so it is run by hand. Checked on the test machine. Against the build before this change the new test failed 4 of 7, as predicted: the ninth login and the one after the logout were both answered OK. With this change the build is clean, with no compiler warnings and no yacc conflicts, mandoc reports only its two existing STYLE notes, the suite passes, 20 scripts of 20, the parser baseline is byte-identical to the reference, and the new test passes 7 of 7. maillog shows each of its two refusals as the parent's "refusing login: uid ... already has 8 sessions (account sessions)" followed by the listener's "closed ... reason=limit-exceeded". Change 19 of 25: limit the connections held at once: "connections max" Nothing bounded how many connections imapd accepted. Each costs a listener process, and one more until it logs in, and the parent and keymgr keep a descriptor for each. On OpenBSD's defaults, the process table and the parent's descriptors run out at about a thousand connections, long before memory does, and would run out in the middle of spawning a session. The new directive "connections max", default 256, bounds them. The parent counts every session it tracks, logged in or not, and checks the count in its accept loop before MaxStartups and before any fork. Past the limit a connection on the cleartext port gets RFC 9051's rejected-connection greeting, "* BYE [UNAVAILABLE]", written best effort as sshd writes its own refusal, and is closed. On the TLS port that greeting would have to travel inside TLS, which the parent does not speak, so the connection is closed with nothing sent, as httpd does at its client limit. The parent logs once when the limit is reached and once when it accepts again, as smtpd does. C connections, L of them not yet logged in, for A accounts need up to C + L + 2A processes plus two, and the parent keeps a descriptor to each. The default leaves room for the worst case, every connection a different account, within OpenBSD's default kern.maxproc and the daemon login class's openfiles. imapd.8 gives that rule, and says to raise those limits before raising this one. STORE_CHILD_MAX goes. It capped store children, one per account, at 64, so it refused the 65th account to log in, well inside the new default, and "connections max" bounds the accounts anyway. accept(2) failing for descriptors now pauses the listening sockets for a second, as httpd does. Before, the listen event stayed armed and fired again at once. The comment on the parent's duplicate-grant path said it was reachable by an ordinary retry after a failed store spawn. It has not been since the auth-worker began exiting after its grant; only a broken auth-worker gets there. The directive is reloaded on SIGHUP and documented in imapd.8 and imapd.conf.example. A wire test came first. It needs a low "connections max" in the running imapd.conf, so it is run by hand. It opens connections until one is refused, and checks the refusal on port 143 is the BYE greeting and a close, the refusal on port 993 a close with nothing sent, that the connections held still work, and that once one logs out a new connection is accepted and the one after it refused again. Checked on the test machine. Against the build before this change the new test failed 4 of 6, as predicted: nothing was refused, and port 143 answered with its OK greeting. With this change the build is clean, with no compiler warnings and no yacc conflicts, mandoc reports only its two existing STYLE notes, the suite passes, 20 scripts of 20, and the parser baseline is byte-identical to the reference. With "connections max 6" the new test passes 6 of 6, itself holding 3 connections with 3 already open, and maillog shows "connections max reached (6 open), refusing new connections", then "below connections max again (5 open), accepting", then the first line again. Change 20 of 25: cap connections not yet logged in per address: "startups per-source" MaxStartups counts the connections not yet logged in across all addresses. One address could therefore hold every one of its slots, a hundred by default, with bare TCP connections and no password, and every other client was refused until the login grace closed them. The new directive "startups per-source", as sshd's PerSourceMaxStartups, caps what one address may hold, default 5, below "startups begin" so one address cannot even start the ramp; "none" turns it off. The parent keeps each session's address and counts, at accept and before any fork, the sessions from that address not yet logged in, as sshd's srclimit_check_allow() does. A session stops counting once it logs in. Each address counts on its own, IPv6 ones included, as sshd's default grouping does; imapd.8 says what that leaves open for IPv6, and that users behind one NAT address share the limit. sshd refuses PerSourceMaxStartups and MaxStartups through one function. All three admission limits now share one refusal here too: "connections max", the new cap, and MaxStartups, which until now closed silently on both ports. On the cleartext port a refused connection gets RFC 9051's rejected-connection greeting, "* BYE [UNAVAILABLE]", best effort, and is closed. On the TLS port it is closed with nothing sent, since the greeting would have to travel inside TLS. The directive is reloaded on SIGHUP and documented in imapd.8 and imapd.conf.example. A wire test came first, and joins the suite, which now runs twenty-one scripts, since the cap is on by default. From one address it holds the cap's worth of connections without logging in, and checks that one more is refused on each port in its own way, that the connections held still work, that one of them logging in makes room for another, that the next is then refused again, and that one logging out makes room too. Checked on the test machine. Against the build before this change the new test failed 3 of 7, as predicted: one more connection on each port was accepted. With this change the build is clean, with no compiler warnings and no yacc conflicts, mandoc reports only its two existing STYLE notes, the suite passes, 20 scripts of 20 before the new test joined it, the parser baseline is byte-identical to the reference, and the new test passes 7 of 7. maillog has no line for its refusals, which are logged at debug level, as MaxStartups' are. Once it had joined, the suite passed 21 scripts of 21. Change 21 of 25: set SO_KEEPALIVE on client connections After login nothing ended a session but its client. A client that vanished without closing its connection, a laptop put to sleep or a phone that lost its network, left its session open until the server next wrote to it, and a session that was never written to stayed for ever. With "account sessions" that is also a lockout: every such session counts against the account's limit, so a client that dropped off often enough could fill it until imapd was restarted. The parent now sets SO_KEEPALIVE on every connection it accepts, as sshd does by default for TCPKeepAlive, and as syslogd does for its TLS connections, so the kernel probes a quiet connection and drops it when the probes go unanswered. How long that takes is the kernel's: the net.inet.tcp.keepidle sysctl plus eight net.inet.tcp.keepintvl intervals, 7200 and 75 seconds by default, so 2 hours 10 minutes. imapd.8 says so under "account sessions". There is no directive: a connection is always probed, as syslogd's are. A failure to set the option is logged and the connection goes on. No inactivity autologout is added. RFC 9051 section 5.4 would allow one of at least 30 minutes, but once keepalive ends the sessions whose client has gone, all it would add is logging out clients that are alive and quiet, each of which then reconnects. A wire test cannot make its own client vanish, so the check is ktrace. Checked on the test machine. Against the build before this change, ktrace(1) on the parent while one connection was made showed accept(2) and the two fcntl(2) calls that set O_NONBLOCK, and no setsockopt(2). With this change the build is clean, mandoc reports only its two existing STYLE notes, the suite passes, 21 scripts of 21, the parser baseline is byte-identical to the reference, and the same ktrace shows "setsockopt(9,SOL_SOCKET,SO_KEEPALIVE,...,4)" returning 0 after the fcntl(2) calls. Change 22 of 25: parse the SEARCH content keys' operands before refusing them RFC 9051 section 6.4.4 lists eleven search keys that need a message's header or body: BCC, BODY, CC, FROM, HEADER, SENTBEFORE, SENTON, SENTSINCE, SUBJECT, TEXT and TO. imapd refused each by name with NO as soon as it read the key, so it never read the operand that follows. The listener now parses those operands, as the first step to answering the keys. The string keys take an RFC 9051 section 9 astring: an atom or a quoted string, with only the two quoted-specials escaped. HEADER takes a field name and a string. The three SENT keys take a date, read by the same function as BEFORE, ON and SINCE, now shared. Having parsed the whole program, SEARCH still answers NO for any of the eleven, with the same text as before, because nothing yet answers them. Two answers change now, because they can be given without matching: A literal operand is refused BAD, "literals are not supported in SEARCH, send a quoted string". A synchronizing literal is never sent its continuation, which section 2.2.1 allows for a command refused with BAD. Literals in commands other than APPEND are a change of their own. The string operands of one SEARCH may hold 4096 octets in all, counted after unquoting and including HEADER's field name. More is refused "NO [LIMIT]", the section 7.1 response code for an implementation limit. 4096 is section 4.3's cap on a non-synchronizing literal, and it leaves room for the operands beside the search program in one imsg. A malformed operand, such as an unterminated quoted string or a missing one, is now BAD rather than the old NO. Checked on the test machine. The build is clean and mandoc reports only its two existing STYLE notes; the suite passes, 21 scripts of 21; the parser baseline is byte-identical to the reference. A new wire test of the header keys, written before this change, makes 38 checks: before it 30 failed, each on the old NO; after it 27 fail, the same refusal, and the three that pass are the two over-cap SEARCHes answered NO [LIMIT] and the literal operand answered BAD with no continuation. Change 23 of 25: answer SEARCH SUBJECT, HEADER and SENT* from the message header RFC 9051 section 6.4.4's SUBJECT, HEADER, SENTBEFORE, SENTON and SENTSINCE are now answered, where before they were refused with NO. FROM, TO, CC, BCC, BODY and TEXT are still refused as before. The matching runs in the parser-worker, the process that already parses message content for FETCH, confined by pledge "stdio recvfd" in a chroot: the store never reads a header for SEARCH. A new request, IMSG_PARSER_SEARCH, carries the message's descriptor and the SEARCH's content keys with their strings; the reply carries one byte per key, 1 where the message matches it. The store checks that the reply is exactly that, one byte of 0 or 1 per key, before using it. The strings travel from the listener in a pool beside the search program, in IMSG_MBOX_SEARCH's fixed part, so the lock-wait record keeps them with the request as it is. A node names its string by offset and length, and the store refuses one that lies outside the pool. The store evaluates each message over three values first. A content key is unknown until the parser answers, and AND, OR and NOT carry that through, so a message its flags already decide is never sent to the parser: UNSEEN SUBJECT x asks only about unseen messages, and a SEARCH with no content key never starts a parser. The SEARCH walk can now pause, as the FETCH walk can: to wait for a parser to start, and every 64 parser requests to let the event loop serve the account's other sessions. A message the parser cannot check, because it missed its deadline, died, has struck out, or cannot read the header within "attachment max", ends the SEARCH NO. Answering as if it had not matched would look complete and not be. The matching itself, in search_match.c: A field's value is unfolded (RFC 5322 section 2.2.3) and its RFC 2047 encoded-words are decoded, B and Q, with the white space between two adjacent words dropped (section 6.2). The decoded octets are compared as they are: no charset is converted, since base has no iconv. Matching is a substring match, case folded in the ASCII range only, as RFC 9051 section 6.4.4 says it SHOULD be. HEADER matches any field of that name; an empty string tests that the field is present. SENT* take the Date: field's own calendar day, disregarding its time and zone, and accept RFC 5322 section 4.3's two- and three-digit years. A message with no Date:, or one that cannot be read, matches none of them. The header is read in blocks up to its blank line. A NUL in it does not stop the match, which goes by length. mime.c gains header_next_field(), which walks a header's fields for a caller outside the file, over the existing hdr_next_field(). parser_request() sends with imsg_composev(), as relayd's ca.c does, so a request can carry data after its fixed part; the four FETCH callers pass none. Checked on the test machine. The build is clean and mandoc reports only its two existing STYLE notes; the suite passes, 21 scripts of 21; the parser baseline is byte-identical to the reference. The header-key test, written before the grammar change, makes 38 checks; before this change 27 failed, after it 16, each a check that names FROM, TO, CC or BCC, still refused. SUBJECT "fill" matched all 10,000 messages of a bench mailbox in 2.4 s and 99,999 of 100,000 in 44 s; NOT SUBJECT "fill" matched the other one, whose Subject is a test's own. During the 100,000-message search, a second session of the same account logged in, selected and fetched in 0.36 s. ps showed a parser for the account during the search, replaced once as it retired, and maillog has no failed check, bad request or parser failure. Change 24 of 25: answer SEARCH FROM, TO, CC and BCC RFC 9051 section 6.4.4's FROM, TO, CC and BCC are now answered, where before they were refused with NO. Only BODY and TEXT are still refused. Section 6.4.4 matches these keys against "the envelope structure's" field, and an address in that structure keeps its display name, local part and domain apart (section 7.5.2). Each address is matched twice: as its display name, with its quoted-pairs undone and its RFC 2047 encoded-words decoded, and as mailbox@host rejoined, as smtpd rejoins an address as user@domain before matching a rule (ruleset.c, mailaddr_to_text()). So a whole address matches as it is written, and so does any part of it or of the name. Comments are not taken out: one after an address is matched as part of its host, and one in a display name as part of the name, as ENVELOPE shows them. What does not match: the angle brackets, the commas between addresses, and a group's name, since a group is no address. HEADER From still matches the field's text as it stands. The address parse is the one ENVELOPE already uses. It is split out of envbuf_append_one_address() as address_split(), and the top-level comma split out of envbuf_append_address_list() as address_list_next(); both formatters now call them, and what they produce is unchanged. The matching runs in the parser-worker, as SUBJECT's does, in search_match.c. Checked on the test machine. The build is clean and mandoc reports only its two existing STYLE notes; the suite passes, 21 scripts of 21; the parser baseline, which fetches ENVELOPE, is byte-identical to the reference. The header-key test makes 38 checks; before this change 16 failed, each naming FROM, TO, CC or BCC, and after it none. FROM "filler@example.com" matched all 10,000 messages of a bench mailbox in 2.4 s, as SUBJECT did, and 99,999 of 100,000 in 44 s. A FROM that matches nothing took the same 2.4 s, and SEEN FROM on a mailbox with nothing seen took 0.52 s, the time of a flag-only search, so no message went to the parser. maillog has no failed check, bad request or parser failure. Change 25 of 25: check the SEARCH matcher against references, and fix what that found search_match.c, the parser-worker's matcher for SUBJECT, HEADER, SENT*, FROM, TO, CC and BCC, now has a fuzzing harness. A sanitizer finds an overrun, not a wrong answer, so it also checks the answers against references written in the harness: ASCII-only case folding, RFC 2047 B and Q words decoding back to their octets, with the white space between two adjacent words dropped (section 6.2) and kept elsewhere, Date: days computed without timegm(), every obsolete year of RFC 5322 section 4.3, SENTBEFORE and SENTSINCE partitioning the dated messages, every mailbox@host of a generated address list, and headers read across the 64K block edge and the cap. A table carries the header-key test's messages and keys, and the address cases. Before the fixes below it failed, each time for a wrong answer: A word that fails to decode overwrote the space before it. decode_value() took the space back before decode_word(), which writes as it goes, so "=?x?Q?a?= =?x?B?QQ!A?=" decoded to "aA=?x?B?...". The word is now decoded after the space and moved back over it only if it decodes. A folded Date: was no date. RFC 5322 section 3.3 allows FWS between a date's tokens, and date_field_day() took only SP and HTAB. It now takes CR and LF too; the value's only CR and LF are folds. "attachment max" was rounded up to the next 64K block, and a message of header fields alone exactly that long was refused. read_header() now reads at most one octet past the cap and refuses a header longer than it, as read_body_from_fd() refuses a message longer than it. The header-key test joins the suite, now twenty-two scripts. Checked on the test machine. The build is clean; the suite passes, 22 scripts of 22, the header-key test among them; the parser baseline is byte-identical to the reference; the header test makes 38 checks and none fails. The matcher's harness, built there with the minimal UBSan runtime, ran 3,201,074 cases and exited 0; before the three fixes it failed on each of them, in a run elsewhere. maillog has no failed check, bad request or parser failure.

subscriptions, large messages, lock timeout, login grace, logging, IDLE SUBSCRIBE and UNSUBSCRIBE are implemented, and LIST takes the SUBSCRIBED selection option of RFC 9051 section 6.3.9.1. Both commands used to reply NO, and LSUB returned every mailbox the account had, because no subscription state was kept at all: the deprecated command worked and its replacement was missing. Subscriptions now live in one file per account, imapd.subscriptions, at the maildir root beside imapd.uidvalidity, one mailbox name per line. An absent file means every mailbox is subscribed. Only UNSUBSCRIBE creates one, and the first UNSUBSCRIBE writes out every other mailbox on its way past, so upgrading from 0.1.4 cannot leave an account subscribed to nothing, and a client that hides unsubscribed mailboxes cannot come back to an empty tree. A file that cannot be read is treated the same way, so damage shows too many mailboxes rather than too few. INBOX is never named in the file: it always exists and cannot be deleted, so it is permanently subscribed and UNSUBSCRIBE INBOX replies NO. A name outlives its mailbox, which RFC 9051 section 6.3.8 requires. LIST (SUBSCRIBED) reports such a name \Subscribed \NonExistent, and LSUB reports it \Noselect, which is RFC 3501 section 6.3.9's spelling of the same fact. A plain LIST reports no attributes at all and never opens the subscription file, so the request a client makes on every connection costs what it did before any of this existed. Neither CREATE nor RENAME changes the file; what is subscribed is left to the client to say, and the clients tested say it. Unsupported LIST selection options, REMOTE and RECURSIVEMATCH, are refused rather than ignored, since accepting one silently would misreport which names the response contains. A mailbox name the server refuses now draws NO rather than BAD. RFC 9051 section 7.1.3 reserves a tagged BAD for a protocol-level error in the client's command, and these are not errors of that kind: SELECT "" parses, since an empty quoted string is well-formed, and so does a name carrying the hierarchy delimiter. Each of SELECT, EXAMINE, CREATE, RENAME, SUBSCRIBE and COPY carries an explicit "NO - ... that name" in its own result table. Five sites changed: NO [NONEXISTENT] for SELECT's empty name, which is what the store's own no-such-mailbox reply already said, and NO [CANNOT] for the rest, which is what the non-UTF-8 refusal one line above four of them already said. BAD is untouched wherever a command genuinely fails to parse. Apple Mail sends SELECT "" before renaming a selected mailbox, which is how this surfaced. imapd now logs to LOG_MAIL rather than LOG_DAEMON. Upgrading from 0.1.4, this is the change that needs attention: lines that landed in /var/log/messages now land in /var/log/maillog under the rule set syslog.conf(5) documents, so anyone routing daemon.* somewhere particular will need to adjust. The syslog tag is now the daemon name rather than the per-process role, it read "listener[29527]:" before, naming no daemon at all, with the role moved into the message text, where smtpd and ntpd keep theirs. imapd.8 states the facility. Every session now gets one connect line and one close line at LOG_INFO, where a default-verbosity daemon previously said nothing about any session it served. The close line carries the peer address, the authenticated user where there was one, the duration, whether the session ended up TLS-protected, and why it closed: one of logout, client-closed, io-error, tls-error, protocol-error, limit-exceeded, login-grace or shutdown. The connect line cannot report the first of those, since it describes how the connection was accepted and a port-143 session may issue STARTTLS afterwards. ps(1) now shows one line per session, "imapd: session 42 [authenticated]" beside "session 42 auth", "session 42 search" and "session 42 store", since the daemon forks a worker group per connection. Titles carry the session id and lifecycle stage only, never the username or peer address: a process title is readable by any local user, and unlike an ssh login an IMAP login is not otherwise visible locally. Identity is recovered by joining the session id to the log, which is not world-readable. The parent is deliberately left untitled, since rc.d finds a daemon by matching the command line setproctitle(3) overwrites, and a titled parent could not be stopped by rcctl. rcctl stop now stops the daemon. The parent used to signal every child and exit without waiting, so it returned before they were gone and every open session lost its record at the moment an operator would most want one. It now closes each listener worker's imsg channel, which that worker takes as the cue to tear its own session down: the store child is asked to exit through a message it already understood rather than shot, the client gets its TLS close_notify, and the session gets a close line with reason=shutdown. The parent then waits for the last child, bounded at five seconds, after which anything still there is killed and counted; a second SIGTERM skips the wait. keymgr is asked to exit rather than signalled, so a bare EOF still means the parent died, and it frees the loaded TLS key on the way out. Starting as a non-root user now says so, and names the uid. It previously failed reading the root-only imapd.conf and reported that instead, naming the wrong problem entirely. Under -d, a log line is now written with one write(2), so concurrent sessions no longer interleave mid-line. The store child no longer addresses mailboxes by changing its working directory. It chdir(2)s once at startup, to the account's maildir root, and afterwards names every mailbox by an open descriptor: one for the root, one for the SELECTed mailbox, and a temporary one for any command that visits a second mailbox. The commands that used to save the working directory, switch away and switch back do none of those things now, and the thirteen places that switched back are gone. Eleven of those thirteen only logged a failure to switch back, and nothing else re-established the directory. A session whose switch back failed therefore kept serving with the client's selection naming one mailbox and the process standing in another, so the next FETCH read the wrong mailbox and the next EXPUNGE deleted from it, both under a tagged OK. Triggering it needs no bug, only a second session renaming or deleting the mailbox between the switch away and the switch back, which two ordinary IMAP commands arrange; done that way it destroyed every message in a mailbox the client had never named. A regression test does exactly that and is in the suite. A COPY into a mailbox another session renames mid-command now succeeds, where before it failed and left the session working in the destination. MOVE loses the error path that reported messages duplicated in both mailboxes because it could not get back to the source, since there is no longer anything to get back to. The can't-happen truncation recovery in the old switching function is gone with the function, having been unable to do what it claimed: it walked into the truncated name it had just written. unveil(2) covers a lookup relative to a descriptor exactly as it covers one relative to the working directory, and every *at call this uses is already permitted by the store's pledge. A FETCH that cannot return an item it was asked for now ends in NO rather than OK. A message whose body could not be read came back as a bare "* 1 FETCH (UID 42)" under a tagged OK, which a client can only take to mean the message has no body. RFC 9051 section 6.4.5 gives FETCH the result "NO - fetch error: can't fetch that data". The untagged responses for whatever could be produced are still sent, and a section-part the message does not have is still an empty item rather than a failure. A corrupt index can no longer widen a UID sequence set. The function that finds the highest UID, which resolves "*" for UID FETCH, STORE, EXPUNGE, COPY, MOVE and SEARCH, parsed the index's last line with its own code and no sign check, so a last line of "-1:name::1" made "*" 4294967295 and "1:*" the whole UID space; for STORE, EXPUNGE, COPY and MOVE that decides which messages are changed. It now uses the index's one line parser. That parser, and the header's UIDVALIDITY and UIDNEXT, now refuse a value above 4294967295 instead of truncating it: a UIDNEXT of 4294967297 had become 1, so the next APPEND would have reused UIDs already given out, which RFC 9051 section 2.3.1.1 does not allow. The index lives in the user's own maildir, which is why it is checked at all. A connection that never authenticates is now closed, after 60 seconds by default, set by a new imapd.conf directive, login grace, from 1 to 3600 seconds or 0 to disable. Every accepted connection costs three processes and nothing closed one that completed TCP and then sent nothing, so about a hundred silent connections reached the startups full limit and every later client was refused. The timer covers a connection on the implicit-TLS port that never starts its handshake as well as one on port 143 that never sends a command, and is cancelled the moment a session authenticates. RFC 9051 section 5.4 permits exactly this: servers "are allowed to use a shortened pre-authentication timer to protect themselves from Denial-of-Service attacks". It is not an autologout for authenticated sessions, which imapd does not have. Such a connection closes with reason=login-grace. A STORE or EXPUNGE that answers NO now changes nothing. System flags live in the maildir filename and keywords and modseq in the index, and both commands used to change files message by message and save the index once at the end, reporting each message to the client as they went. A STORE whose merged keyword set did not fit on a later message was refused with earlier messages already renamed: their system flags changed on disk while their keywords and modseq were discarded, so the client was shown flags that were never stored, and a client syncing by CONDSTORE modseq never learned of the change that did stick. STORE now works out every message before renaming any, undoes its renames if a later rename or the index save fails, and reports only after the save. EXPUNGE told the client each message was gone before saving the index, and RFC 9051 section 7.5.1 has the client renumber on each such response. If the save then failed, client and server numbered messages differently for the rest of the session, so a later STORE and EXPUNGE by sequence number could delete a message the user never chose, and the index kept entries for files already unlinked, which nothing ever removed and which made any COPY including them fail. EXPUNGE now saves the index first, then removes the files, then reports. Neither command reports a HIGHESTMODSEQ the index does not hold, and a failed command no longer moves the value a session later reports on ENABLE CONDSTORE. Finding a message's file no longer rescans cur/. Every message access looked for new/<name> and then read cur/ until it matched, and APPEND and STORE both leave messages in cur/, so FETCH 1:* (FLAGS) over 10,000 messages did 10,000 scans of 10,000 entries and took 19.9 s. The store now reads cur/ once per command into a sorted array; a miss still falls back to a real scan, so the snapshot can cost a lookup but never change an answer. The same FETCH takes 0.87 s. A large FETCH now starts answering at once. The store used to compose the reply for every matching message before sending any of it, so FETCH 1:* (BODY.PEEK[]) over 10,000 messages waited 1.46 s for its first line and grew the store child by 92 MB. It now pauses after composing 1 MB, resumes when that has drained, and answers the first line in 0.06 s with the store growing 3.7 MB. A message larger than 12000 octets can now be uploaded and downloaded. APPEND used to refuse one with NO [LIMIT] and FETCH returned no BODY[] for one, so a client could neither save nor read an ordinary message with an attachment. The cap was there because a message travelled between the store child and the listener inside one imsg, which libutil bounds at 16384 octets. APPEND now streams. The listener tells the store to open the tmp/ file when the literal is announced, passes the literal on in pieces as it arrives, and tells the store to commit once the command's closing CRLF is in, so neither process holds the whole message. A failure part way through is reported when the command ends, and a session that dies mid-upload leaves no tmp/ file behind. A new imapd.conf directive, append max, bounds what APPEND accepts, 35 MiB by default, the same as the max-message-size default of smtpd.conf(5). A larger literal draws NO [LIMIT] stating the limit as a number; the old reply had an unmatched parenthesis and named a C macro. imapd refuses a configuration whose append max exceeds attachment max, since a message larger than that could be stored but its BODYSTRUCTURE never fetched. FETCH of BODY[], BODY[TEXT] and BODY[<part>] no longer copies the message into an imsg. The store finds the octet range, refusing a message that contains NUL, which a literal cannot carry (RFC 9051 section 9, CHAR8), and passes the listener a read-only descriptor on the message file with an offset and length; the listener reads and writes and parses nothing. The store's pledge gains sendfd, and the listener already held recvfd. A FETCH walk also pauses after passing 16 descriptors, each of which holds a slot in the system-wide file table until it is sent, and a store that runs out of descriptors with some of its own still queued waits and retries the message rather than failing it. On premio, a 10 W Celeron J1900, FETCH 1:* (BODY.PEEK[]) over 1,000 messages of 64 KB answered in 1.97 s, with the store growing 260 KiB and holding at most 13 descriptors in flight. The partial-fetch clamp at 12000 octets is gone with the cap. BODY[]<0.1000000> of a large message used to report the text as 12000 octets long, which RFC 9051 section 6.4.5 does not allow: it truncates a partial fetch at the end of the text, not at a server limit. Nothing in src/ exceeds 80 columns; 750 lines did, across 29 of 33 files. IDLE now reports flag changes, and a refresh costs what the change cost rather than what the mailbox holds. A client in IDLE heard about new mail and about expunges, but never about a flag another session set, so a message marked read on a phone stayed unread on a desktop until that desktop asked. RFC 9051 section 6.3.13 names flag changes among the things IDLE exists to report. The server now sends an untagged FETCH carrying the UID and the flags for a message whose flags changed elsewhere, and the mod-sequence as well once the session has issued a CONDSTORE enabling command, which RFC 7162 section 3.1 requires. A message that arrived since the last poll is still announced by EXISTS alone, since its flags are news to nobody. The refresh behind IDLE also stopped sending the whole mailbox. The store child used to send the listener one imsg per message on any change, and the listener compared that list against the previous one by rescanning it from the start for each message, which is quadratic. The store child now keeps the list it last reported, compares in a single walk of two ascending lists, and sends only what the listener is to print: the EXPUNGE sequence numbers, the new count, and the flag changes. Nothing about the mailbox is kept in the listener any more. On premio, a 10 W Celeron J1900, with a mailbox of 100,000 messages and a session idling on it, the work behind one push fell from between 6.33 and 7.33 seconds to under 0.52 seconds. At 10,000 messages it was already under 0.12 seconds. What remains at 100,000 is reading and parsing the index, not comparing it. testing/imap_idle_flags_test.py is new and joins the regression set: two sessions on one scratch mailbox, one idling, with the flag push asserted and an APPEND and an EXPUNGE as controls, and a third session that selects with the CONDSTORE parameter to assert the mod-sequence is there for it and absent for the session that did not ask. testing/idle_diff_test.c, the standalone test for the sequence-number diff, now carries both the old rescan and the new walk and checks every case through both, including RFC 9051 section 6.4.3's EXPUNGE example and section 6.3.13's IDLE transcript. An IDLE refresh that gives up no longer loses the change it was about to report. The cheap probe in front of the refresh records the mailbox directory's inode and modification time whether or not its caller then succeeds, so a refresh that failed after the probe had said "something moved" had already consumed that change, and the next poll found nothing to report. The client was never told. The refresh now clears the probe on any reply it does not mark ok, so the following poll looks properly. Reachable today only through an index read or an allocation failing, which is why nothing exercises it. SELECT no longer moves this session's state before it knows the selection succeeded. It closed the old mailbox descriptor, installed the new one and recorded the new name, and only then took the lock and read the index, so a failure in either left the store child's descriptor and recorded name pointing at a mailbox whose SELECT had failed. The gate that guards the commands using that descriptor was never at risk: store_dispatch() clears mailbox_selected before dispatching a SELECT, so FETCH, STORE, EXPUNGE, SEARCH, COPY, MOVE and the IDLE refresh are all refused after one fails. What was left was the recorded name, which DELETE and RENAME read without consulting that gate, so either could act on bookkeeping for a mailbox this session never selected. The descriptor and the name now move with the gate, after the index has been read, and every failure before that point leaves all three alone. Not exercised by a test. Reaching the interesting failures needs the index read or the lock to fail, which a client cannot arrange, and the one failure a client can arrange returns before any of this. Waiting for another session's mailbox index lock is now bounded. The lock was taken with a blocking flock(2) and nothing anywhere gave up, so a session waited as long as the holder took. Measured three times on a 100,000 message mailbox: an ordinary STORE 1:* holds the lock for about 46 seconds, and a second session of the same user issuing a one-message STORE was blocked for about 45 of those, with no error and nothing to show for it. Two clients on one account is ordinary, so this is the common case rather than a corner. index_lock_acquire() grows a third answer for a caller that adds LOCK_NB: another process holds it. That is an ordinary outcome rather than a failure, so it is not logged, and blocking callers cannot see it because flock(2) does not report EWOULDBLOCK without LOCK_NB. A command whose handler cannot take the lock is kept by the store child and run again from the top on a timer, backing off from 50ms to a cap of 250ms, until it can or until the new "lock timeout" passes. That cap is also the worst case delay between the lock coming free and the command noticing; at it, a command waiting behind a 46 second STORE costs about 190 wakeups, four a second. One slot is enough, since the listener holds a session's next command while one is in flight. The child never blocks, so it keeps serving its listener while it waits. Re-running rather than resuming means a handler that defers must not have touched the mailbox or the session first, which is checked per handler. At the deadline the command answers NO with the RFC 9051 section 7.1 INUSE response code, whose description is this case exactly: someone else may be holding an exclusive lock needed for this operation, and the operation may succeed if the client tries again later. The default is deliberately generous at 120 seconds, because it is a safety net for a holder that is stuck rather than a cure for one that is slow: an ordinary command over a large mailbox legitimately runs for the better part of a minute, and a deadline under that would refuse ordinary concurrent use. A client that does not retry INUSE would discard the user's change with nothing on screen, which is worse than waiting. Setting it to 0 restores the unbounded wait. Every command that takes a mailbox's index lock is served this way: STORE, EXPUNGE, CLOSE, FETCH, SEARCH, SELECT, STATUS, COPY, MOVE and APPEND, each answered with its own reply type so that the client sees NO [INUSE] rather than a generic failure or "no such mailbox". What each handler has to clean up before returning busy differs by command: SELECT and STATUS close the directory they opened, FETCH takes the lock before resetting its walk, and APPEND skips the fsync(2) an earlier try already did and removes its tmp/ file when it gives up. COPY and MOVE still take their two locks in name order, but when the second is busy they now release the first as well and start again from nothing, since the command is run again from the top. Holding the first while waiting would keep every other session off that mailbox for as long as the deadline, and the ordering already rules out the deadlock that releasing might otherwise risk. A CLOSE that fails now leaves the mailbox selected. It removes nothing unless its index save succeeds, so a NO always meant the mailbox was untouched, yet the session was deselected anyway, and a client retrying the CLOSE as the INUSE text invites got BAD. The listener and the store child now both deselect only on success. The IDLE refresh skips a poll when the lock is busy, and the next poll tries again. The first refresh of an IDLE cannot skip: it records the mailbox as the client now knows it, and without it the next poll would compare against an older record and resend EXPUNGE responses the client has already processed, renumbering its view wrongly (RFC 9051 section 7.5.1). So a busy first refresh reads the committed index without the lock, which is safe because index_save() only ever replaces it whole by rename(2), and the change it found the lock held for is reported once the holder saves. One window remains: a holder that has saved but not yet released has committed its change, and a first refresh in that window adopts it unreported, as the blocking refresh it replaces did for every change made under the lock. A busy skip is logged at debug rather than as a failed refresh. Every call to index_lock_acquire() now passes LOCK_NB. testing/imap_lock_timeout_test.py covers eleven commands, choosing the waiter from STORE, EXPUNGE, FETCH, SEARCH, CLOSE, SELECT, STATUS, COPY, MOVE, APPEND and IDLE. For the first ten it asserts the bound, that it does not fire early, the response code, and that the same command succeeds afterwards, which for CLOSE is the check that the refused one left the mailbox selected. For IDLE it asserts that the idling client is told of every message the holder changed. It needs a lock timeout far below the default and so is not in run-all-tests.sh, as the login grace test is not.

libtls fake-key invariant checks, a MODIFIED fix, a credentials-file permission tightening, and a dedup pass listener terminates TLS without holding the private key: it installs libtls's placeholder key via tls_config_use_fake_private_key() and an RSA_METHOD/EC_KEY_METHOD override that forwards every private-key operation to keymgr. Three properties of libtls internals make that work, and until now all three held by inspection only. Nothing in the daemon checked them, and nothing underneath does either, libtls skips SSL_CTX_check_private_key() whenever the fake key is in use. keymgr_assert_fake_key() now checks all three on every private-key operation, before anything is composed for keymgr: A the key object carries no private component. Under the fake key libtls builds it from X509_get_pubkey(), and a public-key decode never writes d or priv_key, so this is a NULL-pointer test rather than a value test. If it fires, libtls has begun loading real key material into the process that parses hostile TLS records, and the separation this mechanism exists for is not in effect. B the pubkey-hash tag on ex_data slot 0 is present. smtpd's ca.c treats an unset tag as an unrelated key belonging to some other part of the process and falls through to the real method. That is right for smtpd's dispatcher and wrong here: listener configures exactly one keypair, once, at spawn, does no client-certificate verification, and cannot reach these callbacks with an ephemeral ECDHE key, since only sign_sig is overridden and ECDH agreement uses a different method slot. With no legitimate unrelated-key case, an absent tag is the regression, so the three fall-through branches are gone. C the tag is a NUL-terminated string within KEYMGR_HASH_MAX bytes, checked before anything reads it as one. Not merely a consistency check: strlcpy(3) walks its source to the NUL to compute its return value, unbounded once the destination is full, so keymgr_forward_rsa()'s own length guard could only fire after an over-read had already happened. libtls stores a struct tls_config pointer in the adjacent ex_data slot 1, so a slot renumbering was all it would have taken to point that walk at a C struct. Each failure is fatalx(). listener is a per-connection worker, so a regression costs that connection rather than the daemon, and costs it before any key operation is performed or forwarded. New testing/keymgr_fakekey_test.c asserts the same three properties against the real installed libtls, so a libtls change is caught by running a test rather than by a handshake misbehaving in production. It generates its own RSA and EC self-signed certificates in process, installs the same engine override, and drives four real handshakes over a non-blocking socketpair: RSA and EC, TLS 1.2 and 1.3. Two faults are injectable, so each check is seen to fire rather than assumed to. "-f realkey" configures a genuine private key using public API alone and requires A and B to fire; "-f untermtag" places an unterminated tag against a guard page, where C rejects it safely and a forked child running strlcpy(3) on the same tag dies with SIGSEGV. RSA_PRIVDEC is not exercised, reaching it needs static-RSA key exchange, which TLS 1.3 does not have and the "secure" cipher selection excludes, and the program says so in its own output rather than implying the coverage. No new dependency surface. The checks use RSA_get0_d(), EC_KEY_get0_private_key() and the ex_data getters that listener.c already includes <openssl/rsa.h> and <openssl/ec.h> for, all public and exported, and add no libtls-internal declaration beyond the tls_config_use_fake_private_key() extern already in the file. README now states which OpenBSD this builds on: -current, not 7.9. Building on the most recent stable release is a goal for 1.0. The credentials-file permission check auth.c's cred_lookup() applies now also rejects a file that is group-executable or not owned by root or the process's own (post-chroot, post-setresuid) uid. It already refused a world-accessible or group-writable file; a credentials file left mode 0650, or owned by neither root nor _imapauth, was accepted without comment. cred_file_secure() replaces the inline check and mirrors parse.y's check_file_secrecy(), the policy imapd.conf itself is already held to. New testing/cred_file_perm_test.c (generated by testing/gen_cred_perm_test.py, same splice-and-verify shape as mailbox_name_test.c) drives cred_file_secure() over 14 synthetic (mode, owner) cases, no real files or root needed, and fails on exactly the three the old check missed before this fix, passing all fourteen after it. An incomplete RFC 7162 MODIFIED set is no longer sent. SS3.1.3 requires the set to list every message that failed the UNCHANGEDSINCE test, and SS3.1.3's own client guidance is that a client re-checks and retries what it finds there, so a message missing from the set is one the client believes was stored and will never revisit. Three paths could produce that. A failed realloc(3) in session_handle_store_modified() dropped entries silently, and if it failed on the first entry the count stayed 0, the MODIFIED block was skipped entirely, and the client was told "STORE completed". A failed malloc(3) of the response buffers dropped the response code, which per SS3.1.3 says the same thing. A formatter truncation logged a warning and sent the short list anyway. All three now set one sticky flag and answer NO [UNAVAILABLE], RFC 5530 SS3, marking it transient so a client retries rather than treating the STORE as rejected. That matches what every other variable-length list here already does on a failed grow: search_alloc_failed, copy_alloc_failed and qresync_alloc_failed all refuse rather than send a short answer, and store_ipc.c says why for VANISHED, "a dropped range would leave the client holding a phantom UID it can never be told about". store_do() also gains the defensive pre-command reset that search_dispatch() has and it lacked. New mboxname.c/mboxname.h hold the mailbox-name rules once. The listener's copy of store.c's syntax check had already drifted once, missing the rejection of "." / ".." and of the on-disk index filenames, and utf8.c exists because the UTF-8 half of the same rule drifted before that. The four reserved filenames move into that header too, since spelling them as literals on one side and constants on the other is how the drift happened; store_internal.h includes it, so the store side is unchanged. Both validators survive as separate entry points, the store does not trust the listener, and re-checking on the far side of the imsg boundary is the point, but what they check is now one function. mailbox_name_is_inbox() joins it, replacing a byte-identical one-liner on each side. The rest is duplication with no behaviour attached. hdr_next_field() (mime.c) walks one RFC 5322 header field, replacing the line-end, CRLF-vs-LF, blank-line and obs-fold logic written out twice; its tri-state return also makes explicit the difference between "header ended cleanly" and "ran out", which read_message_header_fields() previously encoded as two different breaks setting two different values twenty lines apart. seqset_position() (index.c) answers "does this command apply to this message?" for FETCH, STORE and COPY. append_range_token() (store_cmd.c) carries the token/comma/budget tail format_seq_list() and format_range_list() both spelled out. send_mbox_request() now composes CREATE/DELETE/RENAME/LIST/STATUS too, taking every listener-to-store request through one function, and grew a no-trailing-array fast path so a fixed-size request no longer mallocs a copy of itself. session_writef() (listener.c) carries the CRLF- preservation fixup session_reply() and session_untagged() shared. mbox_root_enter() (mbox_manage.c) opens CREATE/DELETE/RENAME/LIST. read_file_capped() (parent.c) carries the fgetc(3) probe that tells "a file exactly the buffer's size" from "there is more". setup_peer_send() absorbs its search-oracle twin. keymgr_try_reload() and index_lines_grow() are byte-identical extractions.

privsep hardening, IDLE polling, and a full security review pass Two new privilege-separated children. keymgr holds the TLS private key and performs every private-key operation on behalf of listener, which no longer has the key in its address space; listener reaches it through an OpenSSL RSA_METHOD/EC_KEY_METHOD engine override that forwards to keymgr over imsg. search-oracle parses the SEARCH grammar -- the largest and most attacker-reachable parser in the daemon -- in a per-connection process that has no descriptors, no filesystem and nothing to steal. Every child role's pledge(2) promise is narrower as a result: listener stdio recvfd sendfd inet -> stdio recvfd auth stdio rpath recvfd sendfd -> stdio rpath store stdio rpath wpath cpath flock recvfd sendfd -> stdio rpath wpath cpath flock keymgr (new) stdio recvfd search (new) stdio parent is now the only process in the daemon that can pass a descriptor at all, and search-oracle is down to bare stdio. IDLE is now a poll, and the README no longer claims otherwise. It said "real cross-session push"; what it did was tell an IDLEing session nothing until DONE. A session now rechecks its selected mailbox every "idle poll" seconds (default 5, 1-300, or 0 to disable), so mail arriving from an external MTA or another IMAP session is announced within one interval. Most polls cost two stat(2) calls and no lock: store only re-reads the index and streams UIDs when the mailbox directory or new/ has actually changed. New "startups begin N rate N full N" admission control on concurrent unauthenticated connections, following sshd_config(5)'s MaxStartups algorithm and its 10:30:100 default. A "listen" directive now selects which listeners run, not just how they are configured: a file whose only listen directive names tls port 993 binds nothing on 143, which is the deployment RFC 8314 section 3 asks for. With no listen directive at all, both are bound as before. Mailbox names are validated as UTF-8 (new utf8.c): RFC 3629 well-formedness, C1 controls refused, U+FEFF refused anywhere, per RFC 5198 Net-Unicode as far as it can be enforced without Unicode tables. The three mailbox-name parsers that had drifted apart are now one, so the mailbox and list-mailbox grammars of RFC 9051 section 9 differ in exactly one place instead of three. Correctness and robustness fixes from the review, each with a test: DELETE of the selected mailbox no longer leaves the session pointing at a mailbox that is gone; CREATE/DELETE/RENAME reject trailing garbage; INBOX is quoted in LIST output; the credential store is refused if it is world-accessible or group-writable, rather than serving every user's bcrypt hash from a 0644 file; control characters in logged names are escaped, so a mailbox name cannot forge log lines; keymgr exits when its parent dies instead of lingering. Migrated off imsg_get(), removed from OpenBSD libutil in commit 348f1fc0836b, and retired the hand-maintained "imsgev_add() after every imsg_compose()" pairing at 36 call sites in favour of an imsg close callback (imsgbuf_set_close_callback(3), added by that same commit). The read re-arm that shared the name is now imsgev_rearm_read(), so the two jobs are distinguishable. rc.d/imapd's relink step wrote a fixed /tmp path as root on every rcctl start, which is a symlink target; it now writes inside the mktemp -d directory it already creates. Found by checking the tree against the OpenBSD ports guide's security recommendations. Also: imapd.8 documents the new directives and the maildir-ownership deployment assumption; README records that keymgr depends on tls_config_use_fake_private_key() and ex_data behavior inside tls_keypair_load(), neither of which is declared in libtls's installed tls.h nor covered by any compatibility promise.

trim comments throughout tree

Land post-refactor source tree (listener.c/store.c split into 17 files); auth sends uid/gid/maildir to parent directly instead of via listener; bump to 0.1.1

README: document real repo hosting

Initial import