]/BODY.PEEK[

] + * response (whole message, TEXT-only, or a numeric section-part) can + * carry on one IMSG_MBOX_FETCH_BODY, reusing APPEND_LITERAL_MAX's own + * value and "comfortable headroom under imsg's MAX_IMSGSIZE" reasoning + * rather than a new constant, since it's the same underlying limit + * (one struct imsg_mbox_fetch_body's fixed fields plus this many + * trailing bytes have to fit under MAX_IMSGSIZE alongside the imsg + * header itself). Before this pass, only whole/TEXT fetches existed + * and this cap was simply "the message is too big, fail the whole + * item" (APPEND_LITERAL_MAX's original framing). Section-part fetches + * change the picture: MIME parts (attachments especially) routinely + * exceed this on their own -- that's the entire reason BODYSTRUCTURE_ + * READ_MAX exists as a separate, much larger cap on what store.c is + * willing to *read* off disk. A client fetching a large part without + * a <> range still can't get more than this many bytes back + * in one response (treated as "not found" for that item, same reject- + * not-truncate precedent as everywhere else) -- real clients handle + * this by re-fetching in <> ranged chunks instead (confirmed + * against real Apple Mail traffic this pass, which already issues + * BODY.PEEK[TEXT]<0.16384>-style ranged fetches unprompted). A + * <> request's own requested count is silently clamped down + * to this cap rather than rejected outright if it's larger -- RFC 9051 + * SS6.4.5's BODY[]<> semantics already require truncating a + * range that runs past the end of the available text, so a server- + * side response-size cap truncating a too-large *count* the same way + * (returning fewer bytes than asked, letting the client re-fetch the + * remainder at a later origin octet) is consistent with that existing + * "truncate the count, don't fail the fetch" spirit, not a new kind of + * behavior this cap invents. + */ +#define FETCH_PART_MAX APPEND_LITERAL_MAX + +/* + * Cap on the space-joined header-field-name list a BODY.PEEK[HEADER. + * FIELDS (...)]/BODY.PEEK[HEADER.FIELDS.NOT (...)] fetch-att carries from + * listener.c to store.c (struct imsg_mbox_fetch's header_fields below). + * 256 bytes comfortably covers any realistic request (RFC 9051 SS6.4.5's + * own example asks for two: "DATE FROM"; even a dozen longish field names + * like "Content-Type"/"Message-Id" fit easily) -- listener.c's parse_ + * header_fields_att() rejects (BAD) a field-name list that would exceed + * this rather than truncate it, same "reject rather than silently do + * something the client didn't ask for" precedent as every other MAX + * constant in this file. + */ +#define HEADER_FIELDS_MAX 256 + +/* + * Cap on the fully-formatted ENVELOPE parenthesized-list text store.c's + * build_envelope() can carry on one IMSG_MBOX_FETCH_ENVELOPE (struct imsg_ + * mbox_fetch_envelope below). Sized off the same reasoning as FETCH_HEADER_ + * MAX (8192): every field in an envelope is extracted from a header that + * itself can't exceed FETCH_HEADER_MAX, and while IMAP quoted-string + * escaping (backslash/double-quote doubling) plus the address-structure + * parenthesization overhead can inflate the formatted size somewhat versus + * the raw header, a header dense enough with backslashes/quotes/addresses + * to actually approach 8192 bytes of *formatted* envelope text from a + * header that's itself under 8192 bytes is already an extreme case -- + * matching FETCH_HEADER_MAX's own "essentially always well under" framing. + * A message whose formatted envelope somehow exceeds this is treated as + * "not found" for ENVELOPE purposes (same reject-rather-than-truncate + * precedent as FETCH_HEADER_MAX/APPEND_LITERAL_MAX), not truncated. + */ +#define ENVELOPE_MAX 8192 + +/* + * Caps on store.c's recursive BODYSTRUCTURE builder (build_body_structure(), + * store.c), bounding both the work it does and the size of what it can ever + * produce -- a message is entirely attacker/sender-controlled data (MIME + * part count and multipart nesting depth are both just numbers the message's + * own headers claim), so both need a hard ceiling rather than trusting + * whatever a message says about its own structure. MIME_MAX_DEPTH (10) is + * generous for any real mail this personal-use server will see -- deeply + * nested multipart is already unusual beyond 2-3 levels (e.g. multipart/ + * mixed containing a multipart/alternative) -- while still bounding + * recursion (and hence worst-case stack use) at a small, fixed number. + * MIME_MAX_PARTS (64) similarly bounds total part count across the whole + * recursive walk (a running counter threaded through every recursive call, + * not a per-multipart-parent limit), bounding both output size and total + * work independent of depth. Exceeding either cap is treated as "not found" + * for this message's BODYSTRUCTURE (reject, not silently truncate the part + * tree into something that no longer accurately describes the message) -- + * same precedent as every other MAX constant in this header. + */ +#define MIME_MAX_DEPTH 10 +#define MIME_MAX_PARTS 64 + +/* + * Cap on the fully-formatted BODYSTRUCTURE parenthesized-list text store.c's + * build_bodystructure() can carry on one IMSG_MBOX_FETCH_BODYSTRUCTURE + * (struct imsg_mbox_fetch_bodystructure below). Unlike ENVELOPE_MAX, this + * isn't derived from a single header's own size cap -- a BODYSTRUCTURE's + * size instead scales with MIME_MAX_PARTS (each part contributing its own + * type/subtype/parameter-list/encoding/octet-count fields) and MIME_MAX_ + * DEPTH (each nesting level adding its own wrapping parens and multipart + * subtype). 12000 mirrors APPEND_LITERAL_MAX's own reasoning: comfortable + * headroom under imsg(3)'s MAX_IMSGSIZE (16384) alongside this struct's own + * fixed fields and the imsg header itself, generous enough that MIME_MAX_ + * PARTS/MIME_MAX_DEPTH -- not this byte cap -- are expected to be the + * limiting factor in practice for any real message. Same reject-rather- + * than-truncate handling as every other MAX constant here if somehow + * exceeded anyway. + */ +#define BODYSTRUCTURE_MAX 12000 + +/* + * Cap on the raw on-disk bytes store.c's build_bodystructure() (via read_ + * message_body(), store.c) will read into memory while deriving a + * message's MIME structure. Deliberately independent from APPEND_LITERAL_ + * MAX/FETCH_BODY_MAX: that constant's "no on-disk message can legally + * exceed this" reasoning only holds for messages that arrived through + * this server's own APPEND command, not for mail delivered by an + * external MTA into the spool directly, which this server doesn't + * control the size of at all. Real-hardware testing (an Apple Mail + * message with a small image attachment) confirmed this in practice -- + * base64-encoded attachment data routinely pushes even a modest image + * past 12000 bytes, so sharing that cap made BODYSTRUCTURE fail on + * essentially any real attachment-bearing mail. + * + * build_bodystructure() only *derives* a small, MIME_MAX_PARTS/MIME_MAX_ + * DEPTH/BODYSTRUCTURE_MAX-bounded structure summary from these bytes -- + * it never sends the raw bytes themselves back to the client over the + * wire -- so, unlike BODY.PEEK[]/BODY.PEEK[TEXT] (still capped at + * APPEND_LITERAL_MAX, since those responses do carry the raw bytes + * whole on a single imsg and are therefore still bound by imsg's own + * MAX_IMSGSIZE ceiling), this read can afford to be much larger. + * + * 41943040 (40 MiB) is sized off Gmail's own documented 25MB attachment + * limit (https://support.google.com/mail/answer/6584), the most common + * real-world ceiling a personal mailbox is likely to receive mail under, + * plus headroom for base64's ~37% encoding overhead (a 25MB attachment + * becomes roughly 34MB once base64-encoded and wrapped in MIME headers) + * -- not a hard protocol requirement, just a generous, cited real-world + * bound rather than an arbitrary guess. Exceeding it is "not found" for + * this message's BODYSTRUCTURE (reject, not truncate -- same precedent + * as every other MAX constant in this header), same as any other reason + * this message's structure can't be produced. + * + * As of this pass, this value is operator-configurable via imapd.conf's + * "attachment max " directive (parse.y), since the whole reason + * this cap is sized off Gmail's own attachment limit rather than derived + * from any protocol constant is that it's inherently a judgment call + * about the operator's own expected mail, not a fixed property of the + * implementation -- see parse.y's grammar rule for the directive and + * struct openimap_config's bodystructure_read_max field. This macro + * changed meaning accordingly: no code reads it directly anymore (store.c + * uses the runtime value received via IMSG_STORE_INIT instead); it now + * exists solely as config_load()'s default when the directive is absent + * from imapd.conf, so an empty/absent config file still gets the same + * behavior this implementation always had. + */ +#define BODYSTRUCTURE_READ_DEFAULT 41943040 + +struct imsg_mbox_fetch { + uint32_t seq_lo; /* 1-based, inclusive; ignored if + * lo_is_star */ + uint32_t seq_hi; /* 1-based, inclusive; ignored if + * hi_is_star */ + int lo_is_star; + int hi_is_star; /* "*" is resolved by store against + * its own live message count at + * fetch time, not against listener's + * -- possibly stale -- count from the + * last SELECT reply (mail could have + * arrived since) */ + uint32_t attrs; /* bitmask of MBOX_FETCH_* above */ + + /* + * RFC 7162 SS3.1.4.1 CHANGEDSINCE fetch-modifier: "The information + * described by message data items is only returned for messages + * that have a mod-sequence bigger than ." has_ + * changedsince distinguishes "not specified" from a legal value of + * 0, the same has_/value pairing this header already uses for + * APPEND's optional date-time (see imsg_mbox_append's append_has_ + * date, in listener.c's struct session). + */ + int has_changedsince; + uint64_t changedsince; + + /* + * RFC 9051 SS6.4.9 (UID command): "the numbers in the sequence-set + * argument are unique identifiers instead of message sequence + * numbers" for a UID FETCH -- by_uid tells store.c to resolve seq_lo/ + * seq_hi (and "*") against UID space rather than 1-based index + * position. listener.c separately forces MBOX_FETCH_UID into attrs + * whenever by_uid is set (SS6.4.9: "server implementations MUST + * implicitly include the UID message data item as part of any FETCH + * response caused by a UID command"), so store.c itself needs no + * special-casing for *that* part -- meta.uid is already unconditionally + * populated regardless (see imsg_mbox_fetch_meta below). + * + * want_vanished is RFC 7162 SS3.2.6's VANISHED UID FETCH modifier + * (only legal alongside CHANGEDSINCE, and only on UID FETCH -- + * listener.c's parse_fetch_modifiers()/fetch_dispatch() enforce both + * restrictions before this ever reaches store.c). Per this + * implementation's RFC 7162 SS5.1 minimal-state QRESYNC decision + * (see imsg_mbox_select_vanished below), store.c doesn't track + * expunge-event history, so it can't actually filter by CHANGEDSINCE + * here either -- it reports every UID in [seq_lo, seq_hi] that isn't + * currently present, unconditionally, which SS3.2.6's own note + * explicitly sanctions ("A server that receives a mod-sequence + * smaller than ... MUST behave as if it was requested to + * report all expunged messages from the provided UID set parameter" -- + * this implementation always behaves that way, having no memory of + * at all). + */ + int by_uid; + int want_vanished; + + /* + * BODY.PEEK[HEADER.FIELDS (...)]/BODY.PEEK[HEADER.FIELDS.NOT (...)] + * (see MBOX_FETCH_HEADER_FIELDS above): header_fields_not is 0 for + * HEADER.FIELDS (include only the listed names), 1 for HEADER. + * FIELDS.NOT (exclude the listed names). header_fields is the + * requested field-name list, space-joined, exactly as the client + * typed each name (matching is ASCII-range case-insensitive per + * SS6.4.5.1, done by store.c's read_message_header_fields() -- + * this field is not itself normalized to any particular case). + * Both are only meaningful alongside attrs & MBOX_FETCH_HEADER_ + * FIELDS; listener.c's parse_header_fields_att() has already fully + * validated the header-list grammar before either field is ever + * populated, so store.c can assume header_fields is well-formed + * (non-empty, space-separated, no embedded quotes -- see that + * function's comment for why a bare-atom-only field name is this + * implementation's own scope cut). + */ + int header_fields_not; + char header_fields[HEADER_FIELDS_MAX]; + + /* + * BODY[]/BODY.PEEK[] (MBOX_FETCH_BODY_ + * PART above): section_part is the client-typed dotted-numeric part + * path verbatim (e.g. "3.1"), non-empty only alongside attrs & + * MBOX_FETCH_BODY_PART -- listener.c's tokenizer has already + * validated it's 1*(digit) *("." 1*digit) with no leading zeros + * before this is ever populated, so store.c can assume it parses + * cleanly into a path of nz-numbers. Single shared field, same "one + * instance assumed per FETCH command" simplification as header_ + * fields above -- a client requesting two different section-parts + * in one FETCH (legal per SS6.4.5.1, unseen from any real client so + * far) only gets the first one honored, matching that same + * precedent rather than restructuring this struct into an array for + * a case no real client actually does. + * + * has_partial/partial_start/partial_count carry a <> range + * (SS6.4.5's "" suffix, e.g. "BODY.PEEK[3.1]<0.65536>" + * or "BODY.PEEK[TEXT]<0.16384>" -- the latter is real, observed + * Apple Mail traffic this project's own real-hardware testing + * turned up, previously silently unhandled since no "<...>" parsing + * existed anywhere in listener.c at all before this pass). Applies + * uniformly to whichever BODY[...]/BODY.PEEK[...] variant attrs + * selects (whole, TEXT, or section_part) -- store.c slices the + * already-extracted content to [partial_start, partial_start + + * partial_count) before ever composing the response imsg, clamped + * to FETCH_PART_MAX and to the content's own actual length (RFC + * 9051 SS6.4.5: "If the starting octet is beyond the end of the + * text, an empty string is returned... Any partial fetch that + * attempts to read beyond the end of the text is truncated as + * appropriate"). has_partial distinguishes "no range requested" + * from a legal partial_start of 0, same has_/value pairing this + * header already uses for CHANGEDSINCE above. + */ + char section_part[SECTION_PART_MAX]; + int has_partial; + uint32_t partial_start; + uint32_t partial_count; +}; + +struct imsg_mbox_fetch_meta { + uint32_t seqno; /* 1-based */ + uint32_t uid; + uint64_t size; /* on-disk file size, octets -- + * RFC822.SIZE (RFC 9051 SS2.3.4). + * F13 fix: 64-bit so a message larger + * than 4 GiB reports a correct size. */ + int64_t internaldate; /* Unix timestamp, parsed from the + * maildir basename's own leading + * delivery-time field -- see store.c's + * handle_mbox_fetch() comment for why + * that's used instead of the file's + * mtime */ + char flags[MBOX_FLAGS_MAX]; /* space-separated IMAP flag + * names, e.g. "\Seen \Flagged foo" -- + * pre-formatted by store.c, the only + * side that has both the maildir + * flag-suffix letters and the index's + * keyword list */ + uint64_t modseq; /* RFC 7162 per-message mod-sequence -- + * always populated by store.c (cheap: + * it's already parsed the index line), + * same "always compute, let listener.c + * decide whether to print it" split as + * every other struct imsg_mbox_fetch_ + * meta field; shared by FETCH, STORE's + * FETCH echo, and QRESYNC SELECT + * resync's FETCH-with-UID responses, + * all three of which reuse this struct + * (see imsg_mbox_selected's comment) */ +}; + +/* + * IMSG_MBOX_FETCH_HEADER: sent by store.c immediately before the + * IMSG_MBOX_FETCH_META for the same message, if and only if req->attrs & + * MBOX_FETCH_BODY_HEADER -- listener.c relies on this exact ordering + * (rather than matching on seqno/uid) to fold the header bytes into the + * same untagged "* N FETCH (...)" response line as the message's other + * requested data items, per RFC 9051's own SS6.4.5/SS7.5.2 worked examples, + * which always show every FETCH data item for a message on one response + * line, not spread across several. This mirrors imsg_mbox_append's + * "fixed struct, then trailing variable-length bytes on the same imsg" + * shape (listener.c's session_finish_append(): malloc(sizeof(struct) + + * len), memcpy both pieces in, one imsg_compose() call) -- just sent in + * the opposite direction (store -> listener) and read back the same way + * imsg_mbox_append already is (store.c's handle_mbox_append(): + * imsg_get_buf() for the fixed struct, then imsg_get_len()/imsg_get_buf() + * again for whatever trailing bytes remain). + */ +struct imsg_mbox_fetch_header { + uint32_t seqno; /* 1-based -- matches the seqno on the + * IMSG_MBOX_FETCH_META that follows */ + uint32_t uid; + int found; /* 0 if locate_message_file() (well, + * store.c's own open_message_file(), + * see its comment for why this is a + * separate lookup rather than a shared + * one) couldn't find the message file, + * or its header exceeded + * FETCH_HEADER_MAX -- listener.c omits + * BODY[HEADER] from this one message's + * response rather than failing the + * whole FETCH, same as a metadata + * lookup failure already does. hdrlen + * and the trailing bytes are only + * meaningful when found is 1. */ + uint32_t hdrlen; /* length of the trailing raw header + * bytes on this same imsg, capped at + * FETCH_HEADER_MAX */ +}; + +/* + * IMSG_MBOX_FETCH_BODY: sent by store.c immediately before the + * IMSG_MBOX_FETCH_META for the same message, if and only if req->attrs & + * (MBOX_FETCH_BODY_WHOLE | MBOX_FETCH_BODY_TEXT) -- same ordering contract, + * same "fixed struct + trailing variable-length bytes on one imsg" shape, + * as struct imsg_mbox_fetch_header above (see that struct's comment); this + * is the same wire idiom used a third time, not a new one. + */ +struct imsg_mbox_fetch_body { + uint32_t seqno; /* 1-based -- matches the seqno on the + * IMSG_MBOX_FETCH_META that follows */ + uint32_t uid; + int found; /* 0 if open_message_file() couldn't find + * the message file, the file exceeded + * APPEND_LITERAL_MAX (listener.c), the + * content contained a NUL byte, or (for + * is_text only) no header/body separator + * could be found within the file -- + * listener.c omits BODY[]/BODY[TEXT] from + * this one message's response rather than + * failing the whole FETCH, same precedent + * as imsg_mbox_fetch_header's found field. + * bodylen and the trailing bytes are only + * meaningful when found is 1. */ + int is_text; /* 0 -- these are BODY.PEEK[] bytes (whole + * message, header and body together); 1 -- + * these are BODY.PEEK[TEXT] bytes (body + * only, header omitted). Set by store.c + * based on which of MBOX_FETCH_BODY_WHOLE/ + * MBOX_FETCH_BODY_TEXT req->attrs actually + * had set (WHOLE wins if a client somehow + * requested both, see MBOX_FETCH_BODY_ + * TEXT's comment) -- listener.c uses this + * to label the literal "BODY[]" vs + * "BODY[TEXT]" correctly rather than + * re-deriving it from its own fetch_attrs, + * which could have both bits set. */ + uint32_t bodylen; /* length of the trailing raw body bytes on + * this same imsg, capped at + * APPEND_LITERAL_MAX (listener.c) -- same + * cap APPEND itself already enforces when + * writing a message, reused here rather + * than inventing a second constant, since + * no message on disk can legally exceed it + * in the first place */ +}; + +/* + * IMSG_MBOX_FETCH_ENVELOPE: sent by store.c immediately before the + * IMSG_MBOX_FETCH_META for the same message, if and only if req->attrs & + * MBOX_FETCH_ENVELOPE -- same ordering contract as struct imsg_mbox_fetch_ + * header/imsg_mbox_fetch_body above. Unlike those two, whose trailing bytes + * are raw message content that listener.c wraps in a `{n}` literal, the + * trailing bytes here are the *complete*, already-formatted RFC 9051 + * SS7.5.2 envelope parenthesized-list text -- e.g. `("Wed, 17 Jul 1996 + * 02:23:25 -0700 (PDT)" "IMAP4rev1 WG mtg summary and minutes" (("Terry + * Gray" NIL "gray" "cac.washington.edu")) (("Terry Gray" NIL "gray" + * "cac.washington.edu")) (("Terry Gray" NIL "gray" "cac.washington.edu")) + * ((NIL NIL "imap" "cac.washington.edu")) ((NIL NIL "minutes" + * "CNRI.Reston.VA.US")("John Klensin" NIL "KLENSIN" "MIT.EDU")) NIL NIL + * "")`, RFC 9051's own SS7.5.2 worked + * example -- fully quoted/escaped and ready to splice verbatim into the + * FETCH response right after the literal string "ENVELOPE ". store.c does + * all the RFC 5322 header parsing, field unfolding, and address-list + * decomposition (build_envelope(), store.c) -- that's where the raw header + * bytes already live and where BODY.PEEK[HEADER.FIELDS...]'s folding-aware + * parsing already lives, so this keeps parsing logic in one place rather + * than splitting RFC 5322 semantics across both processes; listener.c's job + * stays purely wire framing, same division of labor as every other FETCH + * content item this project has implemented. No literal-block wrapping + * needed (unlike BODY[HEADER]/BODY[]/BODY[TEXT]): envelope fields are short + * quoted strings built by build_envelope()'s own IMAP-quoted-string escaper, + * never raw message bytes, so they're never CRLF-bearing and always fit + * directly on the response line. + */ +struct imsg_mbox_fetch_envelope { + uint32_t seqno; /* 1-based -- matches the seqno on the + * IMSG_MBOX_FETCH_META that follows */ + uint32_t uid; + int found; /* 0 if open_message_file()/read_message_ + * header() couldn't find or read the + * message's header, or the formatted + * envelope text exceeded ENVELOPE_MAX -- + * listener.c omits ENVELOPE from this one + * message's response rather than failing + * the whole FETCH, same precedent as + * imsg_mbox_fetch_header/imsg_mbox_fetch_ + * body's own found fields. envlen and the + * trailing bytes are only meaningful when + * found is 1. */ + uint32_t envlen; /* length of the trailing, already- + * formatted envelope text on this same + * imsg, capped at ENVELOPE_MAX */ +}; + +/* + * IMSG_MBOX_FETCH_BODYSTRUCTURE: sent by store.c immediately before the + * IMSG_MBOX_FETCH_META for the same message, if and only if req->attrs & + * MBOX_FETCH_BODYSTRUCTURE -- same ordering contract and same "already- + * formatted response text, not raw bytes" shape as struct imsg_mbox_fetch_ + * envelope above (see that struct's comment). The trailing bytes are the + * *complete* RFC 9051 SS7.5.2 BODYSTRUCTURE parenthesized-list text, e.g. + * `("TEXT" "PLAIN" ("CHARSET" "US-ASCII") NIL NIL "7BIT" 2279 48)` for a + * simple message (RFC 9051's own SS7.5.2 example), or, for a multipart + * message, a nested structure like `(("TEXT" "PLAIN" ("CHARSET" "US-ASCII") + * NIL NIL "7BIT" 1152 23)("TEXT" "PLAIN" ("CHARSET" "US-ASCII" "NAME" + * "cc.diff") "<...>" "Compiler diff" "BASE64" 4554 73) "MIXED")` (also RFC + * 9051's own example) -- built entirely by store.c's build_bodystructure()/ + * build_body_structure(), which do all the MIME parsing (Content-Type/ + * Content-Transfer-Encoding/Content-ID/Content-Description extraction, + * multipart boundary splitting, recursion into sub-parts bounded by MIME_ + * MAX_DEPTH/MIME_MAX_PARTS) -- same "all the semantic parsing lives in one + * process, listener.c does pure wire framing" division of labor as + * envelope. This implementation deliberately never emits RFC 9051's + * optional extension data (body MD5/disposition/language/location -- see + * MBOX_FETCH_BODYSTRUCTURE's own comment for why), so a BODYSTRUCTURE fetch + * and a bare BODY fetch (the explicitly non-extensible form) produce + * identical text in this server. No literal-block wrapping needed, same + * reasoning as envelope (build_body_structure()'s own IMAP-quoted-string + * escaping, reused from envbuf_append_nstring(), guarantees no raw CRLF). + */ +struct imsg_mbox_fetch_bodystructure { + uint32_t seqno; /* 1-based -- matches the seqno on the + * IMSG_MBOX_FETCH_META that follows */ + uint32_t uid; + int found; /* 0 if the message's raw bytes couldn't + * be read, MIME_MAX_DEPTH/MIME_MAX_PARTS + * was exceeded, the message contains a + * message/rfc822 or message/global part + * (see MBOX_FETCH_BODYSTRUCTURE's comment + * for why those are scoped out), or the + * formatted text exceeded BODYSTRUCTURE_ + * MAX -- listener.c omits BODYSTRUCTURE + * from this one message's response rather + * than failing the whole FETCH, same + * precedent as every other content item's + * own found field. bslen and the trailing + * bytes are only meaningful when found is + * 1. */ + uint32_t bslen; /* length of the trailing, already- + * formatted BODYSTRUCTURE text on this + * same imsg, capped at BODYSTRUCTURE_MAX */ +}; + +/* Generic per-operation completion signal -- FETCH is the first user, + * but the name (and the enum's own placement of IMSG_MBOX_RESULT after + * every other IMSG_MBOX_* type) suggests it's meant to close out + * STORE/APPEND/etc. too once those exist. */ +struct imsg_mbox_result { + int ok; + uint32_t count; /* number of IMSG_MBOX_FETCH_META (or + * equivalent, for a future op) messages that + * preceded this one */ + + /* + * RFC 7162: the mailbox's HIGHESTMODSEQ after this operation. + * Always populated for STORE/EXPUNGE (the two operations that can + * change it); left 0 for a plain FETCH, which never mutates + * anything. listener.c is the one that decides whether/how to + * surface it: EXPUNGE's tagged OK MAY include it (SS3.2.7, "If at + * least one message got expunged and QRESYNC was enabled, the + * server MUST send" it -- this implementation does so whenever + * CONDSTORE is enabled at all, a safe superset, see cmd_expunge()'s + * comment), STORE's tagged OK/NO doesn't need to (SS3.1.3's own + * examples show it appearing on some responses and not others -- + * "presumably because this was the first CONDSTORE enabling + * command", i.e. it's the same "first enabling command" case + * covered by session_condstore_enable(), not something every STORE + * repeats), and CLOSE's tagged OK MUST NOT include it (SS3.2.8, + * explicit "MUST NOT ... as this might cause loss of + * synchronization on the client" -- cmd_close()'s existing + * was_close branch just never reads this field). + */ + uint64_t highestmodseq; + + /* + * RFC 9051 SS7.1's COPYUID response code: "the UIDVALIDITY of the + * destination mailbox". Populated only for COPY/MOVE (see imapd.h's + * imsg_mbox_copy comment) -- v1 has no CREATE, so the destination is + * always the same mailbox as the source (the currently selected + * mailbox), making this identical to that mailbox's own UIDVALIDITY; + * left 0 for FETCH/STORE/EXPUNGE/SEARCH, which have no COPYUID to + * report. Same "populated for the operations that need it, 0 + * otherwise, listener.c decides what to do with it" split as + * highestmodseq above. + */ + uint32_t uidvalidity; + + /* + * COPY/MOVE only (0 for every other operation that shares this + * struct, same "populated for the operations that need it" split as + * highestmodseq/uidvalidity above): 1 distinguishes "destination + * mailbox doesn't exist" from any other failure -- listener.c must + * send the tagged NO with a "[TRYCREATE]" prefix per SS6.4.7/SS6.4.8 + * for this case specifically, same distinction imsg_mbox_appended's + * own no_such_mailbox field already makes for APPEND. + */ + int no_such_mailbox; +}; + +/* + * IMSG_MBOX_STORE (listener -> store): RFC 9051 SS6.4.6 STORE command -- + * `store = "STORE" SP sequence-set SP store-att-flags`, `store-att-flags = + * (["+" / "-"] "FLAGS" [".SILENT"]) SP (flag-list / (flag *(SP flag)))`. + * Replies reuse IMSG_MBOX_FETCH_META (one per modified message, sent only + * if !silent) and the terminal IMSG_MBOX_RESULT -- SS6.4.6 itself says + * STORE's only response is "untagged responses: FETCH", the exact same + * shape FETCH already produces ("* FETCH (FLAGS (...))"), so + * there was no reason to invent a second reply pair. + * + * System flags are a fixed 5-bit set (v1 supports exactly the five RFC + * 9051 SS2.3.2 system flags: \Answered \Flagged \Deleted \Seen \Draft -- + * \Recent is explicitly excluded from the `flag` ABNF production itself, + * and any other "\"-prefixed token is a `flag-extension` this server + * doesn't define, so listener.c rejects both before ever building this + * struct). Keywords (arbitrary non-"\" atoms, SS2.3.2's `flag-keyword`) + * are carried separately as a comma-separated list -- matching the + * index's own on-disk keyword delimiter (openimap-storage-backend.md) so + * store.c can merge them directly without a format conversion; listener.c + * rejects any client-supplied keyword containing ':' or ',' (both valid + * in IMAP's `atom` grammar, but the index format has no escaping + * mechanism for its own field/record delimiters -- see cmd_store_cmd()'s + * comment) rather than silently corrupting the index. + */ +#define MBOX_FLAG_ANSWERED (1U << 0) +#define MBOX_FLAG_FLAGGED (1U << 1) +#define MBOX_FLAG_DELETED (1U << 2) +#define MBOX_FLAG_SEEN (1U << 3) +#define MBOX_FLAG_DRAFT (1U << 4) + +#define MBOX_STORE_SET 0 /* FLAGS -- replace outright */ +#define MBOX_STORE_ADD 1 /* +FLAGS -- union in */ +#define MBOX_STORE_REMOVE 2 /* -FLAGS -- subtract out */ + +struct imsg_mbox_store { + uint32_t seq_lo; + uint32_t seq_hi; + int lo_is_star; + int hi_is_star; + int mode; /* MBOX_STORE_* above */ + int silent; /* 1 if the ".SILENT" suffix was given + * -- suppress the untagged FETCH + * response per message */ + uint32_t sysflags; /* MBOX_FLAG_* bitmask named in this + * STORE (the flags being set/added/ + * removed, not the message's + * resulting flags -- store.c computes + * that) */ + char keywords[MBOX_FLAGS_MAX]; /* comma-separated keyword + * atoms named in this STORE, "" if + * none */ + + /* + * RFC 7162 SS3.1.3 UNCHANGEDSINCE store-modifier. has_unchangedsince + * distinguishes "not specified" from the legal value 0 (SS3.1.3 + * Example 8: "Use of UNCHANGEDSINCE with a modification sequence of + * 0 always fails if the metadata item exists" -- a deliberate, + * always-fails conditional test, not the same as omitting the + * modifier entirely). + */ + int has_unchangedsince; + uint64_t unchangedsince; + + /* + * RFC 9051 SS6.4.9: same UID-vs-sequence-number resolution switch as + * imsg_mbox_fetch's by_uid, for UID STORE. meta.uid (in the shared + * imsg_mbox_fetch_meta STORE echo) is already unconditionally + * populated regardless of this flag -- only listener.c's decision to + * *print* it changes based on whether the in-flight command was UID + * STORE (see struct session's cmd_by_uid in listener.c). + */ + int by_uid; +}; + +/* + * IMSG_MBOX_STORE_MODIFIED (store -> listener, zero or more, only when + * req->has_unchangedsince, before the terminal IMSG_MBOX_RESULT): one + * message whose mod-sequence exceeded the UNCHANGEDSINCE value, so the + * requested STORE operation was *not* performed for it (RFC 7162 SS3.1.3: + * "the message number (or unique identifier in the case of the UID STORE + * command) is added to the list of messages that failed the UNCHANGEDSINCE + * test"). listener.c range-compacts these into the tagged response's + * MODIFIED response code, same compaction helper SEARCH/VANISHED already + * use. + */ +struct imsg_mbox_store_modified { + uint32_t seqno; + uint32_t uid; +}; + +/* + * IMSG_MBOX_EXPUNGE (listener -> store) / IMSG_MBOX_EXPUNGED (store -> + * listener, one per removed message, streamed in the order store.c + * actually removes them) / IMSG_MBOX_RESULT (store -> listener, exactly + * once, terminal -- same generic completion signal FETCH/STORE already + * use). + * + * RFC 9051 SS6.4.3: EXPUNGE "permanently removes all messages that have + * the \Deleted flag set from the currently selected mailbox," sending one + * untagged EXPUNGE response per removed message before the tagged OK. + * SS7.5.1 (the EXPUNGE response itself): "The message sequence number for + * each successive message in the mailbox is immediately decremented by 1" + * -- so which sequence number gets reported for each removal depends on + * removal order; this server removes lowest-numbered first ("a 'lower to + * higher' server," SS7.5.1's own term), matching SS6.4.3's own worked + * example exactly (message 3, then 3 again, then 5, then 8, for original + * positions 3/4/7/11) -- see store.c's handle_mbox_expunge() for the + * compaction algorithm that produces those numbers. + * + * silent exists so CLOSE (RFC 9051 SS6.4.1: "permanently removes all + * messages that have the \Deleted flag set ... No untagged EXPUNGE + * responses are sent") can reuse this exact request/reply pair instead of + * inventing a second one -- the same ".SILENT"-suffix pattern STORE + * already uses for the same "same operation, client doesn't want the + * per-message notifications" reason. + */ +struct imsg_mbox_expunge { + int silent; /* 1 for CLOSE, 0 for a real EXPUNGE command */ + + /* + * RFC 9051 SS6.4.9's second UID command form: "the UID command takes + * an EXPUNGE command with an extra parameter that specifies a + * sequence set of UIDs to operate on... permanently removes all + * messages that have both the \Deleted flag set and a UID that is + * included in the specified sequence set... If a message either does + * not have the \Deleted flag set or has a UID that is not included in + * the specified sequence set, it is not affected." by_uid is never + * set together with silent=1 -- CLOSE has no UID-restricted form + * (there is no "UID CLOSE"), only plain EXPUNGE does. seq_lo/seq_hi/ + * lo_is_star/hi_is_star mirror imsg_mbox_fetch's own naming exactly + * (same seq-range/UID-range resolution convention throughout this + * header); meaningless when !by_uid, since a plain EXPUNGE takes no + * arguments at all (RFC 9051 SS6.4.3: "Arguments: none"). + */ + int by_uid; + uint32_t seq_lo; + uint32_t seq_hi; + int lo_is_star; + int hi_is_star; +}; + +struct imsg_mbox_expunged { + uint32_t seqno; /* the message's sequence number at the moment + * of removal, per SS7.5.1's "immediately + * decremented" rule -- NOT its UID, and NOT + * its pre-EXPUNGE sequence number */ + uint32_t uid; /* RFC 7162 addition: the same message's UID, + * needed once QRESYNC is enabled -- SS3.2.10.2 + * replaces the untagged EXPUNGE (seqno-based) + * response with VANISHED (UID-based) in that + * case, and listener.c has no other way to + * learn the removed message's UID (the index + * line is already gone by the time this imsg + * is sent -- see store.c's handle_mbox_ + * expunge()) */ +}; + +/* + * IMSG_MBOX_COPY (listener -> store, RFC 9051 SS6.4.7 COPY) / IMSG_MBOX_MOVE + * (listener -> store, SS6.4.8 MOVE) share this exact request shape -- same + * "one struct, two message types, differ only in which store.c handler + * fires" pattern imsg_mbox_expunge already established for EXPUNGE/CLOSE + * (distinguished there by .silent; here by which IMSG_MBOX_* type arrived). + * + * destname: the destination mailbox, resolved and validated by listener.c's + * copy_move_dispatch() exactly the way cmd_rename()'s oldname/newname + * already are -- mailbox_name_is_inbox()/mailbox_name_valid() first, with + * zero store round trip for a syntactically invalid name. Before flat + * multi-mailbox support (RFC 9051 SS6.3.4-SS6.3.6, docs/openimap-storage- + * backend.md item 10) this struct had no destination field at all: v1 had + * no CREATE, so the destination was *always* the same mailbox as the + * source (the currently selected mailbox, itself always INBOX) -- + * explicitly RFC-sanctioned even then (SS6.4.8: "moving a message to the + * currently selected mailbox... is allowed when copying the message to the + * currently selected mailbox is allowed"). Now that named mailboxes are + * real, a genuinely different destination is real too; store.c's handle_ + * mbox_copy()/handle_mbox_move() still fast-path the "destname names the + * mailbox already selected" case as a single-index operation identical to + * that original v1 code, and only take the new two-index cross-mailbox + * path (see those functions' own header comments, including the lock- + * ordering discussion) when it genuinely differs. + */ +struct imsg_mbox_copy { + int by_uid; + uint32_t seq_lo; + uint32_t seq_hi; + int lo_is_star; + int hi_is_star; + char destname[MBOX_NAME_MAX]; +}; + +/* + * IMSG_MBOX_COPY_MAPPING (store -> listener, zero or more, before the + * terminal IMSG_MBOX_RESULT): one message's COPYUID mapping -- RFC 9051 + * SS7.1's COPYUID response code carries "a UID set containing the UIDs of + * the message(s) in the source mailbox that were copied... followed by + * another UID set containing the UIDs assigned... in the destination + * mailbox... in the order the message(s) was copied." Streamed one pair + * per message, ascending, rather than pre-compacted into ranges here -- + * matching this header's own established "store streams raw values, + * listener compacts into ranges at formatting time" split (format_seq_ + * list() already does exactly this for ESEARCH/MODIFIED); listener.c + * accumulates src_uid/dest_uid into two parallel growable arrays and + * range-compacts each independently once the terminal reply arrives. + * + * Used for both COPY and MOVE. For MOVE, listener.c also buffers each + * IMSG_MBOX_EXPUNGED that arrives during the same round trip (rather than + * writing it to the client immediately, which is what happens for a real + * EXPUNGE/CLOSE) and flushes both buffers, in order, only once the + * terminal reply arrives -- COPYUID first, then EXPUNGE/VANISHED -- per + * SS6.4.8: "servers are also REQUIRED to send the COPYUID response code in + * an untagged OK before sending EXPUNGE". This mirrors session_handle_ + * mbox_selected()'s existing vanished_ranges/qresync_fetches dual-buffer- + * then-flush-in-fixed-order pattern for a QRESYNC SELECT resync -- same + * underlying problem (control final wire order across two streamed + * sub-types that arrive interleaved with other traffic), same solution. + */ +struct imsg_mbox_copy_mapping { + uint32_t src_uid; + uint32_t dest_uid; +}; + +/* + * IMSG_MBOX_APPEND (listener -> store) / IMSG_MBOX_APPENDED (store -> + * listener, exactly once). APPEND handles exactly one message per + * request in v1 (no [MULTIAPPEND]), so unlike FETCH/STORE/EXPUNGE there's + * no per-message streaming reply -- one request, one reply. + * + * RFC 9051 SS6.3.12: `append = "APPEND" SP mailbox [SP flag-list] [SP + * date-time] SP literal`. The literal -- the message body itself -- is + * carried as variable-length trailing data appended directly after this + * fixed struct within the SAME imsg, not as a separate message and not + * fd-passed. Verified directly against the real imsg.c/imsg-buffer.c this + * session: imsg_get_data() requires an *exact* length match (its own + * source: `if (ibuf_size(imsg->buf) != len) { errno = EBADMSG; return + * (-1); }`), so it cannot be used to read a fixed header out of a longer + * imsg. imsg_get_buf() is the sequential-read alternative (no length + * check, just `ibuf_get()`, which advances the ibuf's internal read + * position); imsg_get_len() reflects bytes *remaining*, not total size, + * because it calls ibuf_size(), which is literally `wpos - rpos`. So + * store.c's handle_mbox_append() reads this struct with imsg_get_buf(), + * then treats whatever imsg_get_len() reports afterward as the message + * body length and reads that with a second imsg_get_buf() call. + * + * This one-message-one-imsg design only works because the message is + * capped at APPEND_LITERAL_MAX (listener.c) to fit comfortably under + * MAX_IMSGSIZE (16384, imsg.h) alongside this struct's own ~550 bytes -- + * see APPEND_LITERAL_MAX's comment in listener.c for the exact arithmetic + * and the imsg_create() source check ("datalen += IMSG_HEADER_SIZE; if + * (datalen > imsgbuf->maxsize) ... return NULL") it's based on. A message + * larger than that needs real fd-passing -- the same mechanism BODY[] + * FETCH still needs and doesn't have -- not implemented this pass; + * listener.c rejects an oversized literal announcement with a plain NO + * (RFC 9051 defines no response code for a size cap) before ever reading + * it, rather than truncating or crashing. + */ +struct imsg_mbox_append { + char mailbox[MBOX_NAME_MAX]; + uint32_t sysflags; /* MBOX_FLAG_* bitmask -- an omitted or + * empty "()" flag-list both mean 0, + * per SS6.3.12: "otherwise the flag + * list of the resulting message is + * set to 'empty' by default" */ + char keywords[MBOX_FLAGS_MAX]; /* comma-separated, same + * convention as imsg_mbox_store's */ + int has_date; /* 0 -- SS6.3.12: "otherwise the + * internal date... is set to the + * current date and time" -- store.c + * uses time(NULL) at delivery time + * in that case */ + int64_t date; /* Unix timestamp; meaningful only if + * has_date */ + uint32_t msglen; /* length of the trailing message + * bytes -- redundant with what + * imsg_get_len() reports after the + * header is read, kept anyway as an + * explicit value store.c cross-checks + * against that, rather than trusting + * a single source for something this + * consequential (a mismatch likely + * means a build-time struct-layout + * skew between listener and store, + * or a truncated imsg). */ +}; + +struct imsg_mbox_appended { + int ok; + int no_such_mailbox; /* 1 distinguishes "not INBOX" -- + * listener.c must send the tagged NO + * with a "[TRYCREATE]" prefix per + * SS6.3.12 -- from any other failure + * (plain NO, no response code) */ + uint32_t uidvalidity; + uint32_t uid; /* the appended message's own UID -- + * together with uidvalidity, this is + * SS7.1's APPENDUID response code */ + uint32_t exists; /* mailbox's new total message count, + * so listener.c can send SS6.3.12's + * "SHOULD notify the client + * immediately via an untagged EXISTS + * response" -- sent only if this + * session currently has the mailbox + * selected; see cmd_append()'s and + * session_handle_mbox_appended()'s + * comments */ +}; + +/* + * IMSG_MBOX_SEARCH (listener -> store) / IMSG_MBOX_SEARCH_MATCH (store -> + * listener, one per matching message, streamed in ascending sequence + * order -- same per-message streaming shape as IMSG_MBOX_FETCH_META and + * IMSG_MBOX_EXPUNGED) / IMSG_MBOX_RESULT (store -> listener, exactly + * once, terminal -- the same generic completion signal FETCH/STORE/ + * EXPUNGE/APPEND already reuse; `count` is the number of matches + * streamed, which listener.c uses directly as COUNT if requested). + * + * RFC 9051 SS6.4.4 SEARCH's `search-key` grammar nests arbitrarily (NOT + * wraps one key, OR takes two, a parenthesized list ANDs N of them), so + * the parsed criteria can't be a single fixed-size struct the way + * STORE's flag-list or FETCH's fetch-att bitmask can. Instead, + * listener.c's parse_search_key()/parse_search_key_list() compile the + * whole search-program into a flat postfix (reverse Polish) array of + * struct search_node, sent as variable-length trailing data on this + * imsg after a small fixed header -- the same imsg_get_buf()/imsg_get_ + * len() wire technique IMSG_MBOX_APPEND already established and had + * verified against the real imsg-buffer.c this project (see that + * struct's own comment above): the header's nnodes times sizeof(struct + * search_node) tells store_dispatch() exactly how many trailing bytes + * to expect. SEARCH_PROGRAM_MAX_NODES bounds both the array's wire size + * (comfortably under imsg's MAX_IMSGSIZE even with every node using its + * full fixed-size keyword field) and store.c's postfix-evaluation stack + * depth -- generous for a hand-typed v1 query, not a hard IMAP-mandated + * ceiling; listener.c rejects a query that compiles to more nodes than + * this with a plain BAD ("search criteria too complex"), the same "cap + * and refuse cleanly" choice APPEND makes for an oversized message. + * + * v1 scope, matching this project's smaller-feature-set design + * philosophy (openimap-privsep-design.md): search keys that need actual + * message content or headers -- BCC/BODY/CC/FROM/HEADER/SENTBEFORE/ + * SENTON/SENTSINCE/SUBJECT/TEXT/TO -- are rejected by listener.c's + * parser with a specific NO before this imsg is ever built, the same + * "recognized, can't do it right now" category FETCH's BODY[] rejection + * already uses; store.c never sees a SEARCH_OP_* for any of them because + * none exists. The "UID SEARCH" command-level wrapper (RFC 9051 + * SS6.4.9's generic UID prefix, which would make ESEARCH's data refer to + * UIDs instead of sequence numbers) is also out of scope this pass -- + * IMSG_MBOX_SEARCH_MATCH always carries both seqno and uid, but + * listener.c's v1 ESEARCH formatter only ever uses seqno. The "UID + * " *search key* (filtering by UID range, one of many + * possible criteria within an ordinary sequence-number-space SEARCH) is + * unrelated to that command-level wrapper and is fully supported + * (SEARCH_OP_UIDSET) -- SS6.4.4's own example list includes it as a + * plain search key, not as a UID-command variant. + * + * Exactly one sequence-set range per SEQSET/UIDSET node (no internal + * comma-separated list) -- the same v1 restriction already established + * for FETCH/STORE's own sequence-set argument (see imsg_mbox_fetch's + * comment above); listener.c rejects a comma inside a SEARCH sequence- + * set token before ever building a node for it. + */ +#define SEARCH_PROGRAM_MAX_NODES 100 +#define SEARCH_KEYWORD_MAX 64 /* one flag-keyword atom, not + * a list -- MBOX_FLAGS_MAX is + * sized for STORE's comma- + * joined keyword *list* and + * would be the wrong constant + * to reuse here */ + +#define SEARCH_OP_ALL 0 /* leaf: matches every message */ +#define SEARCH_OP_ANSWERED 1 +#define SEARCH_OP_UNANSWERED 2 +#define SEARCH_OP_DELETED 3 +#define SEARCH_OP_UNDELETED 4 +#define SEARCH_OP_DRAFT 5 +#define SEARCH_OP_UNDRAFT 6 +#define SEARCH_OP_FLAGGED 7 +#define SEARCH_OP_UNFLAGGED 8 +#define SEARCH_OP_SEEN 9 +#define SEARCH_OP_UNSEEN 10 +#define SEARCH_OP_KEYWORD 11 /* operand: keyword */ +#define SEARCH_OP_UNKEYWORD 12 /* operand: keyword */ +#define SEARCH_OP_BEFORE 13 /* operand: num (UTC day-start epoch) */ +#define SEARCH_OP_ON 14 /* operand: num (UTC day-start epoch) */ +#define SEARCH_OP_SINCE 15 /* operand: num (UTC day-start epoch) */ +#define SEARCH_OP_LARGER 16 /* operand: num (octets) */ +#define SEARCH_OP_SMALLER 17 /* operand: num (octets) */ +#define SEARCH_OP_SEQSET 18 /* operand: seq_lo/seq_hi/lo_is_star/ + * hi_is_star, matched against sequence + * number */ +#define SEARCH_OP_UIDSET 19 /* operand: seq_lo/seq_hi/lo_is_star/ + * hi_is_star, matched against UID */ +#define SEARCH_OP_AND 20 /* postfix binary combinator */ +#define SEARCH_OP_OR 21 /* postfix binary combinator */ +#define SEARCH_OP_NOT 22 /* postfix unary combinator */ +#define SEARCH_OP_MODSEQ 23 /* RFC 7162 SS3.1.5 -- operand: num + * (mod-sequence-valzer threshold, + * message matches if its own + * mod-sequence is >= this). The + * optional / prefix (SS3.1.5: "If the server + * doesn't store separate mod-sequences + * for different metadata items, it + * MUST ignore and ") is parsed by listener.c's + * parse_search_key() for syntax only + * and never reaches this struct -- v1's + * per-message mod-sequence (see store.c's + * struct mbox_index comment) is exactly + * the "doesn't store separate mod- + * sequences per metadata item" case the + * RFC anticipates, so there's nothing + * for store.c to narrow by */ + +struct search_node { + int op; /* SEARCH_OP_* above */ + int64_t num; /* BEFORE/ON/SINCE/LARGER/SMALLER/MODSEQ + * operand */ + uint32_t seq_lo; /* SEQSET/UIDSET operand -- see + * imsg_mbox_fetch's seq_lo/seq_hi/ + * lo_is_star/hi_is_star comment for + * the "*" resolution convention this + * mirrors; store.c resolves it here + * against its own live idx.nlines + * (SEQSET) or highest in-use UID + * (UIDSET) up front, once, before + * scanning messages */ + uint32_t seq_hi; + int lo_is_star; + int hi_is_star; + char keyword[SEARCH_KEYWORD_MAX]; /* KEYWORD/UNKEYWORD + * operand */ +}; + +struct imsg_mbox_search { + uint32_t nnodes; /* number of struct search_node entries + * in this imsg's trailing data */ +}; + +struct imsg_mbox_search_match { + uint32_t seqno; + uint32_t uid; + uint64_t modseq; /* RFC 7162 SS3.1.6: "If a client + * specifies a MODSEQ criterion in a + * SEARCH ... command and the server + * returns a non-empty SEARCH result, + * the server MUST also append ... the + * highest mod-sequence for all messages + * being returned." Always populated + * (cheap, store.c already has it + * per-message) -- listener.c only + * tracks the running max across + * matches when the client's search + * program actually used SEARCH_OP_ + * MODSEQ, same "store always computes, + * listener decides whether to use it" + * split as imsg_mbox_fetch_meta.modseq */ +}; + +/* + * RFC 9051 SS6.3.13 (IDLE). IMSG_MBOX_IDLE_REFRESH (listener -> store, no + * payload) / IMSG_MBOX_IDLE_UID (store -> listener, one per currently- + * existing message, in ascending UID order) / IMSG_MBOX_IDLE_REFRESHED + * (store -> listener, exactly once, terminal). + * + * Used two ways by listener.c: once, synchronously after "+ idling" is + * sent, purely to seed s->idle_known_uids with a baseline (no diffing or + * pushing yet, since there's nothing to diff against the first time); and + * again, any time session_notify_idle_peers() (triggered by another same- + * uid session's successful EXPUNGE/UID EXPUNGE/CLOSE-with-removal/APPEND/ + * MOVE) asks an idling session to recheck its mailbox. Both cases reuse + * the identical request/reply shape -- only what listener.c does with the + * result differs. store.c doesn't need to know or care which case this + * is; it just reports current, authoritative state, via the same refresh_ + * index() helper handle_mbox_select() itself uses (including that + * function's new/ directory scan for undiscovered message files), so an + * idle-refresh is exactly as fresh as a fresh SELECT would be -- including + * picking up mail placed directly in new/ by something other than this + * daemon's own APPEND, if an idle-refresh happens to run after it landed. + * + * v1 scope decision (confirmed with the user): EXISTS and EXPUNGE only. + * RFC 9051 SS6.3.13 says the server is merely "free to" send EXISTS/ + * EXPUNGE/FETCH while idling, none of the three is mandatory -- pushing + * unsolicited FETCH (e.g. for a flag changed by another session) is + * deferred to a follow-up pass, since it needs the same per-message + * mod-sequence diffing this server already has for QRESYNC, just re- + * triggered the same way, and RFC 9051 doesn't require it. That's why + * this reply carries only the UID list (enough for listener.c to compute + * seqno-based EXPUNGE lines and an EXISTS count) and not per-message + * flags/modseq. + */ +struct imsg_mbox_idle_uid { + uint32_t uid; +}; + +struct imsg_mbox_idle_refreshed { + int ok; + uint32_t exists; + uint32_t uidvalidity; + uint32_t uidnext; + uint64_t highestmodseq; +}; + +/* + * RFC 9051 SS6.3.4/SS6.3.5 (CREATE/DELETE) and SS6.3.9 (LIST), this pass -- + * see docs/openimap-storage-backend.md's "Open items" #10 for the full + * design (flat, non-nested mailboxes as sibling subdirectories of the + * session's own per-user maildir root; CREATE/DELETE/RENAME all reply with + * the existing struct imsg_mbox_result, only "ok" meaningful). listener.c + * has already validated the name syntactically (non-empty, not "INBOX", + * no hierarchy-delimiter character, within MBOX_NAME_MAX) before either of + * these is ever sent -- store.c re-validates independently rather than + * trusting that, the same defense-in-depth every other mailbox-name- + * carrying imsg in this file already gets across the privsep boundary. + */ +struct imsg_mbox_create { + char mailbox[MBOX_NAME_MAX]; +}; + +struct imsg_mbox_delete { + char mailbox[MBOX_NAME_MAX]; +}; + +/* + * RFC 9051 SS6.3.6 (RENAME). oldname/newname, not "mailbox"/"destination", + * to avoid any confusion with imsg_mbox_copy's identically-purposed but + * differently-named destination field -- RENAME's two names are peers + * (both mailbox names in the same flat namespace), not a source-message- + * range-plus-destination-mailbox pairing the way COPY/MOVE's is. + */ +struct imsg_mbox_rename { + char oldname[MBOX_NAME_MAX]; + char newname[MBOX_NAME_MAX]; +}; + +/* + * RFC 9051 SS6.3.9 (LIST). One IMSG_MBOX_LIST_ITEM per real, on-disk + * mailbox subdirectory store.c finds under the session's maildir root + * (INBOX itself excluded -- see the IMSG_MBOX_LIST_ITEM enum comment + * above), in no particular guaranteed order; listener.c does its own + * wildcard matching (list_pattern_match()) against each name plus the + * literal "INBOX" it already handles locally. Terminal reply reuses struct + * imsg_mbox_result ("count" = number of names streamed, "ok" = 0 only on a + * real I/O error opening the maildir root itself, not on finding zero + * mailboxes -- an account with no named mailboxes yet is not a failure). + */ +struct imsg_mbox_list_item { + char mailbox[MBOX_NAME_MAX]; +}; + +/* main.c */ +const char *log_procname(enum openimap_proc_type); + +/* parse.y */ +int config_load(const char *, struct openimap_config *); +int cmdline_symset(char *); + +/* parent.c */ +/* + * First argument is the config file path, threaded through from main.c's + * "conffile" local (default "/etc/imapd.conf" or -f's argument) -- added + * for SIGHUP reload support: parent.c's sighup_handler() needs to re-run + * config_load() against the exact same path main() originally used, and + * had no way to reach it before (main()'s "conffile" was a local, never + * passed down; parent.c only ever received the already-parsed struct). + */ +__dead void parent_main(const char *, int, char *[], + struct openimap_config *); + +/* + * listener.c / auth.c / store.c take no struct openimap_config * -- + * none of the three re-exec'd child roles read imapd.conf, and as of + * this pass none of them receive it as a stub parameter either (an + * earlier draft did, unused/misleadingly for listener and store, and + * actively wrong for auth -- see the imsg_listener_init/imsg_auth_init + * comment above). Each gets exactly the config it needs over its fd-3 + * channel from parent instead: IMSG_LISTENER_INIT, IMSG_AUTH_INIT, + * IMSG_STORE_INIT respectively. + */ +__dead void listener_main(void); +__dead void auth_main(void); +__dead void store_main(void); + +/* + * imsg helpers shared by all roles -- see each role's setup_proc(). The + * "handler" passed to imsgev_init() is the real libevent callback: it's + * expected to do its own imsgbuf_read()/imsg_get() loop, matching + * parent.c's parent_dispatch_child() as the reference shape. This mirrors + * the imsg_event_add()-style pattern common to OpenBSD privsep daemons + * (relayd, httpd) -- not itself quoted from any uploaded source file this + * session, flagged as a standard-pattern choice, not a sourced one. + */ +void imsgev_init(struct imsgev *, int, + void (*)(int, short, void *), void *); +void imsgev_init_from_ibuf(struct imsgev *, struct imsgbuf *, + void (*)(int, short, void *), void *); +void imsgev_add(struct imsgev *); + +/* + * Boot-time setup-loop helpers, sourced against smtpd.c's setup_proc() + * shape (see openimap-privsep-design.md's "Peer-wiring handshake" + * section): a freshly exec'd child blocks reading fd 3 for zero or more + * IMSG_SETUP_PEER messages, then an IMSG_SETUP_DONE, and acks. v1's + * boot-time wiring is 1:1 (listener gets exactly one peer, auth gets + * exactly one) so setup_recv_one_peer() covers both; store's later, + * per-session peer wiring happens post-boot, through the normal event + * loop, not through these blocking helpers -- see store.c. + */ +int setup_recv_one_peer(struct imsgbuf *); +void setup_recv_done_and_ack(struct imsgbuf *); + +#endif /* IMAPD_H */ blob - /dev/null blob + 2afc427465067cbf551fb84c000c437ffdacda0a (mode 644) --- /dev/null +++ src/imsgev.c @@ -0,0 +1,214 @@ +/* + * Copyright (c) 2026 David Williams + * + * Permission to use, copy, modify, and distribute this software for any + * purpose with or without fee is hereby granted, provided that the above + * copyright notice and this permission notice appear in all copies. + * + * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + */ + +/* + * Small shared wrapper around imsgbuf + event(3), used identically by + * parent.c, listener.c, auth.c, and store.c. Not itself sourced from any + * uploaded file this session -- see the comment on struct imsgev in + * openimap.h. + * + * API NAMES: checked against the real src/imsg.h this session (see + * parent.c's header comment) -- imsgbuf_set_maxsize() and + * imsgbuf_queuelen() both match the real header exactly. + * MAX_IMSGSIZE itself *is* sourced (imsg_init(3), cited throughout + * openimap-privsep-design.md). + */ + +#include + +#include +#include +#include + +#include "imapd.h" +#include "log.h" + +void +imsgev_init(struct imsgev *iev, int fd, void (*handler)(int, short, void *), + void *data) +{ + if (imsgbuf_init(&iev->ibuf, fd) == -1) + fatal("imsgbuf_init"); + imsgbuf_set_maxsize(&iev->ibuf, MAX_IMSGSIZE); + + /* + * Every channel wrapped by struct imsgev either sends or receives + * fd-passed messages somewhere in its lifetime (listening sockets, + * SETUP_PEER peer fds, later store-peer fds) -- imsgbuf_allow_fdpass() + * is real (confirmed against src/imsg.h) and was missing from this + * skeleton entirely until this pass. Unconditional here rather than + * per-caller since there's no channel in this design that *never* + * passes an fd. + */ + imsgbuf_allow_fdpass(&iev->ibuf); + + iev->handler = handler; + iev->data = data != NULL ? data : iev; + iev->events = EV_READ; + + event_set(&iev->ev, fd, iev->events, iev->handler, iev->data); + event_add(&iev->ev, NULL); +} + +/* + * Like imsgev_init(), but for a channel whose struct imsgbuf has already + * been imsgbuf_init()'d and read from -- e.g. a fd-3 boot channel that a + * child process drained synchronously (before event_init() was even + * callable) and now wants to hand off to the event loop for the rest of + * its life. Takes ownership of *ibuf by copying it into iev->ibuf, NOT by + * calling imsgbuf_init() again. + * + * This distinction matters: unlike struct imsgev (which embeds a struct + * event -- see the struct store_child comment in parent.c for why THAT + * must never be copied once registered with libevent), struct imsgbuf + * itself holds no libevent registration and its only heap state (`w`) is + * a pointer, so copying it by value is safe and standard -- but calling + * imsgbuf_init() a *second* time on the same fd would silently discard + * any bytes imsgbuf_read() had already buffered beyond the messages the + * caller happened to have consumed so far (parent doesn't wait for acks + * between sends, so more than one message can already be sitting in the + * kernel socket buffer -- and possibly already pulled into ibuf's + * internal state -- by the time a child gets around to reading it). + * imsgev_init() alone would have that bug; this function exists so + * listener.c's fd-3 channel (parent) doesn't hit it. See listener.c's + * listener_main() for the caller. + */ +void +imsgev_init_from_ibuf(struct imsgev *iev, struct imsgbuf *ibuf, + void (*handler)(int, short, void *), void *data) +{ + iev->ibuf = *ibuf; + + iev->handler = handler; + iev->data = data != NULL ? data : iev; + iev->events = EV_READ; + + event_set(&iev->ev, iev->ibuf.fd, iev->events, iev->handler, + iev->data); + event_add(&iev->ev, NULL); +} + +/* + * Re-arm after composing an outgoing message: watch EV_WRITE too if the + * imsgbuf has queued, unflushed output. Callers' dispatch handlers should + * call this at the end of any code path that calls imsg_compose(). + */ +void +imsgev_add(struct imsgev *iev) +{ + iev->events = EV_READ; + if (imsgbuf_queuelen(&iev->ibuf) > 0) + iev->events |= EV_WRITE; + + event_del(&iev->ev); + event_set(&iev->ev, iev->ibuf.fd, iev->events, iev->handler, + iev->data); + event_add(&iev->ev, NULL); +} + +/* + * Blocks (no event loop running yet) for exactly one IMSG_SETUP_PEER on + * ibuf3 (already imsgbuf_init()'d on fd 3 by the caller) and returns the + * fd-passed peer fd. fatalx()s on anything else -- matches smtpd's + * setup_proc() treating an unexpected message during setup as fatal + * ("bad imsg %d"). + * + * Real deadlock caught on first real-hardware run (OpenBSD, not this + * sandbox): parent.c sends IMSG_SETUP_PEER and IMSG_SETUP_DONE back to + * back on the same socketpair fd (setup_peer_send() then + * setup_done_send(), both flushed immediately, no wait in between). On a + * SOCK_STREAM socketpair the kernel is free to coalesce both sends into + * one readable chunk -- confirmed via ktrace(1) on the live hang: a + * single recvmsg() here returned 32 bytes (both 16-byte imsg headers) + * instead of 16. imsg_get() correctly peels off just the first message + * and leaves the second one fully buffered in ibuf3's own userspace + * state -- but the *caller* (setup_recv_done_and_ack(), immediately + * next) used to call imsgbuf_read() unconditionally before ever checking + * whether a complete message was already sitting there, so it issued a + * second blocking recvmsg() for bytes that were never coming, while + * parent sat blocked reading this process's now-overdue SETUP_DONE ack. + * Fixed by checking imsg_get() FIRST in both this loop and + * setup_recv_done_and_ack()'s below -- imsgbuf_read() (an actual + * blocking syscall) only happens when nothing is already buffered. + */ +int +setup_recv_one_peer(struct imsgbuf *ibuf3) +{ + struct imsg imsg; + ssize_t n; + int fd; + + for (;;) { + if ((n = imsg_get(ibuf3, &imsg)) == -1) + fatal("imsg_get"); + if (n != 0) + break; + if ((n = imsgbuf_read(ibuf3)) == -1) + fatal("imsgbuf_read"); + if (n == 0) + fatalx("setup_recv_one_peer: parent closed channel"); + } + + if (imsg_get_type(&imsg) != IMSG_SETUP_PEER) + fatalx("setup_recv_one_peer: expected IMSG_SETUP_PEER, got %d", + imsg_get_type(&imsg)); + + if ((fd = imsg_get_fd(&imsg)) == -1) + fatalx("setup_recv_one_peer: IMSG_SETUP_PEER carried no fd"); + + imsg_free(&imsg); + return (fd); +} + +/* + * Blocks for IMSG_SETUP_DONE on ibuf3, then sends one back as an ack. + * Sourced against smtpd.c's setup_proc() loop (IMSG_SETUP_DONE case sets + * a "done" flag and exits the loop; the ack-back send is the same + * imsg_compose(ibuf, IMSG_SETUP_DONE, 0, 0, -1, NULL, 0) shape quoted in + * the design doc from setup_proc()'s tail). + * + * imsg_get()-before-imsgbuf_read() ordering: see setup_recv_one_peer()'s + * header comment above -- this is the specific call site where the real + * deadlock happened (IMSG_SETUP_DONE arrives already-buffered, coalesced + * with the preceding IMSG_SETUP_PEER read by the caller). + */ +void +setup_recv_done_and_ack(struct imsgbuf *ibuf3) +{ + struct imsg imsg; + ssize_t n; + + for (;;) { + if ((n = imsg_get(ibuf3, &imsg)) == -1) + fatal("imsg_get"); + if (n != 0) + break; + if ((n = imsgbuf_read(ibuf3)) == -1) + fatal("imsgbuf_read"); + if (n == 0) + fatalx("setup_recv_done_and_ack: parent closed channel"); + } + + if (imsg_get_type(&imsg) != IMSG_SETUP_DONE) + fatalx("setup_recv_done_and_ack: expected IMSG_SETUP_DONE, " + "got %d", imsg_get_type(&imsg)); + imsg_free(&imsg); + + if (imsg_compose(ibuf3, IMSG_SETUP_DONE, 0, 0, -1, NULL, 0) == -1) + fatal("imsg_compose IMSG_SETUP_DONE"); + if (imsgbuf_flush(ibuf3) == -1) + fatal("imsgbuf_flush"); +} blob - /dev/null blob + a42a003990cf0f34ffcf910cd350f587df7783e9 (mode 644) --- /dev/null +++ src/listener.c @@ -0,0 +1,10619 @@ +/* + * Copyright (c) 2026 David Williams + * + * Permission to use, copy, modify, and distribute this software for any + * purpose with or without fee is hereby granted, provided that the above + * copyright notice and this permission notice appear in all copies. + * + * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + */ + +/* + * listener.c -- protocol/network process. Implements the "listener" + * section of openimap-privsep-design.md: owns client TCP sockets, runs + * the IMAP command parser/dispatch table for the "any state" and "not + * authenticated state" command sets, and now terminates TLS for real + * (implicit-TLS port 993 per RFC 8314, and STARTTLS on the cleartext + * port) via libtls -- tls_server()/tls_configure() built once at boot + * from cert/key bytes parent sends (IMSG_TLS_CERT/IMSG_TLS_KEY), then + * tls_accept_socket()+tls_handshake() per connection, sourced directly + * against src/lib/libtls/tls.h and httpd's server_tls_init()/server_ + * tls_handshake() (openbsd_source/src/usr.sbin/httpd/server.c). AUTHENTICATE + * PLAIN is now real too: cmd_authenticate() handles both the inline- + * initial-response and continuation-request forms (RFC 9051 SS6.2.2), + * decodes via b64_pton() (, sourced against openbsd_source/src/ + * lib/libc/net/base64.c), splits per RFC 4616's authzid/authcid/passwd + * framing (fetched directly from https://www.rfc-editor.org/rfc/rfc4616.txt + * this session -- not present in research/ or openbsd_source/), and sends + * IMSG_AUTH_REQUEST to auth over iev_auth; the tagged OK/NO back to the + * client is sent from whichever of listener_dispatch_auth()'s IMSG_AUTH_ + * RESULT or listener_dispatch_parent()'s IMSG_SETUP_PEER/IMSG_STORE_FORK + * cases actually resolves the async round trip. A session can now + * genuinely reach SESSION_AUTHENTICATED. imap_cmds[] also now has a real, + * correctly state-gated entry for every command-auth (RFC 9051 SS6.3) and + * command-select (SS6.4) command -- ENABLE, SELECT, LIST, FETCH, STORE, + * EXPUNGE, CLOSE, UNSELECT, APPEND, and SEARCH are fully implemented + * (cmd_enable()/cmd_select()/cmd_list()/cmd_fetch()/cmd_store_cmd()/cmd_ + * expunge()/cmd_close()/cmd_unselect()/cmd_append()/cmd_search(), FETCH + * message-metadata-only: FLAGS/UID/INTERNALDATE/RFC822.SIZE, not BODY[] + * -- see cmd_fetch()'s own comment for why). STORE reuses FETCH's + * IMSG_MBOX_FETCH_META/IMSG_MBOX_RESULT reply pair, since RFC 9051 + * SS6.4.6 says STORE's only response is itself an untagged FETCH; CLOSE + * reuses EXPUNGE's IMSG_MBOX_EXPUNGE/IMSG_MBOX_EXPUNGED/IMSG_MBOX_RESULT + * wholesale (silent=1, no untagged EXPUNGE responses, and a different + * post-completion state -- see session_request_expunge()/session_handle_ + * mbox_result()); UNSELECT needs no store round trip at all, since + * store.c holds no per-selection state to free. LIST (SS6.3.9, basic + * syntax only -- extended selection/return options get a flagged NO) + * needs no store round trip either: v1 has no CREATE, so INBOX's + * existence is never in question, making LIST pure string/wildcard + * matching against the fixed name "INBOX" (list_pattern_match()). + * APPEND (SS6.3.12) required this file's first real IMAP literal + * ({n}/{n+}) support -- a raw-byte read phase (s->literal_pending, in + * session_dispatch_client()'s read loop, intercepted before the CRLF + * line parser since literal bytes can contain embedded CRLFs) followed + * by a store round trip in the new SESSION_APPENDING state; message + * bytes travel to store.c as variable-length trailing data on a single + * imsg (imsg_get_buf()/imsg_get_len(), verified against the real imsg- + * buffer.c this session), capped at APPEND_LITERAL_MAX (12000 bytes) to + * stay under imsg's MAX_IMSGSIZE -- larger messages need real fd-passing, + * not implemented this pass, same deferral as BODY[] FETCH. SEARCH + * (SS6.4.4, ESEARCH responses per SS7.3.4) compiles the flag/date/size/ + * sequence-number/UID-range/NOT/OR/parenthesized-list search-key grammar + * into a flat postfix bytecode (parse_search_key()/parse_search_key_ + * list()) sent to store.c as variable-length trailing data on IMSG_MBOX_ + * SEARCH (the same imsg technique APPEND established), which streams + * back matching sequence numbers via IMSG_MBOX_SEARCH_MATCH for session_ + * finish_search() to assemble into MIN/MAX/ALL/COUNT per RFC 9051's own + * worked examples; content-and-header-based search keys (BCC/BODY/CC/ + * FROM/HEADER/SENTBEFORE/SENTON/SENTSINCE/SUBJECT/TEXT/TO), the SAVE/"$" + * result variable, and the "UID SEARCH" command wrapper are all + * deliberately out of scope this pass -- see imapd.h's imsg_mbox_ + * search comment. Everything else in those two command sets still + * replies NO via stub_not_implemented() pending store.c's still- + * undesigned IMSG_MBOX_* wire protocol for that operation. A session can + * now genuinely reach SESSION_SELECTED, issue a LIST from either + * Authenticated or Selected, and round-trip a FETCH, STORE, EXPUNGE, + * CLOSE, UNSELECT, APPEND, or SEARCH. + * + * The boot-time setup handshake, receiving the listening socket fds + + * TLS cert/key from parent, the per-session table, requesting + wiring + * a per-session store child on successful auth (IMSG_STORE_FORK / + * IMSG_SETUP_PEER / IMSG_SETUP_DONE from parent, demuxed by session_id), + * its own privilege drop, and full session teardown (including + * notifying a wired store child via IMSG_STORE_SHUTDOWN, and closing + * a TLS session correctly -- see session_teardown()'s comment on why + * tls_close() sometimes needs a manual close(2) fallback and sometimes + * doesn't) are all implemented for real too. + * + * Two distinct persistent channels exist here, fixed this pass (see the + * "channel-identity gap" note that used to be here as a flagged TODO): + * - iev_auth: the boot-time SETUP_PEER-wired channel to the AUTH + * process (peer_fd below). Carries IMSG_AUTH_REQUEST out / + * IMSG_AUTH_RESULT in. + * - iev_parent: this process's own fd-3 channel back to PARENT, kept + * alive for the process's whole lifetime (parent never closes its + * end -- "parent isn't on this path once wiring completes" only + * applies to STORE, not to listener/auth themselves). Carries + * IMSG_STORE_FORK out, and IMSG_SETUP_PEER (a new store child's + * peer fd, demuxed by session_id via imsg_get_id()) / IMSG_STORE_FORK + * (parent replying with a *failure*) in. + * + * An earlier draft of this file used ONE struct imsgev (also named + * iev_parent) fed from peer_fd -- i.e. it was actually the AUTH channel + * despite the name -- and never turned fd 3 itself into a persistent, + * event-driven channel at all past the synchronous boot-time drain loop. + * That meant IMSG_STORE_FORK was being sent to auth, not parent, and + * nothing ever read fd 3 again after boot. Fixed below by keeping fd 3's + * already-populated struct imsgbuf alive across the transition into the + * event loop (imsgev_init_from_ibuf() in imsgev.c) instead of a second, + * fresh imsgbuf_init() on the same fd, which would have silently dropped + * any bytes already buffered from parent (parent doesn't wait for acks + * between sends, so more than one message can already be in flight by + * the time this process gets around to reading it). + * + * API NAMES: checked against the real src/imsg.h this session -- see + * parent.c's header comment for the full verification note. + */ + +#include +#include +#include + +#include + +/* + * (below, for b64_pton()) uses "struct sockaddr_in", + * "struct in_addr", and "struct in6_addr" without defining them itself -- + * confirmed against openbsd_source/src/include/resolv.h, which only + * includes , , and . + * must come first, matching the ordering smtpd's util.c and httpd's + * server_http.c both use for the same pairing. + */ +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "imapd.h" +#include "log.h" + +enum session_state { + SESSION_NOT_AUTH, + SESSION_AUTHENTICATING, /* IMSG_AUTH_REQUEST sent, awaiting reply */ + SESSION_STORE_PENDING, /* IMSG_STORE_FORK sent, awaiting peer */ + SESSION_AUTHENTICATED, + SESSION_SELECTING, /* IMSG_MBOX_SELECT sent, awaiting reply -- + * see cmd_select()/session_store_dispatch() */ + SESSION_SELECTED, + SESSION_FETCHING, /* IMSG_MBOX_FETCH sent, awaiting the + * IMSG_MBOX_FETCH_META stream + terminal + * IMSG_MBOX_RESULT -- see cmd_fetch()/ + * session_handle_mbox_result() */ + SESSION_STORING, /* IMSG_MBOX_STORE sent, awaiting the same + * IMSG_MBOX_FETCH_META stream + terminal + * IMSG_MBOX_RESULT shape SESSION_FETCHING + * uses -- see cmd_store_cmd()'s comment on + * why STORE reuses FETCH's reply types */ + SESSION_EXPUNGING, /* IMSG_MBOX_EXPUNGE sent (by EXPUNGE itself, + * or by CLOSE with silent=1), awaiting the + * IMSG_MBOX_EXPUNGED stream + terminal + * IMSG_MBOX_RESULT -- see cmd_expunge()/ + * cmd_close()/session_handle_mbox_result() */ + SESSION_APPENDING, /* IMSG_MBOX_APPEND sent, awaiting the single + * terminal IMSG_MBOX_APPENDED reply -- see + * cmd_append()/session_finish_append()/ + * session_handle_mbox_appended(). Distinct + * from the *client-literal-read* phase that + * precedes it (s->literal_pending -- see that + * field's comment): this state only covers + * the store round trip, once the full + * message has already been read off the + * wire. */ + SESSION_SEARCHING, /* IMSG_MBOX_SEARCH sent, awaiting the + * IMSG_MBOX_SEARCH_MATCH stream + terminal + * IMSG_MBOX_RESULT -- see cmd_search()/ + * session_finish_search(). Same per-match- + * stream-then-terminal-result reply shape as + * SESSION_FETCHING/STORING/EXPUNGING, so it's + * handled by the same session_handle_mbox_ + * result() entry point, which branches to + * session_finish_search() first. */ + SESSION_STATUSING, /* IMSG_MBOX_STATUS sent, awaiting the single + * terminal IMSG_MBOX_STATUS_RESULT reply -- + * see cmd_status()/session_handle_mbox_ + * status_result(). Same single-request/single- + * reply shape as SESSION_SELECTING, but + * (unlike SELECT) never changes s->state's + * SELECTED-ness -- STATUS "does not change the + * currently selected mailbox" (RFC 9051 + * SS6.3.11) -- so s->status_prev_state records + * whichever ST_AUTH state was current when + * cmd_status() was called, for session_handle_ + * mbox_status_result() to restore. */ + SESSION_COPYING, /* IMSG_MBOX_COPY or IMSG_MBOX_MOVE sent + * (s->cmd_is_move says which), awaiting the + * IMSG_MBOX_COPY_MAPPING stream (plus, for a + * MOVE, an interleaved IMSG_MBOX_EXPUNGED + * stream -- see s->move_expunged's comment) + + * terminal IMSG_MBOX_RESULT -- see cmd_copy()/ + * cmd_move()/session_finish_copy_or_move(). + * Same per-message-stream-then-terminal-result + * shape as SESSION_FETCHING/STORING/EXPUNGING/ + * SEARCHING, so it's handled by the same + * session_handle_mbox_result() entry point, + * which branches to session_finish_copy_or_ + * move() first, the same way it already + * branches to session_finish_search(). */ + + /* + * RFC 9051 SS6.3.4-SS6.3.6/SS6.3.9 additions (flat multi-mailbox + * support, docs/openimap-storage-backend.md item 10). All four are + * command-auth (valid in Authenticated or Selected state) and never + * change s->state's SELECTED-ness -- same "record whichever ST_AUTH + * state was current before, restore it after" pattern SESSION_ + * STATUSING's s->status_prev_state already established, reused here + * as s->mbox_op_prev_state (one shared field: only one of these four + * can ever be in flight for a given session at once, so there's no + * need for four separate fields the way SESSION_APPENDING has its + * own append_prev_state). + */ + SESSION_CREATING, /* IMSG_MBOX_CREATE sent, awaiting the single + * terminal IMSG_MBOX_RESULT reply -- see + * cmd_create()/session_finish_mbox_op(). */ + SESSION_DELETING, /* IMSG_MBOX_DELETE sent, same single-terminal- + * reply shape as SESSION_CREATING -- see + * cmd_delete()/session_finish_mbox_op(). */ + SESSION_RENAMING, /* IMSG_MBOX_RENAME sent, same single-terminal- + * reply shape as SESSION_CREATING -- see + * cmd_rename()/session_finish_mbox_op(). */ + SESSION_LISTING /* IMSG_MBOX_LIST sent, awaiting the + * IMSG_MBOX_LIST_ITEM stream + terminal + * IMSG_MBOX_RESULT -- see list_dispatch()/ + * session_finish_list(). Same per-item-stream- + * then-terminal-result shape as SESSION_ + * SEARCHING/COPYING, so it's handled by the + * same session_handle_mbox_result() entry + * point too, branching to session_finish_ + * list() first. */ +}; + +/* + * Generous line-length cap for the raw CRLF-delimited read buffer below. + * RFC 9051 doesn't mandate a specific limit -- SS7.1.3's "* BAD Command + * line too long" is example text, not a normative value -- but any real + * server needs one to bound memory for a client that never sends CRLF. + * Revisit once IMAP literals ({n}-prefixed octet counts, SS4.3) are + * implemented: those need a different, larger mechanism than a flat line + * buffer, since a literal's byte count is part of the command syntax + * itself, not just "a longer line". + */ +#define SESSION_INBUF_MAX 8192 + +/* + * Generous bound for a client-chosen tag we need to remember across an + * async IMSG_AUTH_REQUEST/IMSG_AUTH_RESULT (and, on success, IMSG_STORE_ + * FORK/IMSG_SETUP_PEER) round trip -- RFC 9051 SS9 doesn't specify a tag + * length limit (`tag = 1*`), so this is the + * same kind of deliberate, flagged simplification as session_reply()'s + * 512-byte response buffer: truncated via strlcpy() rather than rejected, + * matching this file's existing truncate-rather-than-overflow style. + */ +#define IMAP_TAG_MAX 64 + +/* + * Cap on the verbatim label text (e.g. "HEADER.FIELDS (DATE FROM)" or + * "HEADER.FIELDS.NOT (X-SPAM-STATUS)") stashed on struct session's + * pending_header_label and echoed back into the FETCH response's + * "BODY[]" -- see that field's own comment for the full + * reasoning. Defined up here (rather than down near parse_fetch_atts(), + * where it's actually used) because struct session, just below, + * declares a fixed array of this size -- needs to be visible before that + * point. Never crosses the imsg boundary (listener.c has this text + * straight from the client's own command line), so this is a + * listener.c-local constant, not a shared imapd.h one like HEADER_ + * FIELDS_MAX. Sized generously above HEADER_FIELDS_MAX (256, imapd.h) + * to cover the "HEADER.FIELDS.NOT (...)" wrapper text around whatever + * field-name list already fits under that cap. + */ +#define HEADER_FIELDS_LABEL_MAX 288 + +/* One RFC 7162 VANISHED (EARLIER) range -- see struct session's vanished_ + * ranges comment and format_range_list(). Named (not anonymous) so format_ + * range_list() can take it as a parameter type. */ +struct vanished_range { + uint32_t lo; + uint32_t hi; +}; + +struct session { + uint32_t id; + int client_fd; + struct event client_ev; + enum session_state state; + int implicit_tls; /* accepted on port 993, per + * RFC 8314 -- vs. port 143, + * where TLS only starts after + * STARTTLS */ + struct imsgev *store_iev; /* NULL until STORE_PENDING + * resolves */ + int client_ev_added; /* client_ev is only safe to + * event_del() once this is + * set -- see listener_accept()'s + * implicit-TLS early-teardown + * path, which never reaches + * the event_set()/event_add() + * call below it. */ + int tls_active; /* 1 once a real TLS handshake + * (implicit-TLS accept or + * STARTTLS) has completed -- + * threaded through CAPABILITY's + * state-appropriate response and + * AUTHENTICATE's TLS gate. */ + struct tls *tls_ctx; /* NULL until tls_accept_socket() + * -- non-NULL during handshake + * too, not just once tls_active */ + int pending_greeting; /* implicit-TLS only: the + * greeting can't be sent until + * after the handshake completes + * (RFC 8314 -- no protocol bytes + * before TLS), so listener_accept() + * defers it and session_tls_ + * handshake() sends it on success */ + char inbuf[SESSION_INBUF_MAX]; + size_t inbuflen; /* bytes of unparsed input + * currently in inbuf */ + int auth_cont; /* 1 while the next raw client + * line is a SASL continuation- + * response (base64 or "*"), not + * a fresh tagged command -- set + * by cmd_authenticate() when the + * client sends bare "AUTHENTICATE + * PLAIN" with no inline initial + * response, cleared by session_ + * handle_auth_continuation() as + * soon as that line arrives. */ + int idling; /* 1 while the next raw client + * line is IDLE's "DONE" + * continuation, not a fresh + * tagged command -- set by + * cmd_idle(), cleared by + * session_handle_idle_ + * continuation() (RFC 9051 + * SS6.3.13). Same shape as + * auth_cont above, checked + * second in session_dispatch_ + * client()'s read loop. */ + uid_t uid; /* set once, in session_request_ + * store(), from the same imsg_ + * auth_result the store-fork + * request itself uses -- exists + * purely so session_notify_ + * idle_peers() can find this + * session's *other* concurrent + * sessions (same user, v1 has + * no shared mailboxes so that's + * the only case that can ever + * matter) without a second round + * trip through auth just to ask + * "whose session is this." */ + char pending_tag[IMAP_TAG_MAX]; /* copy of whichever + * command's tag is waiting on an + * async imsg round trip right now + * (AUTHENTICATE -> auth, or SELECT + * -> store) -- tag itself is a + * pointer into inbuf, which gets + * overwritten by the next read() + * long before IMSG_AUTH_RESULT (or + * the later IMSG_SETUP_PEER/ + * IMSG_STORE_FORK store-spawn + * reply, or IMSG_MBOX_SELECTED) + * comes back, so it has to be + * saved out. Only ever one such + * round trip in flight at a time + * per session (SESSION_ + * AUTHENTICATING/STORE_PENDING/ + * SELECTING/FETCHING are all + * mutually exclusive states), so + * one field is enough. */ + uint32_t fetch_attrs; /* MBOX_FETCH_* bitmask for + * the FETCH currently in flight + * (SESSION_FETCHING) -- needed + * because struct imsg_mbox_ + * fetch_meta's fields (uid, + * size, ...) are always + * populated by store regardless + * of what was actually + * requested, so session_send_ + * fetch_response() needs to + * know which ones to print. */ + char *pending_header_buf; /* malloc(3)'d raw header + * bytes from the most recent + * IMSG_MBOX_FETCH_HEADER, held + * here until the very next + * IMSG_MBOX_FETCH_META (same + * message, always sent + * immediately after -- see that + * struct's comment in + * imapd.h) folds it into one + * FETCH response line; freed and + * NULL'd there. Never allocated + * for more than one message at a + * time -- store.c's ordering + * contract means this is never + * still set when the next + * IMSG_MBOX_FETCH_HEADER + * arrives, but session_teardown() + * frees it defensively anyway in + * case a FETCH is torn down + * mid-stream. */ + uint32_t pending_header_len; /* valid only alongside + * pending_header_buf != NULL */ + int pending_header_found; /* 0 -- this message's + * header couldn't be read; 1 + * -- pending_header_buf/_len are + * valid. Distinguishes "BODY[HEADER] + * was requested but this message + * has none to give" from "BODY[HEADER] + * wasn't requested at all", the + * latter never populating this or + * pending_header_buf in the first + * place. */ + char pending_header_label[HEADER_FIELDS_LABEL_MAX]; + /* verbatim client-typed section-spec + * text -- "HEADER" for plain BODY.PEEK + * [HEADER], or e.g. "HEADER.FIELDS + * (DATE FROM)" for that variant -- + * echoed back as-is in the FETCH + * response's "BODY[]" rather than + * reconstructed, since RFC 9051 doesn't + * mandate an exact echo format and + * verbatim is simplest and trivially + * correct. Set once in fetch_dispatch(), + * read by session_send_fetch_response() + * -- never needs its own free()/reset + * since it's a fixed-size buffer, not a + * malloc(3)'d pointer like pending_ + * header_buf. */ + char *pending_body_buf; /* same stash-until- + * the-next-IMSG_MBOX_FETCH_META + * shape as pending_header_buf above, + * just for IMSG_MBOX_FETCH_BODY + * (BODY.PEEK[]/BODY.PEEK[TEXT]) + * instead -- see struct imsg_mbox_ + * fetch_body's comment in + * imapd.h. Freed and NULL'd by + * session_send_fetch_response(); + * session_teardown() frees it + * defensively too, same reasoning as + * pending_header_buf. */ + uint32_t pending_body_len; /* valid only alongside + * pending_body_buf != NULL */ + int pending_body_found; /* same found/not-found + * distinction as pending_header_ + * found, for BODY.PEEK[]/ + * BODY.PEEK[TEXT] */ + char pending_body_label[SECTION_PART_MAX]; + /* verbatim section text for the + * "BODY[]" response -- + * "" for BODY.PEEK[] (whole + * message), "TEXT" for BODY.PEEK + * [TEXT], or the client-typed + * dotted-numeric path (e.g. "3.1") + * for BODY.PEEK[] -- + * same "precompute the full + * verbatim label at dispatch time" + * idiom as pending_header_label, + * unifying what used to be a + * pending_body_is_text boolean + * (adequate when only WHOLE/TEXT + * existed, not once a third, + * client-supplied-text variant -- + * PART -- joined them). Set once in + * fetch_dispatch() from whichever of + * MBOX_FETCH_BODY_WHOLE/_TEXT/_PART + * attrs selects (same WHOLE > TEXT > + * PART precedence store.c's + * handle_mbox_fetch() applies -- + * see that function's comment), + * read by session_send_fetch_ + * response(). */ + int pending_body_has_partial; /* 1 if the + * client's own BODY.PEEK[...] + * token carried a <> + * suffix (RFC 9051 SS6.4.5) for + * whichever of WHOLE/TEXT/PART was + * actually selected -- see struct + * imsg_mbox_fetch's has_partial + * comment in imapd.h. Only + * pending_body_partial_origin (the + * *requested* start octet, not + * store.c's own bodylen -- SS6.4.5: + * "The origin octet facility MUST + * NOT be used ... unless the client + * specifically requested it", and + * the response echoes only the + * origin, never the count) is echoed + * back; set alongside pending_body_ + * label in fetch_dispatch(). */ + uint32_t pending_body_partial_origin; + char *pending_envelope_buf; /* same stash- + * until-the-next-IMSG_MBOX_ + * FETCH_META shape as pending_ + * header_buf/pending_body_buf + * above, for IMSG_MBOX_FETCH_ + * ENVELOPE -- see struct imsg_ + * mbox_fetch_envelope's comment + * in imapd.h. Unlike those + * two, holds already-formatted + * "(...)" envelope response + * text, not raw message bytes, + * so session_send_fetch_ + * response() writes it directly + * rather than wrapping it in a + * `{n}` literal. Freed and + * NULL'd by session_send_fetch_ + * response(); session_teardown() + * frees it defensively too, same + * reasoning as pending_header_buf. */ + uint32_t pending_envelope_len; /* valid only + * alongside pending_envelope_buf + * != NULL */ + int pending_envelope_found; /* same found/ + * not-found distinction as + * pending_header_found, for + * ENVELOPE */ + char *pending_bodystructure_buf; /* same stash- + * until-the-next-IMSG_MBOX_ + * FETCH_META shape, same + * already-formatted-text shape, + * as pending_envelope_buf above, + * for IMSG_MBOX_FETCH_ + * BODYSTRUCTURE -- see struct + * imsg_mbox_fetch_bodystructure's + * comment in imapd.h. */ + uint32_t pending_bodystructure_len; /* valid only + * alongside pending_ + * bodystructure_buf != NULL */ + int pending_bodystructure_found; /* same found/ + * not-found distinction as + * pending_header_found, for + * BODYSTRUCTURE */ + char pending_bodystructure_label[16]; /* + * "BODY" or "BODYSTRUCTURE" -- + * RFC 9051 SS9's msg-att-static: + * `"BODY" ["STRUCTURE"] SP body` + * means the response label + * itself echoes whichever bare + * token the client used (both + * set the same MBOX_FETCH_ + * BODYSTRUCTURE bit and produce + * identical body text in this + * implementation -- see that + * bit's imapd.h comment -- + * but the label still has to + * match what was asked for). + * Set once in fetch_dispatch(), + * read by session_send_fetch_ + * response() -- same fixed- + * buffer, no-free-needed shape + * as pending_header_label. */ + int close_after_expunge; /* 1 if the in-flight + * SESSION_EXPUNGING round trip + * was started by CLOSE, not a + * real EXPUNGE command -- both + * send the identical IMSG_MBOX_ + * EXPUNGE (CLOSE with silent=1) + * and get back the identical + * IMSG_MBOX_RESULT, so this is + * the only way session_handle_ + * mbox_result() can tell which + * tagged completion text/next + * state applies: CLOSE goes to + * SESSION_AUTHENTICATED, a real + * EXPUNGE goes back to SESSION_ + * SELECTED. */ + int literal_pending; /* 1 while the next bytes off + * the wire are a client + * literal's raw octets (RFC 9051 + * SS4.3), not a CRLF-delimited + * line -- set by cmd_append() + * when it finds a trailing + * "{n}"/"{n+}" on the APPEND + * command line, cleared once + * literal_buf_len reaches + * literal_len. Checked at the + * very top of session_dispatch_ + * client()'s read loop, *before* + * the normal CRLF search -- so + * unlike every async-imsg-wait + * state above, this needs no + * ST_* dispatch-table exclusion: + * while it's set, session_ + * handle_line() is never reached + * at all, regardless of s->state, + * so no new complete "tag SP + * command CRLF" can ever be + * mis-parsed out of literal + * bytes. */ + char *literal_buf; /* malloc(3)'d accumulator for + * the literal currently being + * read, literal_len bytes, freed + * once handed off to session_ + * finish_append() (or on early + * teardown -- see session_ + * teardown()) */ + uint64_t literal_len; /* total announced literal size + * (what "{n}" both this and + * literal_remaining start from) */ + uint64_t literal_remaining; /* bytes of the literal still + * needed -- literal_len minus + * however much has landed in + * literal_buf so far, across + * possibly many reads */ + char append_mailbox[MBOX_NAME_MAX]; /* APPEND's + * arguments, parsed by + * cmd_append() *before* the + * literal is read (mailbox/ + * flags/date all precede the + * literal in RFC 9051 SS6.3.12's + * grammar) and held here across + * the literal-read phase until + * session_finish_append() needs + * them to build IMSG_MBOX_ + * APPEND */ + uint32_t append_sysflags; + char append_keywords[MBOX_FLAGS_MAX]; + int append_has_date; + int64_t append_date; + enum session_state append_prev_state; /* SESSION_AUTHENTICATED or + * SESSION_SELECTED -- whichever + * s->state was when cmd_append() + * was first called (APPEND is + * valid in either, RFC 9051 + * command-auth), so session_ + * handle_mbox_appended() knows + * which one to restore once + * IMSG_MBOX_APPENDED arrives -- + * unlike FETCH/STORE/EXPUNGE, + * which are only ever dispatched + * from (and so only ever return + * to) SESSION_SELECTED, APPEND's + * "go back to" state isn't + * fixed. */ + uint32_t status_attrs; /* STATUS_ATT_* bitmask for the + * STATUS currently in flight + * (SESSION_STATUSING) -- which + * status-att values to include + * in the response, in listener.c's + * fixed canonical order; store.c + * always computes MESSAGES/ + * UIDNEXT/UIDVALIDITY/ + * HIGHESTMODSEQ regardless (see + * imapd.h's imsg_mbox_status + * comment), same "store always + * computes, listener decides + * whether to print" split as + * s->fetch_attrs. */ + char status_mailbox[MBOX_NAME_MAX]; /* the mailbox + * name STATUS was asked about + * (RFC 9051 SS6.3.4-SS6.3.6's + * flat multi-mailbox support -- + * before that, STATUS could only + * ever mean INBOX, so this field + * didn't need to exist) -- + * stashed here so session_ + * handle_mbox_status_result() can + * echo it back in the untagged + * "* STATUS (...)" + * response; store.c's own reply + * (struct imsg_mbox_status_ + * result) has no mailbox field of + * its own to read it back from, + * since listener.c already knows + * what it asked. */ + enum session_state status_prev_state; /* SESSION_AUTHENTICATED or + * SESSION_SELECTED -- same + * "STATUS is command-auth, valid + * in either state" reasoning as + * s->append_prev_state above, and + * for the identical reason: + * session_handle_mbox_status_ + * result() needs to know which + * one to restore. */ + enum session_state mbox_op_prev_state; /* SESSION_AUTHENTICATED or + * SESSION_SELECTED -- same + * "record whichever ST_AUTH + * state was current before, + * restore it after" pattern as + * s->status_prev_state just + * above, shared across all four + * of SESSION_CREATING/DELETING/ + * RENAMING/LISTING (see that enum + * block's own comment) since only + * one of the four can ever be in + * flight at once for a given + * session. session_finish_mbox_ + * op()/session_finish_list() both + * restore it. */ + char list_pattern[2 * MBOX_NAME_MAX]; /* canonical + * LIST/LSUB pattern (reference + * concatenated with the mailbox + * pattern by list_dispatch(), + * same construction the old + * synchronous 'canon' local + * used) stashed across the + * SESSION_LISTING round trip so + * session_handle_mbox_list_ + * item() can test each IMSG_ + * MBOX_LIST_ITEM name against it + * as it streams in, one at a + * time -- unlike s->search_ + * matches, nothing needs to be + * accumulated first, since each + * match can be turned into its + * untagged response + * immediately. */ + int list_is_lsub; /* 1 if the in-flight SESSION_ + * LISTING round trip was started + * by LSUB rather than LIST -- + * picks the untagged response + * keyword and the tagged + * completion text, same is_lsub + * parameter list_dispatch() + * already threads through + * synchronously today. */ + char rename_oldname[MBOX_NAME_MAX]; /* RENAME's + * two arguments, stashed by + * cmd_rename() across the + * SESSION_RENAMING round trip + * purely so session_finish_ + * mbox_op() can compare + * rename_oldname against s-> + * selected_mailbox on success -- + * if this session had the + * just-renamed mailbox itself + * selected, its own s->selected_ + * mailbox needs to follow along + * to rename_newname, the same + * way store.c's handle_mbox_ + * rename() already makes its own + * cwd/current_mailbox_dir follow + * (see that function's own real- + * hardware-bug comment). Neither + * name is otherwise needed here + * -- req.oldname/req.newname + * already made the trip to + * store.c in the imsg itself. */ + char rename_newname[MBOX_NAME_MAX]; + uint32_t search_return_opts; /* SEARCH_RETURN_* bitmask + * for the SEARCH currently in + * flight (SESSION_SEARCHING) -- + * which of MIN/MAX/ALL/COUNT + * session_finish_search() should + * include in the ESEARCH + * response; store.c never needs + * this, since it only computes + * *which* messages match, not + * how the client wants that + * reported. */ + uint32_t *search_matches; /* malloc(3)'d/realloc(3)'d, + * growable -- ascending sequence + * numbers streamed in via + * IMSG_MBOX_SEARCH_MATCH, kept + * in full (not range-compacted + * until formatting time) so + * MIN/MAX/COUNT/ALL can all be + * derived from one array. Freed + * by session_finish_search() once + * the ESEARCH response is sent, + * or by session_teardown() on + * early disconnect. */ + uint32_t search_nmatches; /* entries actually in + * search_matches */ + uint32_t search_matches_cap; /* allocated capacity of + * search_matches, >= search_ + * nmatches */ + int search_alloc_failed; /* 1 if a realloc(3) in + * session_handle_mbox_search_ + * match() ever failed for this + * SEARCH -- makes session_ + * finish_search() send a NO + * instead of a silently + * incomplete ESEARCH response; + * see that function's comment on + * why "some matches dropped" is + * treated as a hard failure, not + * a best-effort partial result. */ + int search_used_modseq; /* 1 if the SEARCH program + * currently in flight + * contained a MODSEQ + * search-key -- RFC 7162 + * SS3.1.5/SS3.1.8: makes + * this SEARCH a CONDSTORE- + * enabling command, and + * makes session_finish_ + * search() append "(MODSEQ + * n)" using the running + * max below. */ + uint64_t search_max_modseq; /* highest modseq seen + * across all matches of + * the SEARCH currently in + * flight, tracked by + * session_handle_mbox_ + * search_match() -- + * store.c always sends + * modseq per match (cheap, + * see imapd.h's struct + * imsg_mbox_search_match + * comment), so this is + * just a running max, not + * conditional on search_ + * used_modseq itself. */ + + /* + * RFC 7162 (CONDSTORE/QRESYNC) session state, added this pass. + * condstore_enabled/qresync_enabled are both sticky for the whole + * connection once set (SS3.1: "Once a client issues a CONDSTORE + * enabling command, it has announced itself... until the connection + * is closed"; SS3.2.3 says the same for QRESYNC) -- never cleared + * back to 0 anywhere in this file. qresync_enabled implies + * condstore_enabled is also 1 (SS3.2.3: "the presence of the + * 'QRESYNC' capability implies support for the CONDSTORE IMAP + * extension"), enforced at every place qresync_enabled gets set. + */ + int condstore_enabled; + int qresync_enabled; + + /* + * Best-known HIGHESTMODSEQ of the currently selected mailbox -- + * cached from every IMSG_MBOX_SELECTED/IMSG_MBOX_RESULT reply that + * carries one (store.c always populates it; see imapd.h's + * imsg_mbox_selected/imsg_mbox_result comments), regardless of + * whether this session is CONDSTORE-aware yet. Exists specifically + * for session_condstore_enable()'s "CONDSTORE enabling command + * issued while a mailbox is already selected" case (RFC 7162 SS3.1: + * the server MUST emit an unsolicited HIGHESTMODSEQ OK response at + * that moment) -- without a cached value there would be nothing to + * report without an extra store round trip just for this edge case. + */ + uint64_t mbox_highestmodseq; + + /* + * RFC 9051 SS6.3.3: whether the currently selected mailbox was + * opened via EXAMINE (1) rather than SELECT (0) -- meaningful only + * while s->state == SESSION_SELECTED (and, transiently, SESSION_ + * SELECTING/SESSION_EXPUNGING while a select_or_examine()/session_ + * request_expunge() round trip is in flight). Set synchronously by + * select_or_examine() before the IMSG_MBOX_SELECT round trip even + * starts -- like s->state = SESSION_SELECTING itself, this doesn't + * need to wait for store.c's reply, since it's derived purely from + * which command the client typed, not from anything store.c knows. + * Consulted by session_handle_mbox_selected() (README/OK response + * code, PERMANENTFLAGS) and by every command that would otherwise + * mutate the selected mailbox's permanent state -- store_do(), + * session_request_expunge(), copy_move_dispatch()'s is_move case -- + * to refuse (or, for CLOSE specifically, silently no-op) per SS6.3.3 + * ("No changes to the permanent state of the mailbox... are + * permitted") and SS6.4.1's CLOSE exception. + */ + int mbox_readonly; + char selected_mailbox[MBOX_NAME_MAX]; /* which + * mailbox SESSION_SELECTED actually + * refers to -- didn't need to exist + * before RFC 9051 SS6.3.4-SS6.3.6's flat + * multi-mailbox support (docs/openimap- + * storage-backend.md item 10), since + * "selected" and "INBOX is selected" + * were the same fact. Written + * optimistically by select_or_examine() + * at the same point s->state moves to + * SESSION_SELECTING -- safe even though + * the SELECT/EXAMINE might still fail, + * because every reader of this field + * also gates on s->state == + * SESSION_SELECTED first, and a failed + * SELECT/EXAMINE never reaches that + * state (RFC 9051 SS6.3.2: "the session + * is in authenticated state" on + * failure, deselected same as before + * the attempt). Consulted so far by + * session_handle_mbox_appended()'s own + * "is the mailbox APPEND just targeted + * the one actually selected right now" + * check -- see that function's + * comment. */ + + /* + * RFC 9051 SS6.3.13 (IDLE) v1 scope: EXISTS and EXPUNGE only (see + * imapd.h's imsg_mbox_idle_uid comment for the sourcing/reasoning + * behind not also pushing unsolicited FETCH). idle_known_uids is this + * session's own cached, ordered snapshot of "which UIDs existed as of + * the last time this session's view was authoritative" -- populated + * from an IMSG_MBOX_IDLE_REFRESH round trip, either the baseline one + * cmd_idle() triggers right after sending "+ idling" (idle_baseline_ + * valid is 0 until that first one lands, so session_handle_idle_ + * refreshed() knows not to diff against garbage), or a later one + * triggered by session_notify_idle_peers() when some *other* session + * belonging to the same uid successfully mutates the mailbox. + * Diffing this array against a freshly streamed IMSG_MBOX_IDLE_UID + * list is what lets session_handle_idle_refreshed() compute correct + * seqno-based "* N EXPUNGE" lines for whatever UIDs dropped out -- + * same "position in the shrinking list, removed in ascending order" + * algorithm handle_mbox_expunge() already needs on the store side, + * just recomputed here in listener.c from two UID arrays instead of + * from the index file directly. + * + * idle_refresh_pending/idle_refresh_again exist because a refresh is + * itself an async imsg round trip: if a second trigger arrives while + * one is already in flight for this session (e.g. two sibling + * sessions both mutate in quick succession), there's no way to ask + * store.c to hurry up or to safely start a second overlapping + * request on the same store_iev channel -- idle_refresh_again just + * remembers to immediately re-request once the in-flight one's + * IMSG_MBOX_IDLE_REFRESHED reply lands, rather than either blocking + * or silently dropping the second trigger. + */ + uint32_t *idle_known_uids; + uint32_t idle_known_nuids; + uint32_t idle_known_cap; + int idle_baseline_valid; + int idle_refresh_pending; + int idle_refresh_again; + + /* + * Accumulates the freshly streamed IMSG_MBOX_IDLE_UID list for an + * in-flight refresh -- deliberately a *separate* array from idle_ + * known_uids above, not built in place over it, since session_ + * handle_idle_refreshed() needs both the old and the new list at + * once to diff them. Same "held back until the terminal reply" + * shape as s->vanished_ranges/s->qresync_fetches during a QRESYNC + * SELECT resync. + */ + uint32_t *idle_incoming_uids; + uint32_t idle_incoming_n; + uint32_t idle_incoming_cap; + + /* + * Accumulates VANISHED (EARLIER) ranges streamed in via zero or more + * IMSG_MBOX_SELECT_VANISHED during an in-flight QRESYNC SELECT + * resync (s->state == SESSION_SELECTING) -- store.c already reports + * these pre-compacted into ranges (see imsg_mbox_select_vanished's + * comment in imapd.h), so this array just collects them in + * arrival order (already ascending -- store.c streams its single + * forward pass in increasing-UID order) for session_handle_mbox_ + * selected() to join into one combined "* VANISHED (EARLIER) ..." + * line once IMSG_MBOX_SELECTED arrives -- RFC 7162 SS3.2.6 requires + * every VANISHED (EARLIER) response to precede any FETCH response + * in the same resync, which is only guaranteed by holding all of it + * (and the FETCH data below) until the terminal reply, not by + * relaying each piece to the client as it streams in. + */ + struct vanished_range *vanished_ranges; + uint32_t vanished_nranges; + uint32_t vanished_cap; + + /* + * Accumulates the QRESYNC resync's per-message FETCH-with-UID-and- + * MODSEQ data, streamed in via IMSG_MBOX_FETCH_META while s->state + * == SESSION_SELECTING (see session_store_dispatch()'s IMSG_MBOX_ + * FETCH_META case) -- held back for the same "VANISHED must precede + * FETCH" ordering reason vanished_ranges is. Holds full struct + * imsg_mbox_fetch_meta copies (not just seqno/uid, unlike search_ + * matches) because session_send_qresync_fetch_response() needs + * FLAGS too -- heavier per-entry than search_matches, but v1-scale + * resyncs are small (this project's usual "personal use, modest + * mailbox size" tolerance). + */ + struct imsg_mbox_fetch_meta *qresync_fetches; + uint32_t qresync_nfetches; + uint32_t qresync_fetches_cap; + + /* + * COPY/MOVE (RFC 9051 SS6.4.7/SS6.4.8), SESSION_COPYING. Two parallel + * growable arrays accumulating each IMSG_MBOX_COPY_MAPPING streamed + * in during the round trip -- src_uid/dest_uid pairs, already + * ascending (store.c streams its single forward pass in increasing + * order, same guarantee vanished_ranges above relies on), range- + * compacted independently via format_seq_list() into COPYUID's two + * UID sets once the terminal reply arrives. cmd_is_move distinguishes + * which of COPY/MOVE is in flight (same session-state-plus-flag + * pattern as s->close_after_expunge for EXPUNGE/CLOSE) -- both share + * SESSION_COPYING since their in-flight bookkeeping is otherwise + * identical, differing only in the terminal formatting (see session_ + * finish_copy_or_move()). move_expunged buffers each IMSG_MBOX_ + * EXPUNGED that arrives during a MOVE specifically (rather than + * writing it to the client immediately, the way a real EXPUNGE/CLOSE + * does) -- SS6.4.8 requires COPYUID to precede any EXPUNGE/VANISHED + * for the same operation, which (like vanished_ranges/qresync_fetches + * above) is only guaranteed by holding everything until the terminal + * reply, not by relaying each piece as it streams in. + */ + uint32_t *copy_src_uids; + uint32_t *copy_dest_uids; + uint32_t copy_n; + uint32_t copy_cap; + int copy_alloc_failed; /* same "don't silently drop + * a mapping" reasoning as + * s->search_alloc_failed */ + int cmd_is_move; + struct imsg_mbox_expunged *move_expunged; + uint32_t move_expunged_n; + uint32_t move_expunged_cap; + + /* + * Accumulates messages that failed a STORE's UNCHANGEDSINCE test, + * streamed in via zero or more IMSG_MBOX_STORE_MODIFIED while + * s->state == SESSION_STORING, for session_handle_mbox_result() to + * range-compact (format_seq_list(), same helper SEARCH/ESEARCH + * already use) into the tagged response's MODIFIED response code + * (RFC 7162 SS3.1.3). Holds sequence numbers for a plain STORE, UIDs + * for a UID STORE (SS3.1.3: "the message number (or unique + * identifier in the case of the UID STORE command)") -- a single + * array, not a parallel uid array, since exactly one of the two + * values is ever meaningful for a given STORE; see session_handle_ + * store_modified()'s use of s->cmd_by_uid to pick which. + */ + uint32_t *store_modified; + uint32_t store_modified_n; + uint32_t store_modified_cap; + + /* + * RFC 9051 SS6.4.9 (UID command), added this pass: 1 if the async + * operation currently in flight (SESSION_FETCHING/STORING/SEARCHING/ + * EXPUNGING) was dispatched via UID rather than the bare + * command. Every entry point that starts one of those four states + * sets this explicitly (fetch_dispatch()/store_do()/search_dispatch()/ + * session_request_expunge()), so a stale value from a previous, + * different-mode command can never leak into the next one -- there is + * no "clear it back to 0" step anywhere else, matching this struct's + * existing condstore_enabled/qresync_enabled precedent of "every + * setter is unconditional, so no separate reset is needed." + * + * Consulted by: session_send_store_fetch_response() (UID STORE's + * FETCH echo must include UID, SS6.4.9's "MUST implicitly include + * the UID message data item"), session_handle_mbox_search_match()/ + * session_finish_search() (UID SEARCH reports UIDs instead of + * sequence numbers, plus the "UID" ESEARCH correlator token), + * session_handle_store_modified() (UID STORE's MODIFIED response + * code lists UIDs, not sequence numbers -- RFC 7162 SS3.1.3: "the + * message number (or unique identifier in the case of the UID STORE + * command)"), and session_handle_mbox_result() (every one of the four + * commands' tagged completion text becomes "UID completed" -- + * RFC 9051's own SS6.4.9 example shows "UID FETCH completed" + * verbatim; UID EXPUNGE's own worked example shows "UID EXPUNGE + * completed"; UID STORE/UID SEARCH aren't shown as literal worked + * examples but follow the same "UID completed" pattern by + * direct symmetry with those two, not fabricated). Plain FETCH + * doesn't need to consult this at all -- forcing MBOX_FETCH_UID into + * s->fetch_attrs at dispatch time (see fetch_dispatch()) already + * makes session_send_fetch_response()'s existing attrs-driven UID + * printing do the right thing with no further changes there. + */ + int cmd_by_uid; + + TAILQ_ENTRY(session) entry; +}; + +static TAILQ_HEAD(, session) sessions = TAILQ_HEAD_INITIALIZER(sessions); +static uint32_t next_session_id = 1; + +/* see this file's header comment for what each of these actually is */ +static struct imsgev iev_auth; +static struct imsgev iev_parent; +/* + * Dual-stack ("listen on *") support: up to LISTENER_MAX_ADDRS bound + * sockets per port purpose now, one per resolved address family, instead + * of exactly one -- see imapd.h's LISTENER_MAX_ADDRS comment for why + * OpenBSD needs two separate sockets rather than one dual-mapped one. + * n_cleartext_fd/n_tls_fd (set once, at the end of the boot-time drain + * loop below) say how many of each array's slots are actually live. + */ +static int cleartext_fd[LISTENER_MAX_ADDRS] = { -1, -1 }; +static int tls_fd[LISTENER_MAX_ADDRS] = { -1, -1 }; +static int n_cleartext_fd, n_tls_fd; +static struct event ev_accept_cleartext[LISTENER_MAX_ADDRS]; +static struct event ev_accept_tls[LISTENER_MAX_ADDRS]; + +/* + * The server-wide TLS context, built once at boot in listener_main() + * from the cert/key bytes parent sends over IMSG_TLS_CERT/IMSG_TLS_KEY. + * tls_server()+tls_configure(), sourced directly against httpd's + * server_tls_init() (openbsd_source/src/usr.sbin/httpd/server.c) and + * src/lib/libtls/tls.h. NULL if TLS setup failed (bad cert/key, etc.) -- + * checked before every tls_accept_socket() call rather than treated as + * fatal, so a cert problem degrades to "no TLS" (implicit-TLS port + * closes, STARTTLS/AUTHENTICATE keep replying NO) instead of taking the + * whole daemon down; cleartext CAPABILITY/NOOP/LOGOUT/ID still work. + * + * Also rebuilt post-boot, on a live SIGHUP reload -- see listener_reload_ + * tls() below and parent.c's sighup_handler(). tls_accept_socket() + * (man.openbsd.org/tls_accept_socket.3: "these functions create a new + * context... and return it in *cctx") gives each already-connected + * session's own struct tls * (s->tls_ctx), entirely independent of this + * server-wide pair, so swapping these two pointers here on reload has no + * effect on sessions already mid-handshake or established -- only new + * tls_accept_socket() calls from this point on see the new cert/key. + */ +static struct tls_config *listener_tls_config; +static struct tls *listener_tls_ctx; + +/* Matches parent.c's send_tls_certs() read buffer size -- see that + * function's comment for the (flagged, unsolved) chain-size limitation + * this shares. Moved up from just before listener_main() (its original, + * boot-only location) since the SIGHUP reload staging buffers just below + * need these two constants as well now. */ +#define TLS_CERT_MAX 8192 +#define TLS_KEY_MAX 8192 + +/* + * Staging area for a SIGHUP reload's IMSG_TLS_CERT/IMSG_TLS_KEY bytes, + * received post-boot on the same parent channel (iev_parent) as the + * ongoing IMSG_STORE_FORK/IMSG_SETUP_PEER traffic listener_dispatch_ + * parent() already handles. Parent sends these back to back (send_tls_ + * certs(), unchanged since boot) but each arrives as its own imsg, and + * nothing forces both to land in the same event-loop dispatch call -- + * these persist across calls the same way got_cert/got_key are local to + * listener_main()'s one-shot boot drain loop, except here as file-scope + * statics since the reload path runs repeatedly, from the event loop. + * listener_reload_tls() is only invoked once both have arrived; then both + * flags reset so a later reload starts clean. + */ +static char reload_cert_buf[TLS_CERT_MAX], reload_key_buf[TLS_KEY_MAX]; +static size_t reload_cert_len, reload_key_len; +static int reload_got_cert, reload_got_key; + +static void listener_accept(int, short, void *); +static void listener_dispatch_auth(int, short, void *); +static void listener_dispatch_parent(int, short, void *); +static void listener_reload_tls(const char *, size_t, const char *, + size_t); +static void session_dispatch_client(int, short, void *); +static void session_store_dispatch(int, short, void *); +static void session_handle_mbox_selected(struct session *, + struct imsg_mbox_selected *); +static void session_handle_mbox_status_result(struct session *, + struct imsg_mbox_status_result *); +static void session_send_fetch_response(struct session *, + struct imsg_mbox_fetch_meta *); +static void session_send_store_fetch_response(struct session *, + struct imsg_mbox_fetch_meta *); +static void session_send_expunge_response(struct session *, + struct imsg_mbox_expunged *); +static void session_handle_mbox_result(struct session *, + struct imsg_mbox_result *); +static int session_request_expunge(struct session *, const char *, + int, int, uint32_t, uint32_t, int, int); +static int session_finish_append(struct session *); +static void session_handle_mbox_appended(struct session *, + struct imsg_mbox_appended *); +static void format_internaldate(int64_t, char *, size_t); +static int parse_nz_number(const char *, uint32_t *); +static int parse_seq_range(const char *, uint32_t *, uint32_t *, + int *, int *); +static int parse_fetch_atts(char *, uint32_t *, int *, int *, char *, + size_t, char *, size_t, int *, char *, size_t, int *, + uint32_t *, uint32_t *, const char **); +static int parse_header_fields_att(const char *, int *, char *, size_t); +static int section_part_valid(const char *); +static int parse_partial_suffix(const char *, int *, uint32_t *, + uint32_t *); +static int parse_store_flags(char *, uint32_t *, char *, size_t, + const char **); +static int parse_date_time(const char *, int64_t *); +struct append_parsed; +static int parse_append_args(char *, struct append_parsed *, + const char **); +static int parse_search_date(const char *, int64_t *); +struct search_parse_ctx; +static int parse_search_key(char **, struct search_parse_ctx *, + const char **); +static int parse_search_key_inner(char **, struct search_parse_ctx *, + const char **); +static int parse_search_key_list(char **, struct search_parse_ctx *, + const char **, int); +static int parse_search_return_opts(char **, uint32_t *, const char **); +static void session_handle_mbox_search_match(struct session *, + struct imsg_mbox_search_match *); +static void session_finish_search(struct session *, + struct imsg_mbox_result *); +static void session_request_store(struct session *, + struct imsg_auth_result *); +static void session_send_greeting(struct session *); +static void session_teardown(struct session *); +static struct session *session_find(uint32_t); + +/* RFC 7162 (CONDSTORE/QRESYNC) helpers, added this pass. */ +static void session_condstore_enable(struct session *); +static size_t format_seq_list(char *, size_t, const uint32_t *, uint32_t, + int *); +static size_t format_range_list(char *, size_t, + const struct vanished_range *, uint32_t, int *); +static void session_send_qresync_fetch_response(struct session *, + struct imsg_mbox_fetch_meta *); +static void session_handle_select_vanished(struct session *, + struct imsg_mbox_select_vanished *); +static void session_handle_select_fetch(struct session *, + struct imsg_mbox_fetch_meta *); +static void session_handle_store_modified(struct session *, + struct imsg_mbox_store_modified *); +static void session_handle_idle_uid(struct session *, + struct imsg_mbox_idle_uid *); +static void session_handle_idle_refreshed(struct session *, + struct imsg_mbox_idle_refreshed *); +static void session_request_idle_refresh(struct session *); +static void session_push_idle_expunges(struct session *, + const uint32_t *, uint32_t, const uint32_t *, uint32_t); +static void session_notify_idle_peers(struct session *); +static int session_handle_idle_continuation(struct session *, char *); +static int parse_select_params(char *, struct imsg_mbox_select *, + struct session *, int *, const char **); +static int parse_qresync_group(char *, struct imsg_mbox_select *, + const char **); +static char *split_trailing_modifiers(char *); +static int parse_fetch_modifiers(char *, struct imsg_mbox_fetch *, + struct session *, int, int *, const char **); +static int parse_store_modifiers(char *, struct imsg_mbox_store *, + const char **); + +/* RFC 9051 SS6.4.9 (UID command) helpers, added this pass. */ +static int fetch_dispatch(struct session *, const char *, char *, int); +static int store_do(struct session *, const char *, char *, int); +static int search_dispatch(struct session *, const char *, char *, int); +static int uid_expunge_dispatch(struct session *, const char *, char *); +static void session_handle_fetch_vanished(struct session *, + struct imsg_mbox_select_vanished *); + +/* RFC 9051 SS6.4.7/SS6.4.8 (COPY/MOVE) helpers, added this pass. */ +static int copy_move_dispatch(struct session *, const char *, char *, + int, int); +static int mailbox_name_valid(const char *); +static int mailbox_name_is_inbox(const char *); +static int parse_list_token(char **, char *, size_t, const char **); +static void session_finish_mbox_op(struct session *, + struct imsg_mbox_result *); +static void session_finish_list(struct session *, + struct imsg_mbox_result *); +static void session_handle_mbox_list_item(struct session *, + struct imsg_mbox_list_item *); +static void session_handle_mbox_copy_mapping(struct session *, + struct imsg_mbox_copy_mapping *); +static void session_finish_copy_or_move(struct session *, + struct imsg_mbox_result *); + +static void session_tls_start(struct session *); +static void session_tls_handshake(int, short, void *); +static void session_arm_client_read(struct session *); + +static void session_write(struct session *, const char *, size_t); +static void session_reply(struct session *, const char *, const char *, + const char *); +static void session_untagged(struct session *, const char *); +static int parse_command_line(char *, char **, char **, char **); +static int session_handle_line(struct session *, char *); +static int session_handle_auth_continuation(struct session *, char *); +static int sasl_plain_finish(struct session *, const char *, + const char *, int); + +static int cmd_capability(struct session *, const char *, char *); +static int cmd_noop(struct session *, const char *, char *); +static int cmd_logout(struct session *, const char *, char *); +static int cmd_id(struct session *, const char *, char *); +static int cmd_login(struct session *, const char *, char *); +static int cmd_starttls(struct session *, const char *, char *); +static int cmd_authenticate(struct session *, const char *, char *); + +/* command-auth (RFC 9051 SS6.3 -- valid in Authenticated or Selected + * state): ENABLE is real (see cmd_enable()'s comment for why that's a + * complete implementation, not a stub, for v1). Everything else here + * needs a mailbox layer that doesn't exist yet -- store.c's IMSG_MBOX_* + * family is entirely unimplemented (see store.c's own header comment), + * and its wire payload shapes aren't designed, so these all reply a + * generic NO via stub_not_implemented() rather than doing anything real. + * They're in the dispatch table anyway, correctly gated by session + * state, because "recognized command, can't do it right now" (NO) and + * "not a real IMAP command at all" (BAD, session_handle_line()'s unknown- + * command fallback) are different, both spec-meaningful responses -- see + * openimap-v1-dispatch.md's dispatch table for the same distinction. */ +static int stub_not_implemented(struct session *, const char *, + const char *); +static int cmd_enable(struct session *, const char *, char *); +static int cmd_select(struct session *, const char *, char *); +static int cmd_examine(struct session *, const char *, char *); +static int cmd_create(struct session *, const char *, char *); +static int cmd_delete(struct session *, const char *, char *); +static int cmd_rename(struct session *, const char *, char *); +static int cmd_subscribe(struct session *, const char *, char *); +static int cmd_unsubscribe(struct session *, const char *, char *); +static int cmd_list(struct session *, const char *, char *); +static int cmd_lsub(struct session *, const char *, char *); +static int cmd_namespace(struct session *, const char *, char *); +static int cmd_status(struct session *, const char *, char *); +static int cmd_append(struct session *, const char *, char *); +static int cmd_idle(struct session *, const char *, char *); + +/* command-select (RFC 9051 SS6.4 -- valid only in Selected state). + * SESSION_SELECTED is currently unreachable (SELECT itself is a stub + * above), so none of these can be dispatched to yet in practice -- listed + * for the same "correctly gated, honestly stubbed" reason as command-auth + * above. */ +static int cmd_close(struct session *, const char *, char *); +static int cmd_unselect(struct session *, const char *, char *); +static int cmd_expunge(struct session *, const char *, char *); +static int cmd_search(struct session *, const char *, char *); +static int cmd_fetch(struct session *, const char *, char *); +static int cmd_store_cmd(struct session *, const char *, char *); +static int cmd_copy(struct session *, const char *, char *); +static int cmd_move(struct session *, const char *, char *); +static int cmd_uid(struct session *, const char *, char *); + +/* + * v1 CAPABILITY strings, originally quoted verbatim from openimap-v1- + * dispatch.md's "v1 CAPABILITY strings" section (itself sourced against + * RFC 9051 SS6.1.1 and the IANA IMAP Capabilities Registry). Selected via + * session->tls_active, which a real STARTTLS/implicit-TLS handshake now + * sets -- see cmd_starttls() and session_tls_handshake(). + * + * CONDSTORE and QRESYNC added this pass (RFC 7162). Both listed in both + * strings, not gated on tls_active/auth state the way AUTH=PLAIN is -- + * SS3.1.1/SS3.2.2 define support for each purely in terms of whether the + * server returns the token in CAPABILITY at all, with no mention of a TLS + * or authentication precondition (unlike AUTH=PLAIN, which this server + * deliberately withholds pre-TLS to keep LOGINDISABLED honest). Listing + * QRESYNC implies CONDSTORE support too (SS3.2.3), but SS3.2.2 separately + * "SHOULD"s advertising CONDSTORE explicitly as well for CONDSTORE-only + * clients, so both tokens are listed rather than relying on the implication + * alone. + * + * IDLE (added this pass, RFC 9051 SS6.3.13) needs no new token in either + * string: unlike RFC 2177 (where IDLE was an extension gated on an "IDLE" + * capability token), SS6.3.13 redefines IDLE natively in IMAP4rev2 with no + * capability-gating language at all -- it's base spec, covered by the + * existing "IMAP4rev2" token already in both strings. + */ +#define CAPABILITY_PRE_TLS "IMAP4rev2 STARTTLS LOGINDISABLED ID CONDSTORE QRESYNC" +#define CAPABILITY_POST_TLS "IMAP4rev2 AUTH=PLAIN ID CONDSTORE QRESYNC" + +/* + * v1 command dispatch table. "any state" (RFC 9051 command-any, plus RFC + * 2971's ID, which command-any is explicitly extended to include) and + * "not authenticated state" (command-nonauth) are fully implemented. + * command-auth and command-select now have real table entries too -- + * ENABLE is a genuine, complete implementation (see cmd_enable()); every + * other command-auth/command-select entry is an honest stub + * (stub_not_implemented()) pending store.c's still-undesigned IMSG_MBOX_* + * wire protocol. See session_handle_line()'s "unknown command" fallback + * for anything not listed here at all. + */ +struct imap_cmd_entry { + const char *name; + unsigned int states; /* bitmask of 1U << SESSION_* */ + int (*handler)(struct session *, const char *, char *); +}; + +#define ST_ANY \ + ((1U << SESSION_NOT_AUTH) | (1U << SESSION_AUTHENTICATING) | \ + (1U << SESSION_STORE_PENDING) | (1U << SESSION_AUTHENTICATED) | \ + (1U << SESSION_SELECTING) | (1U << SESSION_SELECTED) | \ + (1U << SESSION_FETCHING) | (1U << SESSION_STORING) | \ + (1U << SESSION_EXPUNGING) | (1U << SESSION_APPENDING) | \ + (1U << SESSION_SEARCHING) | (1U << SESSION_STATUSING) | \ + (1U << SESSION_COPYING) | (1U << SESSION_CREATING) | \ + (1U << SESSION_DELETING) | (1U << SESSION_RENAMING) | \ + (1U << SESSION_LISTING)) +#define ST_NOTAUTH (1U << SESSION_NOT_AUTH) + +/* + * RFC 9051 SS9's `command-auth`/`command-select` ABNF productions, whose + * inline comments are quoted directly: command-auth is "Valid only in + * Authenticated or Selected state"; command-select is "Valid only when + * in Selected state". SESSION_AUTHENTICATING/SESSION_STORE_PENDING (the + * async window between a successful IMSG_AUTH_RESULT and the client's + * tagged OK -- see sasl_plain_finish()/session_request_store()) are + * deliberately excluded from ST_AUTH: the client hasn't been told + * AUTHENTICATE succeeded yet, so from its point of view it's still in the + * Not Authenticated state and shouldn't be able to pipeline a command-auth + * command into that window. SESSION_SELECTING (the same kind of async + * window, this time between IMSG_MBOX_SELECT and IMSG_MBOX_SELECTED -- see + * select_or_examine()/session_store_dispatch()) is excluded for the identical + * reason: the client hasn't been told SELECT succeeded or failed yet. + * SESSION_FETCHING (IMSG_MBOX_FETCH sent, awaiting the IMSG_MBOX_FETCH_META + * stream + terminal IMSG_MBOX_RESULT) is excluded from ST_SELECTED for a + * related but distinct reason: s->pending_tag and s->fetch_attrs belong to + * the in-flight FETCH until that terminal reply arrives, so a second + * command-select command dispatched into that window would clobber both + * before session_handle_mbox_result() gets to use them. SESSION_STORING + * (IMSG_MBOX_STORE sent) is excluded for the identical reason -- it shares + * the same s->pending_tag and the same IMSG_MBOX_FETCH_META/IMSG_MBOX_ + * RESULT reply shape as SESSION_FETCHING (see cmd_store_cmd()'s comment). + * SESSION_EXPUNGING (IMSG_MBOX_EXPUNGE sent, by either EXPUNGE or CLOSE) + * is excluded for the same reason again -- s->pending_tag and s->close_ + * after_expunge belong to that in-flight request until IMSG_MBOX_RESULT + * arrives. SESSION_APPENDING (IMSG_MBOX_APPEND sent, once the client's + * literal has been fully read) is excluded for the same reason -- s-> + * pending_tag and s->append_prev_state belong to that in-flight request + * until IMSG_MBOX_APPENDED arrives; note this is a *different* window + * than s->literal_pending (the client-literal-read phase that precedes + * it), which is intercepted at the raw-byte level in session_dispatch_ + * client() before command dispatch is ever reached at all, and so needs + * no ST_* exclusion of its own -- see that field's comment. SESSION_ + * SEARCHING (IMSG_MBOX_SEARCH sent) is excluded for the same reason + * again -- s->pending_tag, s->search_return_opts, and s->search_matches + * (accumulating across the IMSG_MBOX_SEARCH_MATCH stream) all belong to + * the in-flight SEARCH until IMSG_MBOX_RESULT arrives. SESSION_STATUSING + * (IMSG_MBOX_STATUS sent) is excluded from ST_AUTH for the same reason + * again -- s->pending_tag, s->status_attrs, and s->status_prev_state + * belong to the in-flight STATUS until IMSG_MBOX_STATUS_RESULT arrives; + * note STATUS itself is dispatched *from* ST_AUTH (it's valid in either + * Authenticated or Selected state, RFC 9051 SS6.3.11), so this exclusion + * only matters for a second command arriving while one STATUS is still + * outstanding, exactly like SELECTING/SEARCHING/etc. above. SESSION_ + * COPYING (IMSG_MBOX_COPY or IMSG_MBOX_MOVE sent) is excluded from + * ST_SELECTED for the same reason as SESSION_SEARCHING -- s->pending_tag, + * s->copy_src_uids/copy_dest_uids, s->cmd_is_move, and (for a MOVE) + * s->move_expunged all belong to the in-flight COPY/MOVE until IMSG_MBOX_ + * RESULT arrives. + */ +#define ST_AUTH \ + ((1U << SESSION_AUTHENTICATED) | (1U << SESSION_SELECTED)) +#define ST_SELECTED (1U << SESSION_SELECTED) + +static const struct imap_cmd_entry imap_cmds[] = { + { "CAPABILITY", ST_ANY, cmd_capability }, + { "NOOP", ST_ANY, cmd_noop }, + { "LOGOUT", ST_ANY, cmd_logout }, + { "ID", ST_ANY, cmd_id }, + { "LOGIN", ST_NOTAUTH, cmd_login }, + { "STARTTLS", ST_NOTAUTH, cmd_starttls }, + { "AUTHENTICATE", ST_NOTAUTH, cmd_authenticate }, + + /* command-auth, RFC 9051 SS6.3 order */ + { "ENABLE", ST_AUTH, cmd_enable }, + { "SELECT", ST_AUTH, cmd_select }, + { "EXAMINE", ST_AUTH, cmd_examine }, + { "CREATE", ST_AUTH, cmd_create }, + { "DELETE", ST_AUTH, cmd_delete }, + { "RENAME", ST_AUTH, cmd_rename }, + { "SUBSCRIBE", ST_AUTH, cmd_subscribe }, + { "UNSUBSCRIBE", ST_AUTH, cmd_unsubscribe }, + { "LIST", ST_AUTH, cmd_list }, + { "LSUB", ST_AUTH, cmd_lsub }, + { "NAMESPACE", ST_AUTH, cmd_namespace }, + { "STATUS", ST_AUTH, cmd_status }, + { "APPEND", ST_AUTH, cmd_append }, + { "IDLE", ST_AUTH, cmd_idle }, + + /* command-select, RFC 9051 SS6.4 order */ + { "CLOSE", ST_SELECTED, cmd_close }, + { "UNSELECT", ST_SELECTED, cmd_unselect }, + { "EXPUNGE", ST_SELECTED, cmd_expunge }, + { "SEARCH", ST_SELECTED, cmd_search }, + { "FETCH", ST_SELECTED, cmd_fetch }, + { "STORE", ST_SELECTED, cmd_store_cmd }, + { "COPY", ST_SELECTED, cmd_copy }, + { "MOVE", ST_SELECTED, cmd_move }, + { "UID", ST_SELECTED, cmd_uid }, +}; +#define NUM_IMAP_CMDS (sizeof(imap_cmds) / sizeof(imap_cmds[0])) + +__dead void +listener_main(void) +{ + struct imsgbuf ibuf3; + struct passwd *pw; + int peer_fd; + struct imsg imsg; + struct imsg_listener_init init; + ssize_t n; + char cert_buf[TLS_CERT_MAX], key_buf[TLS_KEY_MAX]; + size_t cert_len = 0, key_len = 0; + int got_cert = 0, got_key = 0, got_init = 0; + int recv_cleartext = 0, recv_tls = 0, i; + + memset(&init, 0, sizeof(init)); + + if (imsgbuf_init(&ibuf3, 3) == -1) + fatal("imsgbuf_init"); + imsgbuf_allow_fdpass(&ibuf3); /* this channel receives fd-passed + * IMSG_LISTENER_SOCKET_CLEARTEXT/_TLS + * and IMSG_SETUP_PEER messages below + * -- see imsgev.c's + * imsgev_init() comment. */ + + /* boot-time handshake: one peer (auth), then SETUP_DONE+ack. */ + peer_fd = setup_recv_one_peer(&ibuf3); + setup_recv_done_and_ack(&ibuf3); + + /* + * IMSG_LISTENER_SOCKET_CLEARTEXT/_TLS (one each normally, two each + * for "listen on *" -- see imapd.h's LISTENER_MAX_ADDRS comment), + * IMSG_TLS_CERT, IMSG_TLS_KEY, and IMSG_LISTENER_INIT still arrive + * on fd 3 from parent, sent right after the handshake above (see + * parent.c's parent_main()) -- read them synchronously here, before + * entering the event loop, since we need the socket fds to exist + * before we can event_set() an accept handler on them. ibuf3 itself + * (not a fresh imsgbuf) is later handed to imsgev_init_from_ibuf() + * below -- see this file's header comment for why a second + * imsgbuf_init() on fd 3 would be wrong. + * + * How many socket messages to expect isn't known until + * IMSG_LISTENER_INIT itself arrives (it carries n_cleartext_addrs/ + * n_tls_addrs), so the loop below only checks recv_cleartext/ + * recv_tls against those counts once got_init is true -- the + * short-circuit "!got_init ||" keeps the loop going regardless of + * what those still-zeroed counts would otherwise say. Every message + * type here can arrive in any order relative to the others (parent + * fires them off back to back with no synchronization forcing a + * read in between), including sockets before init. + */ + /* + * imsg_get() before imsgbuf_read(), same reasoning throughout this + * loop as imsgev.c's setup_recv_one_peer() header comment: the + * SETUP_DONE-ack exchange immediately preceding this loop + * (setup_recv_done_and_ack(), just above) can leave more than one + * of these five messages already sitting fully buffered in ibuf3 + * -- parent fires them off back to back with no synchronization + * forcing a read in between -- so checking imsg_get() first avoids + * ever issuing a real blocking recvmsg() for bytes that already + * arrived. Restructured from the previous "outer imsgbuf_read(), + * inner drain-while-imsg_get()" shape into one flat loop that + * always tries imsg_get() first, since that outer/inner split had + * the exact same bug on its very first iteration. + */ + while (!got_init || !got_cert || !got_key || + recv_cleartext < init.n_cleartext_addrs || + recv_tls < init.n_tls_addrs) { + if ((n = imsg_get(&ibuf3, &imsg)) == -1) + fatal("imsg_get"); + if (n == 0) { + if ((n = imsgbuf_read(&ibuf3)) == -1) + fatal("imsgbuf_read"); + if (n == 0) + fatalx("listener: parent closed channel " + "during boot"); + continue; + } + switch (imsg_get_type(&imsg)) { + case IMSG_LISTENER_SOCKET_CLEARTEXT: + if (recv_cleartext >= LISTENER_MAX_ADDRS) + fatalx("listener: too many cleartext " + "listener sockets (max %d)", + LISTENER_MAX_ADDRS); + cleartext_fd[recv_cleartext++] = imsg_get_fd(&imsg); + break; + case IMSG_LISTENER_SOCKET_TLS: + if (recv_tls >= LISTENER_MAX_ADDRS) + fatalx("listener: too many tls listener " + "sockets (max %d)", LISTENER_MAX_ADDRS); + tls_fd[recv_tls++] = imsg_get_fd(&imsg); + break; + case IMSG_TLS_CERT: + cert_len = imsg_get_len(&imsg); + if (cert_len > sizeof(cert_buf)) { + log_warnx("listener: TLS cert too " + "large (%zu > %zu)", cert_len, + sizeof(cert_buf)); + cert_len = 0; + } else if (imsg_get_data(&imsg, cert_buf, + cert_len) == -1) { + log_warnx("bad IMSG_TLS_CERT"); + cert_len = 0; + } + got_cert = 1; + break; + case IMSG_TLS_KEY: + key_len = imsg_get_len(&imsg); + if (key_len > sizeof(key_buf)) { + log_warnx("listener: TLS key too " + "large (%zu > %zu)", key_len, + sizeof(key_buf)); + key_len = 0; + } else if (imsg_get_data(&imsg, key_buf, + key_len) == -1) { + log_warnx("bad IMSG_TLS_KEY"); + key_len = 0; + } + got_key = 1; + break; + case IMSG_LISTENER_INIT: + /* see imapd.h's imsg_listener_init comment -- + * n_cleartext_addrs/n_tls_addrs are load-bearing + * now (this loop's own exit condition reads + * them), not just a log line. The listening + * sockets themselves are never rebuilt from + * listen_addr here; that already happened in + * parent before this process even existed. */ + if (imsg_get_data(&imsg, &init, sizeof(init)) + == -1) { + log_warnx("bad IMSG_LISTENER_INIT"); + break; + } + got_init = 1; + break; + default: + log_debug("listener boot: unhandled %d", + imsg_get_type(&imsg)); + break; + } + imsg_free(&imsg); + } + n_cleartext_fd = recv_cleartext; + n_tls_fd = recv_tls; + + log_info("listening on %s:%u (cleartext, %d socket%s) and " + "%s:%u (implicit TLS, %d socket%s)", + init.listen_addr, init.port_cleartext, n_cleartext_fd, + n_cleartext_fd == 1 ? "" : "s", + init.listen_addr, init.port_implicit_tls, n_tls_fd, + n_tls_fd == 1 ? "" : "s"); + + if ((pw = getpwnam("_imapd")) == NULL) + fatalx("getpwnam _imapd: no such user " + "(expected, not yet provisioned by an install script)"); + + if (chroot("/var/empty") == -1) + fatal("chroot /var/empty"); + if (chdir("/") == -1) + fatal("chdir /"); + + /* + * listener's own privilege drop -- previously flagged as entirely + * missing (this process stayed at whatever uid execvp()'d it, i.e. + * root). _imapd here is an ordinary system daemon user, the + * listener-role counterpart to auth.c's _imapauth (renamed from + * _openimap/_openimapd along with the rest of the daemon's own + * on-disk/system identity -- see imapd.h's header comment for the + * rename-scoping policy) -- unrelated to the credential-file/ + * mailbox-user design decision, same distinction auth.c's header + * comment makes for its own daemon user. + */ + if (setgroups(1, &pw->pw_gid) == -1 || + setresgid(pw->pw_gid, pw->pw_gid, pw->pw_gid) == -1 || + setresuid(pw->pw_uid, pw->pw_uid, pw->pw_uid) == -1) + fatal("cannot drop privileges to _imapd"); + + /* + * Build the server-wide TLS context from the cert/key bytes read + * above, sourced directly against httpd's server_tls_init() + * (openbsd_source/src/usr.sbin/httpd/server.c) and src/lib/libtls/ + * tls.h: tls_config_new()+tls_server()+tls_config_set_keypair_mem() + * +tls_configure(), then tls_config_clear_keys() and scrubbing our + * own copy -- httpd does the equivalent with freezero(), we don't + * have that in this sandbox's stub environment so explicit_bzero()+ + * the buffer simply going out of scope at function return serves + * the same purpose (it's a fixed-size stack buffer, not a heap + * allocation, so there's nothing to free()). + * + * A failure here is NOT fatal to the whole daemon -- see + * listener_tls_ctx's file-scope comment for why: cleartext + * CAPABILITY/NOOP/LOGOUT/ID still work without TLS, so a bad + * cert/key degrades to "no TLS" rather than killing listener + * entirely. tls_accept_socket() call sites below all check for + * listener_tls_ctx == NULL first. + * + * tls_config_set_ciphers(..., "secure") is explicit here rather + * than left at whatever tls_config_new()'s own unset default + * happens to be -- prompted by relayd(8)/httpd(8) both explicitly + * moving their own default cipher sets to "secure" this same + * OpenBSD development cycle (rsadowski.de's "Dead Software + * Walking" writeup on the two daemons' ongoing modernization). + * "secure" is a real, documented named cipher-list value + * (tls_config_set_ciphers(3): "secure (or alias default)"), so + * this removes any ambiguity about which list actually applies + * rather than relying on an implicit library default that could + * change across libtls versions. + */ + if (cert_len == 0 || key_len == 0) { + log_warnx("listener: no TLS cert/key received -- TLS " + "disabled for this run"); + } else if ((listener_tls_config = tls_config_new()) == NULL) { + log_warnx("listener: tls_config_new failed -- TLS disabled"); + } else if ((listener_tls_ctx = tls_server()) == NULL) { + log_warnx("listener: tls_server failed -- TLS disabled"); + tls_config_free(listener_tls_config); + listener_tls_config = NULL; + } else if (tls_config_set_ciphers(listener_tls_config, "secure") + != 0) { + log_warnx("listener: tls_config_set_ciphers: %s -- " + "TLS disabled", tls_config_error(listener_tls_config)); + tls_free(listener_tls_ctx); + tls_config_free(listener_tls_config); + listener_tls_ctx = NULL; + listener_tls_config = NULL; + } else if (tls_config_set_keypair_mem(listener_tls_config, + (const uint8_t *)cert_buf, cert_len, + (const uint8_t *)key_buf, key_len) != 0) { + log_warnx("listener: tls_config_set_keypair_mem: %s -- " + "TLS disabled", tls_config_error(listener_tls_config)); + tls_free(listener_tls_ctx); + tls_config_free(listener_tls_config); + listener_tls_ctx = NULL; + listener_tls_config = NULL; + } else if (tls_configure(listener_tls_ctx, listener_tls_config) + != 0) { + log_warnx("listener: tls_configure: %s -- TLS disabled", + tls_error(listener_tls_ctx)); + tls_free(listener_tls_ctx); + tls_config_free(listener_tls_config); + listener_tls_ctx = NULL; + listener_tls_config = NULL; + } else { + tls_config_clear_keys(listener_tls_config); + log_info("listener: TLS configured"); + } + explicit_bzero(key_buf, sizeof(key_buf)); + + event_init(); + + imsgev_init(&iev_auth, peer_fd, listener_dispatch_auth, NULL); + + /* + * Hands off fd 3's already-populated ibuf3 to the event loop + * without a second imsgbuf_init() -- see this file's header + * comment and imsgev.c's imsgev_init_from_ibuf() comment for why + * that distinction is load-bearing here, not just tidiness. This + * is the channel parent uses for the ongoing per-session + * store-fork protocol (IMSG_STORE_FORK out, IMSG_SETUP_PEER / + * IMSG_STORE_FORK-as-failure in). + */ + imsgev_init_from_ibuf(&iev_parent, &ibuf3, listener_dispatch_parent, + NULL); + + /* Each listening socket gets its own struct event -- reusing + * an imsg channel's .ev here (an earlier draft of this function + * did) would clobber that channel's own event registration, since + * event_set()/event_add() on a struct event that's already + * registered elsewhere silently repurposes it rather than erroring. + * That now also means each *additional* dual-stack socket needs its + * own struct event too, not just each port purpose -- ev_accept_ + * cleartext[]/ev_accept_tls[] arrays, one slot per bound fd, rather + * than one struct event per array. listener_accept()'s own arg + * ((void *)0 cleartext, (void *)1 tls) is unchanged: it only ever + * needed to know which *port purpose* accepted the connection, not + * which address family did. */ + for (i = 0; i < n_cleartext_fd; i++) { + event_set(&ev_accept_cleartext[i], cleartext_fd[i], + EV_READ | EV_PERSIST, listener_accept, (void *)0); + event_add(&ev_accept_cleartext[i], NULL); + } + for (i = 0; i < n_tls_fd; i++) { + event_set(&ev_accept_tls[i], tls_fd[i], + EV_READ | EV_PERSIST, listener_accept, (void *)1); + event_add(&ev_accept_tls[i], NULL); + } + +#ifdef __OpenBSD__ + if (pledge("stdio recvfd sendfd inet", NULL) == -1) + fatal("pledge"); +#endif + + event_dispatch(); + fatalx("listener: exited event loop"); +} + +/* + * Accepts a connection and allocates its struct session. On the + * cleartext port, wires session_dispatch_client() and sends the RFC + * 9051 greeting immediately. On the implicit-TLS port (RFC 8314), no + * protocol bytes -- not even the greeting -- may cross the wire before + * TLS is established, so the handshake starts immediately instead (see + * session_tls_start()) and the greeting is deferred until it completes + * (s->pending_greeting, sent from session_tls_handshake()). + */ +static void +listener_accept(int fd, short event, void *arg) +{ + struct sockaddr_storage ss; + socklen_t sslen = sizeof(ss); + int client_fd; + struct session *s; + + (void)event; + if ((client_fd = accept(fd, (struct sockaddr *)&ss, &sslen)) == -1) { + log_warn("accept"); + return; + } + + s = calloc(1, sizeof(*s)); + if (s == NULL) { + log_warn("calloc"); + close(client_fd); + return; + } + s->id = next_session_id++; + s->client_fd = client_fd; + s->state = SESSION_NOT_AUTH; + s->implicit_tls = (arg != (void *)0); /* see the two event_set() + * calls in listener_main() -- + * (void *)1 for the port-993 + * listener */ + TAILQ_INSERT_TAIL(&sessions, s, entry); + + log_debug("session %u: accepted (%s)", s->id, + s->implicit_tls ? "implicit TLS" : "cleartext/STARTTLS"); + + if (s->implicit_tls) { + if (listener_tls_ctx == NULL) { + /* TLS didn't configure successfully at boot -- see + * listener_tls_ctx's file-scope comment. Nothing + * correct to do on this port without it. */ + log_warnx("session %u: implicit-TLS port, but TLS " + "isn't configured -- closing", s->id); + session_teardown(s); + return; + } + s->pending_greeting = 1; + session_tls_start(s); + return; + } + + session_arm_client_read(s); + session_send_greeting(s); +} + +/* + * (Re-)registers client_ev in normal EV_READ|EV_PERSIST mode pointing at + * session_dispatch_client() -- the steady-state read handler once no TLS + * handshake is in progress (either because there's no TLS at all, or + * because one just completed). Guards the event_del() with client_ev_ + * added the same way session_teardown() does, since this can be the + * very first registration for a session (the plaintext accept path) or + * a re-registration after a handshake (which repurposed client_ev to a + * different handler/flags -- see session_tls_start()). + */ +static void +session_arm_client_read(struct session *s) +{ + if (s->client_ev_added) + event_del(&s->client_ev); + event_set(&s->client_ev, s->client_fd, EV_READ | EV_PERSIST, + session_dispatch_client, s); + event_add(&s->client_ev, NULL); + s->client_ev_added = 1; +} + +/* + * Starts a TLS handshake on s->client_fd: tls_accept_socket() creates + * the per-connection struct tls (non-blocking, doesn't itself perform + * handshake I/O), then client_ev is armed for EV_READ pointing at + * session_tls_handshake() to drive the actual handshake once the socket + * is readable -- sourced against httpd's server_input()/tls_accept_ + * socket() call site and server_tls_handshake() (openbsd_source/src/ + * usr.sbin/httpd/server.c), which arms EV_READ and waits for the first + * callback rather than calling tls_handshake() synchronously here. + * Called both from listener_accept() (implicit-TLS port, client_ev not + * yet registered) and cmd_starttls() (cleartext port, client_ev already + * in normal read-dispatch mode) -- the client_ev_added guard, same + * pattern as session_arm_client_read() and session_teardown(), covers + * both. + */ +static void +session_tls_start(struct session *s) +{ + if (tls_accept_socket(listener_tls_ctx, &s->tls_ctx, s->client_fd) + != 0) { + log_warnx("session %u: tls_accept_socket: %s", s->id, + tls_error(listener_tls_ctx)); + session_teardown(s); + return; + } + + if (s->client_ev_added) + event_del(&s->client_ev); + event_set(&s->client_ev, s->client_fd, EV_READ, session_tls_handshake, + s); + event_add(&s->client_ev, NULL); + s->client_ev_added = 1; +} + +/* + * Drives a non-blocking TLS handshake to completion, re-arming client_ev + * for whichever direction tls_handshake() reports it's waiting on + * (TLS_WANT_POLLIN/TLS_WANT_POLLOUT can each occur regardless of which + * direction the *previous* call was armed for -- a TLS handshake isn't + * a simple read-then-write sequence). Sourced directly against httpd's + * server_tls_handshake(): ret == 0 is success, TLS_WANT_POLLIN/POLLOUT + * re-arm and wait for another callback, anything else is a hard + * failure. This is the one place in this codebase where blocking would + * be a real bug rather than just a simplification -- listener is a + * single event loop serving every session, so blocking here on one + * client's slow handshake would stall all the others; unlike + * session_write()'s short-message blocking-write simplification, there + * is no equivalent shortcut available for a multi-round-trip handshake. + */ +static void +session_tls_handshake(int fd, short event, void *arg) +{ + struct session *s = arg; + int ret; + + (void)fd; + (void)event; + + ret = tls_handshake(s->tls_ctx); + if (ret == 0) { + s->tls_active = 1; + log_debug("session %u: TLS handshake complete (%s %s)", + s->id, tls_conn_version(s->tls_ctx), + tls_conn_cipher(s->tls_ctx)); + session_arm_client_read(s); + if (s->pending_greeting) { + s->pending_greeting = 0; + session_send_greeting(s); + } + return; + } + if (ret == TLS_WANT_POLLIN) { + event_del(&s->client_ev); + event_set(&s->client_ev, s->client_fd, EV_READ, + session_tls_handshake, s); + event_add(&s->client_ev, NULL); + return; + } + if (ret == TLS_WANT_POLLOUT) { + event_del(&s->client_ev); + event_set(&s->client_ev, s->client_fd, EV_WRITE, + session_tls_handshake, s); + event_add(&s->client_ev, NULL); + return; + } + + log_warnx("session %u: tls_handshake: %s", s->id, + tls_error(s->tls_ctx)); + session_teardown(s); +} + +/* + * "* OK IMAP4rev2 server ready" -- the exact example text from RFC 9051 + * SS7.1.1's OK-response section, used verbatim rather than paraphrased. + * Goes through session_write() like every other response, so it's + * transparently TLS-aware for the implicit-TLS accept path (see + * session_tls_handshake(), which calls this once the handshake + * completes) without needing its own TLS-vs-plaintext branch here. + */ +static void +session_send_greeting(struct session *s) +{ + static const char greeting[] = "* OK IMAP4rev2 server ready\r\n"; + + session_write(s, greeting, sizeof(greeting) - 1); +} + +/* + * Reads raw bytes into s->inbuf and splits them into CRLF-terminated + * lines, per RFC 9051 SS2.2's "all interactions ... are in the form of + * lines" -- a bare LF (some clients/testing tools are sloppy about this) + * is deliberately NOT treated as a line ending, matching the ABNF's + * literal CRLF requirement rather than being lenient about it. Each + * complete line goes to session_handle_line() for tag/command parsing + * and dispatch. + * + * Correctness note: session_handle_line() can tear down (free) *s* -- + * currently only LOGOUT does this deliberately, see cmd_logout(). Once + * that happens this loop MUST NOT touch s again, which is why `consumed` + * is computed *before* the call (pure pointer arithmetic on data already + * in the buffer, no s-> field mutation needed yet) and the loop returns + * immediately rather than falling through to the memmove/inbuflen + * update below. Write failures do NOT tear down the session here (see + * session_write()'s comment) specifically to avoid needing this same + * care in every command handler -- LOGOUT is the one deliberate, + * well-understood exception. + */ +static void +session_dispatch_client(int fd, short event, void *arg) +{ + struct session *s = arg; + ssize_t n; + char *crlf; + + (void)event; + + if (s->tls_active) { + n = tls_read(s->tls_ctx, s->inbuf + s->inbuflen, + sizeof(s->inbuf) - s->inbuflen); + if (n == TLS_WANT_POLLIN || n == TLS_WANT_POLLOUT) { + /* + * tls_read() can want to WRITE (renegotiation, + * session ticket rotation -- see httpd's server_ + * tls_readcb() for the same pattern) even though + * this is a read path. Re-arm client_ev for whichever + * direction it actually needs and wait for the next + * callback -- unlike session_write()'s bounded + * blocking poll() retry, this is already inside the + * event loop, so there's no need to block at all, + * just re-register and return. Not EV_PERSIST: this + * is a one-shot retry registration, restored to the + * normal EV_READ|EV_PERSIST steady state below once + * real data (or EOF) is actually obtained. + */ + event_del(&s->client_ev); + event_set(&s->client_ev, s->client_fd, + (n == TLS_WANT_POLLIN) ? EV_READ : EV_WRITE, + session_dispatch_client, s); + event_add(&s->client_ev, NULL); + return; + } + if (n == -1) { + log_warnx("session %u: tls_read: %s", s->id, + tls_error(s->tls_ctx)); + session_teardown(s); + return; + } + /* + * Got real data or EOF (n == 0, handled below) -- the event + * that delivered us here might have been a temporary, + * non-persistent WANT_POLLIN/WANT_POLLOUT retry registration + * (above) rather than the steady-state EV_READ|EV_PERSIST + * one, so restore that steady state unconditionally rather + * than tracking which case we're in. Redundant but harmless + * in the common case where it was already correct. + */ + session_arm_client_read(s); + } else { + n = read(fd, s->inbuf + s->inbuflen, + sizeof(s->inbuf) - s->inbuflen); + if (n == -1) { + log_warn("session %u: read", s->id); + session_teardown(s); + return; + } + } + + if (n == 0) { + log_debug("session %u: client closed connection", s->id); + session_teardown(s); + return; + } + s->inbuflen += (size_t)n; + + for (;;) { + size_t consumed; + int alive; + + /* + * RFC 9051 SS4.3 literal in flight (cmd_append() found a + * trailing "{n}"/"{n+}" and is waiting for its raw octets -- + * see s->literal_pending's comment). Checked *before* the + * CRLF search below on purpose: a literal's bytes can + * contain CRLF sequences of their own (SS4.3: "a sequence of + * zero or more octets (including CR and LF)"), so treating + * them as line-oriented input here would both misparse the + * literal itself and, worse, could let embedded bytes that + * happen to look like "tag SP command CRLF" get dispatched + * as a bogus command mid-literal. This block runs first, + * every iteration, until the full literal (and its + * mandatory trailing CRLF) has been consumed -- no ST_* + * dispatch-table exclusion is needed for this phase, unlike + * every async-imsg-wait state, because session_handle_line() + * is simply never reached while it's active. + */ + if (s->literal_pending) { + uint64_t want, take; + + want = s->literal_remaining; + take = (uint64_t)s->inbuflen < want ? + (uint64_t)s->inbuflen : want; + + if (take > 0) { + memcpy(s->literal_buf + + (s->literal_len - s->literal_remaining), + s->inbuf, (size_t)take); + s->literal_remaining -= take; + memmove(s->inbuf, s->inbuf + take, + s->inbuflen - (size_t)take); + s->inbuflen -= (size_t)take; + } + + if (s->literal_remaining > 0) + break; /* need more data -- wait for the + * next read */ + + /* + * Literal body fully received. RFC 9051 SS9's + * `command = tag SP ... CRLF` still requires a + * trailing CRLF after the literal's raw octets -- + * SS4.3's `literal` production ends with the octets + * themselves, not a CRLF; the CRLF belongs to the + * outer `command` production, same as after any + * other argument. APPEND's own grammar (SS6.3.12: + * `append = ... SP literal`) puts the literal last, + * so nothing else can follow it on the line -- v1 + * doesn't support a literal anywhere but the final + * argument, a deliberate simplification flagged in + * parse_append_args()'s comment. + */ + if (s->inbuflen < 2) + break; /* trailing CRLF hasn't arrived yet */ + if (s->inbuf[0] != '\r' || s->inbuf[1] != '\n') { + /* + * Genuinely hard to resync from here -- we + * have no reliable way to know where the + * next real command boundary is inside + * whatever the client actually sent instead + * of CRLF. Same "give up rather than guess" + * call session_teardown() elsewhere in this + * file makes for comparably confused + * protocol states, rather than risk + * mis-parsing arbitrary subsequent bytes as + * commands. + */ + log_warnx("session %u: expected CRLF after " + "literal data, closing", s->id); + session_teardown(s); + return; + } + memmove(s->inbuf, s->inbuf + 2, s->inbuflen - 2); + s->inbuflen -= 2; + + s->literal_pending = 0; + alive = session_finish_append(s); + if (!alive) + return; + continue; + } + + crlf = memmem(s->inbuf, s->inbuflen, "\r\n", 2); + if (crlf == NULL) + break; + + *crlf = '\0'; + consumed = (size_t)(crlf - s->inbuf) + 2; + + /* + * A bare SASL continuation-response line (base64, or "*" to + * cancel -- RFC 9051 SS6.2.2) is NOT a tagged command and + * must not go through session_handle_line()'s tag/name/args + * parser -- s->auth_cont, set by cmd_authenticate() when the + * client sent "AUTHENTICATE PLAIN" with no inline initial + * response, routes it to session_handle_auth_continuation() + * instead. Likewise, a bare "DONE" continuation line while + * idling (RFC 9051 SS6.3.13) is NOT a tagged command either + * -- s->idling, set by cmd_idle(), routes it to session_ + * handle_idle_continuation() the same way. All three share + * the same alive/torn-down (1/0) return convention this loop + * already relies on. + */ + alive = s->auth_cont ? + session_handle_auth_continuation(s, s->inbuf) : + s->idling ? + session_handle_idle_continuation(s, s->inbuf) : + session_handle_line(s, s->inbuf); + if (alive == 0) + return; /* s was torn down (LOGOUT) -- must not + * touch it again, see this function's + * header comment. */ + + /* + * A handler can mutate s->inbuflen itself -- cmd_starttls() + * deliberately zeroes it to discard any plaintext pipelined + * past the STARTTLS line (see its comment). Clamp `consumed` + * (computed above, before the handler ran) against the + * *current* inbuflen so the subtraction below can't + * underflow a size_t into a huge value. A no-op in the + * ordinary case, since inbuflen only otherwise grows via new + * reads between session_dispatch_client() invocations, never + * shrinks except through this same loop. + */ + if (consumed > s->inbuflen) + consumed = s->inbuflen; + + memmove(s->inbuf, s->inbuf + consumed, s->inbuflen - consumed); + s->inbuflen -= consumed; + } + + if (s->inbuflen == sizeof(s->inbuf)) { + /* Buffer full with no CRLF found -- matches the spirit of + * RFC 9051 SS7.1.3's own example ("* BAD Command line too + * long"), though that exact string is example text, not a + * normative response -- see SESSION_INBUF_MAX's comment. */ + static const char bad[] = "* BAD command line too long\r\n"; + + log_warnx("session %u: command line too long, closing", + s->id); + (void)write(s->client_fd, bad, sizeof(bad) - 1); + session_teardown(s); + } +} + +/* + * Writes to the client socket -- plain blocking write(2) for a + * plaintext session, or tls_write() for a TLS one. Every response in + * this file (including the greeting -- see session_send_greeting()) + * goes through this one function. + * + * Blocking is a deliberate v1 simplification: worth revisiting together + * (event-driven EV_WRITE + a per-session outbound queue, mirroring + * imsgev.c's pattern) once any response needs to be larger than one + * send (multi-line FETCH output, etc.) -- v1's responses are all short + * and fixed, so this has always been an accepted shortcut, not a claim + * of correctness at scale. + * + * For TLS specifically: tls_write() can return TLS_WANT_POLLIN or + * TLS_WANT_POLLOUT even for a plain short write (renegotiation, session + * ticket rotation -- see httpd's server_tls_writecb() for the same + * pattern in event-driven form), so this retries via poll(2) with a + * BOUNDED timeout rather than looping forever. Unlike session_tls_ + * handshake() (which MUST be event-driven, since listener is one event + * loop serving every session and a slow multi-round-trip handshake + * would stall all of them), a bounded blocking retry here is a much + * smaller, deliberately accepted risk: it can only stall this one + * session's own short write, and only in the rare renegotiation case, + * and only for SESSION_WRITE_POLL_TIMEOUT_MS at worst -- not the + * unbounded, whole-daemon-hanging risk an infinite poll() timeout would + * be against a client that simply never reads its socket. + * + * Deliberately does NOT call session_teardown() on failure (including a + * poll(2) timeout) -- unlike the "line too long" path. A write failure + * here is logged and otherwise ignored; the read side will discover the + * same dead connection on its next read()/tls_read() (EOF or error) and + * tear down then. This avoids every command handler needing to guard + * against *s* being freed out from under it mid-dispatch -- LOGOUT is + * the one place that tears a session down deliberately, and it does so + * explicitly, not via a write failure. See session_dispatch_client()'s + * header comment for the reentrancy hazard this sidesteps. + */ +#define SESSION_WRITE_POLL_TIMEOUT_MS 5000 + +static void +session_write(struct session *s, const char *buf, size_t len) +{ + size_t sent = 0; + + if (!s->tls_active) { + /* + * F12 fix: loop on partial writes and retry EINTR/EAGAIN + * (poll like the TLS branch below). Previously a single + * write() ignored short counts, silently truncating a + * multi-write response under back-pressure. + */ + while (sent < len) { + ssize_t n; + struct pollfd pfd; + + n = write(s->client_fd, buf + sent, len - sent); + if (n == -1) { + if (errno == EINTR) + continue; + if (errno == EAGAIN || errno == EWOULDBLOCK) { + pfd.fd = s->client_fd; + pfd.events = POLLOUT; + if (poll(&pfd, 1, + SESSION_WRITE_POLL_TIMEOUT_MS) <= 0) { + log_warnx("session %u: write: " + "timed out or poll error", + s->id); + return; + } + continue; + } + log_warn("session %u: write", s->id); + return; + } + sent += (size_t)n; + } + return; + } + + while (sent < len) { + ssize_t n; + struct pollfd pfd; + int pret; + + n = tls_write(s->tls_ctx, buf + sent, len - sent); + if (n == TLS_WANT_POLLIN || n == TLS_WANT_POLLOUT) { + pfd.fd = s->client_fd; + pfd.events = (n == TLS_WANT_POLLIN) ? POLLIN : POLLOUT; + pret = poll(&pfd, 1, SESSION_WRITE_POLL_TIMEOUT_MS); + if (pret == -1) { + log_warn("session %u: poll (tls_write retry)", + s->id); + return; + } + if (pret == 0) { + log_warnx("session %u: tls_write: timed out " + "waiting for socket", s->id); + return; + } + continue; + } + if (n == -1) { + log_warnx("session %u: tls_write: %s", s->id, + tls_error(s->tls_ctx)); + return; + } + sent += (size_t)n; + } +} + +static void +session_reply(struct session *s, const char *tag, const char *status, + const char *text) +{ + char buf[512]; + int len; + + len = snprintf(buf, sizeof(buf), "%s %s %s\r\n", tag, status, text); + if (len < 0) + return; + if ((size_t)len >= sizeof(buf)) + len = sizeof(buf) - 1; /* truncate rather than overflow -- + * fine for v1's short fixed messages, + * none of which approach this bound */ + session_write(s, buf, (size_t)len); +} + +static void +session_untagged(struct session *s, const char *text) +{ + char buf[512]; + int len; + + len = snprintf(buf, sizeof(buf), "* %s\r\n", text); + if (len < 0) + return; + if ((size_t)len >= sizeof(buf)) + len = sizeof(buf) - 1; + session_write(s, buf, (size_t)len); +} + +/* + * RFC 7162 SS3.1: marks this session as a "CONDSTORE-aware client" -- + * called from every CONDSTORE-enabling command this server recognizes + * (ENABLE CONDSTORE/QRESYNC, FETCH with the MODSEQ fetch-att or CHANGEDSINCE + * modifier, STORE with UNCHANGEDSINCE, SEARCH with a MODSEQ criterion). + * SELECT/EXAMINE with a CONDSTORE or QRESYNC select-param is deliberately + * NOT routed through this function -- see cmd_select()'s comment -- because + * that command's own response already includes HIGHESTMODSEQ as part of its + * normal reply sequence (session_handle_mbox_selected()), so calling this + * from there would emit a redundant, spec-violating second copy. + * + * Idempotent (does nothing once already enabled) and, per SS3.1's own + * parenthetical ("A first CONDSTORE enabling command executed in the + * session with a mailbox selected MUST cause the server to return + * HIGHESTMODSEQ... (if any is selected)"), only emits the unsolicited + * HIGHESTMODSEQ OK response when a mailbox is actually selected right now + * -- e.g. ENABLE is normally issued pre-SELECT (RFC 9051 SS6.3.1), where + * this is correctly a no-op beyond setting the flag. Uses s->mbox_ + * highestmodseq (cached from the last SELECT/STORE/EXPUNGE reply) rather + * than asking store.c for a fresh value -- see that field's comment in + * struct session for why a cached value is an acceptable, deliberate + * choice for this specific edge case. + */ +static void +session_condstore_enable(struct session *s) +{ + char buf[48]; + + if (s->condstore_enabled) + return; + s->condstore_enabled = 1; + + if (s->state == SESSION_SELECTED) { + snprintf(buf, sizeof(buf), "OK [HIGHESTMODSEQ %llu]", + (unsigned long long)s->mbox_highestmodseq); + session_untagged(s, buf); + } +} + +/* + * Splits one already CRLF-stripped line into tag/name/args, per RFC + * 9051's ABNF: `command = tag SP (command-any / ...) CRLF`. Tokens are + * split on single spaces; RFC 9051 SS2.2.1 says extraneous spaces are + * technically a client syntax error, but this parser is deliberately + * lenient about repeated spaces between tokens (skips them) rather than + * rejecting -- a simplification, not a claim of full ABNF conformance. + * Tag characters themselves aren't validated against the real ASTRING- + * CHAR class (RFC 9051 SS9, `tag = 1*`) -- + * another deliberate simplification, flagged rather than silently + * assumed correct. + * + * Returns -1 if no tag could be found at all (line was empty or all + * spaces) -- the caller sends RFC 9051 SS7.1.3's own example response + * for exactly this case, untagged "* BAD Empty command line". Returns 0 + * otherwise; *name may be NULL if a tag was found but no command name + * followed it (caller sends a tagged BAD). + */ +static int +parse_command_line(char *line, char **tag, char **name, char **args) +{ + char *p = line; + + while (*p == ' ') + p++; + if (*p == '\0') + return (-1); + *tag = p; + + while (*p != '\0' && *p != ' ') + p++; + if (*p == '\0') { + *name = NULL; + *args = NULL; + return (0); + } + *p++ = '\0'; + + while (*p == ' ') + p++; + if (*p == '\0') { + *name = NULL; + *args = NULL; + return (0); + } + *name = p; + + while (*p != '\0' && *p != ' ') + p++; + if (*p == '\0') { + *args = NULL; + return (0); + } + *p++ = '\0'; + + while (*p == ' ') + p++; + *args = (*p != '\0') ? p : NULL; + return (0); +} + +/* + * Parses and dispatches one command line. Returns 1 if the session is + * still alive afterward, 0 if it was torn down (LOGOUT only, in v1) -- + * see session_dispatch_client()'s header comment for why callers must + * check this before touching *s* again. + */ +static int +session_handle_line(struct session *s, char *line) +{ + char *tag, *name, *args; + size_t i; + + log_debug("session %u: <<< %s", s->id, line); + + if (parse_command_line(line, &tag, &name, &args) == -1) { + session_reply(s, "*", "BAD", "Empty command line"); + return (1); + } + if (name == NULL) { + session_reply(s, tag, "BAD", "Missing command"); + return (1); + } + + for (i = 0; i < NUM_IMAP_CMDS; i++) { + if (strcasecmp(name, imap_cmds[i].name) == 0) + break; + } + if (i == NUM_IMAP_CMDS) { + session_reply(s, tag, "BAD", "Unknown command"); + return (1); + } + if (!(imap_cmds[i].states & (1U << s->state))) { + /* RFC 9051 SS3: "the server will respond with a BAD or NO + * (depending upon server implementation)" for a command + * received in the wrong state -- BAD chosen here. */ + session_reply(s, tag, "BAD", + "Command not permitted in this state"); + return (1); + } + return (imap_cmds[i].handler(s, tag, args)); +} + +static int +cmd_capability(struct session *s, const char *tag, char *args) +{ + (void)args; /* RFC 9051: "Arguments: none" -- extra args are + * silently ignored rather than rejected, a + * leniency simplification like parse_command_line()'s. */ + + session_untagged(s, s->tls_active ? + "CAPABILITY " CAPABILITY_POST_TLS : "CAPABILITY " CAPABILITY_PRE_TLS); + session_reply(s, tag, "OK", "CAPABILITY completed"); + return (1); +} + +static int +cmd_noop(struct session *s, const char *tag, char *args) +{ + (void)args; + session_reply(s, tag, "OK", "NOOP completed"); + return (1); +} + +static int +cmd_logout(struct session *s, const char *tag, char *args) +{ + (void)args; + + /* Exact example text from RFC 9051 SS6.1.3. */ + session_untagged(s, "BYE IMAP4rev2 Server logging out"); + session_reply(s, tag, "OK", "LOGOUT completed"); + session_teardown(s); + return (0); +} + +static int +cmd_id(struct session *s, const char *tag, char *args) +{ + /* + * RFC 2971 SS3.1: full parsing of the client's parenthesized + * field/value list isn't implemented (real IMAP list/string + * argument parsing -- quoted strings, literals -- doesn't exist + * anywhere in this codebase yet, and nothing here needs to look + * inside the list per RFC 2971's own "MUST NOT make operational + * changes based on the data" rule). Logged raw and discarded, + * matching openimap-v1-dispatch.md's stated v1 behavior ("accepts + * client ID params, logs them, no persistence needed for v1"). + * Always replies NIL, per SS3.2's "a server MAY send NIL in place + * of the list" -- simplest spec-compliant response, and matches + * the RFC's own minimal example (C: ID NIL / S: * ID NIL). + */ + log_debug("session %u: ID params: %s", s->id, + args != NULL ? args : "(none)"); + session_untagged(s, "ID NIL"); + session_reply(s, tag, "OK", "ID completed"); + return (1); +} + +static int +cmd_login(struct session *s, const char *tag, char *args) +{ + (void)args; + + /* + * Permanently disabled regardless of TLS state -- resolved design + * decision, openimap-v1-dispatch.md's LOGIN row: RFC 9051 SS6.2.3 + * only mandates refusing LOGIN when unprotected (a floor, not a + * ceiling); we rely solely on AUTHENTICATE PLAIN under TLS. + */ + session_reply(s, tag, "NO", + "LOGIN disabled -- use AUTHENTICATE PLAIN under TLS"); + return (1); +} + +static int +cmd_starttls(struct session *s, const char *tag, char *args) +{ + if (args != NULL) { + /* RFC 9051 SS6.2.1 Result: "BAD - ... arguments invalid". */ + session_reply(s, tag, "BAD", "STARTTLS takes no arguments"); + return (1); + } + if (s->tls_active) { + /* RFC 9051 SS6.2.1 Result: "BAD - STARTTLS received after a + * successful TLS negotiation ...". */ + session_reply(s, tag, "BAD", "TLS already active"); + return (1); + } + if (listener_tls_ctx == NULL) { + /* RFC 9051 SS6.2.1 Result: "NO - TLS negotiation can't be + * initiated, due to server configuration error" -- exactly + * this codebase's situation if cert/key loading failed at + * boot (see listener_tls_ctx's file-scope comment). RFC + * 5530: UNAVAILABLE -- "Temporary failure because a + * subsystem is down," which is exactly what a boot-time + * cert/key failure leaves TLS in for this run. */ + session_reply(s, tag, "NO", + "[UNAVAILABLE] TLS negotiation unavailable"); + return (1); + } + + /* + * RFC 9051 SS6.2.1: "A TLS negotiation begins immediately after + * the CRLF at the end of the tagged OK response from the server." + * -- matches the RFC's own example exactly ("C: a002 STARTTLS" / + * "S: a002 OK Begin TLS negotiation now"). Must go out in + * cleartext BEFORE the handshake starts. + */ + session_reply(s, tag, "OK", "Begin TLS negotiation now"); + + /* + * RFC 9051 SS6.2.1's STARTTLS command-injection mitigation: any + * plaintext bytes already sitting in our own read buffer past the + * STARTTLS line itself (a pipelining client, or an attacker) are + * discarded rather than reinterpreted as the start of the TLS + * handshake -- the "throw it away" option the RFC explicitly + * permits as an alternative to "treat as start of handshake", + * chosen here for simplicity. session_dispatch_client()'s caller + * loop clamps its own `consumed` calculation against the + * (possibly now smaller) s->inbuflen after this handler returns, + * specifically to make this safe -- see its comment. + */ + s->inbuflen = 0; + + session_tls_start(s); + return (1); +} + +/* + * RFC 4616 SS2: authzid/authcid/passwd are each "MUST accept up to and + * including 255 octets", joined by two single-octet NUL delimiters -- + * 255*3+2 = 767, rounded up. Sourced by fetching the RFC's own text + * (https://www.rfc-editor.org/rfc/rfc4616.txt) this session, since it + * wasn't present in research/ or openbsd_source/ -- see this file's + * git history / session notes for that lookup. + */ +#define SASL_PLAIN_MAX 768 + +/* + * Decodes and verifies one SASL PLAIN client message (RFC 4616 SS2: + * "message = [authzid] UTF8NUL authcid UTF8NUL passwd"), then sends + * IMSG_AUTH_REQUEST to auth.c over iev_auth. b64 is the raw base64 token + * from the wire -- either an inline initial response on the AUTHENTICATE + * line itself, or the client's line following our "+ " continuation + * request (RFC 9051 SS6.2.2). + * + * b64_pton() (declared in ; OpenBSD's __b64_pton, the same + * codec smtpd.c uses -- see openbsd_source/src/usr.sbin/smtpd/util.c) + * does the actual decode. Its full implementation (openbsd_source/src/ + * lib/libc/net/base64.c) was read this project: it returns -1 on any + * character outside the base64 alphabet, on target-buffer overflow, and + * on misplaced/non-terminal "=" padding -- exactly RFC 9051 SS6.2.2's own + * required rejection condition ("if it receives an invalid base64 string + * ... it MUST reject the AUTHENTICATE command by sending a tagged BAD + * response", explicitly calling out "characters outside the base64 + * alphabet" and non-terminal "=" as the two examples). + * + * allow_empty_equals is set only by the inline-initial-response call site + * (cmd_authenticate()) -- RFC 9051's initial-resp ABNF is literally + * "(base64 / \"=\")", a single pad character meaning "response present, + * but zero-length" (SS6.2.2: "To send a zero-length initial response, the + * client MUST send a single pad character"). That shorthand is NOT part + * of the plain `base64` grammar used for continuation-response lines, so + * session_handle_auth_continuation() passes 0 here -- a lone "=" from a + * continuation line falls through to b64_pton() and is correctly rejected + * as invalid (non-terminal, in fact orphaned) padding. + * + * Never tears down the session -- an AUTHENTICATE failure keeps the + * connection open for a retry, per RFC 9051 SS6.2.2's own "the client MAY + * try another authentication mechanism" -- so this always returns 1; the + * int return exists only to match session_handle_line()'s alive/torn- + * down calling convention that session_dispatch_client()'s loop relies + * on. + */ +static int +sasl_plain_finish(struct session *s, const char *tag, const char *b64, + int allow_empty_equals) +{ + unsigned char raw[SASL_PLAIN_MAX]; + unsigned char *authcid, *passwd, *nul; + int rawlen; + size_t off, authcidlen, passwdlen; + struct imsg_auth_request req; + + if (allow_empty_equals && strcmp(b64, "=") == 0) { + rawlen = 0; + } else { + rawlen = b64_pton(b64, raw, sizeof(raw)); + if (rawlen < 0) { + session_reply(s, tag, "BAD", "invalid base64"); + return (1); + } + } + + nul = memchr(raw, '\0', (size_t)rawlen); + if (nul == NULL) { + session_reply(s, tag, "BAD", "malformed SASL PLAIN message"); + explicit_bzero(raw, sizeof(raw)); + return (1); + } + off = (size_t)(nul - raw) + 1; + authcid = raw + off; + nul = memchr(authcid, '\0', (size_t)rawlen - off); + if (nul == NULL) { + session_reply(s, tag, "BAD", "malformed SASL PLAIN message"); + explicit_bzero(raw, sizeof(raw)); + return (1); + } + authcidlen = (size_t)(nul - authcid); + passwd = nul + 1; + passwdlen = (size_t)rawlen - off - authcidlen - 1; + + /* + * RFC 4616 SS2: "if preparation fails or results in an empty + * string, verification SHALL fail". An empty authcid or passwd is + * a syntactically well-formed message that can never authenticate + * -- reject it the same way (NO) as a wrong password, not as a + * malformed message (BAD): nothing about the SASL PLAIN framing + * itself is wrong. + */ + if (authcidlen == 0 || passwdlen == 0) { + session_reply(s, tag, "NO", "[AUTHENTICATIONFAILED] authentication failed"); + explicit_bzero(raw, sizeof(raw)); + return (1); + } + /* + * Not a protocol error either -- just bigger than this + * implementation's fixed-size struct imsg_auth_request fields + * (imapd.h) support. Same generic NO as any other credential + * that won't authenticate, rather than a distinct error that + * would hand the client a live oracle for an implementation + * limit -- same spirit as auth.c's auth_verify() comment on not + * short-circuiting differently for an unknown username. + */ + if (authcidlen >= AUTH_USERNAME_MAX || passwdlen >= AUTH_PASSWORD_MAX) { + session_reply(s, tag, "NO", "[AUTHENTICATIONFAILED] authentication failed"); + explicit_bzero(raw, sizeof(raw)); + return (1); + } + + memset(&req, 0, sizeof(req)); + req.session_id = s->id; + memcpy(req.username, authcid, authcidlen); + memcpy(req.password, passwd, passwdlen); + explicit_bzero(raw, sizeof(raw)); + + strlcpy(s->pending_tag, tag, sizeof(s->pending_tag)); + s->state = SESSION_AUTHENTICATING; + + if (imsg_compose(&iev_auth.ibuf, IMSG_AUTH_REQUEST, 0, 0, -1, + &req, sizeof(req)) == -1) + log_warn("session %u: imsg_compose IMSG_AUTH_REQUEST", s->id); + imsgev_add(&iev_auth); + /* + * imsg_compose() copies req into its own queue immediately -- safe + * to scrub our own stack copy right after, same reasoning as + * parent.c's send_tls_certs() scrubbing its cert/key stack buffers + * right after the matching imsg_compose() calls. + */ + explicit_bzero(&req, sizeof(req)); + + return (1); +} + +/* + * Handles one line received while s->auth_cont is set -- the client's + * response to our "+ " continuation request after a bare "AUTHENTICATE + * PLAIN" (see cmd_authenticate()). Always clears auth_cont first: whether + * this line succeeds, fails, or cancels, the *next* line is back to being + * an ordinary tagged command either way. + */ +static int +session_handle_auth_continuation(struct session *s, char *line) +{ + s->auth_cont = 0; + + /* RFC 9051 SS6.2.2: "If the client wishes to cancel an + * authentication exchange, it issues a line consisting of a + * single '*'. If the server receives such a response ... it MUST + * reject the AUTHENTICATE command by sending a tagged BAD + * response." */ + if (strcmp(line, "*") == 0) { + session_reply(s, s->pending_tag, "BAD", + "AUTHENTICATE cancelled"); + return (1); + } + + return sasl_plain_finish(s, s->pending_tag, line, 0); +} + +/* + * Handles one line received while s->idling is set -- the client's + * response to our "+ idling" continuation after IDLE (RFC 9051 SS6.3.13; + * see cmd_idle()). Always clears idling first, same "next line is back to + * an ordinary tagged command either way" reasoning as session_handle_auth_ + * continuation() above. + * + * RFC 9051 SS6.3.13: "The IDLE command is terminated by the receipt of a + * 'DONE' continuation from the client... The client MUST NOT send a + * command while the server is waiting for the DONE, since the server will + * not be able to distinguish a command from a continuation." Matched + * case-insensitively -- same leniency this codebase applies to every + * other IMAP keyword (strcasecmp throughout), and RFC 2177's original + * IDLE spec (SS3, worked example) shows literal "DONE" but neither RFC + * spells out case-sensitivity for it specifically. A client that sends + * anything else here has violated the MUST NOT above; there's no + * principled way to recover (we don't know what they meant), so the IDLE + * simply fails with BAD rather than silently accepting or hanging. + */ +static int +session_handle_idle_continuation(struct session *s, char *line) +{ + s->idling = 0; + + if (strcasecmp(line, "DONE") != 0) { + session_reply(s, s->pending_tag, "BAD", + "expected DONE"); + return (1); + } + + session_reply(s, s->pending_tag, "OK", "IDLE terminated"); + return (1); +} + +static int +cmd_authenticate(struct session *s, const char *tag, char *args) +{ + char *mech, *p, *initial; + + if (args == NULL) { + /* RFC 9051 SS6.2.2 Result: "BAD - ... arguments invalid". */ + session_reply(s, tag, "BAD", "Missing SASL mechanism name"); + return (1); + } + mech = args; + for (p = mech; *p != '\0' && *p != ' '; p++) + continue; + if (*p == '\0') { + initial = NULL; + } else { + *p++ = '\0'; + while (*p == ' ') + p++; + initial = (*p != '\0') ? p : NULL; + } + + /* + * openimap-privsep-design.md / RFC 9051 SS6.2.2's quoted "MUST + * implement a configuration in which it does NOT permit any + * plaintext password mechanisms, unless the STARTTLS command has + * been negotiated..." -- s->tls_active reflects a real TLS + * handshake (STARTTLS and implicit-TLS both work as of the TLS- + * wiring pass, see session_tls_handshake()). + */ + if (!s->tls_active) { + /* RFC 5530: PRIVACYREQUIRED -- "If TLS is not in use, the + * client could try STARTTLS ... and then repeat the + * operation," exactly this situation. */ + session_reply(s, tag, "NO", + "[PRIVACYREQUIRED] plaintext authentication requires TLS"); + return (1); + } + + /* v1 only implements PLAIN -- CAPABILITY_POST_TLS only ever + * advertises AUTH=PLAIN, so anything else is a client asking for + * something we never claimed to support. */ + if (strcasecmp(mech, "PLAIN") != 0) { + session_reply(s, tag, "NO", + "authentication mechanism not available"); + return (1); + } + + if (initial != NULL) { + /* RFC 9051 SS6.2.2 "initial response" (SASL SS4): the + * mechanism name was followed by a token on the same line -- + * either a base64 blob, or "=" for a present-but-zero-length + * response (initial-resp = base64 / "=") -- so the exchange + * finishes in one round trip, no continuation request + * needed. sasl_plain_finish() itself saves the tag into + * s->pending_tag once it knows the exchange is actually going + * async (IMSG_AUTH_REQUEST sent) -- no need to duplicate + * that here. */ + return sasl_plain_finish(s, tag, initial, 1); + } + + /* No initial response: RFC 9051 SS6.2.2's continue-req -- "+ SP + * [text] CRLF" (resp-text's text is optionally empty per the + * errata-updated ABNF, RFC 9051 SS8 erratum note 23) -- prompts the + * client for its base64 response on the next line. auth_cont routes + * that next raw line to session_handle_auth_continuation() instead + * of the ordinary tagged-command dispatcher. */ + strlcpy(s->pending_tag, tag, sizeof(s->pending_tag)); + s->auth_cont = 1; + session_write(s, "+ \r\n", 4); + return (1); +} + +/* + * Shared reply for every command-auth/command-select entry that's + * recognized (correctly gated by session state, so this is never + * confused with session_handle_line()'s "Unknown command" BAD) but can't + * actually run yet -- store.c's IMSG_MBOX_* family (imapd.h) has no + * designed wire payloads, so there's nothing for these handlers to send. + * NO (not BAD) is the correct RFC 9051 result category here: the command + * itself is syntactically fine, the server is just failing to perform it + * -- the same distinction cmd_starttls() already draws when TLS isn't + * configured. cmdname is only used for the log line, kept separate from + * the client-facing reply text (which stays generic) the same way + * cmd_login()'s and cmd_starttls()'s replies do. + */ +static int +stub_not_implemented(struct session *s, const char *tag, const char *cmdname) +{ + log_debug("session %u: %s not implemented (store.c's IMSG_MBOX_* " + "wire protocol isn't designed yet)", s->id, cmdname); + session_reply(s, tag, "NO", "not implemented"); + return (1); +} + +/* + * RFC 9051 SS6.3.1: `enable = "ENABLE" 1*(SP capability)` -- at least one + * capability argument is mandatory (Result: "BAD - No arguments, or + * syntax error in an argument"). For each argument, "If the argument is + * not an extension known to the server, the server MUST ignore the + * argument", and the untagged ENABLED response is sent unconditionally, + * "even if no extensions were enabled" -- listing only what THIS command + * actually newly enabled (SS6.3.1's own multi-ENABLE example: a second + * ENABLE "should not" re-list an already-enabled extension). + * + * RFC 7162 addition this pass: CONDSTORE and QRESYNC are now real, + * recognized ENABLE arguments (matched case-insensitively, per SS6.3.1's + * own worked example using "ENABLE CONDSTORE"). Enabling QRESYNC also + * enables CONDSTORE (RFC 7162 SS3.2.3: "the presence of the 'QRESYNC' + * capability implies support for the CONDSTORE IMAP extension"), and -- a + * deliberate implementation choice, matching how several real QRESYNC + * servers behave, though the RFC doesn't explicitly mandate the ENABLED + * echo specifically -- "ENABLE QRESYNC" alone reports both "QRESYNC" and + * "CONDSTORE" as newly enabled, not just QRESYNC, since CONDSTORE really + * is being enabled as a side effect of this exact command. Goes through + * session_condstore_enable() so the "CONDSTORE enabling command issued + * while a mailbox is already selected" unsolicited-HIGHESTMODSEQ case + * (SS3.1) is handled uniformly with every other enabling command -- even + * though RFC 9051 SS6.3.1 says ENABLE is only valid pre-SELECT and clients + * "MUST NOT issue ENABLE" after selecting a mailbox, the same section adds + * "server implementations don't have to check" for that, so this server + * doesn't reject a late ENABLE outright; handling it correctly rather than + * either rejecting or silently misbehaving is the safer choice. + */ +static int +cmd_enable(struct session *s, const char *tag, char *args) +{ + char buf[64]; + char *tok, *save; + int newly_condstore = 0, newly_qresync = 0; + + if (args == NULL) { + session_reply(s, tag, "BAD", + "ENABLE requires at least one capability argument"); + return (1); + } + + for (tok = strtok_r(args, " ", &save); tok != NULL; + tok = strtok_r(NULL, " ", &save)) { + if (strcasecmp(tok, "QRESYNC") == 0) { + if (!s->qresync_enabled) { + s->qresync_enabled = 1; + newly_qresync = 1; + if (!s->condstore_enabled) + newly_condstore = 1; + } + } else if (strcasecmp(tok, "CONDSTORE") == 0) { + if (!s->condstore_enabled) + newly_condstore = 1; + } + /* Anything else: not an extension this server advertises at + * all -- SS6.3.1 says ignore it, so no else-branch needed. */ + } + + if (newly_condstore || newly_qresync) + session_condstore_enable(s); + + buf[0] = '\0'; + if (newly_qresync) + strlcat(buf, "QRESYNC", sizeof(buf)); + if (newly_condstore) { + if (buf[0] != '\0') + strlcat(buf, " ", sizeof(buf)); + strlcat(buf, "CONDSTORE", sizeof(buf)); + } + + if (buf[0] != '\0') { + char untagged[80]; + + snprintf(untagged, sizeof(untagged), "ENABLED %s", buf); + session_untagged(s, untagged); + } else + session_untagged(s, "ENABLED"); + + session_reply(s, tag, "OK", "ENABLE completed"); + return (1); +} + +/* + * Parses the interior of QRESYNC's own parenthesized argument list (RFC + * 7162 SS3.2.5): `uidvalidity SP mod-sequence-value [SP known-uids [SP + * seq-match-data]]`. inner is already NUL-terminated at its closing paren + * by the caller (parse_select_params()) and modified in place. + * + * known-uids gets this codebase's usual single-range restriction (no + * comma-separated sequence-set, matching FETCH/STORE/SEARCH's UID-range + * scope elsewhere) and explicitly rejects "*", which SS3.2.5.1's own + * grammar comment forbids here ("Sequence of UIDs; '*' is not allowed"). + * seq-match-data is parsed only enough to confirm balanced parentheses and + * skipped -- its content is never sent to store.c at all (see imapd.h's + * imsg_mbox_select comment for why: this implementation's chosen minimal + * QRESYNC state model never uses it to narrow anything, which RFC 7162 + * SS5.2 explicitly sanctions as compliant, just less bandwidth-optimal). + */ +static int +parse_qresync_group(char *inner, struct imsg_mbox_select *req, + const char **errmsg) +{ + char *p, *tok; + uint32_t uidvalidity; + uint64_t modseq; + char *ep; + + p = inner; + while (*p == ' ') + p++; + tok = p; + while (*p != '\0' && *p != ' ') + p++; + if (*p != '\0') { + *p = '\0'; + p++; + } + if (parse_nz_number(tok, &uidvalidity) == -1) { + *errmsg = "invalid QRESYNC uidvalidity"; + return (-1); + } + + while (*p == ' ') + p++; + tok = p; + while (*p != '\0' && *p != ' ') + p++; + if (*p != '\0') { + *p = '\0'; + p++; + } + if (*tok == '\0') { + *errmsg = "QRESYNC requires uidvalidity and mod-sequence"; + return (-1); + } + errno = 0; + modseq = strtoull(tok, &ep, 10); + if (*ep != '\0' || errno != 0) { + *errmsg = "invalid QRESYNC mod-sequence"; + return (-1); + } + + req->qresync_uidvalidity = uidvalidity; + req->qresync_modseq = modseq; + req->qresync_has_uids = 0; + + while (*p == ' ') + p++; + if (*p == '\0') + return (0); + + if (*p == '(') { + *errmsg = "QRESYNC seq-match-data requires known-uids first"; + return (-1); + } + + tok = p; + while (*p != '\0' && *p != ' ') + p++; + if (*p != '\0') { + *p = '\0'; + p++; + } + if (strchr(tok, ',') != NULL) { + *errmsg = "comma-separated known-uids not supported in v1"; + return (-1); + } + { + uint32_t lo, hi; + int lo_star, hi_star; + + if (parse_seq_range(tok, &lo, &hi, &lo_star, &hi_star) == -1 || + lo_star || hi_star) { + *errmsg = "invalid known-uids -- '*' is not allowed " + "here (RFC 7162 SS3.2.5.1)"; + return (-1); + } + req->qresync_has_uids = 1; + req->qresync_uid_lo = lo; + req->qresync_uid_hi = hi; + } + + while (*p == ' ') + p++; + if (*p == '\0') + return (0); + + if (*p != '(') { + *errmsg = "expected seq-match-data"; + return (-1); + } + { + char *end = p; + int depth = 0; + + for (;;) { + if (*end == '(') + depth++; + else if (*end == ')') { + depth--; + if (depth == 0) + break; + } else if (*end == '\0') { + *errmsg = "unterminated seq-match-data"; + return (-1); + } + end++; + } + p = end + 1; + } + + while (*p == ' ') + p++; + if (*p != '\0') { + *errmsg = "unexpected data after QRESYNC arguments"; + return (-1); + } + + return (0); +} + +/* + * Parses the interior of SELECT/EXAMINE's select-params list (RFC 4466's + * generic syntax, extended by RFC 7162 SS3.1.8/SS3.2.5 with the CONDSTORE + * and QRESYNC select-params): zero or more space-separated params, where + * CONDSTORE is a bare token and QRESYNC is a token followed by its own + * parenthesized argument group (parsed above). p is already NUL-terminated + * at the outer list's closing paren by the caller (cmd_select()) and + * modified in place. + * + * Sets req->qresync (and fills the rest of req's qresync_* fields via + * parse_qresync_group()) and *want_condstore -- the latter is a plain + * out-parameter rather than something written straight to s->condstore_ + * enabled, since cmd_select() itself decides exactly when to flip that (see + * its own comment on why this doesn't go through session_condstore_ + * enable()). + * + * A QRESYNC select-param requires "ENABLE QRESYNC" to have already + * succeeded this connection (RFC 7162 SS3.2.5: tagged BAD otherwise) -- + * checked here against s->qresync_enabled, since by the time this SELECT's + * response could otherwise be built it would be too late to reject it + * cleanly. + */ +static int +parse_select_params(char *p, struct imsg_mbox_select *req, struct session *s, + int *want_condstore, const char **errmsg) +{ + *want_condstore = 0; + + while (*p != '\0') { + while (*p == ' ') + p++; + if (*p == '\0') + break; + + if (strncasecmp(p, "CONDSTORE", 9) == 0 && + (p[9] == '\0' || p[9] == ' ')) { + *want_condstore = 1; + p += 9; + continue; + } + + if (strncasecmp(p, "QRESYNC", 7) == 0 && + (p[7] == '\0' || p[7] == ' ')) { + char *q = p + 7; + char *end; + int depth; + + while (*q == ' ') + q++; + if (*q != '(') { + *errmsg = "QRESYNC requires a parenthesized " + "argument list"; + return (-1); + } + if (!s->qresync_enabled) { + *errmsg = "QRESYNC select-param requires " + "ENABLE QRESYNC first (RFC 7162 SS3.2.5)"; + return (-1); + } + + depth = 0; + end = q; + for (;;) { + if (*end == '(') + depth++; + else if (*end == ')') { + depth--; + if (depth == 0) + break; + } else if (*end == '\0') { + *errmsg = "unterminated QRESYNC " + "argument list"; + return (-1); + } + end++; + } + *end = '\0'; + + if (parse_qresync_group(q + 1, req, errmsg) == -1) + return (-1); + + req->qresync = 1; + *want_condstore = 1; /* SS3.2.3: QRESYNC implies + * CONDSTORE */ + p = end + 1; + continue; + } + + *errmsg = "unrecognized SELECT parameter"; + return (-1); + } + + return (0); +} + +/* + * RFC 9051 SS6.3.2: `select = "SELECT" SP mailbox`, extended by RFC 4466's + * generic select-param syntax and RFC 7162's CONDSTORE/QRESYNC select- + * params (SS3.1.8/SS3.2.5): `select = "SELECT" SP mailbox [SP "(" select- + * param *(SP select-param) ")"]`. args is whatever parse_command_line() + * left after the command name -- a bare atom (e.g. "INBOX", or, as of RFC + * 9051 SS6.3.4-SS6.3.6's flat multi-mailbox support, any other valid + * mailbox name too -- see handle_mbox_select()'s comment in store.c) in + * every real-world case, but RFC 9051's `mailbox` production also allows + * a quoted string or a literal. Only the quoted- + * string case is handled here (strip a single matching pair of double + * quotes, no backslash-escape decoding) -- a deliberate simplification in + * the same spirit as parse_command_line()'s own "not full ABNF conformance" + * comment, not an oversight; IMAP literals ({n}-prefixed octet counts) + * aren't supported anywhere in this codebase yet (see SESSION_INBUF_MAX's + * comment). + * + * The mailbox token's own boundary is found first (respecting a leading + * quote, so a quoted mailbox name doesn't get accidentally split at an + * internal space), *then* the existing quote-stripping logic runs on just + * that substring, unchanged from before this pass -- anything left over is + * handed to parse_select_params(). + * + * Shared by cmd_select() (readonly=0) and cmd_examine() (readonly=1): RFC + * 9051 SS6.3.3 says "The EXAMINE command is identical to SELECT and returns + * the same output" -- the only difference is that the selected mailbox + * ends up marked read-only, which this function threads through as + * req.readonly (to store.c, which SS6.3.2's own struct comment already + * notes doesn't need to do anything different with it) and s->mbox_readonly + * (to this session, consulted by session_handle_mbox_selected() for the + * READ-ONLY/READ-WRITE response code and PERMANENTFLAGS list, and by every + * command that can mutate the selected mailbox's permanent state). RFC 7162 + * SS3.1.8/SS3.2.5 extend both SELECT and EXAMINE with the identical + * select-param syntax, so CONDSTORE/QRESYNC support doesn't need to + * special-case either command. + */ +static int +select_or_examine(struct session *s, const char *tag, char *args, int readonly) +{ + struct imsg_mbox_select req; + char *mailbox_tok, *params, *p; + size_t len; + int want_condstore = 0; + const char *cmdname = readonly ? "EXAMINE" : "SELECT"; + + if (args == NULL) { + char text[40]; + + snprintf(text, sizeof(text), "%s requires a mailbox name", + cmdname); + session_reply(s, tag, "BAD", text); + return (1); + } + + p = args; + if (*p == '"') { + p++; + while (*p != '\0' && *p != '"') + p++; + if (*p == '"') + p++; + } else { + while (*p != '\0' && *p != ' ') + p++; + } + mailbox_tok = args; + if (*p == ' ') { + *p = '\0'; + p++; + while (*p == ' ') + p++; + params = (*p != '\0') ? p : NULL; + } else if (*p == '\0') { + params = NULL; + } else { + char text[40]; + + snprintf(text, sizeof(text), "malformed %s arguments", cmdname); + session_reply(s, tag, "BAD", text); + return (1); + } + + args = mailbox_tok; + len = strlen(args); + if (len >= 2 && args[0] == '"' && args[len - 1] == '"') { + args[len - 1] = '\0'; + args++; + len -= 2; + } + if (len == 0) { + session_reply(s, tag, "BAD", "empty mailbox name"); + return (1); + } + if (len >= sizeof(req.mailbox)) { + session_reply(s, tag, "BAD", "mailbox name too long"); + return (1); + } + + if (s->store_iev == NULL) { + /* Shouldn't happen -- ST_AUTH only dispatches here once + * SESSION_AUTHENTICATED/SELECTED, both of which require + * store_iev to already be wired (see listener_dispatch_ + * parent()'s IMSG_SETUP_PEER case) -- but this is exactly + * the kind of internal-invariant check this codebase prefers + * to state explicitly rather than silently assume. */ + log_warnx("session %u: %s with no store channel wired", + s->id, cmdname); + session_reply(s, tag, "NO", "[SERVERBUG] internal error"); + return (1); + } + + memset(&req, 0, sizeof(req)); + strlcpy(req.mailbox, args, sizeof(req.mailbox)); + req.readonly = readonly; + + if (params != NULL) { + size_t plen = strlen(params); + const char *errmsg = NULL; + + if (plen < 2 || params[0] != '(' || params[plen - 1] != ')') { + session_reply(s, tag, "BAD", + "malformed select-param list"); + return (1); + } + params[plen - 1] = '\0'; + + if (parse_select_params(params + 1, &req, s, &want_condstore, + &errmsg) == -1) { + session_reply(s, tag, "BAD", errmsg); + return (1); + } + } + + /* + * RFC 7162 SS3.1.8/SS3.2.3: a CONDSTORE or QRESYNC select-param + * enables CONDSTORE (and, for QRESYNC, QRESYNC too -- though that + * half only ever gets here already true, since parse_select_params() + * requires it up front) for this session immediately -- not routed + * through session_condstore_enable(), since that function's + * unsolicited-HIGHESTMODSEQ-if-already-selected behavior would be + * redundant here: this SELECT's own response (session_handle_mbox_ + * selected()) already includes HIGHESTMODSEQ as part of its normal + * sequence once s->condstore_enabled is set, which is exactly what + * this line accomplishes. + */ + if (want_condstore) + s->condstore_enabled = 1; + + /* + * RFC 9051 SS6.3.2: "The SELECT command automatically deselects any + * currently selected mailbox before attempting the new selection + * ... the server MUST return an untagged OK response with the + * '[CLOSED]' response code when the currently selected mailbox is + * closed." Known synchronously, from s->state right now (about to + * be overwritten below) -- no need to wait for store's reply to + * say this, unlike the rest of the required SELECT responses. + */ + if (s->state == SESSION_SELECTED) + session_untagged(s, "OK [CLOSED] Previous mailbox is now closed"); + + strlcpy(s->pending_tag, tag, sizeof(s->pending_tag)); + s->state = SESSION_SELECTING; + s->mbox_readonly = readonly; + strlcpy(s->selected_mailbox, args, sizeof(s->selected_mailbox)); + + if (imsg_compose(&s->store_iev->ibuf, IMSG_MBOX_SELECT, 0, 0, -1, + &req, sizeof(req)) == -1) + log_warn("session %u: imsg_compose IMSG_MBOX_SELECT", s->id); + imsgev_add(s->store_iev); + + return (1); +} + +static int +cmd_select(struct session *s, const char *tag, char *args) +{ + return select_or_examine(s, tag, args, 0); +} + +static int +cmd_examine(struct session *s, const char *tag, char *args) +{ + return select_or_examine(s, tag, args, 1); +} + +/* + * listener.c's own copy of store.c's mailbox_name_valid()/mailbox_name_ + * is_inbox() -- same rules (RFC 9051 SS5.1's CTL-character MAY, SS5.1.1's + * reserved "/" hierarchy delimiter, the length cap, and the "tmp"/"new"/ + * "cur" reserved-name hazard documented at store.c's own copy), so a + * client sending an obviously-invalid CREATE/DELETE/RENAME argument gets a + * fast BAD/NO with zero store round trip, the same "cheap client-side + * check before an async round trip" precedent cmd_status()'s INBOX-only + * check already established. Not a substitute for store.c's own check -- + * store.c re-validates independently once its request arrives (see that + * function's comment for the defense-in-depth rationale across the + * privsep boundary); this is purely a fast-path optimization on this + * side, so the two copies are kept deliberately in sync rather than + * shared via some new cross-file helper that would be its own scope + * creep. + */ +static int +mailbox_name_valid(const char *name) +{ + size_t i, len; + + len = strlen(name); + if (len == 0 || len >= MBOX_NAME_MAX) + return (0); + for (i = 0; i < len; i++) { + unsigned char c = (unsigned char)name[i]; + + if (c == '/') + return (0); + if (c < 0x20 || c == 0x7f) + return (0); + } + + if (strcmp(name, "tmp") == 0 || strcmp(name, "new") == 0 || + strcmp(name, "cur") == 0) + return (0); + + return (1); +} + +static int +mailbox_name_is_inbox(const char *name) +{ + return (strcasecmp(name, "INBOX") == 0); +} + +/* + * RFC 9051 SS6.3.4 CREATE: `create = "CREATE" SP mailbox` -- a single + * mailbox-name argument, same token shape as LIST/STATUS's own mailbox + * argument, so parse_list_token() is reused here too. "It is an error to + * attempt to create INBOX" and "It is an error to attempt to create a + * mailbox with a name that refers to an extant mailbox" -- the first is a + * client-side check (mailbox_name_is_inbox()); the second can only be + * answered by store.c, which owns the filesystem, so an extant-name + * refusal always costs a real round trip (mkdir(2)'s own EEXIST, per + * handle_mbox_create()'s design -- see docs/openimap-storage-backend.md + * item 10). + * + * Flat v1 has no hierarchy for a trailing delimiter to declare "create + * children of" (SS6.3.4's "If the mailbox name is suffixed with the + * hierarchy separator" clause) -- mailbox_name_valid() already refuses any + * name containing "/" outright, which also catches a trailing one, so + * that clause is unreachable here rather than silently ignored partway + * through. + * + * Async round trip: same shape as cmd_status()'s STATUS dispatch -- + * s->mbox_op_prev_state records whichever ST_AUTH state was current (CREATE + * is command-auth, RFC 9051 SS6.3, valid in Authenticated or Selected + * state, and never changes SELECTED-ness), s->pending_tag holds the tag, + * s->state transitions to SESSION_CREATING until store.c's terminal + * IMSG_MBOX_RESULT arrives at session_finish_mbox_op(). + */ +static int +cmd_create(struct session *s, const char *tag, char *args) +{ + struct imsg_mbox_create req; + char mailbox[MBOX_NAME_MAX]; + char *p; + const char *errmsg = NULL; + + if (args == NULL) { + session_reply(s, tag, "BAD", "CREATE requires a mailbox name"); + return (1); + } + + p = args; + if (parse_list_token(&p, mailbox, sizeof(mailbox), &errmsg) == -1) { + session_reply(s, tag, "BAD", errmsg); + return (1); + } + + if (mailbox_name_is_inbox(mailbox)) { + session_reply(s, tag, "NO", "[CANNOT] cannot create INBOX"); + return (1); + } + if (!mailbox_name_valid(mailbox)) { + session_reply(s, tag, "BAD", "invalid mailbox name"); + return (1); + } + + if (s->store_iev == NULL) { + log_warnx("session %u: CREATE with no store channel wired", + s->id); + session_reply(s, tag, "NO", "[SERVERBUG] internal error"); + return (1); + } + + memset(&req, 0, sizeof(req)); + strlcpy(req.mailbox, mailbox, sizeof(req.mailbox)); + + strlcpy(s->pending_tag, tag, sizeof(s->pending_tag)); + s->mbox_op_prev_state = s->state; + s->state = SESSION_CREATING; + + if (imsg_compose(&s->store_iev->ibuf, IMSG_MBOX_CREATE, 0, 0, -1, + &req, sizeof(req)) == -1) + log_warn("session %u: imsg_compose IMSG_MBOX_CREATE", s->id); + imsgev_add(s->store_iev); + + return (1); +} + +/* + * RFC 9051 SS6.3.5 DELETE: `delete = "DELETE" SP mailbox`. "It is an error + * to attempt to delete INBOX" and "It is an error to attempt to delete a + * mailbox that does not exist" -- same split as CREATE: INBOX is a client- + * side check, existence is store.c's (handle_mbox_delete()'s own stat(2) + * check -- see docs/openimap-storage-backend.md item 10). Flat v1's other + * DELETE sub-rules (inferior hierarchical names, the \Noselect-plus- + * children carve-out) are moot by construction -- no mailbox here can ever + * have children -- so nothing else needs checking on either side. + * + * Same async shape as cmd_create() -- see that function's comment. + */ +static int +cmd_delete(struct session *s, const char *tag, char *args) +{ + struct imsg_mbox_delete req; + char mailbox[MBOX_NAME_MAX]; + char *p; + const char *errmsg = NULL; + + if (args == NULL) { + session_reply(s, tag, "BAD", "DELETE requires a mailbox name"); + return (1); + } + + p = args; + if (parse_list_token(&p, mailbox, sizeof(mailbox), &errmsg) == -1) { + session_reply(s, tag, "BAD", errmsg); + return (1); + } + + if (mailbox_name_is_inbox(mailbox)) { + session_reply(s, tag, "NO", "[CANNOT] cannot delete INBOX"); + return (1); + } + if (!mailbox_name_valid(mailbox)) { + /* RFC 5530 NONEXISTENT: an invalid name can never have + * existed, same worked-example fit STATUS's own non-INBOX + * check already uses. */ + session_reply(s, tag, "NO", "[NONEXISTENT] no such mailbox"); + return (1); + } + + if (s->store_iev == NULL) { + log_warnx("session %u: DELETE with no store channel wired", + s->id); + session_reply(s, tag, "NO", "[SERVERBUG] internal error"); + return (1); + } + + memset(&req, 0, sizeof(req)); + strlcpy(req.mailbox, mailbox, sizeof(req.mailbox)); + + strlcpy(s->pending_tag, tag, sizeof(s->pending_tag)); + s->mbox_op_prev_state = s->state; + s->state = SESSION_DELETING; + + if (imsg_compose(&s->store_iev->ibuf, IMSG_MBOX_DELETE, 0, 0, -1, + &req, sizeof(req)) == -1) + log_warn("session %u: imsg_compose IMSG_MBOX_DELETE", s->id); + imsgev_add(s->store_iev); + + return (1); +} + +/* + * RFC 9051 SS6.3.6 RENAME: `rename = "RENAME" SP mailbox SP mailbox` + * (existing-name, new-name). "It is an error to attempt to rename from a + * mailbox name that does not exist or to a mailbox name that already + * exists" -- both existence checks are store.c's (handle_mbox_rename()'s + * own stat(2) checks). Renaming *to* INBOX is already covered by "already + * exists" (INBOX always exists for an authenticated session), so no + * separate client-side check is needed for the destination; renaming + * *from* INBOX is a distinct rule ("using the special name INBOX as the + * source... some servers disallow renaming INBOX") that v1 takes the + * RFC's own sanctioned refusal on -- see docs/openimap-storage-backend.md + * item 10 for the full citation -- checked here, client-side, same as + * CREATE/DELETE's own INBOX checks. + * + * Same async shape as cmd_create()/cmd_delete() -- see cmd_create()'s + * comment. + */ +static int +cmd_rename(struct session *s, const char *tag, char *args) +{ + struct imsg_mbox_rename req; + char oldname[MBOX_NAME_MAX]; + char newname[MBOX_NAME_MAX]; + char *p; + const char *errmsg = NULL; + + if (args == NULL) { + session_reply(s, tag, "BAD", + "RENAME requires two mailbox names"); + return (1); + } + + p = args; + if (parse_list_token(&p, oldname, sizeof(oldname), &errmsg) == -1) { + session_reply(s, tag, "BAD", errmsg); + return (1); + } + if (parse_list_token(&p, newname, sizeof(newname), &errmsg) == -1) { + session_reply(s, tag, "BAD", errmsg); + return (1); + } + + if (mailbox_name_is_inbox(oldname)) { + /* RFC 9051 SS6.3.6 explicitly sanctions this refusal -- see + * this function's header comment. RFC 5530 has no sharper + * fit than CANNOT ("The operation violates some invariant + * of the server and can never succeed"). */ + session_reply(s, tag, "NO", "[CANNOT] cannot rename INBOX"); + return (1); + } + if (!mailbox_name_valid(oldname)) { + session_reply(s, tag, "NO", "[NONEXISTENT] no such mailbox"); + return (1); + } + if (mailbox_name_is_inbox(newname) || !mailbox_name_valid(newname)) { + session_reply(s, tag, "BAD", "invalid mailbox name"); + return (1); + } + + if (s->store_iev == NULL) { + log_warnx("session %u: RENAME with no store channel wired", + s->id); + session_reply(s, tag, "NO", "[SERVERBUG] internal error"); + return (1); + } + + memset(&req, 0, sizeof(req)); + strlcpy(req.oldname, oldname, sizeof(req.oldname)); + strlcpy(req.newname, newname, sizeof(req.newname)); + + strlcpy(s->pending_tag, tag, sizeof(s->pending_tag)); + strlcpy(s->rename_oldname, oldname, sizeof(s->rename_oldname)); + strlcpy(s->rename_newname, newname, sizeof(s->rename_newname)); + s->mbox_op_prev_state = s->state; + s->state = SESSION_RENAMING; + + if (imsg_compose(&s->store_iev->ibuf, IMSG_MBOX_RENAME, 0, 0, -1, + &req, sizeof(req)) == -1) + log_warn("session %u: imsg_compose IMSG_MBOX_RENAME", s->id); + imsgev_add(s->store_iev); + + return (1); +} + +static int +cmd_subscribe(struct session *s, const char *tag, char *args) +{ + (void)args; + return stub_not_implemented(s, tag, "SUBSCRIBE"); +} + +static int +cmd_unsubscribe(struct session *s, const char *tag, char *args) +{ + (void)args; + return stub_not_implemented(s, tag, "UNSUBSCRIBE"); +} + +/* + * RFC 9051 SS6.3.9's wildcard matching ("*" matches zero or more + * characters including a hierarchy delimiter; "%" is the same but does + * NOT match a delimiter), restricted to what it actually needs to do in + * v1's flat, non-nested namespace: no delimiter can ever occur within any + * candidate name (mailbox_name_valid() refuses "/" outright for every + * real mailbox, and the fixed literal "INBOX" obviously has none either), + * so the one behavioral difference between "*" and "%" never has anything + * to bite on -- both are treated identically here as "match zero or more + * of anything." This is a real simplification specific to v1's flat + * namespace, not a general-purpose IMAP wildcard matcher; it would need + * to actually distinguish the two if this server ever grew a real nested + * hierarchy. SS6.3.9's further note that a trailing "%" also causes + * intermediate, not-yet-selectable levels of hierarchy to be returned + * with \Noselect doesn't apply here either, for the same reason: v1 has + * no intermediate levels for a trailing "%" to ever expose. + * + * ci (case-insensitive) is the caller's choice, not baked in: RFC 9051 + * SS5.1's case-insensitivity is scoped to the single reserved name INBOX + * ("The case-insensitive mailbox name INBOX..."), not to mailbox names in + * general -- real, named mailboxes (RFC 9051 SS6.3.4-SS6.3.6) are + * case-sensitive on this server, matching store.c's own strcmp() (not + * strcasecmp()) for every name but INBOX throughout mailbox_name_valid()/ + * handle_mbox_create()/handle_mbox_delete()/handle_mbox_rename(). Callers + * pass ci=1 only when matching against the literal "INBOX". + * + * Iterative two-pointer wildcard match with a single remembered backtrack + * point -- the standard technique behind glob(3)-family matchers, bounded + * at O(n*m) time in the worst case rather than the exponential blowup a + * naive recursive backtracking matcher (what this function used to be) + * hits on adversarial input. + * + * Found during manual security review: the previous implementation tried + * every possible split point for a wildcard via plain recursion, with no + * memoization. An alternating pattern like "a%a%a%...%a" matched against a + * same-shaped, non-fully-matching name is the textbook case for + * catastrophic backtracking in that style of matcher -- each wildcard's + * "try matching here, then here, then here..." loop calls back into a + * function that repeats the same search over the remaining wildcards, + * multiplying out exponentially. Both operands here are attacker-reachable + * by an authenticated user (a mailbox name via CREATE, up to MBOX_NAME_MAX + * bytes; the LIST pattern itself, up to 2*MBOX_NAME_MAX bytes as `canon`), + * and listener.c runs one event loop across every session, so a single + * pathological LIST could have stalled the process for every connected + * session, not just the one that sent it -- not a memory-safety bug, but a + * real CPU-exhaustion DoS. + * + * This rewrite keeps this function's own established, deliberate + * simplifications unchanged: "*" and "%" are still treated identically + * (see the header comment above -- v1's flat namespace makes the RFC's + * delimiter-crossing distinction between them moot), and ci is still the + * caller's choice, not baked in. Only the matching algorithm itself + * changed, not what it matches. + * + * Standard algorithm: walk pat and name together; on a literal mismatch, + * if a wildcard was seen earlier, retry from just after that wildcard + * with one additional character of name absorbed by it (star_s tracks how + * much the most recent wildcard has already absorbed) rather than + * recursing into a fresh search. Because star_s only ever advances + * forward, the whole scan is bounded by name's length times the number of + * wildcards in pat, not exponential in either. + */ +static int +list_pattern_match(const char *pat, const char *name, int ci) +{ + const char *p = pat; + const char *s = name; + const char *star_p = NULL; /* pat position just past the most + * recently seen wildcard run */ + const char *star_s = NULL; /* name position that wildcard has + * absorbed through so far */ + + /* + * Loop is driven by "any name bytes left to consume", exactly like + * the reference iterative algorithm (LeetCode-style "wildcard + * matching", translated to pointers) -- not an unconditional loop + * with internal breaks. That distinction matters: an earlier draft + * of this fix used a for(;;) with breaks and left star_s un-updated + * on an ordinary character match, so a stale star_s could still + * read non-'\0' after p and s both legitimately reached the end, + * triggering a spurious extra backtrack that broke any pattern with + * a literal after a wildcard (e.g. "A*Z" against "AhelloZ") -- + * caught by this fix's own standalone regression test before ever + * reaching store.c/listener.c, not by inspection. + */ + while (*s != '\0') { + if (*p == '*' || *p == '%') { + while (*p == '*' || *p == '%') + p++; + star_p = p; + star_s = s; + } else if (*p != '\0' && (ci ? + toupper((unsigned char)*p) == toupper((unsigned char)*s) : + *p == *s)) { + p++; + s++; + } else if (star_p != NULL) { + star_s++; + s = star_s; + p = star_p; + } else { + return (0); + } + } + + while (*p == '*' || *p == '%') + p++; + return (*p == '\0'); +} + +/* + * Pulls one `list-mailbox`-shaped token (RFC 9051 SS9: `list-mailbox = + * 1*list-char / string`) off *pp, advancing *pp past it -- used for both + * of LIST's basic-syntax positional arguments (reference name, mailbox + * pattern). The quoted-string case strips one matching pair of DQUOTEs + * with no backslash-escape decoding, same deliberate simplification + * cmd_select()'s own mailbox-argument comment already documents; the + * unquoted case is simply "everything up to the next space", which is a + * superset of "1*list-char" (ATOM-CHAR / list-wildcards / resp-specials) + * that doesn't bother validating individual characters -- consistent + * with this file's general practice of not fully policing atom-syntax + * conformance. The `string` grammar alternative also allows an IMAP + * literal ("{n}"-prefixed) reference/pattern; not supported here, same + * gap cmd_select()'s mailbox argument already has. + * + * An empty quoted string ("") is a legal zero-length token and is + * accepted here -- both LIST's basic-syntax empty-mailbox special case + * and an empty reference argument (RFC 9051: "Clients SHOULD use the + * empty reference argument") depend on that. + */ +static int +parse_list_token(char **pp, char *out, size_t outsize, const char **errmsg) +{ + char *p = *pp; + + while (*p == ' ') + p++; + + if (*p == '\0') { + *errmsg = "LIST requires two arguments"; + return (-1); + } + + if (*p == '"') { + char *start = p + 1; + char *end = strchr(start, '"'); + size_t len; + + if (end == NULL) { + *errmsg = "unterminated quoted string"; + return (-1); + } + len = (size_t)(end - start); + if (len >= outsize) { + *errmsg = "argument too long"; + return (-1); + } + memcpy(out, start, len); + out[len] = '\0'; + p = end + 1; + } else { + char *start = p; + size_t len; + + while (*p != '\0' && *p != ' ') + p++; + len = (size_t)(p - start); + if (len >= outsize) { + *errmsg = "argument too long"; + return (-1); + } + memcpy(out, start, len); + out[len] = '\0'; + } + + *pp = p; + return (0); +} + +/* + * RFC 9051 SS6.3.9: basic syntax only -- `list = "LIST" SP mailbox SP + * mbox-or-pat` where mbox-or-pat here is a single list-mailbox, not the + * parenthesized `patterns` alternative, and with no list-select-opts or + * list-return-opts. Extended syntax (detected per SS6.3.9's own three + * conditions: select-opts as the first word, a parenthesized pattern + * list as the second word, or more than two parameters) is recognized + * but rejected with NO, not BAD -- openimap-v1-dispatch.md's own scope + * note for LIST is "v1 should support at minimum the basic form + * cleanly", so this is a deliberate, flagged scope cut, not a syntax + * error. + * + * INBOX itself is answered synchronously, with no store round trip -- + * it always exists unconditionally for any authenticated session (there's + * no CREATE that could make its existence conditional), so matching it is + * pure string/wildcard matching against the fixed name "INBOX", nothing + * store.c needs to be asked about. Real, named mailboxes (RFC 9051 + * SS6.3.4-SS6.3.6 multi-mailbox support, docs/openimap-storage-backend.md + * item 10) are a different story -- only store.c can enumerate what + * actually exists on disk in this session's own maildir root, so those go + * through a real IMSG_MBOX_LIST/SESSION_LISTING round trip (see this + * function's tail and session_finish_list()/session_handle_mbox_list_ + * item()) once the INBOX-only synchronous check above is done. + * + * LSUB (RFC 3501 -- dropped from base IMAP4rev2, but still sent by real + * clients for back-compat, e.g. Apple Mail's very first mailbox-listing + * round trip): real bug caught testing against Apple Mail live on + * premio -- LSUB wasn't in the dispatch table at all, so it fell through + * to session_handle_line()'s generic "Unknown command" BAD, which (paired + * with the FETCH gap fixed just below) left Mail with literally nothing + * to render, hence a blank Inbox. v1 has no subscription list to + * maintain (no UNSUBSCRIBE, no per-mailbox subscribed bit stored + * anywhere) and only one mailbox that could ever be subscribed to, so + * "LSUB matched X" and "LIST matched X, and X happens to always be + * subscribed" are indistinguishable in this server -- is_lsub only + * changes the response keyword (LSUB vs LIST) and the completion text, + * not the matching logic itself. + */ +static int +list_dispatch(struct session *s, const char *tag, char *args, int is_lsub) +{ + char reference[MBOX_NAME_MAX]; + char pattern[MBOX_NAME_MAX]; + char canon[2 * MBOX_NAME_MAX]; + char *p; + const char *errmsg; + const char *cmdname = is_lsub ? "LSUB" : "LIST"; + const char *kw = is_lsub ? "LSUB" : "LIST"; + char text[64]; + + if (args == NULL) { + snprintf(text, sizeof(text), "%s requires two arguments", + cmdname); + session_reply(s, tag, "BAD", text); + return (1); + } + + p = args; + while (*p == ' ') + p++; + + if (*p == '(') { + /* SS6.3.9 condition 1: "the first word after the command + * name begins with a parenthesis" -- list-select-opts. */ + snprintf(text, sizeof(text), + "extended %s selection options not supported", cmdname); + session_reply(s, tag, "NO", text); + return (1); + } + + if (parse_list_token(&p, reference, sizeof(reference), &errmsg) == + -1) { + session_reply(s, tag, "BAD", errmsg); + return (1); + } + + while (*p == ' ') + p++; + + if (*p == '(') { + /* SS6.3.9 condition 2: "the second word after the command + * name begins with a parenthesis" -- the parenthesized + * `patterns` form of mbox-or-pat, not a single + * list-mailbox. */ + snprintf(text, sizeof(text), + "extended %s mailbox-pattern lists not supported", + cmdname); + session_reply(s, tag, "NO", text); + return (1); + } + + if (parse_list_token(&p, pattern, sizeof(pattern), &errmsg) == -1) { + session_reply(s, tag, "BAD", errmsg); + return (1); + } + + while (*p == ' ') + p++; + if (*p != '\0') { + /* SS6.3.9 condition 3: "the LIST command has more than 2 + * parameters" -- trailing list-return-opts. */ + snprintf(text, sizeof(text), + "extended %s return options not supported", cmdname); + session_reply(s, tag, "NO", text); + return (1); + } + + /* + * RFC 9051 SS6.3.9: "In the basic syntax only, an empty ('' string) + * mailbox name argument is a special request to return the + * hierarchy delimiter and the root name of the name given in the + * reference... The value returned as the root MAY be the empty + * string if the reference is non-rooted or is an empty string." + * v1 has no rooting/hierarchy concept at all to resolve a non-empty + * reference against -- there is still no real folder tree to root + * anything in, only INBOX -- so taking the RFC's own "MAY be empty" + * allowance and always returning an empty root, regardless of the + * reference argument's contents, is a real simplification but a + * spec-permitted one, not a violation. "/" is the same real + * hierarchy delimiter SELECT's own untagged LIST response and + * cmd_namespace()'s NAMESPACE response both use -- a resolved + * project decision as of a later pass (openimap-storage-backend.md, + * "Open items carried from this session" #9), not a borrowed + * placeholder anymore. + */ + if (pattern[0] == '\0') { + snprintf(text, sizeof(text), "%s (\\Noselect) \"/\" \"\"", kw); + session_untagged(s, text); + snprintf(text, sizeof(text), "%s completed", cmdname); + session_reply(s, tag, "OK", text); + return (1); + } + + /* + * Canonical LIST pattern: reference concatenated with the mailbox + * pattern. RFC 9051 SS6.3.9: "If a server implementation has no + * concept of break out characters, the canonical form is normally + * the reference name appended with the mailbox name" -- squarely + * true here: v1 has no "current working directory"/break-out- + * character concept at all (a single flat INBOX-only namespace), so + * plain concatenation is the correct reading of that sentence for + * this server, not just the convenient one. + */ + { + size_t n; + + n = strlcpy(canon, reference, sizeof(canon)); + if (n < sizeof(canon)) + n = strlcat(canon, pattern, sizeof(canon)); + if (n >= sizeof(canon)) { + session_reply(s, tag, "BAD", + "combined reference and pattern too long"); + return (1); + } + } + + /* + * SS6.3.9: "Any syntactically valid pattern that is not accepted by + * a server for any reason MUST be silently ignored, i.e., it + * results in no LIST responses, and the LIST command still returns + * a tagged OK response." Answered synchronously and immediately for + * INBOX specifically -- it always exists for any authenticated + * session (there's no CREATE that could make its existence + * conditional), so matching it needs no store round trip, same + * "nothing store.c needs to be asked about" reasoning this + * function's header comment already gives. + */ + if (list_pattern_match(canon, "INBOX", 1)) { + /* "()" -- no attributes -- the same choice, for the same + * reason, as SELECT's own untagged LIST response: INBOX has + * no children and is selectable, and SS7.3.1 makes every + * attribute here optional ("MAY send none of these"). */ + snprintf(text, sizeof(text), "%s () \"/\" INBOX", kw); + session_untagged(s, text); + } + + if (s->store_iev == NULL) { + log_warnx("session %u: %s with no store channel wired", + s->id, cmdname); + session_reply(s, tag, "NO", "[SERVERBUG] internal error"); + return (1); + } + + /* + * RFC 9051 SS6.3.4-SS6.3.6 multi-mailbox support (docs/openimap- + * storage-backend.md item 10): beyond INBOX, real mailbox names now + * live on disk, and only store.c can enumerate them (this session's + * own maildir root, chrooted/unveiled per-session -- listener.c has + * no filesystem access of its own to do this locally the way the + * INBOX-only check above still can). IMSG_MBOX_LIST takes no + * request payload (handle_mbox_list() just opendir(2)s "."), so + * nothing needs to be built here beyond the imsg itself -- s->list_ + * pattern (the same canonical reference+pattern concatenation just + * used for the INBOX check above) and s->list_is_lsub are stashed + * for session_handle_mbox_list_item() to test each streamed name + * against as it arrives. + */ + strlcpy(s->pending_tag, tag, sizeof(s->pending_tag)); + strlcpy(s->list_pattern, canon, sizeof(s->list_pattern)); + s->list_is_lsub = is_lsub; + s->mbox_op_prev_state = s->state; + s->state = SESSION_LISTING; + + if (imsg_compose(&s->store_iev->ibuf, IMSG_MBOX_LIST, 0, 0, -1, + NULL, 0) == -1) + log_warn("session %u: imsg_compose IMSG_MBOX_LIST", s->id); + imsgev_add(s->store_iev); + + return (1); +} + +static int +cmd_list(struct session *s, const char *tag, char *args) +{ + return list_dispatch(s, tag, args, 0); +} + +static int +cmd_lsub(struct session *s, const char *tag, char *args) +{ + return list_dispatch(s, tag, args, 1); +} + +/* + * RFC 9051 SS6.3.10 NAMESPACE: `namespace-response = "NAMESPACE" SP + * namespace SP namespace SP namespace` -- Personal, then Other Users', + * then Shared, each either NIL or a parenthesized list of `(prefix + * delimiter)` pairs. Answered entirely locally, no store round trip, same + * "v1 is single-mailbox, nothing for store.c to be asked" precedent as + * cmd_list() -- NAMESPACE doesn't even take a mailbox-name argument for + * store.c to validate the way LIST/STATUS/SELECT do. + * + * v1's hierarchy delimiter ("/") and Personal Namespace prefix ("") are + * now a real, sourced design decision (openimap-storage-backend.md, "Open + * items carried from this session" #9) rather than the placeholder + * SELECT's own LIST response had been borrowing -- this is RFC 9051 + * SS6.3.10's own Example 1, verbatim: "a server supports a single + * Personal Namespace. No leading prefix is used on personal mailboxes, + * and '/' is the hierarchy delimiter" -> `* NAMESPACE (("" "/")) NIL + * NIL`. No Other Users' Namespace or Shared Namespace (both NIL) -- v1 is + * single-user with no shared-mailbox concept at all, so there's nothing + * for either to expose. Arguments are ignored (RFC 9051 SS6.3.10: + * "Arguments: none" -- session_handle_line()'s generic parser already + * hands cmd_*() functions whatever trailed the command name, if + * anything, same as e.g. cmd_capability() does; a client sending garbage + * after NAMESPACE gets a clean OK rather than a pedantic BAD, consistent + * with this codebase's existing leniency elsewhere for command-any/ + * command-auth commands that formally take no arguments). + */ +static int +cmd_namespace(struct session *s, const char *tag, char *args) +{ + (void)args; + session_untagged(s, "NAMESPACE ((\"\" \"/\")) NIL NIL"); + session_reply(s, tag, "OK", "NAMESPACE command completed"); + return (1); +} + +/* + * RFC 9051 SS6.3.11 STATUS: `status = "STATUS" SP mailbox SP "(" + * status-att *(SP status-att) ")"`. "does not change the currently + * selected mailbox, nor does it affect the state of any messages" -- + * SESSION_STATUSING is a purely transient async-wait state (see that + * enum value's comment); s->status_prev_state records whichever ST_AUTH + * state (SESSION_AUTHENTICATED or SESSION_SELECTED) was current so + * session_handle_mbox_status_result() can restore it, same pattern + * cmd_append() already established for the identical "command-auth, not + * command-select" situation. + * + * Mailbox-name argument reuses parse_list_token() (LIST's own token + * parser -- STATUS's `mailbox` production is the same ABNF shape as one + * of LIST's two arguments). RFC 9051 SS6.3.4-SS6.3.6 multi-mailbox support + * (docs/openimap-storage-backend.md item 10) means a non-INBOX name can now + * genuinely exist, so the only client-side check left is mailbox_name_ + * valid() (obviously-malformed names get a fast NONEXISTENT with zero store + * round trip, same as CREATE/DELETE/RENAME's own fast path) -- actual + * existence is store.c's call now, the same "delegates the check to + * store.c because it needs a round trip regardless" reasoning cmd_select() + * already used, not the old INBOX-only shortcut this comment used to + * describe. + */ +static int +cmd_status(struct session *s, const char *tag, char *args) +{ + struct imsg_mbox_status req; + char mailbox[MBOX_NAME_MAX]; + char *p, *atts, *tok, *save; + char attbuf[256]; + const char *errmsg = NULL; + uint32_t attrs = 0; + size_t plen; + + if (args == NULL) { + session_reply(s, tag, "BAD", + "STATUS requires a mailbox name and status-att list"); + return (1); + } + + p = args; + if (parse_list_token(&p, mailbox, sizeof(mailbox), &errmsg) == -1) { + session_reply(s, tag, "BAD", errmsg); + return (1); + } + + while (*p == ' ') + p++; + atts = p; + plen = strlen(atts); + if (plen < 2 || atts[0] != '(' || atts[plen - 1] != ')') { + session_reply(s, tag, "BAD", + "STATUS requires a parenthesized status-att list"); + return (1); + } + atts[plen - 1] = '\0'; + atts++; + + if (strlcpy(attbuf, atts, sizeof(attbuf)) >= sizeof(attbuf)) { + session_reply(s, tag, "BAD", "status-att list too long"); + return (1); + } + for (tok = strtok_r(attbuf, " ", &save); tok != NULL; + tok = strtok_r(NULL, " ", &save)) { + if (strcasecmp(tok, "MESSAGES") == 0) + attrs |= STATUS_ATT_MESSAGES; + else if (strcasecmp(tok, "UIDNEXT") == 0) + attrs |= STATUS_ATT_UIDNEXT; + else if (strcasecmp(tok, "UIDVALIDITY") == 0) + attrs |= STATUS_ATT_UIDVALIDITY; + else if (strcasecmp(tok, "UNSEEN") == 0) + attrs |= STATUS_ATT_UNSEEN; + else if (strcasecmp(tok, "DELETED") == 0) + attrs |= STATUS_ATT_DELETED; + else if (strcasecmp(tok, "SIZE") == 0) + attrs |= STATUS_ATT_SIZE; + else if (strcasecmp(tok, "HIGHESTMODSEQ") == 0) + attrs |= STATUS_ATT_HIGHESTMODSEQ; + else { + session_reply(s, tag, "BAD", "unknown status-att"); + return (1); + } + } + + /* + * RFC 9051 SS9's `status-att-list` ABNF is `status-att *(SP + * status-att)` on the request side -- at least one is required, + * unlike the *response* side's `mailbox-data` production, `"STATUS" + * SP mailbox SP "(" [status-att-list] ")"`, whose square brackets + * explicitly allow empty parens. An empty "()" here is therefore a + * client syntax error, not a legal "ask for nothing" request. + */ + if (attrs == 0) { + session_reply(s, tag, "BAD", + "STATUS requires at least one status-att"); + return (1); + } + + if (!mailbox_name_is_inbox(mailbox) && !mailbox_name_valid(mailbox)) { + /* RFC 5530: NONEXISTENT -- its own worked example is + * literally "No such mailbox". A malformed name can never + * have existed, so this is answered client-side, same fast + * path CREATE/DELETE/RENAME use -- an otherwise-valid name + * that simply doesn't exist on disk is store.c's call now + * (handle_mbox_status()'s own select_mailbox_dir() failure + * path), not this one's. */ + session_reply(s, tag, "NO", "[NONEXISTENT] no such mailbox"); + return (1); + } + + if (s->store_iev == NULL) { + /* Same internal-invariant check as cmd_select()/cmd_fetch()/ + * cmd_store_cmd() -- ST_AUTH requires store_iev to already + * be wired. */ + log_warnx("session %u: STATUS with no store channel wired", + s->id); + session_reply(s, tag, "NO", "[SERVERBUG] internal error"); + return (1); + } + + /* + * RFC 7162 SS3.1: STATUS (HIGHESTMODSEQ) is one of the six + * CONDSTORE-enabling commands. Called synchronously here, before + * s->state is overwritten below, matching fetch_dispatch()/ + * store_do()'s own ordering -- session_condstore_enable()'s + * unsolicited-HIGHESTMODSEQ-if-already-selected check depends on + * s->state == SESSION_SELECTED still being whatever it was when + * this command was dispatched. + */ + if (attrs & STATUS_ATT_HIGHESTMODSEQ) + session_condstore_enable(s); + + memset(&req, 0, sizeof(req)); + strlcpy(req.mailbox, mailbox, sizeof(req.mailbox)); + req.attrs = attrs; + + strlcpy(s->pending_tag, tag, sizeof(s->pending_tag)); + strlcpy(s->status_mailbox, mailbox, sizeof(s->status_mailbox)); + s->status_attrs = attrs; + s->status_prev_state = s->state; + s->state = SESSION_STATUSING; + + if (imsg_compose(&s->store_iev->ibuf, IMSG_MBOX_STATUS, 0, 0, -1, + &req, sizeof(req)) == -1) + log_warn("session %u: imsg_compose IMSG_MBOX_STATUS", s->id); + imsgev_add(s->store_iev); + + return (1); +} + +/* + * v1's whole-message-in-one-imsg design (imapd.h's imsg_mbox_append + * comment) needs the announced literal size to fit inside a single imsg. + * Confirmed directly against the real imsg.c's imsg_create() this + * session: `datalen += IMSG_HEADER_SIZE; if (datalen > imsgbuf->maxsize) + * ... return NULL`, with maxsize set to MAX_IMSGSIZE (16384, imsg.h) by + * imsgev_init(). struct imsg_mbox_append's own fixed fields (mailbox + * MBOX_NAME_MAX=256, keywords MBOX_FLAGS_MAX=256, plus a handful of + * ints/int64s) come to a bit over 500 bytes; 12000 leaves comfortable + * headroom under that 16384 ceiling for both that and the imsg header + * itself, while still covering v1's expected "personal notes/drafts/ + * small saved messages" use case. A message larger than that needs real + * fd-passing, not implemented this pass; rejected with a plain NO (RFC + * 9051 defines no response code for a size cap) rather than truncating or + * crashing. + * + * Now defined in imapd.h (moved there when BODY.PEEK[]/BODY.PEEK[TEXT] + * were added to store.c's read_message_body()): no message can legally + * exist on disk larger than this cap in the first place, since APPEND is + * v1's only way to create one, so reusing the same symbol for reading a + * message back out -- rather than inventing a second, independently- + * maintained size constant -- keeps the two enforcement points from ever + * silently drifting apart. + */ + +/* Defined further down alongside format_internaldate(); forward-declared + * here so parse_date_time() (which needs the same month-name table, just + * in the reverse direction) can reuse it instead of duplicating it. */ +static const char *fetch_month_names[12]; + +/* + * Reverse of format_internaldate(): parses an RFC 9051 SS9 `date-time` + * (already stripped of its surrounding DQUOTEs by the caller) -- + * `date-day-fixed "-" date-month "-" date-year SP time SP zone`. Uses + * plain C89 sscanf(3), not anything OpenBSD-specific -- unlike strlcpy() + * etc. elsewhere in this file, this doesn't need separate verification + * against real OpenBSD headers. "%2d" on a space-padded single-digit day + * (SS9's `date-day-fixed = (SP DIGIT) / 2DIGIT`) works without special- + * casing: scanf's leading-whitespace skip for %d isn't counted against + * the field width, so " 7" and "07" both parse to day=7. "%3s" for the + * month name relies on month abbreviations always being exactly 3 letters + * (SS9's `date-month` production lists only "Jan".."Dec") to land exactly + * on the following "-" with no whitespace between them. + * + * Builds a struct tm from the fixed-format fields and uses timegm(3) (UTC + * interpretation, no DST, no local-timezone dependency -- appropriate + * since the fields already carry their own explicit zone offset) to get + * a UTC instant for the wall-clock fields alone, then subtracts the + * parsed zone offset to get the true UTC timestamp: a "-0800" zone means + * the wall-clock time is 8 hours behind UTC, so UTC = wall_time - + * (-8h) = wall_time + 8h, i.e. timegm(tm) - zone_offset_seconds where + * zone_offset_seconds is already negative for a "-" zone. + * + * Only coarse range checks (day 1-31, hour/minute/second in range, a + * recognized month name, a 4-digit year not before the Unix epoch) -- + * calendar validity (e.g. "31-Feb-2026") is NOT checked; timegm(3) + * normalizes an out-of-range day forward rather than erroring, which is + * accepted here as reasonable non-error behavior rather than treated as + * a bug worth guarding against, consistent with this file's general + * "reject clearly malformed input, don't chase every edge case" style. + */ +static int +parse_date_time(const char *s, int64_t *out) +{ + struct tm tm; + char mon[4]; + int day, year, hh, mm, ss, zh, zm, i; + char zsign; + time_t t; + int64_t zoff; + + memset(&tm, 0, sizeof(tm)); + + if (sscanf(s, "%2d-%3s-%4d %2d:%2d:%2d %c%2d%2d", &day, mon, &year, + &hh, &mm, &ss, &zsign, &zh, &zm) != 9) + return (-1); + + if (day < 1 || day > 31 || hh < 0 || hh > 23 || mm < 0 || mm > 59 || + ss < 0 || ss > 60 || year < 1970 || + (zsign != '+' && zsign != '-')) + return (-1); + + for (i = 0; i < 12; i++) { + if (strcasecmp(mon, fetch_month_names[i]) == 0) + break; + } + if (i == 12) + return (-1); + + tm.tm_mday = day; + tm.tm_mon = i; + tm.tm_year = year - 1900; + tm.tm_hour = hh; + tm.tm_min = mm; + tm.tm_sec = ss; + + if ((t = timegm(&tm)) == (time_t)-1) + return (-1); + + zoff = (int64_t)zh * 3600 + (int64_t)zm * 60; + if (zsign == '-') + zoff = -zoff; + + *out = (int64_t)t - zoff; + return (0); +} + +/* Parsed result of parse_append_args() below -- kept together as one + * struct, unlike parse_seq_range()'s several out-parameters, simply + * because there are too many fields here for that style to stay + * readable. */ +struct append_parsed { + char mailbox[MBOX_NAME_MAX]; + uint32_t sysflags; + char keywords[MBOX_FLAGS_MAX]; + int has_date; + int64_t date; + uint64_t litlen; + int litnonsync; +}; + +/* + * RFC 9051 SS6.3.12: `append = "APPEND" SP mailbox [SP flag-list] [SP + * date-time] SP literal`. A fixed positional grammar (mailbox, then an + * optional flag-list, then an optional date-time, then a mandatory + * literal, in exactly that order) -- parsed here as a straightforward + * left-to-right scan over args (modified in place), not a generic + * tokenizer, since a quoted date-time string can itself contain spaces + * (ruling out simple whitespace-splitting the way cmd_fetch()'s sequence- + * set/fetch-att parsing gets away with) and a flag-list's own internal + * grammar is already handled by parse_store_flags() (reused here + * directly, since SS6.3.12's flag-list is syntactically identical to + * STORE's). + * + * The mailbox token accepts a bare atom or a single quoted string (no + * backslash-escape decoding) -- same deliberate simplification cmd_ + * select() already documents; APPEND's grammar also allows a literal + * mailbox name, not supported here for the same reason cmd_select() + * doesn't support one either. + * + * The literal is required to be the final thing on the line -- true by + * construction per this grammar (nothing follows `literal` in `append`), + * so this isn't actually a simplification, just an explicit check that + * catches a malformed line (trailing garbage after the "{n}") with a + * clear BAD instead of silently ignoring it. + * + * Returns 0 on success, -1 (BAD) or -2 (NO) with *errmsg set on failure -- + * same convention as parse_fetch_atts()/parse_store_flags(). + */ +static int +parse_append_args(char *args, struct append_parsed *out, const char **errmsg) +{ + char *p = args; + + memset(out, 0, sizeof(*out)); + *errmsg = NULL; + + if (p == NULL || *p == '\0') { + *errmsg = "APPEND requires a mailbox name"; + return (-1); + } + + while (*p == ' ') + p++; + if (*p == '"') { + char *start = p + 1; + char *end = strchr(start, '"'); + size_t len; + + if (end == NULL) { + *errmsg = "unterminated quoted mailbox name"; + return (-1); + } + len = (size_t)(end - start); + if (len == 0) { + *errmsg = "empty mailbox name"; + return (-1); + } + if (len >= sizeof(out->mailbox)) { + *errmsg = "mailbox name too long"; + return (-1); + } + memcpy(out->mailbox, start, len); + out->mailbox[len] = '\0'; + p = end + 1; + } else { + char *start = p; + size_t len; + + while (*p != '\0' && *p != ' ') + p++; + len = (size_t)(p - start); + if (len == 0) { + *errmsg = "empty mailbox name"; + return (-1); + } + if (len >= sizeof(out->mailbox)) { + *errmsg = "mailbox name too long"; + return (-1); + } + memcpy(out->mailbox, start, len); + out->mailbox[len] = '\0'; + } + + while (*p == ' ') + p++; + if (*p == '(') { + char *start = p; + char *end = strchr(p, ')'); + char saved; + int rc; + const char *sub_err; + + if (end == NULL) { + *errmsg = "unterminated flag list"; + return (-1); + } + end++; /* include the ')' itself in the substring below */ + saved = *end; + *end = '\0'; + rc = parse_store_flags(start, &out->sysflags, out->keywords, + sizeof(out->keywords), &sub_err); + *end = saved; + if (rc != 0) { + *errmsg = sub_err; + return (rc); + } + p = end; + while (*p == ' ') + p++; + } + + if (*p == '"') { + char *start = p + 1; + char *end = strchr(start, '"'); + size_t dlen; + char datebuf[64]; + + if (end == NULL) { + *errmsg = "unterminated date-time string"; + return (-1); + } + dlen = (size_t)(end - start); + if (dlen >= sizeof(datebuf)) { + *errmsg = "date-time string too long"; + return (-1); + } + memcpy(datebuf, start, dlen); + datebuf[dlen] = '\0'; + if (parse_date_time(datebuf, &out->date) == -1) { + *errmsg = "malformed date-time string"; + return (-1); + } + out->has_date = 1; + p = end + 1; + while (*p == ' ') + p++; + } + + if (*p != '{') { + *errmsg = "expected a message literal"; + return (-1); + } + { + char *start = p + 1; + char *end = strchr(start, '}'); + char *digits_end; + char *digits_stop; + char digitsbuf[24]; + size_t digits_len; + unsigned long long litlen; + + if (end == NULL) { + *errmsg = "malformed literal announcement"; + return (-1); + } + + out->litnonsync = (end > start && end[-1] == '+'); + digits_stop = out->litnonsync ? end - 1 : end; + digits_len = (size_t)(digits_stop - start); + + if (digits_len == 0 || digits_len >= sizeof(digitsbuf)) { + *errmsg = "malformed literal octet count"; + return (-1); + } + memcpy(digitsbuf, start, digits_len); + digitsbuf[digits_len] = '\0'; + + errno = 0; + litlen = strtoull(digitsbuf, &digits_end, 10); + if (*digits_end != '\0' || errno == ERANGE) { + *errmsg = "malformed literal octet count"; + return (-1); + } + out->litlen = (uint64_t)litlen; + + if (out->litnonsync && out->litlen > 4096) { + /* RFC 9051 SS4.3: "non-synchronizing literals MUST + * NOT be larger than 4096 octets. Any literal larger + * than 4096 bytes MUST be sent as a synchronizing + * literal." A client violating this is a protocol + * error, not merely an oversized message -- BAD, not + * the plain-NO size-cap rejection cmd_append() does + * separately for APPEND_LITERAL_MAX. */ + *errmsg = "non-synchronizing literal exceeds RFC " + "9051 SS4.3's 4096-octet limit -- use a " + "synchronizing literal instead"; + return (-1); + } + + if (end[1] != '\0') { + *errmsg = "literal must be the final argument"; + return (-1); + } + } + + return (0); +} + +/* + * RFC 9051 SS6.3.12: `append = "APPEND" SP mailbox [SP flag-list] [SP + * date-time] SP literal`. Parses everything up through the literal + * announcement via parse_append_args(), then -- if a literal was found -- + * allocates s->literal_buf and switches the session into the client- + * literal-read phase (s->literal_pending; see session_dispatch_client()'s + * header comment on that block for why this needs no new dispatch-table + * state). The actual IMSG_MBOX_APPEND isn't sent from here: that happens + * once the full literal (plus its trailing CRLF) has actually been read, + * in session_finish_append(). + */ +static int +cmd_append(struct session *s, const char *tag, char *args) +{ + struct append_parsed parsed; + int rc; + const char *errmsg; + + rc = parse_append_args(args, &parsed, &errmsg); + if (rc == -1) { + session_reply(s, tag, "BAD", errmsg); + return (1); + } + if (rc == -2) { + session_reply(s, tag, "NO", errmsg); + return (1); + } + + if (parsed.litlen > APPEND_LITERAL_MAX) { + /* RFC 5530: LIMIT -- "The operation ran up against an + * implementation limit of some kind," precisely + * APPEND_LITERAL_MAX's own situation. */ + session_reply(s, tag, "NO", + "[LIMIT] message too large for this server (v1 size " + "limit -- see APPEND_LITERAL_MAX)"); + return (1); + } + + if (s->store_iev == NULL) { + /* Same internal-invariant check as cmd_select()/cmd_fetch()/ + * cmd_store_cmd()/session_request_expunge() -- ST_AUTH + * requires store_iev to already be wired. */ + log_warnx("session %u: APPEND with no store channel wired", + s->id); + session_reply(s, tag, "NO", "[SERVERBUG] internal error"); + return (1); + } + + if (parsed.litlen > 0) { + if ((s->literal_buf = malloc((size_t)parsed.litlen)) == + NULL) { + log_warn("session %u: malloc APPEND literal buffer", + s->id); + session_reply(s, tag, "NO", "[SERVERBUG] internal error"); + return (1); + } + } else + s->literal_buf = NULL; /* zero-length literal -- see + * session_dispatch_client()'s literal + * block, which handles this without + * special-casing */ + + strlcpy(s->append_mailbox, parsed.mailbox, sizeof(s->append_mailbox)); + s->append_sysflags = parsed.sysflags; + strlcpy(s->append_keywords, parsed.keywords, + sizeof(s->append_keywords)); + s->append_has_date = parsed.has_date; + s->append_date = parsed.date; + s->append_prev_state = s->state; + + strlcpy(s->pending_tag, tag, sizeof(s->pending_tag)); + s->literal_len = parsed.litlen; + s->literal_remaining = parsed.litlen; + s->literal_pending = 1; + + /* + * RFC 9051 SS4.3: a synchronizing literal requires the server to + * send a command continuation request ("+") and the client to wait + * for it before sending the literal's octets; a non-synchronizing + * literal (the "+" inside the braces) requires neither -- "the + * server does not generate a command continuation request... and + * clients are not required to wait." Sending "+ Ready for literal + * data" unconditionally would still be harmless protocol-wise (a + * LITERAL+ client just ignores it, since it isn't waiting for + * anything), but would misleadingly claim the server is waiting + * when it isn't, so it's gated on !litnonsync. + */ + if (!parsed.litnonsync) + session_write(s, "+ Ready for literal data\r\n", 27); + + return (1); +} + +/* + * Called once session_dispatch_client() has fully read an APPEND + * literal's octets plus its trailing CRLF (see that function's literal- + * handling block). Builds the combined header-plus-message-bytes imsg + * (imapd.h's imsg_mbox_append comment explains why it's one imsg, not + * two or an fd) and sends it, entering SESSION_APPENDING to await the + * single IMSG_MBOX_APPENDED reply. s->literal_buf is freed here either + * way -- its contents have been copied into the imsg by the time imsg_ + * compose() returns (same copy-not-ownership semantics parent.c's send_ + * tls_certs() comment already documents for imsg_compose() generally), + * so there's nothing left needing it afterward. + */ +static int +session_finish_append(struct session *s) +{ + struct imsg_mbox_append req; + char *combined; + size_t combined_len; + + memset(&req, 0, sizeof(req)); + strlcpy(req.mailbox, s->append_mailbox, sizeof(req.mailbox)); + req.sysflags = s->append_sysflags; + strlcpy(req.keywords, s->append_keywords, sizeof(req.keywords)); + req.has_date = s->append_has_date; + req.date = s->append_date; + req.msglen = (uint32_t)s->literal_len; + + if (s->store_iev == NULL) { + log_warnx("session %u: APPEND with no store channel wired " + "(literal already read)", s->id); + session_reply(s, s->pending_tag, "NO", "[SERVERBUG] internal error"); + free(s->literal_buf); + s->literal_buf = NULL; + s->state = s->append_prev_state; + return (1); + } + + combined_len = sizeof(req) + (size_t)s->literal_len; + if ((combined = malloc(combined_len)) == NULL) { + log_warn("session %u: malloc APPEND imsg buffer", s->id); + session_reply(s, s->pending_tag, "NO", "[SERVERBUG] internal error"); + free(s->literal_buf); + s->literal_buf = NULL; + s->state = s->append_prev_state; + return (1); + } + memcpy(combined, &req, sizeof(req)); + if (s->literal_len > 0) + memcpy(combined + sizeof(req), s->literal_buf, + (size_t)s->literal_len); + + free(s->literal_buf); + s->literal_buf = NULL; + + s->state = SESSION_APPENDING; + + if (imsg_compose(&s->store_iev->ibuf, IMSG_MBOX_APPEND, 0, 0, -1, + combined, combined_len) == -1) + log_warn("session %u: imsg_compose IMSG_MBOX_APPEND", s->id); + free(combined); + imsgev_add(s->store_iev); + + return (1); +} + +/* + * Terminal reply for the APPEND round trip session_finish_append() + * started. Restores s->state to whatever it was before APPEND began + * (s->append_prev_state) -- SESSION_AUTHENTICATED or SESSION_SELECTED, + * since APPEND is valid, and doesn't change the selected/authenticated + * state, in either (RFC 9051 command-auth). + * + * SS6.3.12: "If the destination mailbox does not exist, a server MUST + * return an error... Unless it is certain that the destination mailbox + * cannot be created, the server MUST send the response code + * '[TRYCREATE]'" -- v1 has no CREATE (still a stub), so retrying via + * CREATE would never actually help, but the response code's job is to + * tell the client *why* the append failed (no such mailbox), not to + * promise CREATE will succeed, so it's sent regardless. + * + * SS6.3.12 also: "On successful completion of an APPEND, the server + * returns an APPENDUID response code" (SS7.1: `"APPENDUID" SP nz-number + * SP append-uid` -- uidvalidity then the new UID) and "If the mailbox is + * currently selected... the server SHOULD notify the client immediately + * via an untagged EXISTS response". Before RFC 9051 SS6.3.4-SS6.3.6's flat + * multi-mailbox support (docs/openimap-storage-backend.md item 10), + * append_prev_state == SESSION_SELECTED alone was a sound proxy for "the + * mailbox is currently selected" -- v1 had exactly one mailbox, so + * anything selected was necessarily APPEND's own destination. That's no + * longer true (a session can have "INBOX" selected while APPENDing into + * "Drafts"), so the check now also compares s->append_mailbox against + * s->selected_mailbox -- case-insensitively if both are INBOX (RFC 9051 + * SS5.1), case-sensitively otherwise, same rule mailbox_name_is_inbox()/ + * mailbox_name_valid() apply everywhere else a mailbox name is compared. + */ +static void +session_handle_mbox_appended(struct session *s, + struct imsg_mbox_appended *res) +{ + int appended_to_selected; + + s->state = s->append_prev_state; + + if (!res->ok) { + if (res->no_such_mailbox) + session_reply(s, s->pending_tag, "NO", + "[TRYCREATE] no such mailbox"); + else + session_reply(s, s->pending_tag, "NO", + "APPEND failed"); + return; + } + + if (mailbox_name_is_inbox(s->append_mailbox) && + mailbox_name_is_inbox(s->selected_mailbox)) + appended_to_selected = 1; + else + appended_to_selected = + (strcmp(s->append_mailbox, s->selected_mailbox) == 0); + + /* + * RFC 9051 SS6.3.13 (IDLE): a successful APPEND always adds exactly + * one message, so unlike EXPUNGE/CLOSE this needs no res->count-style + * gate -- wake any other same-uid session idling on this mailbox so + * it can push the new EXISTS. Fired regardless of whether *this* + * session has that same mailbox selected (an APPEND into "Drafts" + * from a session with "INBOX" selected should still wake a different + * session idling on "Drafts") -- session_notify_idle_peers() itself + * doesn't yet filter by which mailbox each peer actually has + * selected (a pre-existing, accepted imprecision -- see that + * function's own comment), so this is a strict improvement over + * today regardless, not a new gap. + */ + session_notify_idle_peers(s); + + if (s->append_prev_state == SESSION_SELECTED && appended_to_selected) { + char buf[32]; + + snprintf(buf, sizeof(buf), "%u EXISTS", res->exists); + session_untagged(s, buf); + } + + { + char buf[96]; + + snprintf(buf, sizeof(buf), + "[APPENDUID %u %u] APPEND completed", res->uidvalidity, + res->uid); + session_reply(s, s->pending_tag, "OK", buf); + } +} + +/* + * RFC 9051 SS6.3.13: "Arguments: none." Valid in the authenticated or + * selected state -- RFC 2177's own formal grammar groups idle under + * "command_auth ::= ... / idle ;; Valid only in Authenticated or Selected + * state", matching this codebase's existing ST_AUTH bitmask (AUTHENTICATED + * | SELECTED) exactly, so the dispatch table entry already in place from + * before this command had a real implementation needed no change. + * + * "The server requests a response to the IDLE command using the + * continuation ('+') response" -- sent synchronously here, same as + * cmd_authenticate()'s own bare-AUTHENTICATE-PLAIN continuation (no imsg + * round trip needed just to produce a continuation prompt). If a mailbox + * is currently selected, also kicks off a background IMSG_MBOX_IDLE_ + * REFRESH purely to seed s->idle_known_uids -- this doesn't delay "+ + * idling" itself; it just means the first real change-triggered refresh + * (session_notify_idle_peers(), triggered by some other same-uid session) + * has an actual baseline to diff against instead of nothing. + */ +static int +cmd_idle(struct session *s, const char *tag, char *args) +{ + (void)args; /* RFC 2177's grammar takes none; same leniency + * toward stray trailing tokens as every other + * zero-argument command in this file. */ + + strlcpy(s->pending_tag, tag, sizeof(s->pending_tag)); + s->idling = 1; + session_write(s, "+ idling\r\n", 10); + + if (s->state == SESSION_SELECTED) + session_request_idle_refresh(s); + + return (1); +} + +/* + * RFC 9051 SS6.4.1: "Arguments: none." Extra arguments are silently + * ignored (like CAPABILITY's) rather than rejected -- see cmd_capability()'s + * comment for the same leniency reasoning. + * + * CLOSE "permanently removes all messages that have the \Deleted flag set + * ... and returns to the authenticated state ... No untagged EXPUNGE + * responses are sent" -- implemented by sending the identical IMSG_MBOX_ + * EXPUNGE request cmd_expunge() sends, just with silent=1, via the shared + * session_request_expunge() helper (see its comment). The SS6.4.1 exception + * for a read-only (EXAMINE'd) mailbox -- "No messages are removed, and no + * error is given, if the mailbox is selected by EXAMINE" -- is handled + * inside session_request_expunge() itself now, via s->mbox_readonly: CLOSE + * short-circuits to a plain "OK CLOSE completed" with no store round trip + * at all when read-only (see that function's comment). + */ +static int +cmd_close(struct session *s, const char *tag, char *args) +{ + (void)args; + return session_request_expunge(s, tag, 1, 0, 0, 0, 0, 0); +} + +/* + * RFC 9051 SS6.4.2: "Arguments: none." UNSELECT "frees a session's + * resources associated with the selected mailbox and returns the server + * to the authenticated state... performs the same actions as CLOSE, + * except that no messages are permanently removed." In this codebase + * store.c holds no per-selection state of its own to free -- every + * IMSG_MBOX_* request (SELECT included) is independently self-contained, + * re-opening/re-locking the index as needed rather than caching anything + * mailbox-specific in the store child between requests -- so there is + * nothing to tell store about at all, and no async round trip is needed; + * this is the one command-select handler in this file that's purely a + * local state change. + */ +static int +cmd_unselect(struct session *s, const char *tag, char *args) +{ + (void)args; + + s->state = SESSION_AUTHENTICATED; + session_reply(s, tag, "OK", "Unselect completed"); + return (1); +} + +/* + * RFC 9051 SS6.4.3: "Arguments: none." Same extra-arguments leniency as + * CLOSE above. Sends the real (non-silent) IMSG_MBOX_EXPUNGE via the + * shared session_request_expunge() helper -- see that function's comment, + * and imapd.h's imsg_mbox_expunge comment for the full CLOSE/EXPUNGE + * request-sharing rationale. + */ +static int +cmd_expunge(struct session *s, const char *tag, char *args) +{ + (void)args; + return session_request_expunge(s, tag, 0, 0, 0, 0, 0, 0); +} + +/* RFC 9051 SS6.4.4's SEARCH result options this server understands as + * ESEARCH return items (SS7.3.4) -- SAVE (the "$" search result + * variable, SS6.4.4.1) is recognized but rejected with a flagged NO; see + * parse_search_return_opts()'s comment for why it's out of scope this + * pass. */ +#define SEARCH_RETURN_MIN (1U << 0) +#define SEARCH_RETURN_MAX (1U << 1) +#define SEARCH_RETURN_ALL (1U << 2) +#define SEARCH_RETURN_COUNT (1U << 3) + +/* + * RFC 9051 SS9: `date = date-text / DQUOTE date-text DQUOTE`, `date-text + * = date-day "-" date-month "-" date-year`, `date-day = 1*2DIGIT` -- + * used by SEARCH's BEFORE/ON/SINCE (internal date, no time-of-day or + * zone component at all, unlike APPEND's `date-time`). Deliberately a + * separate function from parse_date_time(): that one parses APPEND's + * `date-time` (space-padded 2-digit day, full time-of-day, mandatory + * zone offset) -- a different grammar production, not just a formatting + * variant of this one. Reuses fetch_month_names[] (forward-declared + * above, defined alongside format_internaldate() below) for the month + * name, same as parse_date_time() does. + * + * "Disregarding time and timezone" (SS6.4.4's own words for BEFORE/ON/ + * SINCE) is interpreted here as a UTC calendar day: the internaldate + * values being compared against are themselves stored as absolute UTC + * epoch instants with no separate per-message zone metadata retained -- + * parse_maildir_timestamp() reads a delivery timestamp with no zone + * information at all, and APPEND's own parse_date_time() folds any + * client-supplied zone offset into an absolute UTC instant before + * storing it (see that function's comment) -- so there is no remaining + * per-message zone left to "disregard" separately at search time. UTC + * midnight of the given date-text, via timegm(3), is therefore the only + * coherent boundary available given what's actually on disk. This is a + * specific, deliberate interpretation choice, not something RFC 9051 + * spells out explicitly -- flagged here rather than presented as if it + * were an unambiguous reading of the spec text. + * + * Returns 0 and fills *out (Unix timestamp, UTC midnight of that date) + * on success, -1 on any malformed input (bad digit counts, an + * unrecognized month name, a day out of 1-31, a pre-1970 year, or + * trailing garbage after the date-text -- checked via sscanf(3)'s "%n" + * conversion, which records how many characters were consumed). + */ +static int +parse_search_date(const char *s, int64_t *out) +{ + struct tm tm; + char mon[4]; + int day, year, i, n; + time_t t; + + memset(&tm, 0, sizeof(tm)); + n = 0; + if (sscanf(s, "%2d-%3s-%4d%n", &day, mon, &year, &n) != 3) + return (-1); + if (s[n] != '\0') + return (-1); /* trailing garbage after date-text */ + + if (day < 1 || day > 31 || year < 1970) + return (-1); + + for (i = 0; i < 12; i++) { + if (strcasecmp(mon, fetch_month_names[i]) == 0) + break; + } + if (i == 12) + return (-1); + + tm.tm_mday = day; + tm.tm_mon = i; + tm.tm_year = year - 1900; + /* tm_hour/tm_min/tm_sec already 0 from memset -- UTC midnight */ + + if ((t = timegm(&tm)) == (time_t)-1) + return (-1); + + *out = (int64_t)t; + return (0); +} + +/* + * Accumulator for parse_search_key()/parse_search_key_list()'s compiled + * postfix program -- see imapd.h's imsg_mbox_search comment for why + * the wire format is a flat struct search_node array rather than a + * pointer-linked tree. Kept as a small struct (not just a bare array + + * count passed around) purely so search_push() has one thing to take a + * pointer to. + */ +#define SEARCH_MAX_DEPTH 64 /* F7 fix: max parse_search_key() recursion + * depth -- prevents a deeply-nested SEARCH + * from exhausting the listener stack. */ +struct search_parse_ctx { + struct search_node nodes[SEARCH_PROGRAM_MAX_NODES]; + uint32_t n; + int depth; /* F7: current recursion depth */ + int uses_modseq; /* set by parse_search_key() when + * a MODSEQ search-key is pushed -- + * see cmd_search()'s use of it to + * set s->search_used_modseq / + * call session_condstore_enable(). */ +}; + +static int +search_push(struct search_parse_ctx *ctx, const struct search_node *node, + const char **errmsg) +{ + if (ctx->n >= SEARCH_PROGRAM_MAX_NODES) { + *errmsg = "search criteria too complex"; + return (-1); + } + ctx->nodes[ctx->n++] = *node; + return (0); +} + +/* + * Reads one sequence-set token (space- or ')'-delimited) starting at + * *pp, rejects an internal comma (v1's established single-range-only + * restriction -- see imapd.h's imsg_mbox_fetch comment), and parses + * it via the same parse_seq_range() FETCH/STORE already use. Shared by + * both the bare-sequence-set search key and the "UID" SP sequence-set + * search key below -- their token grammar and v1 restrictions are + * identical, only which field of struct search_node the caller stores + * the result into (and which op it tags the node with) differs. + */ +static int +parse_search_seqset_token(char **pp, uint32_t *lo, uint32_t *hi, + int *lo_star, int *hi_star, const char **errmsg) +{ + char *p = *pp; + char *start = p; + + while (*p != '\0' && *p != ' ' && *p != ')') + p++; + + if (p == start) { + *errmsg = "missing sequence set"; + return (-1); + } + + { + char tok[32]; + size_t len = (size_t)(p - start); + + if (len >= sizeof(tok)) { + *errmsg = "sequence set too long"; + return (-1); + } + memcpy(tok, start, len); + tok[len] = '\0'; + + if (strchr(tok, ',') != NULL) { + *errmsg = "comma-separated sequence sets not " + "supported in v1 -- issue separate SEARCH " + "commands"; + return (-1); + } + + if (parse_seq_range(tok, lo, hi, lo_star, hi_star) == -1) { + *errmsg = "invalid sequence set"; + return (-1); + } + } + + *pp = p; + return (0); +} + +/* + * RFC 9051 SS6.4.4's `search-key` grammar (SS9's formal ABNF), recursive- + * descent, one search-key at a time -- pushes exactly one resulting + * value onto ctx's postfix program (a single leaf node for most keys, or + * a leaf/subexpression followed by a combinator for NOT/OR/parenthesized + * lists) and leaves *pp positioned just past what it consumed. + * + * v1 scope: search keys that need actual message content or headers -- + * BCC/BODY/CC/FROM/HEADER/SENTBEFORE/SENTON/SENTSINCE/SUBJECT/TEXT/TO -- + * are recognized by name (so a client seeing NO for one of them gets a + * specific, honest explanation, not a generic BAD) but rejected + * immediately with -2/NO, without attempting to parse or skip their + * operand -- safe because the whole SEARCH command is abandoned on any + * -2/-1 return anyway, so there's no later parse position that still + * needs to be correct. This is the same "recognized, can't do it right + * now" category FETCH's BODY[] rejection already uses, for the same + * underlying reason: real support needs message content access, which + * needs fd-passing from store, not designed yet (see cmd_fetch()'s own + * comment). + * + * Returns 0 on success, -1 (BAD) for malformed syntax, -2 (NO) for a + * recognized-but-unsupported search key -- same convention as parse_ + * fetch_atts()/parse_store_flags()/parse_append_args(). + */ +static int +parse_search_key(char **pp, struct search_parse_ctx *ctx, const char **errmsg) +{ + int rc; + + /* + * F7 fix: bound recursion depth. Every level of the SEARCH-key + * grammar (parenthesized lists, NOT, OR) re-enters this function, + * so a single guard here caps total recursion and prevents a + * crafted deeply-nested SEARCH from exhausting the listener stack. + */ + if (++ctx->depth > SEARCH_MAX_DEPTH) { + ctx->depth--; + *errmsg = "search criteria nested too deeply"; + return (-1); + } + rc = parse_search_key_inner(pp, ctx, errmsg); + ctx->depth--; + return (rc); +} + +static int +parse_search_key_inner(char **pp, struct search_parse_ctx *ctx, + const char **errmsg) +{ + char *p = *pp; + char word[32]; + size_t wlen; + + while (*p == ' ') + p++; + + if (*p == '\0') { + *errmsg = "missing search key"; + return (-1); + } + + if (*p == '(') { + int rc; + + p++; + rc = parse_search_key_list(&p, ctx, errmsg, 1); + if (rc != 0) + return (rc); + while (*p == ' ') + p++; + if (*p != ')') { + *errmsg = "unterminated parenthesized search key list"; + return (-1); + } + p++; + *pp = p; + return (0); + } + + if (isdigit((unsigned char)*p) || *p == '*') { + struct search_node node; + + memset(&node, 0, sizeof(node)); + node.op = SEARCH_OP_SEQSET; + if (parse_search_seqset_token(&p, &node.seq_lo, &node.seq_hi, + &node.lo_is_star, &node.hi_is_star, errmsg) == -1) + return (-1); + if (search_push(ctx, &node, errmsg) == -1) + return (-1); + *pp = p; + return (0); + } + + { + char *start = p; + + while (*p != '\0' && *p != ' ' && *p != ')') + p++; + wlen = (size_t)(p - start); + if (wlen == 0 || wlen >= sizeof(word)) { + *errmsg = "unknown search key"; + return (-1); + } + memcpy(word, start, wlen); + word[wlen] = '\0'; + } + + if (strcasecmp(word, "ALL") == 0) { + struct search_node node; + + memset(&node, 0, sizeof(node)); + node.op = SEARCH_OP_ALL; + if (search_push(ctx, &node, errmsg) == -1) + return (-1); + *pp = p; + return (0); + } + + { + static const struct { + const char *name; + int op; + } boolkeys[] = { + { "ANSWERED", SEARCH_OP_ANSWERED }, + { "UNANSWERED", SEARCH_OP_UNANSWERED }, + { "DELETED", SEARCH_OP_DELETED }, + { "UNDELETED", SEARCH_OP_UNDELETED }, + { "DRAFT", SEARCH_OP_DRAFT }, + { "UNDRAFT", SEARCH_OP_UNDRAFT }, + { "FLAGGED", SEARCH_OP_FLAGGED }, + { "UNFLAGGED", SEARCH_OP_UNFLAGGED }, + { "SEEN", SEARCH_OP_SEEN }, + { "UNSEEN", SEARCH_OP_UNSEEN }, + }; + size_t i; + + for (i = 0; i < sizeof(boolkeys) / sizeof(boolkeys[0]); i++) { + struct search_node node; + + if (strcasecmp(word, boolkeys[i].name) != 0) + continue; + memset(&node, 0, sizeof(node)); + node.op = boolkeys[i].op; + if (search_push(ctx, &node, errmsg) == -1) + return (-1); + *pp = p; + return (0); + } + } + + if (strcasecmp(word, "KEYWORD") == 0 || + strcasecmp(word, "UNKEYWORD") == 0) { + struct search_node node; + char *start; + + memset(&node, 0, sizeof(node)); + node.op = (strcasecmp(word, "KEYWORD") == 0) ? + SEARCH_OP_KEYWORD : SEARCH_OP_UNKEYWORD; + + while (*p == ' ') + p++; + start = p; + while (*p != '\0' && *p != ' ' && *p != ')') + p++; + if (p == start || (size_t)(p - start) >= sizeof(node.keyword)) { + *errmsg = "missing or too-long KEYWORD/UNKEYWORD " + "argument"; + return (-1); + } + memcpy(node.keyword, start, (size_t)(p - start)); + node.keyword[p - start] = '\0'; + + if (search_push(ctx, &node, errmsg) == -1) + return (-1); + *pp = p; + return (0); + } + + if (strcasecmp(word, "BEFORE") == 0 || strcasecmp(word, "ON") == 0 || + strcasecmp(word, "SINCE") == 0) { + struct search_node node; + char datebuf[32]; + + memset(&node, 0, sizeof(node)); + if (strcasecmp(word, "BEFORE") == 0) + node.op = SEARCH_OP_BEFORE; + else if (strcasecmp(word, "ON") == 0) + node.op = SEARCH_OP_ON; + else + node.op = SEARCH_OP_SINCE; + + while (*p == ' ') + p++; + if (*p == '"') { + char *start = p + 1; + char *end = strchr(start, '"'); + size_t len; + + if (end == NULL) { + *errmsg = "unterminated date string"; + return (-1); + } + len = (size_t)(end - start); + if (len >= sizeof(datebuf)) { + *errmsg = "malformed date"; + return (-1); + } + memcpy(datebuf, start, len); + datebuf[len] = '\0'; + p = end + 1; + } else { + char *start = p; + size_t len; + + while (*p != '\0' && *p != ' ' && *p != ')') + p++; + len = (size_t)(p - start); + if (len == 0 || len >= sizeof(datebuf)) { + *errmsg = "malformed date"; + return (-1); + } + memcpy(datebuf, start, len); + datebuf[len] = '\0'; + } + + if (parse_search_date(datebuf, &node.num) == -1) { + *errmsg = "malformed date"; + return (-1); + } + + if (search_push(ctx, &node, errmsg) == -1) + return (-1); + *pp = p; + return (0); + } + + if (strcasecmp(word, "LARGER") == 0 || + strcasecmp(word, "SMALLER") == 0) { + struct search_node node; + char numbuf[24]; + char *start, *numend; + unsigned long long v; + + memset(&node, 0, sizeof(node)); + node.op = (strcasecmp(word, "LARGER") == 0) ? + SEARCH_OP_LARGER : SEARCH_OP_SMALLER; + + while (*p == ' ') + p++; + start = p; + while (*p != '\0' && *p != ' ' && *p != ')') + p++; + if (p == start || (size_t)(p - start) >= sizeof(numbuf)) { + *errmsg = "missing or malformed octet count"; + return (-1); + } + memcpy(numbuf, start, (size_t)(p - start)); + numbuf[p - start] = '\0'; + + errno = 0; + v = strtoull(numbuf, &numend, 10); + if (*numend != '\0' || errno == ERANGE) { + *errmsg = "malformed octet count"; + return (-1); + } + node.num = (int64_t)v; + + if (search_push(ctx, &node, errmsg) == -1) + return (-1); + *pp = p; + return (0); + } + + if (strcasecmp(word, "UID") == 0) { + struct search_node node; + + memset(&node, 0, sizeof(node)); + node.op = SEARCH_OP_UIDSET; + + while (*p == ' ') + p++; + if (parse_search_seqset_token(&p, &node.seq_lo, &node.seq_hi, + &node.lo_is_star, &node.hi_is_star, errmsg) == -1) + return (-1); + + if (search_push(ctx, &node, errmsg) == -1) + return (-1); + *pp = p; + return (0); + } + + if (strcasecmp(word, "MODSEQ") == 0) { + struct search_node node; + char numbuf[24]; + char *start, *numend; + unsigned long long v; + + memset(&node, 0, sizeof(node)); + node.op = SEARCH_OP_MODSEQ; + + while (*p == ' ') + p++; + + /* RFC 7162 SS3.1.5 ABNF: MODSEQ [SP entry-name SP entry- + * type-req] SP mod-sequence-valzer. entry-name/entry-type- + * req come from RFC 5464 METADATA, which this server doesn't + * implement -- SS3.1.5 itself says a server that doesn't + * store separate mod-sequences per metadata item "MUST + * ignore and ", so this only + * needs to parse past them syntactically (see imapd.h's + * SEARCH_OP_MODSEQ comment). Detected by peeking: the + * mod-sequence-valzer itself is always a bare digit string, + * so anything else here must be the optional entry-name. */ + if (*p != '\0' && !isdigit((unsigned char)*p)) { + if (*p == '"') { + char *end = strchr(p + 1, '"'); + + if (end == NULL) { + *errmsg = "unterminated MODSEQ " + "entry-name"; + return (-1); + } + p = end + 1; + } else { + while (*p != '\0' && *p != ' ' && *p != ')') + p++; + } + while (*p == ' ') + p++; + + if (strncasecmp(p, "priv", 4) == 0 && + (p[4] == ' ' || p[4] == '\0' || p[4] == ')')) + p += 4; + else if (strncasecmp(p, "shared", 6) == 0 && + (p[6] == ' ' || p[6] == '\0' || p[6] == ')')) + p += 6; + else if (strncasecmp(p, "all", 3) == 0 && + (p[3] == ' ' || p[3] == '\0' || p[3] == ')')) + p += 3; + else { + *errmsg = "expected priv/shared/all after " + "MODSEQ entry-name"; + return (-1); + } + while (*p == ' ') + p++; + } + + start = p; + while (*p != '\0' && *p != ' ' && *p != ')') + p++; + if (p == start || (size_t)(p - start) >= sizeof(numbuf)) { + *errmsg = "missing or malformed MODSEQ value"; + return (-1); + } + memcpy(numbuf, start, (size_t)(p - start)); + numbuf[p - start] = '\0'; + + errno = 0; + v = strtoull(numbuf, &numend, 10); + if (*numend != '\0' || errno == ERANGE) { + *errmsg = "malformed MODSEQ value"; + return (-1); + } + node.num = (int64_t)v; + + if (search_push(ctx, &node, errmsg) == -1) + return (-1); + ctx->uses_modseq = 1; + *pp = p; + return (0); + } + + if (strcasecmp(word, "NOT") == 0) { + struct search_node node; + int rc; + + rc = parse_search_key(&p, ctx, errmsg); + if (rc != 0) + return (rc); + + memset(&node, 0, sizeof(node)); + node.op = SEARCH_OP_NOT; + if (search_push(ctx, &node, errmsg) == -1) + return (-1); + *pp = p; + return (0); + } + + if (strcasecmp(word, "OR") == 0) { + struct search_node node; + int rc; + + rc = parse_search_key(&p, ctx, errmsg); + if (rc != 0) + return (rc); + rc = parse_search_key(&p, ctx, errmsg); + if (rc != 0) + return (rc); + + memset(&node, 0, sizeof(node)); + node.op = SEARCH_OP_OR; + if (search_push(ctx, &node, errmsg) == -1) + return (-1); + *pp = p; + return (0); + } + + { + static const char *content_keys[] = { + "BCC", "BODY", "CC", "FROM", "HEADER", "SENTBEFORE", + "SENTON", "SENTSINCE", "SUBJECT", "TEXT", "TO", + }; + size_t i; + + for (i = 0; i < sizeof(content_keys) / sizeof(content_keys[0]); + i++) { + if (strcasecmp(word, content_keys[i]) == 0) { + *errmsg = "search keys that require message " + "content/header access are not " + "supported in this pass"; + return (-2); + } + } + } + + *errmsg = "unknown search key"; + return (-1); +} + +/* + * `search-key *(SP search-key)`, ANDed together left to right -- shared + * by the top-level search-program (in_parens == 0, stops at end of + * string) and a parenthesized `patterns`-style list (in_parens == 1, + * stops at, but does not consume, the closing ')') -- see parse_search_ + * key()'s own "(" branch, which consumes the parens themselves. + */ +static int +parse_search_key_list(char **pp, struct search_parse_ctx *ctx, + const char **errmsg, int in_parens) +{ + int rc; + + rc = parse_search_key(pp, ctx, errmsg); + if (rc != 0) + return (rc); + + for (;;) { + char *p = *pp; + + while (*p == ' ') + p++; + + if (in_parens && *p == ')') { + *pp = p; + return (0); + } + if (*p == '\0') { + if (in_parens) { + *errmsg = "unterminated parenthesized search " + "key list"; + return (-1); + } + *pp = p; + return (0); + } + if (!in_parens && *p == ')') { + *errmsg = "unexpected ')'"; + return (-1); + } + + *pp = p; + rc = parse_search_key(pp, ctx, errmsg); + if (rc != 0) + return (rc); + + { + struct search_node combine; + + memset(&combine, 0, sizeof(combine)); + combine.op = SEARCH_OP_AND; + if (search_push(ctx, &combine, errmsg) == -1) + return (-1); + } + } +} + +/* + * RFC 9051 SS6.4.4: `search-return-opts = SP "RETURN" SP "(" [search- + * return-opt *(SP search-return-opt)] ")"`. *pp must already point at + * the opening "(" (caller peeked for it to decide whether a RETURN + * clause is present at all). An empty "()" is valid ABNF and leaves + * *opts_out at 0 -- cmd_search() treats that identically to "no RETURN + * clause at all" (SS6.4.4: "If no result option is specified or empty + * list of options is specified as '()', ALL is assumed"). + * + * SAVE (SS6.4.4.1's "$" search result variable) is recognized but + * rejected with -2/NO: implementing it correctly means resetting the + * variable on SELECT/EXAMINE, adjusting it on EXPUNGE, and teaching + * every command that accepts a sequence-set (FETCH, STORE, COPY, MOVE, + * a future UID SEARCH) to also accept "$" -- real, cross-cutting design + * work spanning multiple already-implemented commands, not a small + * addition to SEARCH alone. Deferred, same category of scope cut as + * APPEND's size cap or LIST's extended syntax. Any other, genuinely + * unrecognized token (not one of MIN/MAX/ALL/COUNT/SAVE) gets -1/BAD -- + * SS6.3.9's own words for LIST options apply equally here: "Any options + * not defined by extensions that the server supports MUST be rejected + * with a BAD response." + */ +static int +parse_search_return_opts(char **pp, uint32_t *opts_out, const char **errmsg) +{ + char *p = *pp; + + *opts_out = 0; + p++; /* skip the '(' the caller already confirmed is there */ + + while (*p == ' ') + p++; + if (*p == ')') { + *pp = p + 1; + return (0); + } + + for (;;) { + char *start = p; + char word[16]; + size_t len; + + while (*p != '\0' && *p != ' ' && *p != ')') + p++; + len = (size_t)(p - start); + if (len == 0 || len >= sizeof(word)) { + *errmsg = "malformed SEARCH RETURN option"; + return (-1); + } + memcpy(word, start, len); + word[len] = '\0'; + + if (strcasecmp(word, "MIN") == 0) + *opts_out |= SEARCH_RETURN_MIN; + else if (strcasecmp(word, "MAX") == 0) + *opts_out |= SEARCH_RETURN_MAX; + else if (strcasecmp(word, "ALL") == 0) + *opts_out |= SEARCH_RETURN_ALL; + else if (strcasecmp(word, "COUNT") == 0) + *opts_out |= SEARCH_RETURN_COUNT; + else if (strcasecmp(word, "SAVE") == 0) { + *errmsg = "SEARCH RETURN (SAVE) -- the \"$\" search " + "result variable -- is not supported in this " + "pass"; + return (-2); + } else { + *errmsg = "unsupported SEARCH RETURN option"; + return (-1); + } + + while (*p == ' ') + p++; + if (*p == ')') { + *pp = p + 1; + return (0); + } + if (*p == '\0') { + *errmsg = "unterminated SEARCH RETURN option list"; + return (-1); + } + } +} + +/* + * RFC 9051 SS6.4.4: `search = "SEARCH" [search-return-opts] SP search- + * program`, `search-program = ["CHARSET" SP charset SP] search-key + * *(SP search-key)`. + * + * v1 scope, summarized (each piece's own comment has the full reasoning): + * basic MIN/MAX/ALL/COUNT result options (SAVE deferred); CHARSET + * accepted only as US-ASCII or UTF-8 (SS6.4.4: "Servers MUST support + * US-ASCII and UTF-8 charsets"), anything else gets NO [BADCHARSET]; the + * full flag/date/size/sequence-number/UID-range/NOT/OR/parenthesized- + * list search-key grammar, except the content-and-header-based keys + * (BCC/BODY/CC/FROM/HEADER/SENTBEFORE/SENTON/SENTSINCE/SUBJECT/TEXT/TO), + * which need message content access this codebase doesn't have yet. + * + * RFC 9051 SS6.4.9 (UID command) is now wired up via cmd_uid()'s SEARCH + * branch calling search_dispatch() below with by_uid=1: "the numbers + * returned in an ESEARCH response for a UID SEARCH command are unique + * identifiers instead of message sequence numbers... the corresponding + * ESEARCH response MUST include the UID indicator" -- see search_ + * dispatch()'s own comment, session_handle_mbox_search_match(), and + * session_finish_search() for how. + */ +static int +cmd_search(struct session *s, const char *tag, char *args) +{ + return search_dispatch(s, tag, args, 0); +} + +/* + * Shared body for cmd_search() (by_uid=0) and cmd_uid()'s SEARCH branch + * (by_uid=1). RFC 9051 SS6.4.9: "the interpretation of the [SEARCH] + * arguments is the same as with SEARCH" -- by_uid does NOT change how a + * bare sequence-set or "UID " search-key is parsed or + * matched (those already go through SEARCH_OP_SEQSET/SEARCH_OP_UIDSET + * exactly as before); it only changes what value session_handle_mbox_ + * search_match()/session_finish_search() report for each match (UID + * instead of sequence number) and adds the "UID" ESEARCH correlator + * token -- see both functions' own comments. No store.c wire-format + * change was needed for this at all: store.c has always sent both seqno + * and uid per match (see imapd.h's imsg_mbox_search_match comment). + */ +static int +search_dispatch(struct session *s, const char *tag, char *args, int by_uid) +{ + struct search_parse_ctx ctx; + char *p; + uint32_t return_opts = 0; + int rc; + const char *errmsg; + + if (args == NULL) { + session_reply(s, tag, "BAD", "SEARCH requires search criteria"); + return (1); + } + + p = args; + while (*p == ' ') + p++; + + if (strncasecmp(p, "RETURN", 6) == 0 && + (p[6] == ' ' || p[6] == '\0')) { + p += 6; + while (*p == ' ') + p++; + if (*p != '(') { + session_reply(s, tag, "BAD", + "malformed SEARCH RETURN option list"); + return (1); + } + rc = parse_search_return_opts(&p, &return_opts, &errmsg); + if (rc == -1) { + session_reply(s, tag, "BAD", errmsg); + return (1); + } + if (rc == -2) { + session_reply(s, tag, "NO", errmsg); + return (1); + } + while (*p == ' ') + p++; + } + + if (return_opts == 0) + return_opts = SEARCH_RETURN_ALL; /* SS6.4.4's default */ + + if (strncasecmp(p, "CHARSET", 7) == 0 && + (p[7] == ' ' || p[7] == '\0')) { + char *start; + char charset[64]; + size_t len; + + p += 7; + while (*p == ' ') + p++; + start = p; + while (*p != '\0' && *p != ' ') + p++; + len = (size_t)(p - start); + if (len == 0 || len >= sizeof(charset)) { + session_reply(s, tag, "BAD", "malformed CHARSET"); + return (1); + } + memcpy(charset, start, len); + charset[len] = '\0'; + + if (strcasecmp(charset, "US-ASCII") != 0 && + strcasecmp(charset, "UTF-8") != 0) { + /* RFC 9051 SS9 resp-text-code: `"BADCHARSET" [SP "(" + * charset *(SP charset) ")"]` -- the parens are + * required by the formal ABNF; SS6.4.4.4's own prose + * example ("NO [BADCHARSET UTF-8] KOI8-R is not + * supported") omits them, an inconsistency in the + * RFC's own text between its worked example and its + * Section 9 grammar -- the ABNF is followed here as + * the normative definition. */ + session_reply(s, tag, "NO", + "[BADCHARSET (US-ASCII UTF-8)] unsupported " + "CHARSET"); + return (1); + } + + while (*p == ' ') + p++; + } + + memset(&ctx, 0, sizeof(ctx)); + rc = parse_search_key_list(&p, &ctx, &errmsg, 0); + if (rc == -1) { + session_reply(s, tag, "BAD", errmsg); + return (1); + } + if (rc == -2) { + session_reply(s, tag, "NO", errmsg); + return (1); + } + + if (ctx.n == 0) { + session_reply(s, tag, "BAD", "SEARCH requires search criteria"); + return (1); + } + + if (s->store_iev == NULL) { + /* Same internal-invariant check as cmd_fetch()/cmd_store_ + * cmd()/session_request_expunge() -- ST_SELECTED requires + * store_iev to already be wired. */ + log_warnx("session %u: %s with no store channel wired", + s->id, by_uid ? "UID SEARCH" : "SEARCH"); + session_reply(s, tag, "NO", "[SERVERBUG] internal error"); + return (1); + } + + /* Defensive cleanup of a previous SEARCH's leftovers, matching the + * ST_SELECTED-exclusion guarantee above (SESSION_SEARCHING can't + * still be in flight when a new SEARCH is dispatched) -- shouldn't + * ever actually find anything here, same belt-and-suspenders + * posture as other cleanup in this file. */ + free(s->search_matches); + s->search_matches = NULL; + s->search_nmatches = 0; + s->search_matches_cap = 0; + s->search_alloc_failed = 0; + s->search_return_opts = return_opts; + s->search_used_modseq = ctx.uses_modseq; + s->search_max_modseq = 0; + s->cmd_by_uid = by_uid; + + /* RFC 7162 SS3.1: "a FETCH or SEARCH command that includes the + * MODSEQ message data item" is a CONDSTORE enabling command. */ + if (ctx.uses_modseq) + session_condstore_enable(s); + + strlcpy(s->pending_tag, tag, sizeof(s->pending_tag)); + s->state = SESSION_SEARCHING; + + { + struct imsg_mbox_search req; + size_t bodylen = (size_t)ctx.n * + sizeof(struct search_node); + char *combined; + + memset(&req, 0, sizeof(req)); + req.nnodes = ctx.n; + + if ((combined = malloc(sizeof(req) + bodylen)) == NULL) { + log_warn("session %u: malloc SEARCH imsg buffer", + s->id); + session_reply(s, tag, "NO", "[SERVERBUG] internal error"); + s->state = SESSION_SELECTED; + return (1); + } + memcpy(combined, &req, sizeof(req)); + if (bodylen > 0) + memcpy(combined + sizeof(req), ctx.nodes, bodylen); + + if (imsg_compose(&s->store_iev->ibuf, IMSG_MBOX_SEARCH, 0, 0, + -1, combined, sizeof(req) + bodylen) == -1) + log_warn("session %u: imsg_compose IMSG_MBOX_SEARCH", + s->id); + free(combined); + imsgev_add(s->store_iev); + } + + return (1); +} + +/* + * RFC 9051 SS9: `nz-number = digit-nz *DIGIT` -- a non-zero unsigned + * decimal number, the ABNF base type behind sequence numbers and UIDs. + * Rejects a leading zero (matches "digit-nz *DIGIT", not the more general + * "number"), non-digit characters, an empty string, and overflow past + * UINT32_MAX (see imapd.h's imsg_mbox_fetch_meta comment and RFC 9051 + * SS2.3.1.1's own "unsigned 32-bit" language for why UINT32_MAX, not + * ULONG_MAX, is the ceiling here). + */ +static int +parse_nz_number(const char *str, uint32_t *out) +{ + unsigned long v; + char *end; + + if (str == NULL || *str == '\0' || *str == '0') + return (-1); + + errno = 0; + v = strtoul(str, &end, 10); + if (*end != '\0' || errno == ERANGE || v == 0 || v > UINT32_MAX) + return (-1); + + *out = (uint32_t)v; + return (0); +} + +/* + * RFC 9051 SS9: `sequence-set = (seq-number / seq-range) *("," sequence- + * set)`, `seq-range = seq-number ":" seq-number`, `seq-number = nz-number / + * "*"`. v1 supports exactly one seq-number or seq-range per FETCH -- + * cmd_fetch() rejects a comma-separated sequence-set with a BAD before this + * function is ever called, so a comma reaching here would be a caller bug, + * not client input this function itself needs to detect. + * + * "*" (SS9: "the largest number in use") can't be resolved here: listener + * doesn't reliably know the live message count (mail can arrive between + * SELECT and this FETCH). lo_star and hi_star instead carry the "this side + * was a star" fact through IMSG_MBOX_FETCH so store.c can resolve it + * against its own up-to-the-moment index length -- see imapd.h's + * imsg_mbox_fetch comment. + * + * A bare seq-number (no ":") normalizes to lo == hi. A backwards range + * (e.g. "4:2") is swapped so lo <= hi, matching SS9's own "it is possible + * to specify a decreasing range (e.g., '4:2')" note -- store.c's + * handle_mbox_fetch() iterates lo..hi ascending and has no other way to + * handle a decreasing range. A star on one side of a range with a literal + * number on the other (e.g. "4:*") is left unswapped -- whether that ends + * up lo <= hi is something only store.c, once it resolves the star, can + * know. + */ +static int +parse_seq_range(const char *tok, uint32_t *lo, uint32_t *hi, int *lo_star, + int *hi_star) +{ + char buf[32]; + char *colon; + const char *loside, *hiside; + + if (tok == NULL || *tok == '\0' || strlen(tok) >= sizeof(buf)) + return (-1); + strlcpy(buf, tok, sizeof(buf)); + + *lo_star = *hi_star = 0; + *lo = *hi = 0; + + if ((colon = strchr(buf, ':')) != NULL) { + *colon = '\0'; + loside = buf; + hiside = colon + 1; + } else { + loside = buf; + hiside = buf; + } + + if (strcmp(loside, "*") == 0) + *lo_star = 1; + else if (parse_nz_number(loside, lo) == -1) + return (-1); + + if (strcmp(hiside, "*") == 0) + *hi_star = 1; + else if (parse_nz_number(hiside, hi) == -1) + return (-1); + + if (!*lo_star && !*hi_star && *lo > *hi) { + uint32_t tmp = *lo; + + *lo = *hi; + *hi = tmp; + } + + return (0); +} + +/* + * Same calling convention as strtok_r(str, " ", &savep) (pass str on the + * first call, NULL thereafter, using the same savep each time), but a + * space is not treated as a delimiter while inside an unclosed '[' or + * '(' -- needed because a single fetch-att can itself contain a + * mandatory embedded space: RFC 9051 SS9's header-list production, + * "(" header-fld-name *(SP header-fld-name) ")", sits inside a bracketed + * section-spec, e.g. the RFC's own example, "BODY[HEADER.FIELDS (DATE + * FROM)]" (SS6.4.5). Plain strtok_r(spec, " ", &save) -- what parse_ + * fetch_atts() used before this pass -- would split that into three + * garbage tokens ("BODY[HEADER.FIELDS", "(DATE", "FROM)]"), the second + * and third of which match nothing and would fail the whole FETCH as + * BAD. This is a real, previously undiscovered bug: a comment in this + * file claimed the generic BODY-prefix catch-all already "recognizes and + * skips the whole item" for a multi-token split like that, but nothing + * actually reassembles tokens 2 and 3 into anything recognizable -- + * found only because implementing HEADER.FIELDS required tracing exactly + * how a bracketed, space-containing fetch-att reaches parse_fetch_atts() + * in the first place, not by inspection of the (incorrect) comment + * alone. No real client request had exercised this path before now. + * + * '[' and '(' share one depth counter since section-spec's own brackets + * and header-list's parens are always properly nested relative to each + * other in this grammar -- nothing here needs to distinguish which kind + * of bracket is currently open, only whether depth is zero. An unclosed + * bracket/paren at end of string is left as trailing unbalanced depth; + * the caller's own field-list parsing (not this function) is what + * rejects that as a syntax error, same "let the specific parser catch + * the specific mistake" split this file already uses elsewhere (e.g. + * split_trailing_modifiers() vs. its own caller). + */ +static char * +fetch_att_tok(char *str, char **savep) +{ + char *p, *start; + int depth = 0; + + p = (str != NULL) ? str : *savep; + + while (*p == ' ') + p++; + if (*p == '\0') { + *savep = p; + return (NULL); + } + + for (start = p; *p != '\0'; p++) { + if (*p == '[' || *p == '(') + depth++; + else if (*p == ']' || *p == ')') { + if (depth > 0) + depth--; + } else if (*p == ' ' && depth == 0) + break; + } + + if (*p != '\0') { + *p = '\0'; + p++; + } + *savep = p; + return (start); +} + +/* + * Parses one "HEADER.FIELDS (name ...)" or "HEADER.FIELDS.NOT (name ...)" + * bracket body -- inner is everything between BODY.PEEK[...]'s brackets, + * e.g. "HEADER.FIELDS (DATE FROM)" (RFC 9051 SS9: section-msgtext = ... / + * "HEADER.FIELDS" [".NOT"] SP header-list, header-list = "(" header-fld- + * name *(SP header-fld-name) ")"). Only called once fetch_att_tok() (see + * its own comment for why a plain strtok_r() split can't handle this + * token's embedded space) has already isolated the whole "BODY.PEEK[...]" + * atom, and parse_fetch_atts() has confirmed it starts with "BODY.PEEK + * [HEADER.FIELDS" case-insensitively. + * + * Returns 0 and fills *not_out, fields_out (space-joined field names -- + * see struct imsg_mbox_fetch's header_fields comment in imapd.h for + * why store.c gets this pre-extracted form rather than the raw bracket + * text) on success. Returns -1 on a syntax error (missing SP, missing/ + * unbalanced parens, empty list, a field name containing a double quote, + * or a field-name list too long for HEADER_FIELDS_MAX) -- unlike an + * *unsupported* item, which parse_fetch_atts() silently degrades, a + * fetch-att that announces itself as HEADER.FIELDS but is malformed is a + * real client error (this implementation does support the item), so the + * caller sends BAD rather than silently dropping it. + * + * Field names are required to be bare atoms (no quoted-string or literal + * form, checked only by rejecting an embedded '"' -- RFC 5322's own + * field-name syntax, 1*ftext, already excludes space/colon/control + * characters, so a bare atom is the only shape any legitimate field name + * can ever take; this implementation simply doesn't bother recognizing + * the quoted-string astring alternative nothing real would need). + */ +static int +parse_header_fields_att(const char *inner, int *not_out, char *fields_out, + size_t fields_outsize) +{ + char listbuf[HEADER_FIELDS_LABEL_MAX]; + char *p = listbuf, *listp, *end; + char *name, *save; + int first = 1; + + *not_out = 0; + fields_out[0] = '\0'; + + if (strlcpy(listbuf, inner, sizeof(listbuf)) >= sizeof(listbuf)) + return (-1); + + if (strncasecmp(p, "HEADER.FIELDS", 13) != 0) + return (-1); + p += 13; + + if (strncasecmp(p, ".NOT", 4) == 0) { + *not_out = 1; + p += 4; + } + + if (*p != ' ') + return (-1); + p++; + + listp = p; + if (*listp != '(') + return (-1); + listp++; + + end = strchr(listp, ')'); + if (end == NULL || end[1] != '\0') + return (-1); + *end = '\0'; + + if (*listp == '\0') + return (-1); /* header-list requires at least one name */ + + for (name = strtok_r(listp, " ", &save); name != NULL; + name = strtok_r(NULL, " ", &save)) { + if (strchr(name, '"') != NULL) + return (-1); + if (!first && + strlcat(fields_out, " ", fields_outsize) >= fields_outsize) + return (-1); + if (strlcat(fields_out, name, fields_outsize) >= fields_outsize) + return (-1); + first = 0; + } + + return (0); +} + +/* + * RFC 9051 SS6.4.5.1: section-part = nz-number *("." nz-number), e.g. + * "3.1". Validates the grammar only -- doesn't parse it into a path + * array, since listener.c never needs the individual part numbers + * itself; it just needs to know whether the bracket body it just + * isolated is a legal section-part before setting MBOX_FETCH_BODY_PART + * and handing the verbatim string on to store.c's own parse_section_ + * part() (store.c and listener.c are separate processes -- this + * validation exists so an *invalid* string is caught here and silently + * degraded like any other unsupported BODY.PEEK[...] shape, rather than + * crossing the imsg boundary and only failing there). Bounded at MIME_ + * MAX_DEPTH components, same as store.c's own parser and for the same + * reason: a path deeper than that could never match anything build_ + * bodystructure() would ever describe. + */ +static int +section_part_valid(const char *s) +{ + int n = 0; + + if (s == NULL || *s == '\0') + return (0); + + while (*s != '\0') { + if (*s < '1' || *s > '9') /* nz-number: digit-nz first */ + return (0); + if (n >= MIME_MAX_DEPTH) + return (0); + n++; + while (*s >= '0' && *s <= '9') + s++; + if (*s == '\0') + return (1); + if (*s != '.') + return (0); + s++; + if (*s == '\0') + return (0); /* trailing dot */ + } + return (1); +} + +/* + * RFC 9051 SS6.4.5's origin-octet-then-count partial-range suffix, + * `"<" number "." nz-number ">"` (a BODY[

]<> request's + * trailing "", e.g. "<0.16384>"). s is whatever followed a + * BODY.PEEK[...] token's closing "]" -- empty (no partial range: *has_ + * partial_out = 0, success) or exactly one "<...>" as above. count is + * only required to be numeric here, not strictly nz-number (a client- + * sent "<5.0>" is a strange request, not a malformed one -- + * apply_partial_range() in store.c already handles a zero count + * gracefully, returning an empty string, same "be liberal" precedent + * this function's own caller uses for unsupported section shapes). + * + * Returns 0 on success (including the "no suffix at all" case), -1 if s + * is non-empty but doesn't match the grammar exactly. + */ +static int +parse_partial_suffix(const char *s, int *has_partial_out, + uint32_t *start_out, uint32_t *count_out) +{ + const char *p; + char *end; + unsigned long start, count; + + *has_partial_out = 0; + *start_out = 0; + *count_out = 0; + + if (s == NULL || *s == '\0') + return (0); + + if (s[0] != '<') + return (-1); + p = s + 1; + + if (*p < '0' || *p > '9') + return (-1); + errno = 0; + start = strtoul(p, &end, 10); + if (errno != 0 || start > UINT32_MAX || *end != '.') + return (-1); + p = end + 1; + + if (*p < '0' || *p > '9') + return (-1); + errno = 0; + count = strtoul(p, &end, 10); + if (errno != 0 || count > UINT32_MAX || *end != '>' || end[1] != '\0') + return (-1); + + *has_partial_out = 1; + *start_out = (uint32_t)start; + *count_out = (uint32_t)count; + return (0); +} + +/* + * RFC 9051 SS6.4.5: `fetch-att`, plus the "ALL" / "FULL" / "FAST" macros. + * spec is the fetch-att portion of the command line, with or without + * surrounding parentheses (a single un-parenthesized item, e.g. bare + * "FLAGS", is valid ABNF too), modified in place (strtok_r()). + * + * Returns 0 and fills *attrs_out on success; -1 (with *errmsg set) for a + * syntax error -- caller sends BAD. + * + * Real bug caught testing against Apple Mail live on premio: this used to + * return -2 (caller sends NO) the instant it saw a single recognized-but- + * unsupported item (BODY[...]/ENVELOPE/RFC822/etc, or the ALL/FULL macros + * that expand to include ENVELOPE) -- rejecting the *entire* fetch-att + * list, even when the same request also asked for several items this + * server fully supports. Apple Mail's actual first FETCH after SELECT was + * "FETCH 1:3 (INTERNALDATE UID RFC822.SIZE FLAGS BODY.PEEK[HEADER])" -- + * four fully-supported items bundled with one unsupported one -- so the + * old behavior meant Mail got nothing at all for any of it, not even the + * flags/dates/sizes it could have had, which (paired with LSUB not being + * implemented at all -- see list_dispatch()'s header comment above) left + * it with literally no data to render, hence a blank Inbox. + * + * Now: an unsupported item is silently skipped (not added to attrs) and + * parsing continues, rather than aborting the whole request -- the same + * "be liberal about what doesn't apply, answer what you can" leniency + * SS6.3.1 already mandates for ENABLE's unrecognized arguments. At the + * time this comment was first written, ALL/FULL both degraded to the FAST + * set once their ENVELOPE component was dropped; ENVELOPE is real now (see + * MBOX_FETCH_ENVELOPE below), so ALL is a complete macro and only FULL + * still degrades, to everything but BODY. *degraded_out is set to 1 + * whenever anything was silently + * dropped this way, purely so the caller can log it -- it doesn't change + * the wire response, which is now a normal OK with whatever data items + * *were* recognized. The -2/NO path is kept for the one case dropping + * still can't paper over: every requested item was unsupported (or the + * list was ALL/FULL alone, which -- unlike bundled with real items above + * -- has nothing left to fall back to without inventing data the client + * didn't ask for), so there would be nothing at all to answer with. + */ +static int +parse_fetch_atts(char *spec, uint32_t *attrs_out, int *degraded_out, + int *header_fields_not_out, char *header_fields_out, + size_t header_fields_outsize, char *header_fields_label_out, + size_t header_fields_label_outsize, int *bodystructure_full_out, + char *section_part_out, size_t section_part_outsize, + int *has_partial_out, uint32_t *partial_start_out, + uint32_t *partial_count_out, const char **errmsg) +{ + char *p, *tok, *save; + size_t len; + uint32_t attrs = 0; + int degraded = 0; + int has_partial = 0; + uint32_t partial_start = 0, partial_count = 0; + + *errmsg = NULL; + *attrs_out = 0; + *degraded_out = 0; + *header_fields_not_out = 0; + header_fields_out[0] = '\0'; + header_fields_label_out[0] = '\0'; + *bodystructure_full_out = 0; + section_part_out[0] = '\0'; + *has_partial_out = 0; + *partial_start_out = 0; + *partial_count_out = 0; + + if (spec == NULL || *spec == '\0') { + *errmsg = "missing message data item(s)"; + return (-1); + } + + p = spec; + len = strlen(p); + if (len >= 2 && p[0] == '(' && p[len - 1] == ')') { + p[len - 1] = '\0'; + p++; + } + if (*p == '\0') { + *errmsg = "empty message data item list"; + return (-1); + } + + for (tok = fetch_att_tok(p, &save); tok != NULL; + tok = fetch_att_tok(NULL, &save)) { + if (strcasecmp(tok, "FAST") == 0) { + /* SS6.4.5: "Macro equivalent to: (FLAGS INTERNALDATE + * RFC822.SIZE)" -- all three are real in v1, so FAST + * is a complete, correct macro here, same as ALL and + * (now that BODYSTRUCTURE is implemented too) FULL. */ + attrs |= MBOX_FETCH_FLAGS | MBOX_FETCH_INTERNALDATE | + MBOX_FETCH_RFC822_SIZE; + } else if (strcasecmp(tok, "ALL") == 0) { + /* SS6.4.5: "Macro equivalent to: (FLAGS INTERNALDATE + * RFC822.SIZE ENVELOPE)" -- all four are real in v1 as + * of the ENVELOPE pass, so ALL is now a complete, + * correct macro, not a degraded one. */ + attrs |= MBOX_FETCH_FLAGS | MBOX_FETCH_INTERNALDATE | + MBOX_FETCH_RFC822_SIZE | MBOX_FETCH_ENVELOPE; + } else if (strcasecmp(tok, "FULL") == 0) { + /* SS6.4.5: "Macro equivalent to: (FLAGS INTERNALDATE + * RFC822.SIZE ENVELOPE BODY)" -- BODY here is the bare, + * non-extensible form (MBOX_FETCH_BODYSTRUCTURE, see + * imapd.h), real as of this pass, so FULL is now a + * complete, correct macro too, not a degraded one. + * bodystructure_full_out stays 0 (the "BODY" label, + * not "BODYSTRUCTURE") since that's literally what the + * macro's own definition expands to. */ + attrs |= MBOX_FETCH_FLAGS | MBOX_FETCH_INTERNALDATE | + MBOX_FETCH_RFC822_SIZE | MBOX_FETCH_ENVELOPE | + MBOX_FETCH_BODYSTRUCTURE; + } else if (strcasecmp(tok, "FLAGS") == 0) { + attrs |= MBOX_FETCH_FLAGS; + } else if (strcasecmp(tok, "UID") == 0) { + attrs |= MBOX_FETCH_UID; + } else if (strcasecmp(tok, "INTERNALDATE") == 0) { + attrs |= MBOX_FETCH_INTERNALDATE; + } else if (strcasecmp(tok, "RFC822.SIZE") == 0) { + attrs |= MBOX_FETCH_RFC822_SIZE; + } else if (strcasecmp(tok, "MODSEQ") == 0) { + /* RFC 7162 SS3.1.4.2 fetch-mod-sequence: "MODSEQ" -- + * causes MODSEQ FETCH response data items, and is + * itself a CONDSTORE-enabling command (SS3.1) -- + * cmd_fetch() checks this bit to decide whether to + * call session_condstore_enable(). */ + attrs |= MBOX_FETCH_MODSEQ; + } else if (strcasecmp(tok, "BODY.PEEK[HEADER]") == 0) { + /* + * The one BODY[...] variant this pass actually + * implements -- see MBOX_FETCH_BODY_HEADER's comment + * in imapd.h for why it's scoped to exactly this + * token (RFC 5322 header, raw and unparsed, via + * store.c's read_message_header()) and not plain + * BODY[HEADER] (would need to implicitly set \Seen, + * not implemented), HEADER.FIELDS/.NOT, TEXT, or any + * MIME-part addressing. An exact strcasecmp() match, + * checked before the generic BODY-prefix catch-all + * below so this one recognized case doesn't fall into + * it and get marked degraded instead. + */ + attrs |= MBOX_FETCH_BODY_HEADER; + } else if (strncasecmp(tok, "BODY.PEEK[", strlen("BODY.PEEK[")) == + 0 && strncasecmp(tok, "BODY.PEEK[HEADER.FIELDS", + strlen("BODY.PEEK[HEADER.FIELDS")) != 0) { + /* + * Reached for every BODY.PEEK[...] shape except + * HEADER (exact match above) and HEADER.FIELDS[.NOT] + * (own prefix branch just below, excluded from this + * one by the strncasecmp() != 0 above so the two + * don't fight over the same token): BODY.PEEK[] + * (whole message, SS6.4.5: "If BODY[] is specified + * ... the FETCH is requesting the [RFC5322] + * expression of the entire message"), BODY.PEEK + * [TEXT] (SS6.4.5.1: "the text body of the message, + * omitting the [RFC5322] header"), or BODY.PEEK + * [] (MIME part-addressed content, + * e.g. "3.1" -- see MBOX_FETCH_BODY_PART's comment + * in imapd.h), each optionally followed by a + * <> partial-range suffix (SS6.4.5, + * e.g. "BODY.PEEK[TEXT]<0.16384>" -- the exact shape + * observed from real Apple Mail traffic; see + * parse_partial_suffix()'s own comment). Unified into + * one branch (replacing this pass's former separate + * exact-match BODY.PEEK[]/BODY.PEEK[TEXT] cases) since + * all three now share the same "parse the bracket + * body, then the optional trailing <...>" shape. + * + * Same .PEEK-only, checked-before-the-generic-catch- + * all scoping as BODY.PEEK[HEADER] above, for the + * same reason (plain BODY[...] implicitly sets \Seen, + * not implemented for any content item). + */ + const char *bracket_start = tok + + strlen("BODY.PEEK["); + char *close; + char inner[SECTION_PART_MAX]; + const char *suffix; + + close = strchr(bracket_start, ']'); + if (close == NULL) { + degraded = 1; /* not even well-bracketed -- + * same lenient "unrecognized + * BODY[...] shape, skip it" + * handling every other + * unsupported form here gets, + * rather than a new BAD path */ + continue; + } + if ((size_t)(close - bracket_start) >= sizeof(inner)) { + degraded = 1; + continue; + } + memcpy(inner, bracket_start, close - bracket_start); + inner[close - bracket_start] = '\0'; + suffix = close + 1; + + if (suffix[0] != '\0' && + parse_partial_suffix(suffix, &has_partial, + &partial_start, &partial_count) == -1) { + *errmsg = "malformed range"; + return (-1); + } + + if (inner[0] == '\0') { + attrs |= MBOX_FETCH_BODY_WHOLE; + } else if (strcasecmp(inner, "TEXT") == 0) { + attrs |= MBOX_FETCH_BODY_TEXT; + } else if (section_part_valid(inner)) { + attrs |= MBOX_FETCH_BODY_PART; + if (strlcpy(section_part_out, inner, + section_part_outsize) >= + section_part_outsize) { + attrs &= ~MBOX_FETCH_BODY_PART; + degraded = 1; + continue; + } + } else { + degraded = 1; /* recognized-shape-but- + * unsupported section, e.g. + * nested MESSAGE/RFC822 + * numbering ("2.1.TEXT") -- + * same silent-skip precedent + * as the generic BODY-prefix + * catch-all below */ + continue; + } + } else if (strncasecmp(tok, "BODY.PEEK[HEADER.FIELDS", + strlen("BODY.PEEK[HEADER.FIELDS")) == 0) { + /* + * "BODY.PEEK[HEADER.FIELDS (...)"/"BODY.PEEK[HEADER. + * FIELDS.NOT (...)" -- checked by prefix (not exact + * match, unlike BODY.PEEK[HEADER]/[]/[TEXT] above) + * since the field-name list itself varies per + * request. tok is already the whole bracketed atom + * here, embedded space and all, thanks to fetch_att_ + * tok() -- see that function's comment for the real + * bug this replaced. Strips the outer "BODY.PEEK[" + * and trailing "]" here (both already confirmed + * present by the strncasecmp() prefix match and the + * closing-bracket check below) and hands the + * "HEADER.FIELDS[...] (...)" interior to parse_ + * header_fields_att() for the real grammar work. + * A second HEADER.FIELDS-shaped item in the same + * FETCH (legal per SS6.4.5, unseen from any real + * client) is silently ignored once one has already + * been captured -- same "first one wins, no error" + * simplification as MBOX_FETCH_BODY_HEADER's + * precedence over this bit, decided below. + */ + size_t toklen = strlen(tok); + char inner[HEADER_FIELDS_LABEL_MAX]; + + if (toklen < strlen("BODY.PEEK[") + 1 || + tok[toklen - 1] != ']') { + *errmsg = "malformed HEADER.FIELDS section"; + return (-1); + } + if (attrs & MBOX_FETCH_HEADER_FIELDS) + continue; /* already captured one -- ignore + any further duplicates */ + + if (toklen - strlen("BODY.PEEK[") - 1 >= + sizeof(inner)) { + *errmsg = "HEADER.FIELDS section too long"; + return (-1); + } + memcpy(inner, tok + strlen("BODY.PEEK["), + toklen - strlen("BODY.PEEK[") - 1); + inner[toklen - strlen("BODY.PEEK[") - 1] = '\0'; + + if (parse_header_fields_att(inner, + header_fields_not_out, header_fields_out, + header_fields_outsize) == -1) { + *errmsg = "malformed HEADER.FIELDS section"; + return (-1); + } + if (strlcpy(header_fields_label_out, inner, + header_fields_label_outsize) >= + header_fields_label_outsize) { + *errmsg = "HEADER.FIELDS section too long"; + return (-1); + } + attrs |= MBOX_FETCH_HEADER_FIELDS; + } else if (strcasecmp(tok, "ENVELOPE") == 0) { + /* + * RFC 9051 SS7.5.2 ENVELOPE -- see MBOX_FETCH_ENVELOPE's + * comment in imapd.h for the full scoping story + * (parsed RFC 5322 header fields + a deliberately + * scoped-down address-list parser, no MIME awareness -- + * BODYSTRUCTURE, just below, is the one with that). + * No .PEEK variant + * and no \Seen side effect to avoid, so -- unlike the + * BODY.PEEK[...] family above -- the bare token is + * enough. + */ + attrs |= MBOX_FETCH_ENVELOPE; + } else if (strcasecmp(tok, "BODY") == 0 || + strcasecmp(tok, "BODYSTRUCTURE") == 0) { + /* + * RFC 9051 SS9's fetch-att: `"BODY" ["STRUCTURE"]` -- + * bare "BODY" (no brackets) and "BODYSTRUCTURE" are + * both requests for the non-extensible body structure + * (see MBOX_FETCH_BODYSTRUCTURE's comment in + * imapd.h for why this implementation's BODY and + * BODYSTRUCTURE produce byte-identical output -- no + * extension data is ever emitted). Checked by exact + * match, before the generic "BODY" prefix catch-all + * below, so these two recognized cases don't fall + * into it and get marked degraded instead -- same + * precedent as BODY.PEEK[HEADER]/[]/[TEXT] above. + * bodystructure_full_out records which literal token + * was used, purely so session_send_fetch_response() + * can echo the same label back (SS9's grammar: `"BODY" + * ["STRUCTURE"] SP body` -- the response label itself + * is "BODY" or "BODYSTRUCTURE", not a separate response + * name). If a client somehow requests both in the same + * FETCH (legal, redundant, unseen from any real + * client), whichever is parsed last simply wins the + * label -- a low-stakes, purely cosmetic difference, + * not worth a "first wins" guard. + */ + attrs |= MBOX_FETCH_BODYSTRUCTURE; + *bodystructure_full_out = + (strcasecmp(tok, "BODYSTRUCTURE") == 0); + } else if (strncasecmp(tok, "BODY", 4) == 0 || + strcasecmp(tok, "RFC822") == 0 || + strcasecmp(tok, "RFC822.HEADER") == 0 || + strcasecmp(tok, "RFC822.TEXT") == 0) { + /* BODY[...]/BODY.PEEK[...] beyond the four items + * already implemented above (whole message, TEXT, + * HEADER, HEADER.FIELDS[.NOT]) -- specifically MIME + * part-addressed content, e.g. BODY[1.2] or BODY.PEEK + * [2.TEXT], which BODYSTRUCTURE (just above) only + * *describes* the existence of, never returns the + * content of -- plus the RFC822(.HEADER/.TEXT) content + * shorthands, need actual per-part content extraction + * beyond what's implemented, not designed this pass. + * strncasecmp() (not strcasecmp()) for the BODY + * prefix specifically catches BODY[...]/BODY.PEEK[...] + * tokens too, even though strtok_r() has already + * split a bracketed fetch-att with embedded spaces + * (e.g. "BODY[HEADER.FIELDS (DATE FROM)]") into + * multiple tokens -- the first such token alone is + * enough to recognize and skip the whole item. Note + * this prefix match would also catch bare "BODY"/ + * "BODYSTRUCTURE" if they ever reached here, but the + * exact-match branch just above already claims both + * first. Silently dropped now (see header comment) + * rather than aborting the whole FETCH. */ + degraded = 1; + } else { + *errmsg = "unknown message data item"; + return (-1); + } + } + + if (attrs == 0) { + /* Every requested item was unsupported -- nothing left to + * answer with, unlike the bundled case this function now + * handles gracefully. ALL/FAST/FULL are all complete macros + * as of the BODYSTRUCTURE pass, so this is only reached by a + * request naming exclusively still-unsupported items, e.g. + * BODY[]/BODY.PEEK[] (MIME part-addressed + * content) or RFC822/RFC822.HEADER/RFC822.TEXT alone. */ + *errmsg = "cannot fetch that message content yet -- " + "supported: FLAGS/UID/INTERNALDATE/RFC822.SIZE/MODSEQ/" + "ENVELOPE/(BODY|BODYSTRUCTURE)/BODY.PEEK[...]"; + return (-2); + } + + /* + * If a client somehow requested both plain BODY.PEEK[HEADER] and a + * BODY.PEEK[HEADER.FIELDS...] variant in the same FETCH (legal per + * SS6.4.5, unseen from any real client so far), HEADER wins -- + * there's only one pending_header_* slot on struct session, and the + * whole header is a strict superset of any subset of it, same "more + * general variant wins" precedent MBOX_FETCH_BODY_WHOLE already + * uses over MBOX_FETCH_BODY_TEXT. + */ + if ((attrs & MBOX_FETCH_BODY_HEADER) && + (attrs & MBOX_FETCH_HEADER_FIELDS)) + attrs &= ~MBOX_FETCH_HEADER_FIELDS; + + *attrs_out = attrs; + *degraded_out = degraded; + /* + * has_partial/partial_start/partial_count were accumulated into + * local variables (not written straight to the out-params) by + * whichever BODY.PEEK[...] branch matched above, mirroring how attrs + * itself is built up in a local before this one final copy-out -- + * done here, not per-branch, so the "if a client requests both + * BODY.PEEK[HEADER] and BODY.PEEK[HEADER.FIELDS...], HEADER wins" + * resolution just above stays the single place attrs gets adjusted + * after parsing, without also needing a matching adjustment to which + * item's partial range should apply. + */ + *has_partial_out = has_partial; + *partial_start_out = partial_start; + *partial_count_out = partial_count; + return (0); +} + +static const char *fetch_month_names[12] = { + "Jan", "Feb", "Mar", "Apr", "May", "Jun", + "Jul", "Aug", "Sep", "Oct", "Nov", "Dec" +}; + +/* + * RFC 9051 SS9: `date-time = DQUOTE date-day-fixed "-" date-month "-" + * date-year SP time SP zone DQUOTE`, `date-day-fixed = (SP DIGIT) / + * 2DIGIT` (space-padded, not zero-padded, below 10), `zone = ("+" / "-") + * 4DIGIT`. Always formats in UTC ("+0000"): ts is a bare Unix timestamp + * (store.c's parse_maildir_timestamp()) with no timezone information + * attached at all, a chroot'd store child can't be assumed to have tzdata + * unveiled, and "+0000" is unambiguous -- a deliberate implementation + * simplification, not something the RFC itself requires (it permits any + * valid zone offset). + */ +static void +format_internaldate(int64_t ts, char *out, size_t outsize) +{ + struct tm tm; + time_t t = (time_t)ts; + + if (gmtime_r(&t, &tm) == NULL) { + strlcpy(out, "01-Jan-1970 00:00:00 +0000", outsize); + return; + } + + snprintf(out, outsize, "%2d-%s-%04d %02d:%02d:%02d +0000", + tm.tm_mday, fetch_month_names[tm.tm_mon], tm.tm_year + 1900, + tm.tm_hour, tm.tm_min, tm.tm_sec); +} + +/* + * snprintf(3)-into-a-growing-buffer helper for session_send_fetch_ + * response(): appends at buf + *len, then advances *len by however much + * was (or would have been) written. Guards against the exact bug flagged + * while this function was being designed -- naively chaining + * `snprintf(buf + len, sizeof(buf) - len, ...)` calls is only safe as long + * as len never exceeds sizeof(buf); if an earlier call were ever truncated, + * snprintf(3)'s return value (the length that *would* have been written) + * can push len past sizeof(buf), and the next call's `sizeof(buf) - len` + * would underflow (size_t is unsigned) into a huge value. Clamping *len to + * bufsize here, plus the "*len >= bufsize" early return, means every + * subsequent call sees a valid, non-negative remaining size instead. + */ +static void +fetch_append(char *buf, size_t bufsize, size_t *len, const char *fmt, ...) +{ + va_list ap; + int n; + + if (*len >= bufsize) + return; + + va_start(ap, fmt); + n = vsnprintf(buf + *len, bufsize - *len, fmt, ap); + va_end(ap); + + if (n < 0) + return; + + *len += (size_t)n; + if (*len > bufsize) + *len = bufsize; +} + +/* + * Formats and sends one untagged "* FETCH (...)" response (RFC + * 9051 SS7.5.2) for a single IMSG_MBOX_FETCH_META reply. Only prints the + * data items s->fetch_attrs actually requested: store.c's handle_mbox_ + * fetch() always populates every field of struct imsg_mbox_fetch_meta it + * can regardless of what was asked for (see that struct's comment in + * imapd.h), so this function is what actually enforces "don't show the + * client attributes it didn't ask for". + * + * Field order (FLAGS, UID, INTERNALDATE, RFC822.SIZE) matches the bit + * order of the MBOX_FETCH_* constants, not anything RFC 9051 requires -- + * SS7.5.2 doesn't mandate a msg-att ordering. + * + * buf is sized generously (MBOX_FLAGS_MAX plus comfortable room for the + * other three items' text) but, like every other fixed buffer in this + * file, truncates rather than overflows if somehow exceeded; session_ + * untagged()'s own 512-byte buffer is the tighter, and ultimately + * governing, bound in that unlikely case -- same "truncate rather than + * overflow" style as session_reply()/session_untagged() themselves. + * + * BODY.PEEK[HEADER]/BODY.PEEK[]/BODY.PEEK[TEXT] addition: when any of these + * were requested and store.c actually found something for this message + * (s->pending_header_found / s->pending_body_found, stashed by the IMSG_ + * MBOX_FETCH_HEADER / IMSG_MBOX_FETCH_BODY cases just before this IMSG_ + * MBOX_FETCH_META arrived -- see struct session's comment), the response + * can't be built as one NUL-terminated string handed to session_untagged() + * the way every other item above is: RFC 9051 SS9's literal syntax puts a + * "{n}\r\n" marker followed by exactly n raw octets (which may contain any + * byte except NUL -- store.c's read_message_header()/read_message_body() + * already reject content containing one) directly in the middle of the + * response, with the closing ")" continuing right after those n octets, + * not on a fresh "line" in the usual CRLF-delimited sense. A client can + * legally request both BODY.PEEK[HEADER] and one of BODY.PEEK[]/BODY.PEEK + * [TEXT] in the same FETCH (SS6.4.5 doesn't forbid combining section + * specs), so this function has to be able to splice in zero, one, or two + * such literal blocks -- not just the single hardcoded case the header- + * only version of this function had. Each literal block gets its own + * "flush buf so far as one raw write, then write the raw payload bytes" + * cycle; buf/len are reused (reset) between blocks. Header is always + * spliced in before body when both are present, matching MBOX_FETCH_* + * bit order. + * + * ENVELOPE addition: s->pending_envelope_buf holds build_envelope()'s + * *already-formatted* "(...)" text (see struct imsg_mbox_fetch_envelope's + * comment in imapd.h), not raw message bytes needing a `{n}` literal + * wrapper -- but it still can't be handed to fetch_append() into buf[768] + * above, since a formatted envelope can be far larger than that (up to + * ENVELOPE_MAX, 8192 bytes) and fetch_append() truncates rather than + * overflows. So ENVELOPE reuses the same "flush buf, then a second raw + * session_write() for the oversized part" mechanism the header/body + * literals use, just without a `{n}\r\n` marker in front of it -- it's + * written as "ENVELOPE " followed directly by the pre-formatted text, no + * literal syntax involved at all. This also means an ENVELOPE-only FETCH + * (no BODY.PEEK[...] items at all) now takes this branch too, where + * before this addition only BODY.PEEK[...] requests ever did -- see + * have_envelope below. + * + * BODYSTRUCTURE addition: same "already-formatted text, no literal, flush + * buf first" shape as ENVELOPE, just written as "BODY " or "BODYSTRUCTURE " + * (s->pending_bodystructure_label -- see struct session's comment on that + * field for why the label has to echo whichever bare token the client + * used) followed by build_bodystructure()'s text. + */ +static void +session_send_fetch_response(struct session *s, + struct imsg_mbox_fetch_meta *meta) +{ + char buf[768]; + char date[40]; + size_t len = 0; + int need_sp = 0; + int have_header = (s->fetch_attrs & (MBOX_FETCH_BODY_HEADER | + MBOX_FETCH_HEADER_FIELDS)) && s->pending_header_found; + int have_body = (s->fetch_attrs & + (MBOX_FETCH_BODY_WHOLE | MBOX_FETCH_BODY_TEXT | + MBOX_FETCH_BODY_PART)) && s->pending_body_found; + int have_envelope = (s->fetch_attrs & MBOX_FETCH_ENVELOPE) && + s->pending_envelope_found; + int have_bodystructure = (s->fetch_attrs & MBOX_FETCH_BODYSTRUCTURE) && + s->pending_bodystructure_found; + + fetch_append(buf, sizeof(buf), &len, "%u FETCH (", meta->seqno); + + if (s->fetch_attrs & MBOX_FETCH_FLAGS) { + fetch_append(buf, sizeof(buf), &len, "FLAGS (%s)", meta->flags); + need_sp = 1; + } + if (s->fetch_attrs & MBOX_FETCH_UID) { + fetch_append(buf, sizeof(buf), &len, "%sUID %u", + need_sp ? " " : "", meta->uid); + need_sp = 1; + } + if (s->fetch_attrs & MBOX_FETCH_INTERNALDATE) { + format_internaldate(meta->internaldate, date, sizeof(date)); + fetch_append(buf, sizeof(buf), &len, "%sINTERNALDATE \"%s\"", + need_sp ? " " : "", date); + need_sp = 1; + } + if (s->fetch_attrs & MBOX_FETCH_RFC822_SIZE) { + fetch_append(buf, sizeof(buf), &len, "%sRFC822.SIZE %llu", + need_sp ? " " : "", (unsigned long long)meta->size); + need_sp = 1; + } + if (s->fetch_attrs & MBOX_FETCH_MODSEQ) { + /* RFC 7162 SS3.1.4.2 fetch-mod-resp: "MODSEQ" SP "(" + * permsg-modsequence ")". */ + fetch_append(buf, sizeof(buf), &len, "%sMODSEQ (%llu)", + need_sp ? " " : "", (unsigned long long)meta->modseq); + need_sp = 1; + } + + if (have_header || have_body || have_envelope || have_bodystructure) { + session_write(s, "* ", 2); + session_write(s, buf, len); + + if (have_envelope) { + len = 0; + fetch_append(buf, sizeof(buf), &len, "%sENVELOPE ", + need_sp ? " " : ""); + session_write(s, buf, len); + if (s->pending_envelope_len > 0) + session_write(s, s->pending_envelope_buf, + s->pending_envelope_len); + need_sp = 1; + } + if (have_bodystructure) { + len = 0; + fetch_append(buf, sizeof(buf), &len, "%s%s ", + need_sp ? " " : "", s->pending_bodystructure_label); + session_write(s, buf, len); + if (s->pending_bodystructure_len > 0) + session_write(s, s->pending_bodystructure_buf, + s->pending_bodystructure_len); + need_sp = 1; + } + if (have_header) { + len = 0; + fetch_append(buf, sizeof(buf), &len, + "%sBODY[%s] {%u}\r\n", need_sp ? " " : "", + s->pending_header_label, s->pending_header_len); + session_write(s, buf, len); + if (s->pending_header_len > 0) + session_write(s, s->pending_header_buf, + s->pending_header_len); + need_sp = 1; + } + if (have_body) { + /* + * RFC 9051 SS6.4.5: "The origin octet facility MUST + * NOT be used by a server in a FETCH response unless + * the client specifically requested it" -- and even + * then, only the *origin* (pending_body_partial_ + * origin, the requested start octet) is echoed, never + * the count store.c actually returned. pending_body_ + * label already holds the right text regardless of + * which of WHOLE/TEXT/PART was selected -- see that + * field's own comment. + */ + len = 0; + if (s->pending_body_has_partial) + fetch_append(buf, sizeof(buf), &len, + "%sBODY[%s]<%u> {%u}\r\n", + need_sp ? " " : "", s->pending_body_label, + s->pending_body_partial_origin, + s->pending_body_len); + else + fetch_append(buf, sizeof(buf), &len, + "%sBODY[%s] {%u}\r\n", need_sp ? " " : "", + s->pending_body_label, s->pending_body_len); + session_write(s, buf, len); + if (s->pending_body_len > 0) + session_write(s, s->pending_body_buf, + s->pending_body_len); + } + session_write(s, ")\r\n", 3); + } else { + fetch_append(buf, sizeof(buf), &len, ")"); + + if (len >= sizeof(buf)) + log_warnx("session %u: FETCH response for seq %u " + "truncated", s->id, meta->seqno); + + session_untagged(s, buf); + } + + /* + * Reset pending_header_* and pending_body_* regardless of whether + * this particular message had anything to give (have_header/have_body + * false just means store.c found nothing -- pending_*_found itself + * still needs resetting so it doesn't leak into the next message's + * response, same reasoning the header-only version of this function + * already documented). + */ + if (have_header) { + free(s->pending_header_buf); + s->pending_header_buf = NULL; + s->pending_header_len = 0; + } + if (s->fetch_attrs & (MBOX_FETCH_BODY_HEADER | MBOX_FETCH_HEADER_FIELDS)) + s->pending_header_found = 0; + + if (have_body) { + free(s->pending_body_buf); + s->pending_body_buf = NULL; + s->pending_body_len = 0; + } + if (s->fetch_attrs & (MBOX_FETCH_BODY_WHOLE | MBOX_FETCH_BODY_TEXT | + MBOX_FETCH_BODY_PART)) + s->pending_body_found = 0; + + if (have_envelope) { + free(s->pending_envelope_buf); + s->pending_envelope_buf = NULL; + s->pending_envelope_len = 0; + } + if (s->fetch_attrs & MBOX_FETCH_ENVELOPE) + s->pending_envelope_found = 0; + + if (have_bodystructure) { + free(s->pending_bodystructure_buf); + s->pending_bodystructure_buf = NULL; + s->pending_bodystructure_len = 0; + } + if (s->fetch_attrs & MBOX_FETCH_BODYSTRUCTURE) + s->pending_bodystructure_found = 0; +} + +/* + * STORE's untagged FETCH response (RFC 9051 SS6.4.6's own example: "* 2 + * FETCH (FLAGS (\Deleted \Seen))") shows FLAGS, regardless of what the + * original STORE actually changed -- unlike FETCH's own response, there's + * no s->fetch_attrs-style bitmask to consult here because STORE only ever + * has one client-requested data item to report. meta->flags is store.c's + * handle_mbox_store()-computed post-STORE value, not the client-supplied + * delta. + * + * RFC 7162 addition this pass: also shows MODSEQ whenever this session is + * CONDSTORE-aware (s->condstore_enabled), independent of whether *this* + * particular STORE used UNCHANGEDSINCE -- SS3.1.3's own Examples 3-10 all + * show MODSEQ in the STORE echo purely because CONDSTORE was already + * enabled for the session, not because each individual STORE requested it + * (STORE has no fetch-att list to request it with in the first place). + * meta->modseq is always populated by store.c regardless (see imapd.h's + * imsg_mbox_fetch_meta comment), so this is purely a "whether to print it" + * decision, same split every other field in this struct already uses. + */ +static void +session_send_store_fetch_response(struct session *s, + struct imsg_mbox_fetch_meta *meta) +{ + char buf[MBOX_FLAGS_MAX + 96]; + size_t len; + + /* + * RFC 9051 SS6.4.9 addition: a UID STORE's echo must include UID too + * (see struct session's cmd_by_uid comment) -- inserted right after + * FLAGS, matching SS6.4.9's own UID FETCH example's FLAGS-then-UID + * order ("FLAGS (\Seen) UID 4827313"); element order within a + * msg-att's parenthesized list isn't itself semantically significant + * (RFC 9051 SS9's msg-att grammar is an unordered set of + * alternatives), so there's no conflict with RFC 7162 SS3.1.3's own + * example showing UID before MODSEQ in a *different* (FLAGS-omitted) + * echo shape. + */ + len = (size_t)snprintf(buf, sizeof(buf), "%u FETCH (FLAGS (%s)", + meta->seqno, meta->flags); + if (s->cmd_by_uid && len < sizeof(buf)) + len += (size_t)snprintf(buf + len, sizeof(buf) - len, + " UID %u", meta->uid); + if (s->condstore_enabled && len < sizeof(buf)) + len += (size_t)snprintf(buf + len, sizeof(buf) - len, + " MODSEQ (%llu)", (unsigned long long)meta->modseq); + if (len < sizeof(buf)) + snprintf(buf + len, sizeof(buf) - len, ")"); + + session_untagged(s, buf); +} + +/* + * Splits a trailing RFC 4466 modifier list -- "[SP '(' modifier *(SP + * modifier) ')']" -- off of spec (a fetch-att or store-att-flags spec that + * may be followed by one), used by both cmd_fetch() (modifiers trail the + * fetch-att list) and cmd_store_cmd() (modifiers instead lead, before + * store-att-flags -- see that function's own comment on why it calls this + * on a different substring). spec is modified in place: NUL-terminated + * right after its own portion, with the returned pointer (or NULL, if + * nothing trails) pointing at the still-parenthesized modifier text for + * the caller's own parser to strip and tokenize. + * + * Depth-counts through spec's own parens (rather than assuming it's never + * parenthesized) so this works whether spec itself is a bare token + * ("FLAGS") or a parenthesized list ("(FLAGS UID)") -- v1 doesn't need to + * handle nested parens *within* spec (no fetch-att or flag needs them), + * but counting depth anyway costs nothing and avoids assuming that stays + * true forever. + */ +static char * +split_trailing_modifiers(char *spec) +{ + char *p = spec; + + if (*p == '(') { + int depth = 0; + + for (;;) { + if (*p == '(') + depth++; + else if (*p == ')') { + depth--; + if (depth == 0) { + p++; + break; + } + } else if (*p == '\0') + return (NULL); /* unterminated -- let the + * caller's own parser produce + * the BAD for this */ + p++; + } + } else { + while (*p != '\0' && *p != ' ') + p++; + } + + if (*p == '\0') + return (NULL); + *p++ = '\0'; + while (*p == ' ') + p++; + if (*p == '\0') + return (NULL); + return (p); +} + +/* + * Parses a FETCH command's trailing fetch-modifier list (RFC 4466's generic + * syntax, extended by RFC 7162 SS3.1.4.1/SS3.2.6). modspec is the still- + * parenthesized text split off by split_trailing_modifiers() above, + * modified in place. + * + * CHANGEDSINCE is the only fetch-modifier this server + * implements; it implicitly sets MBOX_FETCH_MODSEQ (SS3.1.4.1: "implicitly + * adds the MODSEQ FETCH message data item"). VANISHED (RFC 7162 SS3.2.6) is + * only legal on UID FETCH (by_uid) and only once this session has "ENABLE + * QRESYNC"'d -- both a tagged BAD otherwise, matching parse_select_params()'s + * existing pattern for QRESYNC select-params' own "requires ENABLE QRESYNC + * first" check. VANISHED's *other* restriction -- "MUST only be specified + * together with the CHANGEDSINCE UID FETCH modifier" -- can't be checked + * here: CHANGEDSINCE might appear later in the same modifier list (order + * isn't fixed), so fetch_dispatch() checks that combination itself, once + * the whole list has been parsed; *want_vanished just carries "VANISHED was + * present, syntax was fine" out to it. Anything else is an unrecognized + * modifier -- BAD, since v1 defines no other fetch-modifier at all for this + * server to legitimately ignore the way ENABLE ignores unknown capabilities. + */ +static int +parse_fetch_modifiers(char *modspec, struct imsg_mbox_fetch *req, + struct session *s, int by_uid, int *want_vanished, const char **errmsg) +{ + char *p, *tok, *save; + size_t len; + + *errmsg = NULL; + *want_vanished = 0; + len = strlen(modspec); + if (len < 2 || modspec[0] != '(' || modspec[len - 1] != ')') { + *errmsg = "malformed fetch-modifier list"; + return (-1); + } + modspec[len - 1] = '\0'; + p = modspec + 1; + + for (tok = strtok_r(p, " ", &save); tok != NULL; + tok = strtok_r(NULL, " ", &save)) { + if (strcasecmp(tok, "CHANGEDSINCE") == 0) { + char *valtok = strtok_r(NULL, " ", &save); + char *ep; + + if (valtok == NULL) { + *errmsg = "CHANGEDSINCE requires a " + "mod-sequence value"; + return (-1); + } + errno = 0; + req->changedsince = strtoull(valtok, &ep, 10); + if (*ep != '\0' || errno != 0) { + *errmsg = "invalid CHANGEDSINCE mod-sequence"; + return (-1); + } + req->has_changedsince = 1; + req->attrs |= MBOX_FETCH_MODSEQ; + } else if (strcasecmp(tok, "VANISHED") == 0) { + if (!by_uid) { + /* RFC 7162 SS3.2.6: "the VANISHED UID FETCH + * modifier is NOT allowed with a FETCH + * command. The server MUST return a tagged + * BAD response..." */ + *errmsg = "VANISHED is only valid as a UID " + "FETCH modifier (RFC 7162 SS3.2.6)"; + return (-1); + } + if (!s->qresync_enabled) { + *errmsg = "VANISHED requires ENABLE QRESYNC " + "first (RFC 7162 SS3.2.6)"; + return (-1); + } + *want_vanished = 1; + } else { + *errmsg = "unrecognized fetch modifier"; + return (-1); + } + } + + return (0); +} + +/* + * RFC 9051 SS6.4.5: `fetch = "FETCH" SP sequence-set SP ("ALL" / "FULL" / + * "FAST" / fetch-att / "(" fetch-att *(SP fetch-att) ")")`, extended by RFC + * 4466/RFC 7162 with an optional trailing fetch-modifier list: `[SP "(" + * fetch-modifier *(SP fetch-modifier) ")"]`. + * + * v1 scope: message METADATA (FLAGS, UID, INTERNALDATE, RFC822.SIZE, + * MODSEQ), plus, across six real-client/hand-built-testing passes, + * BODY.PEEK[HEADER], BODY.PEEK[]/BODY.PEEK[TEXT], BODY.PEEK[HEADER.FIELDS + * (...)]/BODY.PEEK[HEADER.FIELDS.NOT (...)] (see imapd.h's MBOX_FETCH_ + * BODY_* / MBOX_FETCH_HEADER_FIELDS comments), ENVELOPE (see MBOX_FETCH_ + * ENVELOPE's comment) -- the parsed, structured RFC 5322 header summary + * most clients need for a message list view -- and BODY/BODYSTRUCTURE (see + * MBOX_FETCH_BODYSTRUCTURE's comment), full recursive MIME structure + * parsing bounded by MIME_MAX_DEPTH/MIME_MAX_PARTS, without RFC 9051's + * optional extension data. Everything else content-related -- plain + * BODY[HEADER]/BODY[]/BODY[TEXT]/BODY[HEADER.FIELDS...] without .PEEK + * (would implicitly set \Seen, a real design decision about flag-mutation- + * during-FETCH not taken any of these passes), and BODY[]/BODY.PEEK + * [] (MIME part-*addressed content*, i.e. actually returning one + * specific part's bytes -- distinct from BODYSTRUCTURE, which only + * describes the part tree's existence) -- remains a deliberate, flagged + * scope cut, not full FETCH. parse_fetch_atts() silently drops each of + * those (see its own header comment) rather than rejecting the whole + * request. + * + * Also v1-scoped: exactly one sequence-set range or number per command, + * never a comma-separated list -- rejected below with BAD rather than + * silently fetching only the first sub-range (avoids needing a multi-range + * queue in struct session, whose pending_tag/fetch_attrs fields already + * only track a single in-flight async operation at a time). + */ +static int +cmd_fetch(struct session *s, const char *tag, char *args) +{ + return fetch_dispatch(s, tag, args, 0); +} + +/* + * Shared body for cmd_fetch() (by_uid=0) and cmd_uid()'s FETCH branch + * (by_uid=1) -- see cmd_fetch()'s own comment for the grammar/scope this + * parses. RFC 9051 SS6.4.9 additions this pass, all gated on by_uid: + * + * - The sequence-set argument is resolved as UIDs, not sequence numbers + * (req.by_uid, threaded to store.c -- see handle_mbox_fetch()'s comment + * in store.c for the actual resolution). + * - MBOX_FETCH_UID is forced into attrs regardless of what the client's + * fetch-att list asked for (SS6.4.9: "server implementations MUST + * implicitly include the UID message data item as part of any FETCH + * response caused by a UID command") -- session_send_fetch_response() + * needs no changes at all for this, since it already prints UID whenever + * that bit is set. + * - VANISHED is legal as a fetch-modifier (RFC 7162 SS3.2.6), checked by + * parse_fetch_modifiers() itself for the by_uid/ENABLE QRESYNC + * restrictions; the remaining restriction -- VANISHED requires + * CHANGEDSINCE also be present -- is checked here, after the full + * modifier list has been parsed (order-independent). + * - s->cmd_by_uid is set unconditionally (to by_uid itself) before + * dispatch, same as store_do()/search_dispatch()/session_request_ + * expunge() -- see that field's own comment in struct session. FETCH's + * own response formatting doesn't consult it (see above), but it's set + * anyway for consistency and because a later command in the same + * session must never see a stale value from this one. + */ +static int +fetch_dispatch(struct session *s, const char *tag, char *args, int by_uid) +{ + struct imsg_mbox_fetch req; + char *seqtok, *attspec, *modspec; + uint32_t lo, hi, attrs; + int lo_star, hi_star, rc, want_vanished = 0, degraded; + int header_fields_not = 0; + char header_fields[HEADER_FIELDS_MAX]; + char header_fields_label[HEADER_FIELDS_LABEL_MAX]; + int bodystructure_full = 0; + char section_part[SECTION_PART_MAX]; + int has_partial = 0; + uint32_t partial_start = 0, partial_count = 0; + const char *errmsg; + const char *cmdname = by_uid ? "UID FETCH" : "FETCH"; + + if (args == NULL) { + session_reply(s, tag, "BAD", + "FETCH requires a sequence set and message data item(s)"); + return (1); + } + + seqtok = args; + while (*args != '\0' && *args != ' ') + args++; + if (*args == '\0') { + session_reply(s, tag, "BAD", + "FETCH requires message data item(s)"); + return (1); + } + *args++ = '\0'; + while (*args == ' ') + args++; + attspec = args; + + if (strchr(seqtok, ',') != NULL) { + session_reply(s, tag, "BAD", + "comma-separated sequence sets not supported in v1 -- " + "issue separate FETCH commands"); + return (1); + } + + if (parse_seq_range(seqtok, &lo, &hi, &lo_star, &hi_star) == -1) { + session_reply(s, tag, "BAD", "invalid sequence set"); + return (1); + } + + modspec = split_trailing_modifiers(attspec); + + rc = parse_fetch_atts(attspec, &attrs, °raded, &header_fields_not, + header_fields, sizeof(header_fields), header_fields_label, + sizeof(header_fields_label), &bodystructure_full, section_part, + sizeof(section_part), &has_partial, &partial_start, + &partial_count, &errmsg); + if (rc == -1) { + session_reply(s, tag, "BAD", errmsg); + return (1); + } + if (rc == -2) { + session_reply(s, tag, "NO", errmsg); + return (1); + } + if (degraded) + log_debug("session %u: %s: one or more unsupported message " + "data items silently dropped (plain BODY[...]/BODY[]" + "/BODY.PEEK[] with MESSAGE/RFC822|GLOBAL or MULTIPART" + " nested numbering/RFC822[.HEADER/.TEXT]) -- answering " + "with whatever was recognized", s->id, cmdname); + + memset(&req, 0, sizeof(req)); + req.attrs = attrs; + req.by_uid = by_uid; + req.header_fields_not = header_fields_not; + strlcpy(req.header_fields, header_fields, sizeof(req.header_fields)); + + /* + * has_partial/partial_start/partial_count apply uniformly to + * whichever of WHOLE/TEXT/PART attrs ends up selecting (see store.c's + * handle_mbox_fetch() comment) -- copied through to store.c + * unconditionally here rather than gated on a specific bit, since + * store.c is the one place that already knows the final WHOLE > TEXT + * > PART precedence and applies the range to whichever it picks. + * section_part is similarly always copied through; store.c only + * consults it when MBOX_FETCH_BODY_PART actually wins. + */ + strlcpy(req.section_part, section_part, sizeof(req.section_part)); + req.has_partial = has_partial; + req.partial_start = partial_start; + req.partial_count = partial_count; + + /* + * The verbatim client-typed label ("HEADER" for plain BODY.PEEK + * [HEADER], or e.g. "HEADER.FIELDS (DATE FROM)" for the fields + * variant) never crosses the imsg boundary to store.c -- it's + * purely a listener.c-side echo concern, stashed on the session now + * so session_send_fetch_response() can use it once the matching + * IMSG_MBOX_FETCH_HEADER/IMSG_MBOX_FETCH_META pair arrives per + * message. See struct session's pending_header_label comment. + */ + if (attrs & MBOX_FETCH_BODY_HEADER) + strlcpy(s->pending_header_label, "HEADER", + sizeof(s->pending_header_label)); + else if (attrs & MBOX_FETCH_HEADER_FIELDS) + strlcpy(s->pending_header_label, header_fields_label, + sizeof(s->pending_header_label)); + + /* + * Same idea as pending_header_label just above, for BODY.PEEK[]/ + * BODY.PEEK[TEXT]/BODY.PEEK[] -- WHOLE > TEXT > PART + * precedence matches store.c's handle_mbox_fetch() exactly (see that + * function's comment), since both sides have to agree on which one a + * client that somehow requested more than one of the three actually + * gets. pending_body_has_partial/pending_body_partial_origin are only + * set when the request has_partial applies to *this* winning variant + * -- store.c already ties has_partial/partial_start/partial_count to + * whichever of WHOLE/TEXT/PART it picks with the same precedence, so + * there's nothing further to disambiguate here. + */ + if (attrs & MBOX_FETCH_BODY_WHOLE) + s->pending_body_label[0] = '\0'; + else if (attrs & MBOX_FETCH_BODY_TEXT) + strlcpy(s->pending_body_label, "TEXT", + sizeof(s->pending_body_label)); + else if (attrs & MBOX_FETCH_BODY_PART) + strlcpy(s->pending_body_label, section_part, + sizeof(s->pending_body_label)); + s->pending_body_has_partial = has_partial; + s->pending_body_partial_origin = partial_start; + + /* + * Same idea as pending_header_label just above, for BODYSTRUCTURE: + * RFC 9051 SS9's `"BODY" ["STRUCTURE"] SP body` means the response + * label itself has to echo whichever bare token the client used -- + * see struct session's pending_bodystructure_label comment. + */ + if (attrs & MBOX_FETCH_BODYSTRUCTURE) + strlcpy(s->pending_bodystructure_label, + bodystructure_full ? "BODYSTRUCTURE" : "BODY", + sizeof(s->pending_bodystructure_label)); + + if (modspec != NULL) { + if (parse_fetch_modifiers(modspec, &req, s, by_uid, + &want_vanished, &errmsg) == -1) { + session_reply(s, tag, "BAD", errmsg); + return (1); + } + } + + if (want_vanished && !req.has_changedsince) { + /* RFC 7162 SS3.2.6: "The VANISHED UID FETCH modifier MUST + * only be specified together with the CHANGEDSINCE UID + * FETCH modifier... the server MUST respond with a tagged + * BAD response." */ + session_reply(s, tag, "BAD", + "VANISHED requires CHANGEDSINCE also be specified " + "(RFC 7162 SS3.2.6)"); + return (1); + } + req.want_vanished = want_vanished; + + if (by_uid) + req.attrs |= MBOX_FETCH_UID; + + if (s->store_iev == NULL) { + /* Same internal-invariant check as cmd_select() -- ST_SELECTED + * requires store_iev to already be wired. */ + log_warnx("session %u: %s with no store channel wired", + s->id, cmdname); + session_reply(s, tag, "NO", "[SERVERBUG] internal error"); + return (1); + } + + req.seq_lo = lo; + req.seq_hi = hi; + req.lo_is_star = lo_star; + req.hi_is_star = hi_star; + + /* RFC 7162 SS3.1: the MODSEQ fetch-att and CHANGEDSINCE modifier are + * both CONDSTORE-enabling commands. */ + if (req.attrs & MBOX_FETCH_MODSEQ) + session_condstore_enable(s); + + strlcpy(s->pending_tag, tag, sizeof(s->pending_tag)); + s->fetch_attrs = req.attrs; + s->cmd_by_uid = by_uid; + s->state = SESSION_FETCHING; + + if (imsg_compose(&s->store_iev->ibuf, IMSG_MBOX_FETCH, 0, 0, -1, + &req, sizeof(req)) == -1) + log_warn("session %u: imsg_compose IMSG_MBOX_FETCH", s->id); + imsgev_add(s->store_iev); + + return (1); +} + +/* + * RFC 9051 SS6.4.6: `store-att-flags = (["+" / "-"] "FLAGS" [".SILENT"]) SP + * (flag-list / (flag *(SP flag)))`, `flag = "\Answered" / "\Flagged" / + * "\Deleted" / "\Seen" / "\Draft" / flag-keyword / flag-extension ; Does + * not include "\Recent"`, `flag-keyword = "$MDNSent" / "$Forwarded" / + * "$Junk" / "$NotJunk" / "$Phishing" / atom`, `flag-extension = "\" atom`. + * + * modetok is the "(["+"/"-"] "FLAGS" [".SILENT"])" token (e.g. "+FLAGS", + * "FLAGS.SILENT", "-FLAGS"); flagspec is everything after it, with or + * without surrounding parentheses, modified in place (strtok_r()) -- + * same shape parse_fetch_atts() already accepts for fetch-att lists. + * + * *sysflags_out collects the five supported system flags as a MBOX_FLAG_* + * bitmask. Any other "\"-prefixed token is either "\Recent" (explicitly + * excluded from the `flag` production itself -- not a client error + * exactly, but not something STORE can ever legitimately be asked to set) + * or a flag-extension this server doesn't define (SS2.3.2: system flags + * are "predefined in this specification" -- there is no mechanism here + * for a client to invent a new one), so both are rejected the same way + * FETCH rejects BODY[...]: -2/NO, "recognized syntax, not something this + * server supports" rather than a syntax error. A bare (non-"\") token is + * a keyword and is appended to keywords_out as-is, EXCEPT that a keyword + * containing ':' or ',' is also rejected with -2/NO: both characters are + * valid in IMAP's `atom` grammar, but the index format + * (openimap-storage-backend.md) uses them as its own field/record + * delimiters with no escaping mechanism, so storing such a keyword + * verbatim would corrupt the index -- a v1 storage-format limitation, + * not an IMAP protocol restriction, and treated as "unsupported" rather + * than "invalid" for exactly that reason. + * + * System-flag name matching is case-insensitive (`strcasecmp()`) as a + * deliberate implementation choice for interoperability -- RFC 9051's own + * ABNF for `flag` doesn't state a case-sensitivity rule one way or the + * other, so this isn't a sourced requirement, just this server being + * lenient the same way it already is about repeated spaces in + * parse_command_line(). mode/silent themselves are parsed separately by + * the caller (the store-att-flags prefix, e.g. "+FLAGS.SILENT") -- this + * function only ever sees the flag-list half. + */ +static int +parse_store_flags(char *flagspec, uint32_t *sysflags_out, char *keywords_out, + size_t keywords_out_size, const char **errmsg) +{ + char *p, *tok, *save; + size_t len; + uint32_t sysflags = 0; + int first = 1; + + *errmsg = NULL; + *sysflags_out = 0; + keywords_out[0] = '\0'; + + if (flagspec == NULL || *flagspec == '\0') { + *errmsg = "missing flag list"; + return (-1); + } + + p = flagspec; + len = strlen(p); + if (len >= 2 && p[0] == '(' && p[len - 1] == ')') { + p[len - 1] = '\0'; + p++; + } + /* An empty flag-list, "()" or "", is valid ABNF-wise (flag-list = + * "(" [flag *(SP flag)] ")") but pointless for STORE -- SET with no + * flags would clear everything, which is a real (if unusual) + * request, so this is only rejected when the caller can't tell SET + * from ADD/REMOVE apart at this layer; see cmd_store_cmd(), which + * allows an empty list only for a bare "FLAGS"/"FLAGS.SILENT". */ + if (*p == '\0') + return (0); + + for (tok = strtok_r(p, " ", &save); tok != NULL; + tok = strtok_r(NULL, " ", &save)) { + if (tok[0] == '\\') { + if (strcasecmp(tok, "\\Answered") == 0) + sysflags |= MBOX_FLAG_ANSWERED; + else if (strcasecmp(tok, "\\Flagged") == 0) + sysflags |= MBOX_FLAG_FLAGGED; + else if (strcasecmp(tok, "\\Deleted") == 0) + sysflags |= MBOX_FLAG_DELETED; + else if (strcasecmp(tok, "\\Seen") == 0) + sysflags |= MBOX_FLAG_SEEN; + else if (strcasecmp(tok, "\\Draft") == 0) + sysflags |= MBOX_FLAG_DRAFT; + else if (strcasecmp(tok, "\\Recent") == 0) { + *errmsg = "\\Recent cannot be set -- RFC 9051 " + "deprecates it and excludes it from the " + "flag grammar entirely"; + return (-2); + } else { + *errmsg = "unsupported system flag -- v1 only " + "supports \\Answered/\\Flagged/\\Deleted/" + "\\Seen/\\Draft"; + return (-2); + } + continue; + } + + if (strchr(tok, ':') != NULL || strchr(tok, ',') != NULL) { + *errmsg = "keyword contains ':' or ',' -- not " + "representable in this server's index format"; + return (-2); + } + + if (!first) + strlcat(keywords_out, ",", keywords_out_size); + strlcat(keywords_out, tok, keywords_out_size); + first = 0; + } + + *sysflags_out = sysflags; + return (0); +} + +/* + * Parses a STORE command's leading store-modifier list (RFC 4466's generic + * syntax, extended by RFC 7162 SS3.1.3). modspec is the still-parenthesized + * text cmd_store_cmd() split off before store-att-flags, modified in place + * -- see that function's comment for why store-modifiers lead rather than + * trail here, unlike FETCH's fetch-modifiers. + * + * UNCHANGEDSINCE is the only store-modifier this + * server implements. Unlike parse_fetch_modifiers()'s CHANGEDSINCE, the + * value here is explicitly allowed to be 0 (SS3.1.3 Example 8: "Use of + * UNCHANGEDSINCE with a modification sequence of 0 always fails if the + * metadata item exists" -- a real, distinct case from "not specified", + * hence has_unchangedsince rather than testing unchangedsince != 0). + */ +static int +parse_store_modifiers(char *modspec, struct imsg_mbox_store *req, + const char **errmsg) +{ + char *p, *tok, *save; + size_t len; + + *errmsg = NULL; + len = strlen(modspec); + if (len < 2 || modspec[0] != '(' || modspec[len - 1] != ')') { + *errmsg = "malformed store-modifier list"; + return (-1); + } + modspec[len - 1] = '\0'; + p = modspec + 1; + + for (tok = strtok_r(p, " ", &save); tok != NULL; + tok = strtok_r(NULL, " ", &save)) { + if (strcasecmp(tok, "UNCHANGEDSINCE") == 0) { + char *valtok = strtok_r(NULL, " ", &save); + char *ep; + + if (valtok == NULL) { + *errmsg = "UNCHANGEDSINCE requires a " + "mod-sequence value"; + return (-1); + } + errno = 0; + req->unchangedsince = strtoull(valtok, &ep, 10); + if (*ep != '\0' || errno != 0) { + *errmsg = "invalid UNCHANGEDSINCE mod-sequence"; + return (-1); + } + req->has_unchangedsince = 1; + } else { + *errmsg = "unrecognized store modifier"; + return (-1); + } + } + + return (0); +} + +/* Named cmd_store_cmd(), not cmd_store(), to avoid reading as though it + * belongs to -- or calls into -- the STORE *role* (store.c, s->store_iev, + * session_store_dispatch() etc. elsewhere in this file): same command + * name, unrelated concept. + * + * RFC 9051 SS6.4.6: `store = "STORE" SP sequence-set SP store-att-flags`, + * extended by RFC 4466/RFC 7162 with an optional store-modifier list + * between the sequence-set and store-att-flags: `"STORE" SP sequence-set + * [SP "(" store-modifier *(SP store-modifier) ")"] SP store-att-flags` -- + * store-modifiers *lead*, unlike FETCH's fetch-modifiers, which trail (see + * cmd_fetch()'s comment and RFC 7162 SS3.1.3's own examples, e.g. "STORE * + * (UNCHANGEDSINCE 12121230045) +FLAGS.SILENT (...)"), so this function + * peeks for a leading "(" right after the sequence-set rather than reusing + * split_trailing_modifiers(). + * + * v1 scope matches FETCH's: exactly one sequence-set range or number, + * never a comma-separated list, for the same struct-session-only-tracks- + * one-async-operation reason cmd_fetch()'s comment explains. Replies + * reuse IMSG_MBOX_FETCH_META/IMSG_MBOX_RESULT (see imapd.h's imsg_ + * mbox_store comment) since RFC 9051 SS6.4.6 itself says STORE's only + * response is "untagged responses: FETCH" -- the exact same wire shape + * FETCH already produces, so store.c and this function are what decide + * it's a STORE in flight (s->state == SESSION_STORING), not a different + * imsg type. + */ +static int +cmd_store_cmd(struct session *s, const char *tag, char *args) +{ + return store_do(s, tag, args, 0); +} + +/* + * Shared body for cmd_store_cmd() (by_uid=0) and cmd_uid()'s STORE branch + * (by_uid=1) -- see cmd_store_cmd()'s own comment for the grammar/scope + * this parses. RFC 9051 SS6.4.9 addition this pass: req.by_uid threads the + * UID-vs-sequence-number resolution to store.c (see handle_mbox_store()'s + * comment there); s->cmd_by_uid is set unconditionally before dispatch so + * session_send_store_fetch_response() knows to include UID in the STORE + * echo (SS6.4.9's "MUST implicitly include the UID message data item... + * primarily applies to the UID FETCH and UID STORE commands"). + */ +static int +store_do(struct session *s, const char *tag, char *args, int by_uid) +{ + struct imsg_mbox_store req; + char *seqtok, *modetok, *flagspec, *modspec = NULL; + uint32_t lo, hi, sysflags; + int lo_star, hi_star, mode, silent, rc; + char keywords[MBOX_FLAGS_MAX]; + const char *errmsg; + const char *cmdname = by_uid ? "UID STORE" : "STORE"; + + if (args == NULL) { + session_reply(s, tag, "BAD", + "STORE requires a sequence set and store-att-flags"); + return (1); + } + + seqtok = args; + while (*args != '\0' && *args != ' ') + args++; + if (*args == '\0') { + session_reply(s, tag, "BAD", "STORE requires store-att-flags"); + return (1); + } + *args++ = '\0'; + while (*args == ' ') + args++; + + if (*args == '(') { + char *p = args; + int depth = 0; + + for (;;) { + if (*p == '(') + depth++; + else if (*p == ')') { + depth--; + if (depth == 0) + break; + } else if (*p == '\0') { + session_reply(s, tag, "BAD", + "unterminated store-modifier list"); + return (1); + } + p++; + } + /* p points at the matching ')' for modspec (== args) */ + modspec = args; + p++; /* just past ')' */ + if (*p == '\0') { + session_reply(s, tag, "BAD", + "STORE requires store-att-flags"); + return (1); + } + if (*p != ' ') { + session_reply(s, tag, "BAD", + "expected a space after store-modifier list"); + return (1); + } + *p = '\0'; /* terminate modspec right after ')' */ + p++; + while (*p == ' ') + p++; + args = p; + } + + modetok = args; + while (*args != '\0' && *args != ' ') + args++; + if (*args == '\0') { + session_reply(s, tag, "BAD", "STORE requires a flag list"); + return (1); + } + *args++ = '\0'; + while (*args == ' ') + args++; + flagspec = args; + + if (strchr(seqtok, ',') != NULL) { + session_reply(s, tag, "BAD", + "comma-separated sequence sets not supported in v1 -- " + "issue separate STORE commands"); + return (1); + } + if (parse_seq_range(seqtok, &lo, &hi, &lo_star, &hi_star) == -1) { + session_reply(s, tag, "BAD", "invalid sequence set"); + return (1); + } + + memset(&req, 0, sizeof(req)); + if (modspec != NULL) { + if (parse_store_modifiers(modspec, &req, &errmsg) == -1) { + session_reply(s, tag, "BAD", errmsg); + return (1); + } + } + + if (modetok[0] == '+') { + mode = MBOX_STORE_ADD; + modetok++; + } else if (modetok[0] == '-') { + mode = MBOX_STORE_REMOVE; + modetok++; + } else + mode = MBOX_STORE_SET; + + if (strcasecmp(modetok, "FLAGS") == 0) + silent = 0; + else if (strcasecmp(modetok, "FLAGS.SILENT") == 0) + silent = 1; + else { + session_reply(s, tag, "BAD", + "expected FLAGS, FLAGS.SILENT, +FLAGS, +FLAGS.SILENT, " + "-FLAGS, or -FLAGS.SILENT"); + return (1); + } + + rc = parse_store_flags(flagspec, &sysflags, keywords, sizeof(keywords), + &errmsg); + if (rc == -1) { + session_reply(s, tag, "BAD", errmsg); + return (1); + } + if (rc == -2) { + session_reply(s, tag, "NO", errmsg); + return (1); + } + + if (s->mbox_readonly) { + /* + * RFC 9051 SS6.3.3: "No changes to the permanent state of + * the mailbox, including per-user state, are permitted" for + * an EXAMINE'd mailbox -- STORE's entire purpose is exactly + * that kind of change. RFC 5530 CANNOT, same reasoning as + * session_request_expunge()'s identical check. + */ + session_reply(s, tag, "NO", "[CANNOT] Mailbox is read-only " + "(selected via EXAMINE)"); + return (1); + } + + if (s->store_iev == NULL) { + /* Same internal-invariant check as cmd_select()/cmd_fetch() + * -- ST_SELECTED requires store_iev to already be wired. */ + log_warnx("session %u: %s with no store channel wired", + s->id, cmdname); + session_reply(s, tag, "NO", "[SERVERBUG] internal error"); + return (1); + } + + /* req was already memset(3)'d and populated with has_unchangedsince/ + * unchangedsince (if modspec was present) earlier -- not re-zeroed + * here, or that would silently wipe those two fields back out. */ + req.seq_lo = lo; + req.seq_hi = hi; + req.lo_is_star = lo_star; + req.hi_is_star = hi_star; + req.mode = mode; + req.silent = silent; + req.sysflags = sysflags; + req.by_uid = by_uid; + strlcpy(req.keywords, keywords, sizeof(req.keywords)); + + /* RFC 7162 SS3.1: UNCHANGEDSINCE is a CONDSTORE-enabling command. */ + if (req.has_unchangedsince) + session_condstore_enable(s); + + strlcpy(s->pending_tag, tag, sizeof(s->pending_tag)); + s->cmd_by_uid = by_uid; + s->state = SESSION_STORING; + + if (imsg_compose(&s->store_iev->ibuf, IMSG_MBOX_STORE, 0, 0, -1, + &req, sizeof(req)) == -1) + log_warn("session %u: imsg_compose IMSG_MBOX_STORE", s->id); + imsgev_add(s->store_iev); + + return (1); +} + +/* + * RFC 9051 SS6.4.7 COPY / SS6.4.8 MOVE: `copy = "COPY" SP sequence-set SP + * mailbox`, `move = "MOVE" SP sequence-set SP mailbox` -- identical + * argument grammar, differing only in which IMSG_MBOX_* this sends and + * (per is_move) which store.c handler that becomes. cmd_copy()/cmd_move() + * are thin by_uid=0 wrappers around this, and cmd_uid() calls it again + * with by_uid=1 for UID COPY/UID MOVE -- the same "_dispatch() shared body, + * thin cmd_*() wrappers" pattern fetch_dispatch()/store_do()/search_ + * dispatch() already established for RFC 9051 SS6.4.9's UID command. + * + * Mailbox-name argument reuses parse_list_token() (LIST/STATUS's own + * token parser). Fast client-side validation reuses mailbox_name_is_ + * inbox()/mailbox_name_valid() -- CREATE/DELETE/RENAME's own precedent + * (RFC 9051 SS6.3.4-SS6.3.6 flat multi-mailbox support, docs/openimap- + * storage-backend.md item 10's own follow-up), not cmd_list()/cmd_ + * status()'s "only INBOX can ever match" precedent this function used + * before that support existed: destname is now genuinely any real, + * existing mailbox, not just INBOX, so a syntactically valid name gets a + * real store round trip (IMSG_MBOX_COPY/IMSG_MBOX_MOVE now carry it, see + * imapd.h's imsg_mbox_copy comment) rather than being answered locally. + * + * A syntactically invalid name (BAD, no round trip) is a different + * failure from "syntactically fine but doesn't exist" -- the latter is + * exactly SS6.4.7's TRYCREATE case: "If the destination mailbox does not + * exist, a server MUST return an error... Unless it is certain that the + * destination mailbox can not be created, the server MUST send the + * response code '[TRYCREATE]'". store.c's handle_mbox_copy()/handle_mbox_ + * move() report that case back via struct imsg_mbox_result's no_such_ + * mailbox field (mirroring imsg_mbox_appended's own field for APPEND); + * session_finish_copy_or_move() sends TRYCREATE only then, a plain NO + * "failed" for any other failure. + */ +static int +copy_move_dispatch(struct session *s, const char *tag, char *args, + int by_uid, int is_move) +{ + struct imsg_mbox_copy req; + char mailbox[MBOX_NAME_MAX]; + char *seqtok, *p; + uint32_t lo, hi; + int lo_star, hi_star; + const char *errmsg = NULL; + const char *cmdname = is_move ? + (by_uid ? "UID MOVE" : "MOVE") : (by_uid ? "UID COPY" : "COPY"); + + if (args == NULL) { + char text[64]; + + snprintf(text, sizeof(text), + "%s requires a sequence set and mailbox name", cmdname); + session_reply(s, tag, "BAD", text); + return (1); + } + + seqtok = args; + p = args; + while (*p != '\0' && *p != ' ') + p++; + if (*p == '\0') { + char text[48]; + + snprintf(text, sizeof(text), "%s requires a mailbox name", + cmdname); + session_reply(s, tag, "BAD", text); + return (1); + } + *p++ = '\0'; + while (*p == ' ') + p++; + + if (strchr(seqtok, ',') != NULL) { + session_reply(s, tag, "BAD", + "comma-separated sequence sets not supported in v1 -- " + "issue separate COPY/MOVE commands"); + return (1); + } + if (parse_seq_range(seqtok, &lo, &hi, &lo_star, &hi_star) == -1) { + session_reply(s, tag, "BAD", "invalid sequence set"); + return (1); + } + + if (parse_list_token(&p, mailbox, sizeof(mailbox), &errmsg) == -1) { + session_reply(s, tag, "BAD", errmsg); + return (1); + } + if (*p != '\0') { + session_reply(s, tag, "BAD", "trailing garbage after " + "mailbox name"); + return (1); + } + + if (!mailbox_name_is_inbox(mailbox) && !mailbox_name_valid(mailbox)) { + session_reply(s, tag, "BAD", "invalid mailbox name"); + return (1); + } + + if (is_move && s->mbox_readonly) { + /* + * MOVE permanently removes the copied messages from the + * source (selected) mailbox -- RFC 9051 SS6.4.8 describes it + * as "COPY, followed by removal of the copied messages", + * exactly the kind of permanent-state change SS6.3.3 + * prohibits for an EXAMINE'd mailbox. COPY, by contrast, + * never touches the source mailbox's own state, so it's + * unaffected by mbox_readonly -- only checked for is_move. + */ + session_reply(s, tag, "NO", "[CANNOT] Mailbox is read-only " + "(selected via EXAMINE)"); + return (1); + } + + if (s->store_iev == NULL) { + /* Same internal-invariant check as cmd_select()/cmd_fetch()/ + * cmd_store_cmd() -- ST_SELECTED requires store_iev to + * already be wired. */ + log_warnx("session %u: %s with no store channel wired", + s->id, cmdname); + session_reply(s, tag, "NO", "[SERVERBUG] internal error"); + return (1); + } + + memset(&req, 0, sizeof(req)); + req.by_uid = by_uid; + req.seq_lo = lo; + req.seq_hi = hi; + req.lo_is_star = lo_star; + req.hi_is_star = hi_star; + strlcpy(req.destname, mailbox, sizeof(req.destname)); + + strlcpy(s->pending_tag, tag, sizeof(s->pending_tag)); + s->cmd_by_uid = by_uid; + s->cmd_is_move = is_move; + s->state = SESSION_COPYING; + + if (imsg_compose(&s->store_iev->ibuf, + is_move ? IMSG_MBOX_MOVE : IMSG_MBOX_COPY, 0, 0, -1, &req, + sizeof(req)) == -1) + log_warn("session %u: imsg_compose %s", s->id, cmdname); + imsgev_add(s->store_iev); + + return (1); +} + +static int +cmd_copy(struct session *s, const char *tag, char *args) +{ + return copy_move_dispatch(s, tag, args, 0, 0); +} + +static int +cmd_move(struct session *s, const char *tag, char *args) +{ + return copy_move_dispatch(s, tag, args, 0, 1); +} + +/* + * RFC 9051 SS6.4.9: `uid = "UID" SP (copy / move / fetch / search / + * store)`, extended by RFC 4315/UIDPLUS's UID EXPUNGE (folded into base + * IMAP4rev2 -- confirmed by grepping this server's own copy of RFC 9051 + * SS6.4.9, which documents UID EXPUNGE directly, no separate extension + * capability needed). All six sub-commands are real as of this pass -- + * FETCH/STORE/SEARCH/EXPUNGE (fetch_dispatch()/store_do()/search_ + * dispatch()/uid_expunge_dispatch()) from an earlier pass, and (this + * pass) COPY/MOVE too, via the identical copy_move_dispatch() the base + * (non-UID) COPY/MOVE commands themselves use, just called here with + * by_uid=1 -- same "_dispatch() shared body, by_uid parameter" pattern + * as the other four. + * + * Sub-command name and its own arguments are split the same way parse_ + * command_line() splits tag/command-name/args at the top level (first + * space-delimited token, case-insensitive, rest is that sub-command's own + * argument string, NULL if nothing follows) -- UID's sub-command isn't + * itself a "tag", so it doesn't reuse that function, but the token- + * splitting logic is the same shape. + */ +static int +cmd_uid(struct session *s, const char *tag, char *args) +{ + char *sub, *subargs; + + if (args == NULL) { + session_reply(s, tag, "BAD", "UID requires a sub-command"); + return (1); + } + + sub = args; + while (*args != '\0' && *args != ' ') + args++; + if (*args == ' ') { + *args++ = '\0'; + while (*args == ' ') + args++; + subargs = (*args != '\0') ? args : NULL; + } else + subargs = NULL; + + if (strcasecmp(sub, "FETCH") == 0) + return fetch_dispatch(s, tag, subargs, 1); + if (strcasecmp(sub, "STORE") == 0) + return store_do(s, tag, subargs, 1); + if (strcasecmp(sub, "SEARCH") == 0) + return search_dispatch(s, tag, subargs, 1); + if (strcasecmp(sub, "EXPUNGE") == 0) + return uid_expunge_dispatch(s, tag, subargs); + if (strcasecmp(sub, "COPY") == 0) + return copy_move_dispatch(s, tag, subargs, 1, 0); + if (strcasecmp(sub, "MOVE") == 0) + return copy_move_dispatch(s, tag, subargs, 1, 1); + + session_reply(s, tag, "BAD", + "UID sub-command must be COPY, FETCH, MOVE, SEARCH, or STORE"); + return (1); +} + +/* + * The AUTH channel: only IMSG_AUTH_RESULT arrives here (wired once at + * boot via peer_fd). On success, kicks off the per-session store spawn. + * + * Real bug caught writing SELECT's async round trip: imsgev_init() (see + * imsgev.c) registers every channel it wraps as plain EV_READ, NOT + * EV_READ|EV_PERSIST -- confirmed against the real event.h this pass + * ("The additional flag EV_PERSIST makes an event_add() persistent until + * event_del() has been called", openbsd_source/src/lib/libevent/event.h), + * meaning a one-shot registration fires exactly once and then goes + * inactive until something calls event_add() (imsgev_add(), here) again. + * This function never did that for iev_auth itself -- session_request_ + * store()'s imsgev_add() call re-arms iev_parent, a different channel, + * on the one path that reaches it. The result: the SECOND + * IMSG_AUTH_RESULT ever received on this channel, across the whole + * daemon's lifetime (not just per-session -- iev_auth is one shared + * channel), would never be seen -- every AUTHENTICATE after the very + * first one handled process-wide would silently hang forever. auth.c's + * own auth_dispatch() happens to avoid this by always calling + * imsgev_add(iev) after composing its reply, which incidentally re-arms + * its own read side every time; this function had no equivalent. Fixed + * by unconditionally re-arming at the end below, the same fix applied to + * listener_dispatch_parent() and parent.c's store_child_dispatch(), the + * other two channels with the identical gap. + */ +static void +listener_dispatch_auth(int fd, short event, void *arg) +{ + struct imsgev *iev = arg; + struct imsg imsg; + ssize_t n; + + /* + * EV_WRITE: real bug caught on first real-hardware run -- see + * auth.c's auth_dispatch() header comment for the full citation + * (imsg_init(3)'s own EXAMPLES section) and the live symptoms this + * exact gap produced here specifically: IMSG_AUTH_REQUEST queued by + * cmd_authenticate() via imsg_compose() never actually left this + * process, and this function's own unconditional imsgev_add() at + * the bottom kept re-arming EV_WRITE for a write that never + * happened -- a busy loop, confirmed via `ps` showing 25+ minutes + * of CPU time on an otherwise-idle listener process. + */ + if (event & EV_WRITE) { + if (imsgbuf_write(&iev->ibuf) == -1) + fatal("imsgbuf_write"); + } + + if (event & EV_READ) { + if ((n = imsgbuf_read(&iev->ibuf)) == -1) + fatal("imsgbuf_read"); + if (n == 0) { + log_warnx("auth closed channel"); + event_del(&iev->ev); + return; + } + } + + for (;;) { + if ((n = imsg_get(&iev->ibuf, &imsg)) == -1) + fatal("imsg_get"); + if (n == 0) + break; + + switch (imsg_get_type(&imsg)) { + case IMSG_AUTH_RESULT: { + struct imsg_auth_result res; + struct session *s; + + if (imsg_get_data(&imsg, &res, sizeof(res)) == -1) { + log_warnx("bad IMSG_AUTH_RESULT"); + break; + } + if ((s = session_find(res.session_id)) == NULL) { + log_debug("IMSG_AUTH_RESULT for unknown " + "session %u", res.session_id); + break; + } + if (!res.ok) { + s->state = SESSION_NOT_AUTH; + session_reply(s, s->pending_tag, "NO", + "[AUTHENTICATIONFAILED] authentication failed"); + break; + } + session_request_store(s, &res); + break; + } + default: + log_debug("listener_dispatch_auth: unhandled %d", + imsg_get_type(&imsg)); + break; + } + imsg_free(&imsg); + } + imsgev_add(iev); /* re-arm -- see this function's header comment */ + (void)fd; +} + +/* + * Rebuilds listener_tls_ctx/listener_tls_config from freshly-received + * cert/key bytes, for a live SIGHUP reload -- called from listener_ + * dispatch_parent()'s IMSG_TLS_CERT/IMSG_TLS_KEY cases below once both + * halves of one reload have arrived. Deliberately a separate function + * from listener_main()'s boot-time TLS build rather than a shared helper, + * even though the tls_config_new()+tls_server()+tls_config_set_ciphers()+ + * tls_config_set_keypair_mem()+tls_configure() chain is identical (same + * sourcing as listener_main()'s own copy, against httpd's server_tls_ + * init()): boot failure degrades to "no TLS, cleartext still works" (see + * listener_main()), but a reload failure must NEVER tear down an already- + * working TLS context just because, say, the operator's cron-driven cert + * renewal wrote a momentarily-truncated file. Every failure path here + * frees only the half-built new pair and returns with listener_tls_ctx/ + * listener_tls_config completely untouched; a fully-built replacement + * pair is only ever swapped in after every step succeeds, and the OLD + * pair is freed only once the swap itself is done -- see listener_tls_ + * ctx's file-scope comment for why the swap itself is safe with respect + * to sessions already connected. + */ +static void +listener_reload_tls(const char *cert_buf, size_t cert_len, + const char *key_buf, size_t key_len) +{ + struct tls *new_ctx; + struct tls_config *new_config; + struct tls *old_ctx; + struct tls_config *old_config; + + if (cert_len == 0 || key_len == 0) { + log_warnx("listener: SIGHUP reload: empty cert or key -- " + "keeping previous TLS configuration"); + return; + } + + if ((new_config = tls_config_new()) == NULL) { + log_warnx("listener: SIGHUP reload: tls_config_new failed -- " + "keeping previous TLS configuration"); + return; + } + if ((new_ctx = tls_server()) == NULL) { + log_warnx("listener: SIGHUP reload: tls_server failed -- " + "keeping previous TLS configuration"); + tls_config_free(new_config); + return; + } + if (tls_config_set_ciphers(new_config, "secure") != 0) { + log_warnx("listener: SIGHUP reload: tls_config_set_ciphers: " + "%s -- keeping previous TLS configuration", + tls_config_error(new_config)); + tls_free(new_ctx); + tls_config_free(new_config); + return; + } + if (tls_config_set_keypair_mem(new_config, (const uint8_t *)cert_buf, + cert_len, (const uint8_t *)key_buf, key_len) != 0) { + log_warnx("listener: SIGHUP reload: tls_config_set_keypair_" + "mem: %s -- keeping previous TLS configuration", + tls_config_error(new_config)); + tls_free(new_ctx); + tls_config_free(new_config); + return; + } + if (tls_configure(new_ctx, new_config) != 0) { + log_warnx("listener: SIGHUP reload: tls_configure: %s -- " + "keeping previous TLS configuration", tls_error(new_ctx)); + tls_free(new_ctx); + tls_config_free(new_config); + return; + } + tls_config_clear_keys(new_config); + + old_ctx = listener_tls_ctx; + old_config = listener_tls_config; + listener_tls_ctx = new_ctx; + listener_tls_config = new_config; + if (old_ctx != NULL) + tls_free(old_ctx); + if (old_config != NULL) + tls_config_free(old_config); + + log_info("listener: SIGHUP reload: TLS configuration reloaded"); +} + +/* + * The PARENT channel (this process's fd 3, kept alive for its whole + * lifetime -- see this file's header comment): IMSG_STORE_FORK arrives + * here as parent's *failure* reply to a store-spawn request (see + * parent.c's parent_handle_store_fork()/store_child_teardown() "fail:" + * paths, which reuse this type for that -- a *successful* peer wire-up + * arrives as IMSG_SETUP_PEER instead). IMSG_SETUP_PEER now carries the + * requesting session's id in the imsg header's "id" field (see parent.c's + * setup_peer_send()), read back here via imsg_get_id() to find the right + * session -- this is the fix for the session-demux gap flagged in an + * earlier pass. + * + * IMSG_TLS_CERT/IMSG_TLS_KEY arrive here too now, post-boot, whenever + * parent's sighup_handler() re-pushes freshly-read cert/key bytes (see + * that function's header comment in parent.c) -- reusing the exact same + * two imsg types the boot-time drain loop in listener_main() consumes + * once, synchronously, before the event loop even starts. Each is staged + * into reload_cert_buf/reload_key_buf independently (they can arrive in + * either order, and not necessarily in the same dispatch call -- see + * those statics' own comment), and listener_reload_tls() only runs once + * both reload_got_cert and reload_got_key are set, after which both flags + * reset for the next reload. + * + * Re-arms iev (imsgev_add()) at the end, same fix and same reason as + * listener_dispatch_auth()'s header comment: imsgev_init() registers + * plain EV_READ, not EV_PERSIST, and nothing here previously re-armed + * this channel's OWN read side after servicing it -- the second + * IMSG_STORE_FORK-failure or IMSG_SETUP_PEER, across the daemon's whole + * lifetime, would otherwise never have been seen. + */ +static void +listener_dispatch_parent(int fd, short event, void *arg) +{ + struct imsgev *iev = arg; + struct imsg imsg; + ssize_t n; + + /* EV_WRITE: same real bug, same fix -- see listener_dispatch_auth()'s + * header comment and auth.c's auth_dispatch() for the full citation. */ + if (event & EV_WRITE) { + if (imsgbuf_write(&iev->ibuf) == -1) + fatal("imsgbuf_write"); + } + + if (event & EV_READ) { + if ((n = imsgbuf_read(&iev->ibuf)) == -1) + fatal("imsgbuf_read"); + if (n == 0) { + log_warnx("parent closed channel"); + event_del(&iev->ev); + return; + } + } + + for (;;) { + if ((n = imsg_get(&iev->ibuf, &imsg)) == -1) + fatal("imsg_get"); + if (n == 0) + break; + + switch (imsg_get_type(&imsg)) { + case IMSG_STORE_FORK: { + struct imsg_store_fork fail; + struct session *s; + + if (imsg_get_data(&imsg, &fail, sizeof(fail)) == -1) { + log_warnx("bad IMSG_STORE_FORK reply"); + break; + } + if ((s = session_find(fail.session_id)) != NULL) { + s->state = SESSION_NOT_AUTH; + session_reply(s, s->pending_tag, "NO", + "authentication succeeded but mailbox " + "store unavailable"); + } + break; + } + case IMSG_SETUP_PEER: { + uint32_t sess_id = imsg_get_id(&imsg); + int store_fd = imsg_get_fd(&imsg); + struct session *s; + + if (store_fd == -1) { + log_warnx("IMSG_SETUP_PEER (store) carried " + "no fd"); + break; + } + if ((s = session_find(sess_id)) == NULL) { + log_warnx("IMSG_SETUP_PEER (store) for " + "unknown session %u", sess_id); + close(store_fd); + break; + } + s->store_iev = calloc(1, sizeof(*s->store_iev)); + if (s->store_iev == NULL) { + log_warn("calloc"); + close(store_fd); + break; + } + imsgev_init(s->store_iev, store_fd, + session_store_dispatch, s); + s->state = SESSION_AUTHENTICATED; + log_debug("session %u: store peer wired", sess_id); + /* Exact example text from RFC 9051 SS6.2.2's PLAIN + * example ("S: A03 OK Success (tls protection)") -- + * accurate here too, since AUTHENTICATE PLAIN is only + * ever reachable once s->tls_active is set (see + * cmd_authenticate()'s TLS gate). */ + session_reply(s, s->pending_tag, "OK", + "Success (tls protection)"); + break; + } + case IMSG_TLS_CERT: { + size_t len = imsg_get_len(&imsg); + + if (len > sizeof(reload_cert_buf)) { + log_warnx("listener: SIGHUP reload: TLS cert " + "too large (%zu > %zu)", len, + sizeof(reload_cert_buf)); + len = 0; + } else if (imsg_get_data(&imsg, reload_cert_buf, len) + == -1) { + log_warnx("bad IMSG_TLS_CERT (reload)"); + len = 0; + } + reload_cert_len = len; + reload_got_cert = 1; + if (reload_got_cert && reload_got_key) { + listener_reload_tls(reload_cert_buf, + reload_cert_len, reload_key_buf, + reload_key_len); + explicit_bzero(reload_key_buf, + sizeof(reload_key_buf)); + reload_got_cert = reload_got_key = 0; + } + break; + } + case IMSG_TLS_KEY: { + size_t len = imsg_get_len(&imsg); + + if (len > sizeof(reload_key_buf)) { + log_warnx("listener: SIGHUP reload: TLS key " + "too large (%zu > %zu)", len, + sizeof(reload_key_buf)); + len = 0; + } else if (imsg_get_data(&imsg, reload_key_buf, len) + == -1) { + log_warnx("bad IMSG_TLS_KEY (reload)"); + len = 0; + } + reload_key_len = len; + reload_got_key = 1; + if (reload_got_cert && reload_got_key) { + listener_reload_tls(reload_cert_buf, + reload_cert_len, reload_key_buf, + reload_key_len); + explicit_bzero(reload_key_buf, + sizeof(reload_key_buf)); + reload_got_cert = reload_got_key = 0; + } + break; + } + default: + log_debug("listener_dispatch_parent: unhandled %d", + imsg_get_type(&imsg)); + break; + } + imsg_free(&imsg); + } + imsgev_add(iev); /* re-arm -- see this function's header comment */ + (void)fd; +} + +/* + * The per-session STORE channel, wired once listener_dispatch_parent()'s + * IMSG_SETUP_PEER case sets s->store_iev. IMSG_MBOX_SELECTED (see + * cmd_select() for the request side, store.c's handle_mbox_select() for + * what actually produces this reply) was the first IMSG_MBOX_* reply this + * handled for real; IMSG_MBOX_FETCH_META (zero or more, streamed) and the + * terminal IMSG_MBOX_RESULT (see cmd_fetch()/store.c's handle_mbox_fetch()) + * are the second pair. Everything else in the family still has no request + * path at all (store.c's own TODO), so nothing else can arrive here yet. + * + * A lost store channel (read failure, or a clean close -- the child + * process exiting or crashing) tears down the whole client session rather + * than trying to degrade back to an unselected/unauthenticated state: + * v1's simplification, not a claim that a client-visible reconnect- + * without-losing-the-TCP-connection story wouldn't be better eventually. + * + * Re-arms s->store_iev (imsgev_add()) at the end -- see listener_dispatch_ + * auth()'s header comment for the underlying EV_PERSIST gap this fixes; + * this channel has the exact same shape (imsgev_init(), no re-arm on the + * read side), and without this a second SELECT or FETCH on the same + * session would send its IMSG_MBOX_* request to store just fine but never + * see the reply. + */ +static void +session_store_dispatch(int fd, short event, void *arg) +{ + struct session *s = arg; + struct imsg imsg; + ssize_t n; + + /* + * EV_WRITE: same real bug as listener_dispatch_auth()/auth.c's + * auth_dispatch() (see those header comments for the full + * citation) -- every IMSG_MBOX_* request this session ever sends + * (SELECT, FETCH, STORE, ...) is queued via imsg_compose() and + * needs an actual imsgbuf_write() once the fd is writable, which + * nothing here ever did before this fix. + */ + if (event & EV_WRITE) { + if (imsgbuf_write(&s->store_iev->ibuf) == -1) { + log_warnx("session %u: imsgbuf_write (store)", s->id); + session_teardown(s); + return; + } + } + + if (event & EV_READ) { + if ((n = imsgbuf_read(&s->store_iev->ibuf)) == -1) { + log_warnx("session %u: imsgbuf_read (store)", s->id); + session_teardown(s); + return; + } + if (n == 0) { + log_warnx("session %u: store closed channel", s->id); + session_teardown(s); + return; + } + } + + for (;;) { + if ((n = imsg_get(&s->store_iev->ibuf, &imsg)) == -1) { + log_warnx("session %u: imsg_get (store)", s->id); + session_teardown(s); + return; + } + if (n == 0) + break; + + switch (imsg_get_type(&imsg)) { + case IMSG_MBOX_SELECTED: { + struct imsg_mbox_selected res; + + if (imsg_get_data(&imsg, &res, sizeof(res)) == -1) { + log_warnx("bad IMSG_MBOX_SELECTED"); + break; + } + session_handle_mbox_selected(s, &res); + break; + } + case IMSG_MBOX_STATUS_RESULT: { + struct imsg_mbox_status_result res; + + if (imsg_get_data(&imsg, &res, sizeof(res)) == -1) { + log_warnx("bad IMSG_MBOX_STATUS_RESULT"); + break; + } + session_handle_mbox_status_result(s, &res); + break; + } + case IMSG_MBOX_FETCH_HEADER: { + struct imsg_mbox_fetch_header hdr; + size_t hdrlen; + + /* + * Same fixed-header-plus-variable-trailing-bytes + * technique as IMSG_MBOX_APPEND's read side in + * store.c's handle_mbox_append() case (imsg_get_buf() + * for the fixed part, then imsg_get_len()/ + * imsg_get_buf() for whatever remains) -- just read + * here instead of there, since this message travels + * store -> listener rather than the other way. + */ + if (imsg_get_buf(&imsg, &hdr, sizeof(hdr)) == -1) { + log_warnx("bad IMSG_MBOX_FETCH_HEADER (header)"); + break; + } + free(s->pending_header_buf); + s->pending_header_buf = NULL; + s->pending_header_len = 0; + s->pending_header_found = 0; + + hdrlen = imsg_get_len(&imsg); + if (!hdr.found) + break; /* store.c found nothing for this + * message -- nothing more to read. */ + if (hdrlen != hdr.hdrlen) { + log_warnx("session %u: IMSG_MBOX_FETCH_HEADER " + "length mismatch (header says %u, imsg " + "has %zu)", s->id, hdr.hdrlen, hdrlen); + break; + } + if (hdrlen == 0) { + /* found but empty is a legitimate, if odd, + * message (a header-less file) -- + * pending_header_found alone is enough for + * session_send_fetch_response() to emit a + * zero-length BODY[HEADER] {0} literal. */ + s->pending_header_found = 1; + break; + } + if ((s->pending_header_buf = malloc(hdrlen)) == NULL) { + log_warn("session %u: malloc pending header " + "buffer", s->id); + break; + } + if (imsg_get_buf(&imsg, s->pending_header_buf, hdrlen) + == -1) { + log_warnx("bad IMSG_MBOX_FETCH_HEADER (body)"); + free(s->pending_header_buf); + s->pending_header_buf = NULL; + break; + } + s->pending_header_len = (uint32_t)hdrlen; + s->pending_header_found = 1; + break; + } + case IMSG_MBOX_FETCH_BODY: { + struct imsg_mbox_fetch_body bodyhdr; + size_t bodylen; + + /* Same technique as IMSG_MBOX_FETCH_HEADER just + * above -- see that case's comment. */ + if (imsg_get_buf(&imsg, &bodyhdr, sizeof(bodyhdr)) == + -1) { + log_warnx("bad IMSG_MBOX_FETCH_BODY (header)"); + break; + } + free(s->pending_body_buf); + s->pending_body_buf = NULL; + s->pending_body_len = 0; + s->pending_body_found = 0; + /* pending_body_label/has_partial/partial_origin are + * NOT reset here -- like pending_header_label, they're + * set once in fetch_dispatch() for the whole FETCH, + * not per message; bodyhdr.is_text (struct imsg_mbox_ + * fetch_body) is redundant with what fetch_dispatch() + * already knows it asked for and is intentionally + * unused here. */ + + bodylen = imsg_get_len(&imsg); + if (!bodyhdr.found) + break; /* store.c found nothing for this + * message -- nothing more to read. */ + if (bodylen != bodyhdr.bodylen) { + log_warnx("session %u: IMSG_MBOX_FETCH_BODY " + "length mismatch (header says %u, imsg " + "has %zu)", s->id, bodyhdr.bodylen, + bodylen); + break; + } + if (bodylen == 0) { + /* found but empty is legitimate (a + * zero-length message, or BODY.PEEK[TEXT] on + * a message that's all header) -- + * pending_body_found alone is enough for + * session_send_fetch_response() to emit a + * zero-length literal. */ + s->pending_body_found = 1; + break; + } + if ((s->pending_body_buf = malloc(bodylen)) == NULL) { + log_warn("session %u: malloc pending body " + "buffer", s->id); + break; + } + if (imsg_get_buf(&imsg, s->pending_body_buf, bodylen) + == -1) { + log_warnx("bad IMSG_MBOX_FETCH_BODY (body)"); + free(s->pending_body_buf); + s->pending_body_buf = NULL; + break; + } + s->pending_body_len = (uint32_t)bodylen; + s->pending_body_found = 1; + break; + } + case IMSG_MBOX_FETCH_ENVELOPE: { + struct imsg_mbox_fetch_envelope envhdr; + size_t envlen; + + /* Same technique as IMSG_MBOX_FETCH_HEADER/IMSG_MBOX_ + * FETCH_BODY above -- see IMSG_MBOX_FETCH_HEADER's + * comment. */ + if (imsg_get_buf(&imsg, &envhdr, sizeof(envhdr)) == + -1) { + log_warnx("bad IMSG_MBOX_FETCH_ENVELOPE (header)"); + break; + } + free(s->pending_envelope_buf); + s->pending_envelope_buf = NULL; + s->pending_envelope_len = 0; + s->pending_envelope_found = 0; + + envlen = imsg_get_len(&imsg); + if (!envhdr.found) + break; /* store.c couldn't build an envelope + * for this message -- nothing more to + * read. */ + if (envlen != envhdr.envlen) { + log_warnx("session %u: IMSG_MBOX_FETCH_ENVELOPE " + "length mismatch (header says %u, imsg " + "has %zu)", s->id, envhdr.envlen, envlen); + break; + } + if (envlen == 0) { + /* build_envelope() always emits at least + * "(NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL)" + * -- a zero-length result should never + * actually happen, but handled the same + * defensive way as the header/body cases + * above rather than assumed impossible. */ + s->pending_envelope_found = 1; + break; + } + if ((s->pending_envelope_buf = malloc(envlen)) == NULL) { + log_warn("session %u: malloc pending envelope " + "buffer", s->id); + break; + } + if (imsg_get_buf(&imsg, s->pending_envelope_buf, envlen) + == -1) { + log_warnx("bad IMSG_MBOX_FETCH_ENVELOPE (body)"); + free(s->pending_envelope_buf); + s->pending_envelope_buf = NULL; + break; + } + s->pending_envelope_len = (uint32_t)envlen; + s->pending_envelope_found = 1; + break; + } + case IMSG_MBOX_FETCH_BODYSTRUCTURE: { + struct imsg_mbox_fetch_bodystructure bshdr; + size_t bslen; + + /* Same technique as IMSG_MBOX_FETCH_HEADER/IMSG_MBOX_ + * FETCH_ENVELOPE above -- see IMSG_MBOX_FETCH_HEADER's + * comment. */ + if (imsg_get_buf(&imsg, &bshdr, sizeof(bshdr)) == + -1) { + log_warnx("bad IMSG_MBOX_FETCH_BODYSTRUCTURE " + "(header)"); + break; + } + free(s->pending_bodystructure_buf); + s->pending_bodystructure_buf = NULL; + s->pending_bodystructure_len = 0; + s->pending_bodystructure_found = 0; + + bslen = imsg_get_len(&imsg); + if (!bshdr.found) + break; /* store.c couldn't build a + * BODYSTRUCTURE for this message -- + * nothing more to read. */ + if (bslen != bshdr.bslen) { + log_warnx("session %u: " + "IMSG_MBOX_FETCH_BODYSTRUCTURE length " + "mismatch (header says %u, imsg has %zu)", + s->id, bshdr.bslen, bslen); + break; + } + if (bslen == 0) { + /* build_bodystructure() always emits at + * least a minimal single-part structure -- + * zero-length should never actually happen, + * handled defensively anyway, same as the + * header/body/envelope cases above. */ + s->pending_bodystructure_found = 1; + break; + } + if ((s->pending_bodystructure_buf = malloc(bslen)) == + NULL) { + log_warn("session %u: malloc pending " + "bodystructure buffer", s->id); + break; + } + if (imsg_get_buf(&imsg, s->pending_bodystructure_buf, + bslen) == -1) { + log_warnx("bad IMSG_MBOX_FETCH_BODYSTRUCTURE " + "(body)"); + free(s->pending_bodystructure_buf); + s->pending_bodystructure_buf = NULL; + break; + } + s->pending_bodystructure_len = (uint32_t)bslen; + s->pending_bodystructure_found = 1; + break; + } + case IMSG_MBOX_FETCH_META: { + struct imsg_mbox_fetch_meta meta; + + if (imsg_get_data(&imsg, &meta, sizeof(meta)) == -1) { + log_warnx("bad IMSG_MBOX_FETCH_META"); + break; + } + /* Shared reply type for FETCH, STORE, and (RFC 7162 + * addition this pass) a QRESYNC SELECT resync (see + * imapd.h's imsg_mbox_store/imsg_mbox_selected + * comments) -- s->state says which request is + * actually in flight right now, and therefore which + * formatter applies. The SELECTING case is buffered + * (session_handle_select_fetch()), not formatted + * immediately, per RFC 7162 SS3.2.6's VANISHED-before- + * FETCH ordering requirement -- see that function's + * comment. */ + if (s->state == SESSION_SELECTING) + session_handle_select_fetch(s, &meta); + else if (s->state == SESSION_STORING) + session_send_store_fetch_response(s, &meta); + else + session_send_fetch_response(s, &meta); + break; + } + case IMSG_MBOX_EXPUNGED: { + struct imsg_mbox_expunged exp; + + if (imsg_get_data(&imsg, &exp, sizeof(exp)) == -1) { + log_warnx("bad IMSG_MBOX_EXPUNGED"); + break; + } + /* + * A real EXPUNGE/CLOSE writes each one to the client + * immediately, as it always has. During a MOVE + * (s->state == SESSION_COPYING), handle_mbox_move() + * sends this same message type for each moved + * message's old UID/seqno -- but SS6.4.8 requires + * COPYUID to precede any EXPUNGE/VANISHED for the + * same operation, so these get buffered instead and + * flushed (after COPYUID) only once IMSG_MBOX_RESULT + * arrives -- see s->move_expunged's comment and + * session_finish_copy_or_move(). + */ + if (s->state == SESSION_COPYING) { + if (s->move_expunged_n == s->move_expunged_cap) { + uint32_t newcap = + s->move_expunged_cap ? + s->move_expunged_cap * 2 : 16; + struct imsg_mbox_expunged *enew = + reallocarray(s->move_expunged, + newcap, sizeof(*enew)); + + if (enew == NULL) { + log_warn("session %u: " + "realloc move_expunged", + s->id); + break; + } + s->move_expunged = enew; + s->move_expunged_cap = newcap; + } + s->move_expunged[s->move_expunged_n++] = exp; + } else + session_send_expunge_response(s, &exp); + break; + } + case IMSG_MBOX_COPY_MAPPING: { + struct imsg_mbox_copy_mapping m; + + if (imsg_get_data(&imsg, &m, sizeof(m)) == -1) { + log_warnx("bad IMSG_MBOX_COPY_MAPPING"); + break; + } + session_handle_mbox_copy_mapping(s, &m); + break; + } + case IMSG_MBOX_SEARCH_MATCH: { + struct imsg_mbox_search_match m; + + if (imsg_get_data(&imsg, &m, sizeof(m)) == -1) { + log_warnx("bad IMSG_MBOX_SEARCH_MATCH"); + break; + } + session_handle_mbox_search_match(s, &m); + break; + } + case IMSG_MBOX_SELECT_VANISHED: { + struct imsg_mbox_select_vanished v; + + if (imsg_get_data(&imsg, &v, sizeof(v)) == -1) { + log_warnx("bad IMSG_MBOX_SELECT_VANISHED"); + break; + } + /* Despite the name, also reused for RFC 7162 SS3.2.6's + * VANISHED UID FETCH modifier this pass -- see + * imapd.h's imsg_mbox_select_vanished comment. + * SESSION_SELECTING buffers (ordering against EXISTS/ + * the resync FETCH data); SESSION_FETCHING writes + * immediately, since a plain UID FETCH has only the + * "VANISHED before any FETCH response" ordering + * requirement, already satisfied by store.c's own + * send order (see handle_mbox_fetch()'s comment). */ + if (s->state == SESSION_SELECTING) + session_handle_select_vanished(s, &v); + else + session_handle_fetch_vanished(s, &v); + break; + } + case IMSG_MBOX_STORE_MODIFIED: { + struct imsg_mbox_store_modified m; + + if (imsg_get_data(&imsg, &m, sizeof(m)) == -1) { + log_warnx("bad IMSG_MBOX_STORE_MODIFIED"); + break; + } + session_handle_store_modified(s, &m); + break; + } + case IMSG_MBOX_IDLE_UID: { + struct imsg_mbox_idle_uid item; + + if (imsg_get_data(&imsg, &item, sizeof(item)) == -1) { + log_warnx("bad IMSG_MBOX_IDLE_UID"); + break; + } + session_handle_idle_uid(s, &item); + break; + } + case IMSG_MBOX_IDLE_REFRESHED: { + struct imsg_mbox_idle_refreshed res; + + if (imsg_get_data(&imsg, &res, sizeof(res)) == -1) { + log_warnx("bad IMSG_MBOX_IDLE_REFRESHED"); + break; + } + session_handle_idle_refreshed(s, &res); + break; + } + case IMSG_MBOX_RESULT: { + struct imsg_mbox_result res; + + if (imsg_get_data(&imsg, &res, sizeof(res)) == -1) { + log_warnx("bad IMSG_MBOX_RESULT"); + break; + } + session_handle_mbox_result(s, &res); + break; + } + case IMSG_MBOX_LIST_ITEM: { + struct imsg_mbox_list_item item; + + if (imsg_get_data(&imsg, &item, sizeof(item)) == -1) { + log_warnx("bad IMSG_MBOX_LIST_ITEM"); + break; + } + session_handle_mbox_list_item(s, &item); + break; + } + case IMSG_MBOX_APPENDED: { + struct imsg_mbox_appended res; + + if (imsg_get_data(&imsg, &res, sizeof(res)) == -1) { + log_warnx("bad IMSG_MBOX_APPENDED"); + break; + } + session_handle_mbox_appended(s, &res); + break; + } + default: + log_debug("session %u: store channel: unhandled %d", + s->id, imsg_get_type(&imsg)); + break; + } + imsg_free(&imsg); + } + imsgev_add(s->store_iev); /* re-arm -- see this function's + * header comment */ + (void)fd; +} + +/* + * Finishes the SELECT round trip cmd_select() started: builds and sends + * the RFC 9051 SS6.3.2-required response sequence (or the NO failure + * case) using s->pending_tag, and transitions session state. + * + * RFC 7162 additions this pass: s->mbox_highestmodseq is cached from + * res->highestmodseq unconditionally (store.c always populates it -- see + * imapd.h's imsg_mbox_selected comment), for session_condstore_ + * enable()'s later use regardless of whether *this* session is CONDSTORE- + * aware yet. If it is (s->condstore_enabled, set synchronously by + * cmd_select() when it saw a CONDSTORE or QRESYNC select-param -- see that + * function's comment for why it doesn't go through session_condstore_ + * enable() itself), an OK [HIGHESTMODSEQ] response is included -- NOMODSEQ + * is never emitted, since v1's only mailbox always supports persistent + * mod-sequence storage (see struct mbox_index's comment in store.c). + * Finally, any accumulated QRESYNC resync data (s->vanished_ranges/s-> + * qresync_fetches, populated by session_handle_select_vanished()/session_ + * handle_select_fetch() while this SELECT was in flight) is flushed in the + * RFC-required order: VANISHED (EARLIER) first, then the resync FETCH + * responses, then the tagged OK -- see those two fields' comments in + * struct session for why this can't be streamed live instead. + */ +static void +session_handle_mbox_selected(struct session *s, struct imsg_mbox_selected *res) +{ + char buf[128]; + + if (!res->ok) { + /* RFC 9051 SS6.3.2 Result: "NO - select failure, now in + * authenticated state: no such mailbox, can't access + * mailbox". v1's only failure mode is "not INBOX" (see + * handle_mbox_select() in store.c), which maps directly to + * "no such mailbox" -- RFC 5530 NONEXISTENT's own worked + * example is literally that same text. */ + s->state = SESSION_AUTHENTICATED; + session_reply(s, s->pending_tag, "NO", + "[NONEXISTENT] no such mailbox"); + return; + } + + s->state = SESSION_SELECTED; + s->mbox_highestmodseq = res->highestmodseq; + + /* + * Order matches RFC 9051 SS6.3.2's own worked example (EXISTS, + * OK[UIDVALIDITY], OK[UIDNEXT], FLAGS, OK[PERMANENTFLAGS], LIST), + * though that section explicitly says response order isn't + * significant. + */ + snprintf(buf, sizeof(buf), "%u EXISTS", res->exists); + session_untagged(s, buf); + + snprintf(buf, sizeof(buf), "OK [UIDVALIDITY %u] UIDs valid", + res->uidvalidity); + session_untagged(s, buf); + + snprintf(buf, sizeof(buf), "OK [UIDNEXT %u] Predicted next UID", + res->uidnext); + session_untagged(s, buf); + + if (s->condstore_enabled) { + snprintf(buf, sizeof(buf), "OK [HIGHESTMODSEQ %llu]", + (unsigned long long)res->highestmodseq); + session_untagged(s, buf); + } + + /* + * Standard flags, RFC 9051 SS6.3.2's own example list verbatim -- + * matches the five spec-defined maildir flag-suffix letters + * (D/R/S/T/F) openimap-storage-backend.md sourced against + * Courier's maildir(5). FLAGS itself is unaffected by read-only: + * it just reports what flags exist in the mailbox, not what this + * session may change. + * + * PERMANENTFLAGS differs for EXAMINE: SS6.3.3's own worked example + * shows "OK [PERMANENTFLAGS ()] No permanent flags permitted" for + * an EXAMINE'd mailbox, matching that section's "No changes to the + * permanent state of the mailbox... are permitted" text. For a + * plain SELECT, PERMANENTFLAGS adds "\*" per SS6.3.2's own example: + * the index format supports arbitrary per-message keywords, so the + * client is allowed to create new ones. + */ + session_untagged(s, + "FLAGS (\\Answered \\Flagged \\Deleted \\Seen \\Draft)"); + if (s->mbox_readonly) + session_untagged(s, + "OK [PERMANENTFLAGS ()] No permanent flags permitted"); + else + session_untagged(s, + "OK [PERMANENTFLAGS (\\Answered \\Flagged \\Deleted \\Seen " + "\\Draft \\*)] System flags and keywords allowed"); + + /* + * RFC 9051 SS6.3.2: "The server MUST return a LIST response with + * the mailbox name." "/" as the hierarchy delimiter -- a resolved + * project decision as of a later pass (openimap-storage-backend.md, + * "Open items carried from this session" #9; see also cmd_ + * namespace()'s NAMESPACE response, which reports the identical + * delimiter). "()" -- no attributes -- since no mailbox here can + * ever have children (v1 is flat) and this LIST response only fires + * once SELECT/EXAMINE has already succeeded, i.e. the mailbox is + * selectable by construction. + * + * Real-hardware bug found testing RFC 9051 SS6.3.4-SS6.3.6 multi- + * mailbox support (docs/openimap-storage-backend.md item 10) on + * premio: this used to be the fixed literal "INBOX" unconditionally, + * correct back when INBOX was the only mailbox that could ever be + * selected. s->selected_mailbox (set by select_or_examine() at + * dispatch time -- see that field's own comment) now names whatever + * was actually just selected; quoted, not bare, since an arbitrary + * flat mailbox name can contain a space (same reasoning as session_ + * handle_mbox_status_result()'s identical choice) -- INBOX itself is + * still safe unquoted, but there's no need for a special case since + * a quoted "INBOX" is exactly as valid IMAP as a bare one. + */ + { + char listbuf[MBOX_NAME_MAX + 32]; + + snprintf(listbuf, sizeof(listbuf), "LIST () \"/\" \"%s\"", + s->selected_mailbox); + session_untagged(s, listbuf); + } + + if (s->vanished_nranges > 0) { + char vbuf[8192]; + char full[8192 + 32]; + int truncated; + size_t vlen; + int flen; + + /* + * session_untagged()'s own 512-byte buffer is too small for + * a potentially large VANISHED list -- same bypass session_ + * finish_search() already uses for ESEARCH, and for the same + * reason (truncating range data would be materially wrong, + * not cosmetic). Formats "* VANISHED (EARLIER) \r\n" + * directly, not through the small buf[] above. + */ + vlen = format_range_list(vbuf, sizeof(vbuf), + s->vanished_ranges, s->vanished_nranges, &truncated); + if (truncated) + log_warnx("session %u: VANISHED (EARLIER) list " + "truncated at %zu bytes", s->id, vlen); + + flen = snprintf(full, sizeof(full), "* VANISHED (EARLIER) %s\r\n", + vbuf); + if (flen > 0) + session_write(s, full, (size_t)flen >= sizeof(full) ? + sizeof(full) - 1 : (size_t)flen); + } + free(s->vanished_ranges); + s->vanished_ranges = NULL; + s->vanished_nranges = 0; + s->vanished_cap = 0; + + { + uint32_t i; + + for (i = 0; i < s->qresync_nfetches; i++) + session_send_qresync_fetch_response(s, + &s->qresync_fetches[i]); + } + free(s->qresync_fetches); + s->qresync_fetches = NULL; + s->qresync_nfetches = 0; + s->qresync_fetches_cap = 0; + + /* + * RFC 9051 SS6.3.2: "the server SHOULD prefix the text of the + * tagged OK response with the '[READ-WRITE]' response code" for + * SELECT. SS6.3.3: the tagged OK for EXAMINE "MUST begin with the + * '[READ-ONLY]' response code" -- s->mbox_readonly is a reliable + * proxy for "this was EXAMINE, not SELECT" since v1 has no other + * way for a mailbox to end up read-only (see struct session's + * mbox_readonly comment). + */ + if (s->mbox_readonly) + session_reply(s, s->pending_tag, "OK", + "[READ-ONLY] EXAMINE completed"); + else + session_reply(s, s->pending_tag, "OK", + "[READ-WRITE] SELECT completed"); +} + +/* + * Finishes the STATUS round trip cmd_status() started: formats the + * untagged "* STATUS mailbox (...)" response using s->status_attrs to + * decide which of store.c's always-populated fields to include, restores + * s->state to s->status_prev_state (STATUS never changes SELECTED-ness -- + * RFC 9051 SS6.3.11), and sends the tagged completion. + * + * Attribute order in the response is a fixed canonical order (MESSAGES, + * UIDNEXT, UIDVALIDITY, UNSEEN, DELETED, SIZE, HIGHESTMODSEQ), not the + * client's own request order -- directly sourced from RFC 9051 SS6.3.11's + * own worked example: "C: A042 STATUS blurdybloop (UIDNEXT MESSAGES)" + * answered "S: * STATUS blurdybloop (MESSAGES 231 UIDNEXT 44292)" -- the + * server reordered UIDNEXT/MESSAGES from the client's own request order, + * so there's no conformance reason to preserve request order here either. + */ +static void +session_handle_mbox_status_result(struct session *s, + struct imsg_mbox_status_result *res) +{ + char buf[MBOX_NAME_MAX + 256]; /* F2 fix: was buf[256], too small + * for a near-maximal mailbox name plus + * the STATUS attrs. */ + size_t len; + int n, first = 1; + + s->state = s->status_prev_state; + + if (!res->ok) { + /* cmd_status() already ruled out a malformed name -- what's + * left is either a syntactically valid name that doesn't + * exist on disk (handle_mbox_status()'s select_mailbox_dir() + * failing, RFC 9051 SS6.3.4-SS6.3.6 multi-mailbox support) or + * store.c's own open/flock/index_load failing the same way + * handle_mbox_select() can. v1 has no way to tell those two + * apart from here (struct imsg_mbox_status_result carries no + * distinguishing detail, unlike IMSG_MBOX_APPENDED's no_such_ + * mailbox flag) -- NONEXISTENT is still the more informative + * answer in the common case, so it's used for both, same + * "one plausible RFC 5530 code covers the real failure modes" + * judgment call session_handle_mbox_selected() already makes + * for SELECT's identical ambiguity. */ + session_reply(s, s->pending_tag, "NO", + "[NONEXISTENT] no such mailbox"); + return; + } + + /* + * Quoted, not bare -- unlike the fixed literal "INBOX" this response + * used to always echo, an arbitrary flat mailbox name can contain a + * space (mailbox_name_valid() only refuses "/", CTL bytes, and the + * three reserved maildir-internal names), which would otherwise + * desynchronize a naive client's own response parser. No backslash- + * escaping of an embedded '"' or '\' within the name itself -- same + * "not full ABNF/quoted-string conformance" simplification parse_ + * list_token()'s own comment already documents for the parsing + * direction; a name containing either is an accepted, pre-existing + * gap, not new here. + */ + len = (size_t)snprintf(buf, sizeof(buf), "STATUS \"%s\" (", + s->status_mailbox); + +#define STATUS_APPEND(fmt, val) do { \ + if (len < sizeof(buf)) { /* F2 fix: never index past buf */ \ + n = snprintf(buf + len, sizeof(buf) - len, "%s" fmt, \ + first ? "" : " ", (val)); \ + if (n > 0 && (size_t)n < sizeof(buf) - len) \ + len += (size_t)n; \ + first = 0; \ + } \ +} while (0) + + if (s->status_attrs & STATUS_ATT_MESSAGES) + STATUS_APPEND("MESSAGES %u", res->messages); + if (s->status_attrs & STATUS_ATT_UIDNEXT) + STATUS_APPEND("UIDNEXT %u", res->uidnext); + if (s->status_attrs & STATUS_ATT_UIDVALIDITY) + STATUS_APPEND("UIDVALIDITY %u", res->uidvalidity); + if (s->status_attrs & STATUS_ATT_UNSEEN) + STATUS_APPEND("UNSEEN %u", res->unseen); + if (s->status_attrs & STATUS_ATT_DELETED) + STATUS_APPEND("DELETED %u", res->deleted); + if (s->status_attrs & STATUS_ATT_SIZE) + STATUS_APPEND("SIZE %llu", (unsigned long long)res->size); + if (s->status_attrs & STATUS_ATT_HIGHESTMODSEQ) + STATUS_APPEND("HIGHESTMODSEQ %llu", + (unsigned long long)res->highestmodseq); + +#undef STATUS_APPEND + + if (len < sizeof(buf) - 1) { + buf[len++] = ')'; + buf[len] = '\0'; + } + + session_untagged(s, buf); + session_reply(s, s->pending_tag, "OK", "STATUS completed"); +} + +/* + * Terminal reply shared by CREATE/DELETE/RENAME (cmd_create()/cmd_delete()/ + * cmd_rename()): all three send exactly one struct imsg_mbox_result back, + * with only "ok" meaningful (see imapd.h's imsg_mbox_create/imsg_mbox_ + * delete/imsg_mbox_rename comments) -- "count" is always 0, unlike LIST + * there's no per-item stream preceding this terminal reply. Which of the + * three commands was actually in flight is read from s->state before it's + * overwritten, purely to pick the right command name for the tagged + * completion/failure text -- same "read s->state before restoring it" + * pattern session_handle_mbox_result() itself already uses for STORING/ + * EXPUNGING/FETCHING. s->mbox_op_prev_state (whichever ST_AUTH state was + * current before the round trip -- all three are command-auth and none + * changes SELECTED-ness, RFC 9051 SS6.3.4-SS6.3.6) is restored + * unconditionally, success or failure, same as s->status_prev_state's own + * restore in session_handle_mbox_status_result() above. + */ +static void +session_finish_mbox_op(struct session *s, struct imsg_mbox_result *res) +{ + const char *cmdname; + char text[64]; + + if (s->state == SESSION_CREATING) + cmdname = "CREATE"; + else if (s->state == SESSION_DELETING) + cmdname = "DELETE"; + else + cmdname = "RENAME"; + + s->state = s->mbox_op_prev_state; + + if (!res->ok) { + /* store.c's own failure modes: CREATE's mkdir(2)/ + * ensure_maildir_dirs() failing for a reason other than the + * name already existing (already ruled out client-side by + * mailbox_name_is_inbox()/mailbox_name_valid(), and handled + * as its own EEXIST case -- see handle_mbox_create()), or + * DELETE/RENAME losing a race against the name's own + * existence between listener.c's client-side check and + * store.c's own stat(2) -- no RFC 5530 code fits either + * cleanly, so a plain NO is the honest answer, same as + * session_handle_mbox_result()'s own FETCH/STORE/EXPUNGE + * failure path. */ + snprintf(text, sizeof(text), "%s failed", cmdname); + session_reply(s, s->pending_tag, "NO", text); + return; + } + + /* + * Real-hardware bug found testing RENAME on premio: if this session + * had the just-renamed mailbox itself SELECTed, s->selected_mailbox + * (consulted by session_handle_mbox_appended()'s own-mailbox check + * and echoed in SELECT's/EXAMINE's own untagged LIST response) still + * held the old name -- store.c's handle_mbox_rename() already makes + * its own cwd/current_mailbox_dir follow the rename (see that + * function's comment), so listener.c needs the identical fix on its + * side of the same problem, or the two would silently disagree about + * what this session actually has selected. strcmp(cmdname, "RENAME") + * rather than checking s->state directly since that's already been + * overwritten (restored to s->mbox_op_prev_state) a few lines above. + */ + if (strcmp(cmdname, "RENAME") == 0 && + strcmp(s->selected_mailbox, s->rename_oldname) == 0) + strlcpy(s->selected_mailbox, s->rename_newname, + sizeof(s->selected_mailbox)); + + snprintf(text, sizeof(text), "%s completed", cmdname); + session_reply(s, s->pending_tag, "OK", text); +} + +/* + * One item of the IMSG_MBOX_LIST_ITEM stream store.c's handle_mbox_list() + * sends during a SESSION_LISTING round trip (list_dispatch()'s own tail, + * once its synchronous empty-pattern/INBOX special cases are past) -- + * tested immediately against s->list_pattern (the same canonical + * reference-plus-pattern concatenation list_dispatch() already used for + * the synchronous INBOX check) via list_pattern_match(), case-sensitively + * (ci=0 -- see that function's own comment on why INBOX alone gets ci=1), + * and turned into its own untagged LIST/LSUB response immediately if it + * matches. Streamed straight through rather than buffered first the way + * s->search_matches accumulates SEARCH's results -- there's no sorting, + * deduplication, or MIN/MAX/COUNT-style aggregation LIST needs to do + * across the whole result set, so nothing is gained by waiting. + */ +static void +session_handle_mbox_list_item(struct session *s, + struct imsg_mbox_list_item *item) +{ + char buf[(2 * MBOX_NAME_MAX) + 32]; + const char *kw = s->list_is_lsub ? "LSUB" : "LIST"; + + if (!list_pattern_match(s->list_pattern, item->mailbox, 0)) + return; + + /* Quoted, not bare -- same "an arbitrary flat mailbox name can + * contain a space" reasoning as session_handle_mbox_status_ + * result()'s identical choice; the fixed literal "INBOX" is safe + * unquoted (see list_dispatch()'s own untagged response for it) but + * a real, named mailbox isn't. "()" -- no attributes -- same + * SS7.3.1 "MAY send none of these" choice used throughout. */ + snprintf(buf, sizeof(buf), "%s () \"/\" \"%s\"", kw, item->mailbox); + session_untagged(s, buf); +} + +/* + * Terminal reply for the SESSION_LISTING round trip list_dispatch() + * started: store.c's handle_mbox_list() streams zero or more IMSG_MBOX_ + * LIST_ITEM messages (each already turned into its own untagged response, + * if it matched, by session_handle_mbox_list_item() above) followed by + * exactly one terminal IMSG_MBOX_RESULT. res->count (how many items were + * streamed, matched or not) isn't needed here -- only res->ok, and even + * that can only be 0 from a real I/O error opening this session's own + * maildir root (handle_mbox_list()'s own comment), not from anything + * pattern- or mailbox-name-related; SS6.3.9's "silently ignore" rule for + * an unmatched pattern is already satisfied by simply not having sent an + * untagged response for it, same as the synchronous INBOX case just above + * in list_dispatch(). Restores s->mbox_op_prev_state the same way + * session_finish_mbox_op() does (LIST/LSUB are command-auth, RFC 9051 + * SS6.3.9, and neither changes SELECTED-ness). + */ +static void +session_finish_list(struct session *s, struct imsg_mbox_result *res) +{ + const char *cmdname = s->list_is_lsub ? "LSUB" : "LIST"; + char text[32]; + + s->state = s->mbox_op_prev_state; + + if (!res->ok) { + snprintf(text, sizeof(text), "%s failed", cmdname); + session_reply(s, s->pending_tag, "NO", text); + return; + } + + snprintf(text, sizeof(text), "%s completed", cmdname); + session_reply(s, s->pending_tag, "OK", text); +} + +/* + * EXPUNGE's untagged response (RFC 9051 SS7.5.1): "* EXPUNGE", + * `seqno` already computed by store.c's handle_mbox_expunge() per + * SS7.5.1's "immediately decremented" rule -- this function has nothing + * to compute, only to format. Never called for CLOSE's silent=1 request: + * store.c simply never sends IMSG_MBOX_EXPUNGED when req->silent is set + * (RFC 9051 SS6.4.1: "No untagged EXPUNGE responses are sent"), so there's + * no silent-suppression logic needed here either. + * + * RFC 7162 SS3.2.10.2 addition: once this session has "ENABLE QRESYNC"'d + * (s->qresync_enabled), the server "MUST use the VANISHED response without + * the EARLIER tag instead of the EXPUNGE response... for the duration of + * the connection" -- UID-based (exp->uid, populated by store.c this pass; + * see imapd.h's imsg_mbox_expunged comment), one per message, matching + * this function's existing one-response-per-removed-message shape (RFC + * 7162's own examples combine several UIDs into a single VANISHED line, + * but nothing requires it -- see SS3.2.10's "if necessary, two VANISHED + * responses are sent" for confirmation that multiple VANISHED lines per + * operation are legal; combining these would need the same "buffer until + * the terminal reply" treatment the QRESYNC SELECT resync path uses, + * which isn't necessary here since EXPUNGE has no VANISHED-before-FETCH + * ordering constraint to satisfy -- SS3.2.10.2's ordering rule is specific + * to VANISHED (EARLIER), not this plain form). + */ +static void +session_send_expunge_response(struct session *s, + struct imsg_mbox_expunged *exp) +{ + char buf[32]; + + if (s->qresync_enabled) + snprintf(buf, sizeof(buf), "VANISHED %u", exp->uid); + else + snprintf(buf, sizeof(buf), "%u EXPUNGE", exp->seqno); + session_untagged(s, buf); +} + +/* + * RFC 7162 SS3.2.6's VANISHED UID FETCH modifier: formats one range from + * store.c's IMSG_MBOX_SELECT_VANISHED (see that struct's now-broadened + * comment in imapd.h) as a "* VANISHED (EARLIER) ..." untagged + * response, written immediately rather than buffered -- unlike the QRESYNC + * SELECT resync case (session_handle_select_vanished()), a UID FETCH has + * no other untagged response that needs to interleave in a specific + * relative order with this one; the only ordering requirement (VANISHED + * before any FETCH response for the same command) is already guaranteed + * by handle_mbox_fetch() sending all of its VANISHED ranges, in full, + * before it sends any IMSG_MBOX_FETCH_META. + */ +static void +session_handle_fetch_vanished(struct session *s, + struct imsg_mbox_select_vanished *v) +{ + char buf[64]; + + if (v->uid_lo == v->uid_hi) + snprintf(buf, sizeof(buf), "VANISHED (EARLIER) %u", v->uid_lo); + else + snprintf(buf, sizeof(buf), "VANISHED (EARLIER) %u:%u", + v->uid_lo, v->uid_hi); + session_untagged(s, buf); +} + +/* + * Shared by cmd_expunge(), cmd_close(), and (this pass) cmd_uid()'s + * EXPUNGE branch: all three send the identical IMSG_MBOX_EXPUNGE request + * shape, transition to SESSION_EXPUNGING, and save the tag -- see + * imapd.h's imsg_mbox_expunge comment for why CLOSE reuses EXPUNGE's + * request/reply pair wholesale rather than getting its own. is_close is + * stashed in s->close_after_expunge so session_handle_mbox_result() knows, + * once the terminal IMSG_MBOX_RESULT arrives, which tagged completion text + * to send and which state to return to. + * + * RFC 9051 SS6.4.9 addition (UID EXPUNGE): by_uid/uid_lo/uid_hi/lo_star/ + * hi_star carry UID EXPUNGE's required UID-set argument through to + * store.c (see imapd.h's imsg_mbox_expunge comment) -- meaningless (and + * always zeroed by cmd_close()/cmd_expunge()) when by_uid is 0, since + * plain EXPUNGE takes no arguments and CLOSE has no UID-restricted form at + * all (never called with is_close=1 and by_uid=1 together). Untagged + * EXPUNGE/VANISHED responses stay seqno/UID exactly as store.c's silent + * flag and s->qresync_enabled already decide (RFC 9051 SS6.4.9: "The + * number after the '*' in an untagged FETCH or EXPUNGE response is always + * a message sequence number... even for a UID command response") -- only + * the tagged completion text changes based on by_uid, via s->cmd_by_uid + * and session_handle_mbox_result()'s cmdname. + */ +static int +session_request_expunge(struct session *s, const char *tag, int is_close, + int by_uid, uint32_t uid_lo, uint32_t uid_hi, int lo_star, int hi_star) +{ + struct imsg_mbox_expunge req; + const char *cmdname = is_close ? "CLOSE" : + (by_uid ? "UID EXPUNGE" : "EXPUNGE"); + + if (s->mbox_readonly) { + if (is_close) { + /* + * RFC 9051 SS6.4.1: "No messages are removed, and no + * error is given, if the mailbox is selected by + * EXAMINE." Skip the store round trip entirely -- + * nothing to expunge, s->mbox_highestmodseq is still + * correct since nothing changed, and CLOSE already + * sends no untagged EXPUNGE responses even in the + * read-write case, so there's nothing else this reply + * needs to do besides deselect and reply OK. + */ + s->state = SESSION_AUTHENTICATED; + session_reply(s, tag, "OK", "CLOSE completed"); + return (1); + } + /* + * Unlike CLOSE, plain/UID EXPUNGE has no RFC 9051 text + * excusing it on a read-only mailbox -- SS6.3.3's "No changes + * to the permanent state of the mailbox... are permitted" + * applies directly, since EXPUNGE permanently removes + * messages. RFC 5530 CANNOT ("The operation violates some + * invariant of the server and can never succeed") fits: this + * particular session can never expunge while its selection + * stays read-only, no retry will help. + */ + session_reply(s, tag, "NO", "[CANNOT] Mailbox is read-only " + "(selected via EXAMINE)"); + return (1); + } + + if (s->store_iev == NULL) { + /* Same internal-invariant check as cmd_select()/cmd_fetch()/ + * cmd_store_cmd() -- ST_SELECTED requires store_iev to + * already be wired. */ + log_warnx("session %u: %s with no store channel wired", + s->id, cmdname); + session_reply(s, tag, "NO", "[SERVERBUG] internal error"); + return (1); + } + + memset(&req, 0, sizeof(req)); + req.silent = is_close; + req.by_uid = by_uid; + req.seq_lo = uid_lo; + req.seq_hi = uid_hi; + req.lo_is_star = lo_star; + req.hi_is_star = hi_star; + + strlcpy(s->pending_tag, tag, sizeof(s->pending_tag)); + s->close_after_expunge = is_close; + s->cmd_by_uid = by_uid; + s->state = SESSION_EXPUNGING; + + if (imsg_compose(&s->store_iev->ibuf, IMSG_MBOX_EXPUNGE, 0, 0, -1, + &req, sizeof(req)) == -1) + log_warn("session %u: imsg_compose IMSG_MBOX_EXPUNGE", s->id); + imsgev_add(s->store_iev); + + return (1); +} + +/* + * cmd_uid()'s EXPUNGE branch: RFC 9051 SS6.4.9's second UID command form, + * "UID EXPUNGE " (the sequence set is a required argument + * here, unlike plain EXPUNGE, which takes none at all -- see imapd.h's + * imsg_mbox_expunge comment). Same v1 single-range restriction FETCH/ + * STORE/SEARCH already have (parse_seq_range() rejects a comma internally + * anyway, but the explicit check here gives a clearer BAD reason, matching + * those other three call sites' own style). Never reachable with is_close + * set -- there is no "UID CLOSE" -- so this always calls session_request_ + * expunge() with is_close=0. + */ +static int +uid_expunge_dispatch(struct session *s, const char *tag, char *args) +{ + uint32_t lo, hi; + int lo_star, hi_star; + + if (args == NULL) { + session_reply(s, tag, "BAD", + "UID EXPUNGE requires a sequence set of UIDs"); + return (1); + } + if (strchr(args, ',') != NULL) { + session_reply(s, tag, "BAD", + "comma-separated sequence sets not supported in v1 -- " + "issue separate UID EXPUNGE commands"); + return (1); + } + if (parse_seq_range(args, &lo, &hi, &lo_star, &hi_star) == -1) { + session_reply(s, tag, "BAD", "invalid sequence set"); + return (1); + } + + return session_request_expunge(s, tag, 0, 1, lo, hi, lo_star, hi_star); +} + +/* + * Appends one IMSG_MBOX_SEARCH_MATCH's sequence number to s->search_ + * matches, growing the array as needed (doubling, starting at 32 -- + * generous for a v1-scale personal mailbox without over-allocating for + * the common small-result-set case). store.c streams these in ascending + * index order (see handle_mbox_search()'s comment); since the index + * itself is UID-ordered and sequence number is just live position within + * it, seqno and uid are *both* ascending across the stream, so the array + * stays sorted and duplicate-free by construction regardless of which of + * the two values gets pushed below -- session_finish_search() relies on + * that for both its range-compaction (format_seq_list()) and its MIN/MAX + * shortcuts (first/last element). + * + * A realloc(3) failure here sets s->search_alloc_failed instead of + * silently dropping the match: an ESEARCH response that's missing + * matches the server actually found would be *wrong*, not just + * incomplete, and this codebase's established posture (EXPUNGE's + * "conservatively kept rather than dropped," APPEND's "reject cleanly + * rather than silently truncate") is to never hand a client data it + * can't stand behind. session_finish_search() checks the flag and sends + * a NO instead of a wrong ESEARCH if it's set. m->uid is now used (RFC + * 9051 SS6.4.9's UID SEARCH command wrapper, wired up this pass via + * search_dispatch()'s by_uid) -- see s->cmd_by_uid's comment in struct + * session for which of seqno/uid gets pushed. + */ +static void +session_handle_mbox_search_match(struct session *s, + struct imsg_mbox_search_match *m) +{ + if (s->search_alloc_failed) + return; + + if (s->search_nmatches == s->search_matches_cap) { + uint32_t newcap = s->search_matches_cap ? + s->search_matches_cap * 2 : 32; + uint32_t *n = reallocarray(s->search_matches, newcap, + sizeof(uint32_t)); + + if (n == NULL) { + log_warn("session %u: realloc SEARCH match array", + s->id); + s->search_alloc_failed = 1; + return; + } + s->search_matches = n; + s->search_matches_cap = newcap; + } + + /* RFC 9051 SS6.4.9: UID SEARCH reports UIDs instead of sequence + * numbers in its ESEARCH data -- see s->cmd_by_uid's comment. */ + s->search_matches[s->search_nmatches++] = s->cmd_by_uid ? + m->uid : m->seqno; + + /* RFC 7162 SS3.1.6: track the running max modseq across all matches + * -- store.c always populates m->modseq (see imapd.h's struct + * imsg_mbox_search_match comment), so this costs nothing even when + * the search program didn't use MODSEQ; session_finish_search() + * only actually prints it when s->search_used_modseq is set. */ + if (m->modseq > s->search_max_modseq) + s->search_max_modseq = m->modseq; +} + +/* + * Appends one IMSG_MBOX_COPY_MAPPING's src_uid/dest_uid pair to s-> + * copy_src_uids/copy_dest_uids, growing both arrays together (same + * doubling-from-16 growth as s->move_expunged, generous for v1-scale + * COPY/MOVE ranges without over-allocating the common single-message + * case). store.c streams these in ascending order (both handle_mbox_ + * copy() and handle_mbox_move() walk the index forward), so both arrays + * stay sorted by construction -- session_finish_copy_or_move() relies on + * that for format_seq_list()'s range-compaction, same precondition + * session_handle_mbox_search_match()'s own comment already documents for + * s->search_matches. + * + * A realloc(3) failure here sets s->copy_alloc_failed instead of silently + * dropping the mapping -- same "never hand a client data it can't stand + * behind" reasoning as s->search_alloc_failed; session_finish_copy_or_ + * move() checks the flag and sends a NO instead of a wrong/incomplete + * COPYUID if it's set. + */ +static void +session_handle_mbox_copy_mapping(struct session *s, + struct imsg_mbox_copy_mapping *m) +{ + if (s->copy_alloc_failed) + return; + + if (s->copy_n == s->copy_cap) { + uint32_t newcap = s->copy_cap ? s->copy_cap * 2 : 16; + uint32_t *newsrc = reallocarray(s->copy_src_uids, newcap, + sizeof(uint32_t)); + uint32_t *newdest; + + if (newsrc == NULL) { + log_warn("session %u: realloc COPY src array", s->id); + s->copy_alloc_failed = 1; + return; + } + s->copy_src_uids = newsrc; + + newdest = reallocarray(s->copy_dest_uids, newcap, + sizeof(uint32_t)); + if (newdest == NULL) { + log_warn("session %u: realloc COPY dest array", s->id); + s->copy_alloc_failed = 1; + return; + } + s->copy_dest_uids = newdest; + s->copy_cap = newcap; + } + + s->copy_src_uids[s->copy_n] = m->src_uid; + s->copy_dest_uids[s->copy_n] = m->dest_uid; + s->copy_n++; +} + +/* + * Appends one IMSG_MBOX_SELECT_VANISHED range to s->vanished_ranges -- + * same growable-array shape as session_handle_mbox_search_match() above, + * without an alloc-failure flag: unlike a wrong/incomplete SEARCH result + * (which this codebase treats as a hard failure -- see session_handle_ + * mbox_search_match()'s comment), a dropped VANISHED range only means the + * client doesn't find out about some already-vanished messages this one + * time and will simply learn about them on its next resync -- a real + * degradation, but not a correctness violation the way handing back a + * SEARCH result for a query that was never actually evaluated would be. + * Logged either way. + */ +static void +session_handle_select_vanished(struct session *s, + struct imsg_mbox_select_vanished *v) +{ + if (s->vanished_nranges == s->vanished_cap) { + uint32_t newcap = s->vanished_cap ? + s->vanished_cap * 2 : 16; + struct vanished_range *n = reallocarray(s->vanished_ranges, + newcap, sizeof(*n)); + + if (n == NULL) { + log_warn("session %u: realloc VANISHED range array", + s->id); + return; + } + s->vanished_ranges = n; + s->vanished_cap = newcap; + } + + s->vanished_ranges[s->vanished_nranges].lo = v->uid_lo; + s->vanished_ranges[s->vanished_nranges].hi = v->uid_hi; + s->vanished_nranges++; +} + +/* + * Appends one IMSG_MBOX_FETCH_META (from a QRESYNC SELECT resync -- see + * session_store_dispatch()'s IMSG_MBOX_FETCH_META case, which routes here + * specifically when s->state == SESSION_SELECTING) to s->qresync_fetches. + * Same "held back until the terminal reply" reason as vanished_ranges -- + * RFC 7162 SS3.2.6's VANISHED-before-FETCH ordering requirement. + */ +static void +session_handle_select_fetch(struct session *s, struct imsg_mbox_fetch_meta *m) +{ + if (s->qresync_nfetches == s->qresync_fetches_cap) { + uint32_t newcap = s->qresync_fetches_cap ? + s->qresync_fetches_cap * 2 : 16; + struct imsg_mbox_fetch_meta *n = reallocarray( + s->qresync_fetches, newcap, sizeof(*n)); + + if (n == NULL) { + log_warn("session %u: realloc QRESYNC fetch array", + s->id); + return; + } + s->qresync_fetches = n; + s->qresync_fetches_cap = newcap; + } + + s->qresync_fetches[s->qresync_nfetches++] = *m; +} + +/* + * Appends one IMSG_MBOX_IDLE_UID (RFC 9051 SS6.3.13) to s->idle_incoming_ + * uids -- accumulated during an in-flight refresh, held back until + * session_handle_idle_refreshed() has the complete list to diff against + * s->idle_known_uids. Same realloc-doubling shape and same "log and drop + * this one item on allocation failure" leniency as session_handle_select_ + * vanished()/session_handle_select_fetch() above -- for the same reason: + * losing track of one UID here just means this round's diff might miss + * reporting it, not a crash or a wrong tagged response. + */ +static void +session_handle_idle_uid(struct session *s, struct imsg_mbox_idle_uid *item) +{ + if (s->idle_incoming_n == s->idle_incoming_cap) { + uint32_t newcap = s->idle_incoming_cap ? + s->idle_incoming_cap * 2 : 16; + uint32_t *n = reallocarray(s->idle_incoming_uids, + newcap, sizeof(*n)); + + if (n == NULL) { + log_warn("session %u: realloc IDLE UID array", s->id); + return; + } + s->idle_incoming_uids = n; + s->idle_incoming_cap = newcap; + } + + s->idle_incoming_uids[s->idle_incoming_n++] = item->uid; +} + +/* + * Pushes "* N EXPUNGE" for every UID present in old (length oldn) but + * absent from cur (length curn), in ascending old-list-position order. + * RFC 9051 SS7.5.1: "the server MUST immediately decrement" the sequence + * number of every subsequent message -- so a removed message's reported + * seqno is exactly its position among messages processed so far (both + * already-reported-removed and still-present), which is why seqno only + * advances on a *present* match below, never on a removed one. Matches + * store.c's own handle_mbox_expunge() comment on the identical rule + * ("that message's current sequence number minus one"), and produces the + * exact same output as SS6.4.3's own worked example (removing original + * positions 3, 4, 7, 11 from an 11-message mailbox reports 3, 3, 5, 8) -- + * reimplemented here as a plain array diff, rather than reusing that + * function directly, since this session's own store child isn't the one + * that removed anything; some *other* session's mutation is what + * triggered this refresh (session_notify_idle_peers()), so all this + * session has to go on is two UID snapshots, not a live index to mutate. + * + * O(oldn * curn) -- fine at the mailbox sizes this project targets (see + * openimap-privsep-design.md's "personal use" scope), same non-goal this + * codebase states elsewhere about large-scale performance. + */ +static void +session_push_idle_expunges(struct session *s, const uint32_t *old, + uint32_t oldn, const uint32_t *cur, uint32_t curn) +{ + uint32_t i, j, seqno; + char buf[32]; + + seqno = 1; + for (i = 0; i < oldn; i++) { + int present = 0; + + for (j = 0; j < curn; j++) { + if (cur[j] == old[i]) { + present = 1; + break; + } + } + if (present) { + seqno++; + continue; + } + snprintf(buf, sizeof(buf), "%u EXPUNGE", seqno); + session_untagged(s, buf); + } +} + +/* + * Terminal reply to an IMSG_MBOX_IDLE_REFRESH round trip (RFC 9051 + * SS6.3.13) -- either the baseline one cmd_idle() triggers right after + * sending "+ idling" (s->idle_baseline_valid still 0: nothing to diff + * against yet, this call's only job is to adopt the freshly streamed list + * as the baseline), or a change-triggered one from session_notify_idle_ + * peers() (baseline already valid: diff old vs. new, push EXPUNGE/EXISTS + * for whatever actually changed). + * + * res->ok == 0 (index open/lock/load failure in store.c, same failure + * modes handle_mbox_select() itself can hit) just drops this round on the + * floor -- s->idle_known_uids is left exactly as it was, so a later + * successful refresh can still diff correctly against whatever the last + * good baseline was; the client just doesn't find out about this + * particular change until then. + * + * Pushing anything is gated on s->idling && s->state == SESSION_SELECTED: + * this round trip is asynchronous, so by the time it lands the client may + * already have sent DONE (s->idling cleared) or issued a new SELECT + * (s->state changed) -- in either case, sending untagged EXPUNGE/EXISTS + * now would be at best surprising and at worst interleaved with whatever + * that new command's own response is building. The cache still gets + * updated either way; only the pushing is skipped. + */ +static void +session_handle_idle_refreshed(struct session *s, + struct imsg_mbox_idle_refreshed *res) +{ + uint32_t *newlist = s->idle_incoming_uids; + uint32_t newn = s->idle_incoming_n; + + s->idle_incoming_uids = NULL; + s->idle_incoming_n = 0; + s->idle_incoming_cap = 0; + + s->idle_refresh_pending = 0; + + if (!res->ok) { + log_warnx("session %u: IDLE refresh failed, keeping last " + "known state", s->id); + free(newlist); + goto maybe_again; + } + + if (!s->idle_baseline_valid) { + free(s->idle_known_uids); + s->idle_known_uids = newlist; + s->idle_known_nuids = newn; + s->idle_known_cap = newn; + s->idle_baseline_valid = 1; + goto maybe_again; + } + + if (s->idling && s->state == SESSION_SELECTED) { + session_push_idle_expunges(s, s->idle_known_uids, + s->idle_known_nuids, newlist, newn); + if (newn != s->idle_known_nuids) { + char buf[32]; + + snprintf(buf, sizeof(buf), "%u EXISTS", newn); + session_untagged(s, buf); + } + } + + free(s->idle_known_uids); + s->idle_known_uids = newlist; + s->idle_known_nuids = newn; + s->idle_known_cap = newn; + +maybe_again: + if (s->idle_refresh_again) { + s->idle_refresh_again = 0; + session_request_idle_refresh(s); + } +} + +/* + * Sends IMSG_MBOX_IDLE_REFRESH on s's own store_iev -- either cmd_idle() + * seeding this session's own baseline right after "+ idling", or session_ + * notify_idle_peers() asking a *sibling* session (same uid, currently + * idling) to recheck after this session's own successful mutation. + * Coalesces: if a refresh is already in flight for s, this just remembers + * to run another one immediately after it completes (s->idle_refresh_ + * again) rather than starting a second overlapping request on the same + * store_iev channel. + */ +static void +session_request_idle_refresh(struct session *s) +{ + if (s->idle_refresh_pending) { + s->idle_refresh_again = 1; + return; + } + if (s->store_iev == NULL) + return; /* no store child wired -- nothing to ask */ + + s->idle_refresh_pending = 1; + if (imsg_compose(&s->store_iev->ibuf, IMSG_MBOX_IDLE_REFRESH, 0, 0, + -1, NULL, 0) == -1) + log_warn("session %u: imsg_compose IMSG_MBOX_IDLE_REFRESH", + s->id); + imsgev_add(s->store_iev); +} + +/* + * Called after a successful EXPUNGE/UID EXPUNGE/CLOSE-with-real-removal, + * APPEND, or MOVE/UID MOVE -- the mutations that can change which UIDs + * exist (a plain STORE, by contrast, only changes flags, which this v1 + * scope doesn't push during IDLE at all -- see imapd.h's imsg_mbox_ + * idle_uid comment). Scans every other live session belonging to the + * *same* uid (v1 has no shared mailboxes, so cross-user notification never + * applies -- see struct session's uid comment) that's currently idling + * with a mailbox selected, and asks each one to refresh. s itself (the + * session that just mutated something) is skipped -- it already knows + * about its own change through its own command's ordinary response. + * + * Doesn't check *which* mailbox each peer has selected against which + * mailbox actually changed -- harmless imprecision, not a correctness + * bug: a peer with a different mailbox selected still only gets asked to + * refresh (IMSG_MBOX_IDLE_REFRESH), and that refresh re-diffs the peer's + * own store child's own current_mailbox_dir against its own cached UID + * snapshot (handle_mbox_idle_refresh() in store.c), so a mismatched wakeup + * finds nothing changed and pushes nothing -- at most a wasted round trip, + * never wrong data. Before RFC 9051 SS6.3.4-SS6.3.6's flat multi-mailbox + * support (docs/openimap-storage-backend.md item 10) this distinction + * didn't exist at all (every SELECTED session necessarily had the same, + * only mailbox selected); tightening this to only wake peers actually + * selected on the mailbox that changed is a real, flagged follow-up, not + * done here. + */ +static void +session_notify_idle_peers(struct session *s) +{ + struct session *other; + + TAILQ_FOREACH(other, &sessions, entry) { + if (other == s) + continue; + if (other->uid != s->uid) + continue; + if (!other->idling || other->state != SESSION_SELECTED) + continue; + session_request_idle_refresh(other); + } +} + +/* + * Formats one QRESYNC-resync FETCH response: always UID + FLAGS + MODSEQ, + * matching RFC 7162 SS3.2.5.1's own worked examples exactly (e.g. "* 49 + * FETCH (UID 117 FLAGS (\Seen \Answered) MODSEQ (90060115194045001))") -- + * unlike session_send_fetch_response(), there's no s->fetch_attrs to + * consult here, since this isn't a client-requested attribute list; the + * RFC itself fixes what a resync FETCH response contains. + */ +static void +session_send_qresync_fetch_response(struct session *s, + struct imsg_mbox_fetch_meta *meta) +{ + char buf[MBOX_FLAGS_MAX + 96]; + + snprintf(buf, sizeof(buf), "%u FETCH (UID %u FLAGS (%s) MODSEQ (%llu))", + meta->seqno, meta->uid, meta->flags, + (unsigned long long)meta->modseq); + session_untagged(s, buf); +} + +/* + * Appends one IMSG_MBOX_STORE_MODIFIED (seqno of a message that failed a + * STORE's UNCHANGEDSINCE test) to s->store_modified -- same growable-array + * shape as the two accumulators above, for session_handle_mbox_result() + * to range-compact into the tagged response's MODIFIED response code once + * the terminal IMSG_MBOX_RESULT arrives. + */ +static void +session_handle_store_modified(struct session *s, + struct imsg_mbox_store_modified *m) +{ + if (s->store_modified_n == s->store_modified_cap) { + uint32_t newcap = s->store_modified_cap ? + s->store_modified_cap * 2 : 16; + uint32_t *n = reallocarray(s->store_modified, newcap, + sizeof(*n)); + + if (n == NULL) { + log_warn("session %u: realloc STORE MODIFIED array", + s->id); + return; + } + s->store_modified = n; + s->store_modified_cap = newcap; + } + + /* RFC 7162 SS3.1.3: MODIFIED lists UIDs for UID STORE, sequence + * numbers for plain STORE -- see s->cmd_by_uid's comment in struct + * session. */ + s->store_modified[s->store_modified_n++] = + s->cmd_by_uid ? m->uid : m->seqno; +} + +/* + * Appends nums[i] as either a bare number or (for a run of consecutive + * values) a "lo:hi" range to buf, comma-separating entries -- RFC 9051 + * SS7.3.4's own ESEARCH examples use exactly this compacted form (e.g. + * "ALL 2,10:11"). nums must already be sorted ascending with no + * duplicates (true by construction -- see session_handle_mbox_search_ + * match()'s comment). + * + * Stops cleanly at a token boundary (never writes a partial number or a + * dangling comma) if the next range wouldn't fit in the remaining space, + * and reports that via *truncated -- see session_finish_search()'s + * comment for why a v1-scale response is expected to always fit, and + * what happens on the rare occasion it doesn't. + */ +static size_t +format_seq_list(char *buf, size_t bufsize, const uint32_t *nums, uint32_t n, + int *truncated) +{ + size_t written = 0; + uint32_t i = 0; + int first = 1; + + *truncated = 0; + buf[0] = '\0'; + + while (i < n) { + uint32_t start = nums[i]; + uint32_t end = start; + uint32_t j = i + 1; + char tok[24]; + int toklen; + + while (j < n && nums[j] == end + 1) { + end = nums[j]; + j++; + } + + if (start == end) + toklen = snprintf(tok, sizeof(tok), "%u", start); + else + toklen = snprintf(tok, sizeof(tok), "%u:%u", start, + end); + if (toklen < 0) + break; + + if (written + (size_t)toklen + (first ? 0 : 1) >= bufsize) { + *truncated = 1; + break; + } + + if (!first) + buf[written++] = ','; + memcpy(buf + written, tok, (size_t)toklen); + written += (size_t)toklen; + buf[written] = '\0'; + first = 0; + i = j; + } + + return (written); +} + +/* + * RFC 7162 counterpart to format_seq_list() above, for VANISHED (EARLIER)'s + * known-uids list: ranges is already pre-compacted (store.c's qresync_send_ + * resync() computes minimal maximal ranges directly, see imapd.h's + * imsg_mbox_select_vanished comment), so this just joins ranges[i].lo: + * ranges[i].hi (or a bare number when lo == hi) with commas -- no run- + * detection needed, unlike format_seq_list()'s individual-number input. + * Same truncate-cleanly-at-a-token-boundary contract and *truncated + * signal as format_seq_list(). + */ +static size_t +format_range_list(char *buf, size_t bufsize, const struct vanished_range *ranges, + uint32_t n, int *truncated) +{ + size_t written = 0; + uint32_t i; + int first = 1; + + *truncated = 0; + buf[0] = '\0'; + + for (i = 0; i < n; i++) { + char tok[24]; + int toklen; + + if (ranges[i].lo == ranges[i].hi) + toklen = snprintf(tok, sizeof(tok), "%u", ranges[i].lo); + else + toklen = snprintf(tok, sizeof(tok), "%u:%u", + ranges[i].lo, ranges[i].hi); + if (toklen < 0) + break; + + if (written + (size_t)toklen + (first ? 0 : 1) >= bufsize) { + *truncated = 1; + break; + } + + if (!first) + buf[written++] = ','; + memcpy(buf + written, tok, (size_t)toklen); + written += (size_t)toklen; + buf[written] = '\0'; + first = 0; + } + + return (written); +} + +/* + * Terminal reply for the SEARCH round trip cmd_search() started -- + * called from session_handle_mbox_result() when s->state is SESSION_ + * SEARCHING, before that function's generic FETCH/STORE/EXPUNGE/CLOSE + * handling (SEARCH's reply shape, an ESEARCH response built from the + * accumulated match list rather than a fixed " completed/ + * failed" text, is different enough to warrant its own function rather + * than another branch bolted onto the generic one). + * + * RFC 9051 SS6.4.4/SS7.3.4's MIN/MAX/ALL/COUNT semantics, resolved + * against that section's own worked examples (not just the prose, which + * reads ambiguously on its own -- see the specific examples cited + * below): COUNT is included, unconditionally on match count (even + * "COUNT 0"), if and only if it was requested (RETURN (MIN) UNSEEN's + * own example response, "MIN 4", has no COUNT at all, showing COUNT is + * NOT sent just because it's a valid default -- only when actually + * asked for); MIN/MAX/ALL are each included only if requested AND there + * is at least one match (RETURN (MIN) UNSEEN's response also confirms + * MIN is omitted, not sent as 0, when unrequested attributes are absent, + * and SEARCH's own no-RETURN-clause example, "SEARCH TEXT ...", with no + * matches produces a bare "* ESEARCH (TAG "...")" with nothing after + * the correlator at all -- the default-ALL implied option, with zero + * matches, is simply omitted). The correlator ("(TAG "")") is + * always sent, since every ESEARCH this server produces is a direct + * response to a specific tagged command, never spontaneous. + * + * No "UID" indicator: see imapd.h's imsg_mbox_search comment on why + * ESEARCH data is always in sequence-number space this pass. + * + * Built via session_write() directly with a generously-sized local + * buffer (8192 bytes, matching SESSION_INBUF_MAX's own "generous" + * precedent), bypassing session_untagged()'s ordinary 512-byte cap -- + * that cap is fine for this file's other short, fixed-shape status + * text, but truncating ESEARCH's ALL data at 512 bytes would silently + * hand the client a materially wrong (incomplete) result, not just a + * cosmetically shortened one. format_seq_list() itself never writes a + * partial token even within this larger buffer; on the rare v1-scale + * mailbox where even 8192 bytes isn't enough for a highly non-contiguous + * match set, the response is truncated at a clean token boundary and a + * warning is logged server-side -- a known, flagged v1-scale limitation + * (RFC 9051 has no mechanism to signal a partial ALL list to the + * client), not a silent-corruption risk, since COUNT (when requested) is + * always computed from the true, untruncated match count regardless. + */ +static void +session_finish_search(struct session *s, struct imsg_mbox_result *res) +{ + char buf[8192]; + size_t len; + int truncated = 0; + + s->state = SESSION_SELECTED; + + log_debug("session %u: SEARCH done, ok=%d, %u match(es)", s->id, + res->ok, res->count); + + if (!res->ok || s->search_alloc_failed) { + session_reply(s, s->pending_tag, "NO", + s->cmd_by_uid ? "UID SEARCH failed" : "SEARCH failed"); + goto cleanup; + } + + len = (size_t)snprintf(buf, sizeof(buf), "* ESEARCH (TAG \"%s\")", + s->pending_tag); + + /* RFC 9051 SS9 esearch-response ABNF: `"ESEARCH" [search-correlator] + * [SP "UID"] *(SP search-return-data)` -- the "UID" indicator (RFC + * 9051 SS6.4.9: "the corresponding ESEARCH response MUST include the + * UID indicator") comes right after the correlator, before any of + * MIN/MAX/ALL/COUNT/MODSEQ. Matches SS6.4.9's own worked examples + * exactly: `* ESEARCH (TAG "A285") UID MIN 7 MAX 3800` and + * `* ESEARCH (TAG "A301") UID ALL 17,900,901`. */ + if (s->cmd_by_uid && len < sizeof(buf)) + len += (size_t)snprintf(buf + len, sizeof(buf) - len, " UID"); + + if ((s->search_return_opts & SEARCH_RETURN_MIN) && + s->search_nmatches > 0 && len < sizeof(buf)) + len += (size_t)snprintf(buf + len, sizeof(buf) - len, + " MIN %u", s->search_matches[0]); + + if ((s->search_return_opts & SEARCH_RETURN_MAX) && + s->search_nmatches > 0 && len < sizeof(buf)) + len += (size_t)snprintf(buf + len, sizeof(buf) - len, + " MAX %u", s->search_matches[s->search_nmatches - 1]); + + if ((s->search_return_opts & SEARCH_RETURN_ALL) && + s->search_nmatches > 0 && len < sizeof(buf)) { + size_t listlen; + + len += (size_t)snprintf(buf + len, sizeof(buf) - len, + " ALL "); + if (len < sizeof(buf)) { + listlen = format_seq_list(buf + len, + sizeof(buf) - len, s->search_matches, + s->search_nmatches, &truncated); + len += listlen; + if (truncated) + log_warnx("session %u: SEARCH ALL response " + "truncated at %zu bytes (%u total " + "matches) -- known v1-scale limitation, " + "see session_finish_search()'s comment", + s->id, sizeof(buf), s->search_nmatches); + } + } + + if ((s->search_return_opts & SEARCH_RETURN_COUNT) && len < sizeof(buf)) + len += (size_t)snprintf(buf + len, sizeof(buf) - len, + " COUNT %u", s->search_nmatches); + + /* RFC 7162 SS3.1.10 (Example 19): a MODSEQ SEARCH criterion with a + * non-empty result gets "MODSEQ n" appended to the ESEARCH response, + * using the highest mod-sequence among the returned messages -- + * this codebase always answers SEARCH in ESEARCH form (see the RETURN + * default set above), so SS3.1.10's ESEARCH case is the only one + * that applies here; SS3.1.6's plain "* SEARCH ... (MODSEQ n)" form + * never arises. */ + if (s->search_used_modseq && s->search_nmatches > 0 && + len < sizeof(buf)) + len += (size_t)snprintf(buf + len, sizeof(buf) - len, + " MODSEQ %llu", + (unsigned long long)s->search_max_modseq); + + if (len >= sizeof(buf)) + len = sizeof(buf) - 1; /* truncate rather than overflow -- + * see this function's header comment; + * only reachable if even the fixed + * correlator/MIN/MAX/COUNT portions + * somehow overflowed, which a v1-scale + * tag length never should */ + + buf[len++] = '\r'; + buf[len++] = '\n'; + session_write(s, buf, len); + + /* RFC 9051 SS6.4.9's own worked example shows "UID FETCH completed" + * verbatim; UID EXPUNGE's own worked example shows "UID EXPUNGE + * completed". "UID SEARCH completed" isn't shown as a literal + * example in the RFC text, but follows the same "UID + * completed" pattern by direct symmetry with those two -- not + * fabricated, just not independently quoted. */ + session_reply(s, s->pending_tag, "OK", + s->cmd_by_uid ? "UID SEARCH completed" : "SEARCH completed"); + +cleanup: + free(s->search_matches); + s->search_matches = NULL; + s->search_nmatches = 0; + s->search_matches_cap = 0; + s->search_alloc_failed = 0; + s->search_used_modseq = 0; + s->search_max_modseq = 0; +} + +/* + * Finishes the COPY/MOVE round trip copy_move_dispatch() started. Peeled + * off session_handle_mbox_result() the same way SEARCH is, since COPY/ + * MOVE's reply shape (a COPYUID response code, optionally an untagged OK + * plus buffered EXPUNGE/VANISHED for a MOVE) doesn't fit the generic + * " completed/failed" text FETCH/STORE/EXPUNGE/CLOSE share. + * + * Response placement differs between the two, per RFC 9051 SS6.4.7/ + * SS6.4.8/SS7.1: COPY's COPYUID goes in its own TAGGED OK ("This response + * code is returned in a tagged OK response to the COPY or UID COPY + * command"); MOVE's goes in an UNTAGGED OK sent before any EXPUNGE/ + * VANISHED ("Servers are also REQUIRED to send the COPYUID response code + * in an untagged OK before sending EXPUNGE"), followed by every buffered + * s->move_expunged entry (in original order -- see that field's own + * comment for why these were buffered instead of written live), then a + * plain tagged OK with no COPYUID in it. + * + * If zero messages matched, RFC 9051 SS6.4.7's own worked example shows + * a plain "OK No matching messages, so nothing copied" with no COPYUID + * at all -- COPYUID's uid-set ABNF has no legal empty-set spelling, so + * omitting the bracket code entirely (rather than emitting "[COPYUID v + * ]" with nothing after it) is the only spec-consistent choice; this + * function applies the same omission to a zero-match MOVE by the same + * reasoning, with no RFC worked example to cite for that half but no + * more of a legal spelling for an empty MOVE either. + */ +static void +session_finish_copy_or_move(struct session *s, struct imsg_mbox_result *res) +{ + const char *cmdname = s->cmd_is_move ? + (s->cmd_by_uid ? "UID MOVE" : "MOVE") : + (s->cmd_by_uid ? "UID COPY" : "COPY"); + uint32_t i; + + s->state = SESSION_SELECTED; + + if (!res->ok || s->copy_alloc_failed) { + char text[48]; + + /* + * RFC 9051 SS6.4.7/SS6.4.8: destname resolved to a + * syntactically valid name (BAD would already have been + * sent otherwise, before any store round trip) but store.c + * couldn't find it on disk -- TRYCREATE, not a plain + * failure, same distinction imsg_mbox_appended's own + * no_such_mailbox field already makes for APPEND. + */ + if (!s->copy_alloc_failed && res->no_such_mailbox) + snprintf(text, sizeof(text), "[TRYCREATE] no such " + "mailbox"); + else + snprintf(text, sizeof(text), "%s failed", cmdname); + session_reply(s, s->pending_tag, "NO", text); + goto cleanup; + } + + /* RFC 7162 SS3.1.2.1: cache the mailbox's post-operation + * HIGHESTMODSEQ, same as session_handle_mbox_result() already does + * for STORE/EXPUNGE -- COPY/MOVE both mutate the mailbox (COPY + * always appends; MOVE both appends and removes) and so both change + * it too, even though neither is itself one of RFC 7162 SS3.1's six + * CONDSTORE-enabling commands (no [HIGHESTMODSEQ] is emitted on + * COPY/MOVE's own tagged/untagged OK anywhere below -- this is + * purely for session_condstore_enable()'s later cached-value use if + * CONDSTORE gets enabled afterward). */ + s->mbox_highestmodseq = res->highestmodseq; + + /* + * RFC 9051 SS6.3.13 (IDLE): both COPY and MOVE mutate a mailbox + * some *other* same-uid session could have selected and be idling + * on -- COPY always appends at the destination; MOVE does that too + * and additionally removes s->move_expunged_n message(s) from the + * source. Before cross-mailbox COPY/MOVE existed (RFC 9051 + * SS6.3.4-SS6.3.6, docs/openimap-storage-backend.md item 10's own + * follow-up) the destination was always the same mailbox already + * selected by *this* session, so a peer noticing a COPY's own + * appended messages could only ever be a peer with that same + * mailbox open -- still worth waking, but this call was + * (incorrectly) gated to MOVE only until now, which meant a peer + * idling on that shared mailbox never got woken by a plain COPY + * into it. Now that the destination can be genuinely different + * from anything *this* session has open, notifying unconditionally + * is the only way a peer with the *destination* mailbox selected + * ever finds out. session_notify_idle_peers()'s own comment + * already documents why not filtering by which mailbox each peer + * has selected is harmless imprecision, not a correctness bug -- + * that reasoning covers this unconditional call the same way. + * Gated on copy_n > 0 since the zero-match branch below sends no + * COPYUID/EXPUNGE at all, i.e. neither copied nor moved anything. + */ + if (s->copy_n > 0) + session_notify_idle_peers(s); + + if (s->copy_n > 0) { + char srcbuf[2048], destbuf[2048]; + char text[4096 + 64]; + int truncated; + + format_seq_list(srcbuf, sizeof(srcbuf), s->copy_src_uids, + s->copy_n, &truncated); + if (truncated) + log_warnx("session %u: COPYUID source list truncated", + s->id); + format_seq_list(destbuf, sizeof(destbuf), s->copy_dest_uids, + s->copy_n, &truncated); + if (truncated) + log_warnx("session %u: COPYUID dest list truncated", + s->id); + + if (s->cmd_is_move) { + snprintf(text, sizeof(text), "OK [COPYUID %u %s %s]", + res->uidvalidity, srcbuf, destbuf); + session_untagged(s, text); + + for (i = 0; i < s->move_expunged_n; i++) + session_send_expunge_response(s, + &s->move_expunged[i]); + + session_reply(s, s->pending_tag, "OK", "Done"); + } else { + snprintf(text, sizeof(text), + "[COPYUID %u %s %s] %s completed", + res->uidvalidity, srcbuf, destbuf, cmdname); + session_reply(s, s->pending_tag, "OK", text); + } + } else { + /* RFC 9051 SS6.4.7's own worked example, verbatim: "S: A005 + * OK No matching messages, so nothing copied" -- see this + * function's header comment for why MOVE gets the identical + * text in the zero-match case. */ + session_reply(s, s->pending_tag, "OK", + "No matching messages, so nothing copied"); + } + +cleanup: + free(s->copy_src_uids); + s->copy_src_uids = NULL; + free(s->copy_dest_uids); + s->copy_dest_uids = NULL; + s->copy_n = 0; + s->copy_cap = 0; + s->copy_alloc_failed = 0; + free(s->move_expunged); + s->move_expunged = NULL; + s->move_expunged_n = 0; + s->move_expunged_cap = 0; + s->cmd_is_move = 0; +} + +/* + * Terminal reply for the FETCH, STORE, EXPUNGE/CLOSE, SEARCH, COPY, MOVE, + * CREATE, DELETE, or RENAME round trip cmd_fetch()/cmd_store_cmd()/ + * cmd_expunge()/cmd_close()/cmd_search()/cmd_copy()/cmd_move()/cmd_create()/ + * cmd_delete()/cmd_rename() started: store.c sends exactly one IMSG_MBOX_ + * RESULT after the last (zero or more) per-message reply for a given + * request. SEARCH, COPY/MOVE, and CREATE/DELETE/RENAME are all peeled off + * first, into session_finish_search()/session_finish_copy_or_move()/ + * session_finish_mbox_op() respectively, since none of their reply shapes + * fit the generic " completed/failed" text the remaining four + * share. Which of those four this was is read from s->state (and, for + * EXPUNGE vs. CLOSE specifically, s->close_after_expunge) before either is + * overwritten -- purely to pick the right tagged completion text and the + * right next state; all four otherwise share this handler completely. See + * the ST_SELECTED/ST_AUTH comments above the dispatch table for why + * SESSION_FETCHING/SESSION_STORING/SESSION_EXPUNGING/SESSION_SEARCHING/ + * SESSION_COPYING/SESSION_CREATING/SESSION_DELETING/SESSION_RENAMING all + * have to stay excluded from ST_SELECTED (or ST_AUTH, for the last three) + * until exactly this point. + */ +static void +session_handle_mbox_result(struct session *s, struct imsg_mbox_result *res) +{ + int was_close = 0, was_storing = 0, was_expunging = 0; + const char *cmdname; + + if (s->state == SESSION_SEARCHING) { + session_finish_search(s, res); + return; + } + if (s->state == SESSION_COPYING) { + session_finish_copy_or_move(s, res); + return; + } + if (s->state == SESSION_CREATING || s->state == SESSION_DELETING || + s->state == SESSION_RENAMING) { + /* RFC 9051 SS6.3.4-SS6.3.6 (docs/openimap-storage-backend.md + * item 10) -- same "peel off before the generic FETCH/ + * STORE/EXPUNGE shape below" pattern as SESSION_SEARCHING/ + * SESSION_COPYING just above, since CREATE/DELETE/RENAME's + * reply text ("CREATE completed"/"failed", etc.) doesn't fit + * this function's cmdname table either. */ + session_finish_mbox_op(s, res); + return; + } + if (s->state == SESSION_LISTING) { + /* RFC 9051 SS6.3.9 -- same peel-off reasoning as SESSION_ + * CREATING/DELETING/RENAMING just above; LIST/LSUB's + * "completed"/"failed" text is keyed off s->list_is_lsub, + * not this function's by-state cmdname table. */ + session_finish_list(s, res); + return; + } + + /* + * RFC 9051 SS6.4.9 addition: every one of these becomes "UID " + * when s->cmd_by_uid is set (see that field's own comment in struct + * session for the sourcing on "UID completed"/"failed" text) -- + * CLOSE is the one exception, since there is no "UID CLOSE" (UID + * EXPUNGE's is_close is always 0, enforced by uid_expunge_dispatch() + * only ever calling session_request_expunge() that way). + */ + if (s->state == SESSION_STORING) { + cmdname = s->cmd_by_uid ? "UID STORE" : "STORE"; + was_storing = 1; + } else if (s->state == SESSION_EXPUNGING) { + was_close = s->close_after_expunge; + cmdname = was_close ? "CLOSE" : + (s->cmd_by_uid ? "UID EXPUNGE" : "EXPUNGE"); + was_expunging = 1; + } else + cmdname = s->cmd_by_uid ? "UID FETCH" : "FETCH"; + + log_debug("session %u: %s done, ok=%d, %u response(s) sent", + s->id, cmdname, res->ok, res->count); + + /* + * RFC 7162: cache the mailbox's post-operation HIGHESTMODSEQ for + * session_condstore_enable()'s later use -- only STORE/EXPUNGE (not + * a plain FETCH, which leaves res->highestmodseq at its zeroed + * default; see imapd.h's imsg_mbox_result comment) ever change + * it, so a plain FETCH mustn't blindly overwrite the cached value + * with that default 0. + */ + if (was_storing || was_expunging) + s->mbox_highestmodseq = res->highestmodseq; + + /* + * RFC 9051 SS6.4.1: CLOSE "returns to the authenticated state from + * the selected state" -- unlike FETCH/STORE/a real EXPUNGE, which + * all stay in Selected. + */ + s->state = was_close ? SESSION_AUTHENTICATED : SESSION_SELECTED; + + if (!res->ok) { + /* v1's only failure mode is an index I/O error (see + * store.c's handle_mbox_fetch()/handle_mbox_store()/ + * handle_mbox_expunge() ok = 0 paths) -- RFC 9051 has no + * specific response code for this, so a plain NO is the + * honest answer. */ + char text[32]; + + free(s->store_modified); + s->store_modified = NULL; + s->store_modified_n = 0; + s->store_modified_cap = 0; + + snprintf(text, sizeof(text), "%s failed", cmdname); + session_reply(s, s->pending_tag, "NO", text); + return; + } + + /* + * RFC 7162 SS3.1.3: a STORE that used UNCHANGEDSINCE and had at + * least one message fail the conditional test gets the MODIFIED + * response code on its tagged OK -- v1 never has a reason to use + * the tagged NO form instead (SS3.1.3's Example 11 mixes an + * UNCHANGEDSINCE failure with expunged messages to get that; v1's + * STORE simply never visits an expunged message's index line at + * all -- see handle_mbox_store()'s lo/hi bound against the current + * idx.nlines -- so that combination can't arise here). + */ + if (was_storing && s->store_modified_n > 0) { + char rbuf[2048]; + char text[2048 + 32]; + int truncated; + + format_seq_list(rbuf, sizeof(rbuf), s->store_modified, + s->store_modified_n, &truncated); + if (truncated) + log_warnx("session %u: MODIFIED list truncated", + s->id); + snprintf(text, sizeof(text), "[MODIFIED %s] Conditional " + "%s failed for some messages", rbuf, cmdname); + session_reply(s, s->pending_tag, "OK", text); + + free(s->store_modified); + s->store_modified = NULL; + s->store_modified_n = 0; + s->store_modified_cap = 0; + return; + } + free(s->store_modified); + s->store_modified = NULL; + s->store_modified_n = 0; + s->store_modified_cap = 0; + + /* + * RFC 9051 SS6.3.13 (IDLE): a successful EXPUNGE/UID EXPUNGE, or a + * CLOSE that silently expunged \Deleted messages, may have changed + * what messages exist -- wake any other same-uid session currently + * idling on this mailbox so it can push EXISTS/EXPUNGE. Fired + * unconditionally on was_expunging rather than gated on res->count, + * since (per the HIGHESTMODSEQ comment just below) store.c only + * reports a nonzero count for a real, non-silent EXPUNGE -- CLOSE's + * count is always 0 even when it removed messages, so count can't be + * used to detect that case. An extra, harmless peer refresh on a + * no-op EXPUNGE/CLOSE is the tradeoff. + */ + if (was_expunging) + session_notify_idle_peers(s); + + /* + * RFC 7162 SS3.2.7: a real EXPUNGE (not CLOSE -- SS3.2.8 explicitly + * forbids it there, "as this might cause loss of synchronization on + * the client") that removed at least one message includes + * HIGHESTMODSEQ in its tagged OK once this session is CONDSTORE- + * aware. res->count is exact here: store.c only composes + * IMSG_MBOX_EXPUNGED (which is what increments it) when !silent, + * i.e. for a real EXPUNGE, never for CLOSE. + */ + if (was_expunging && !was_close && s->condstore_enabled && + res->count > 0) { + char text[64]; + + snprintf(text, sizeof(text), "[HIGHESTMODSEQ %llu] %s " + "completed", (unsigned long long)res->highestmodseq, + cmdname); + session_reply(s, s->pending_tag, "OK", text); + return; + } + + { + char text[32]; + + snprintf(text, sizeof(text), "%s completed", cmdname); + session_reply(s, s->pending_tag, "OK", text); + } +} + +static void +session_request_store(struct session *s, struct imsg_auth_result *res) +{ + struct imsg_store_fork req; + + req.session_id = s->id; + req.uid = res->uid; + req.gid = res->gid; + /* + * Also kept on struct session itself, not just forwarded in this + * request -- session_notify_idle_peers() (RFC 9051 SS6.3.13) needs + * to find this session's *other* concurrent sessions later, and + * this is the only point in the whole connection lifecycle where + * the authenticated uid is available at all (auth.c doesn't share + * it any other way, and store.c never sends it back). + */ + s->uid = res->uid; + /* + * res->maildir was resolved by auth.c straight from the credential + * file's fifth field and was, until this pass, silently dropped + * here -- imapd.h's imsg_store_fork comment has the full "this + * was always meant to carry it" citation. Without this, store.c has + * no way to know which subdirectory of the shared spool_root is + * this session's own mailbox. + */ + strlcpy(req.maildir, res->maildir, sizeof(req.maildir)); + + s->state = SESSION_STORE_PENDING; + + if (imsg_compose(&iev_parent.ibuf, IMSG_STORE_FORK, 0, 0, -1, + &req, sizeof(req)) == -1) + log_warn("imsg_compose IMSG_STORE_FORK"); + imsgev_add(&iev_parent); +} + +static struct session * +session_find(uint32_t id) +{ + struct session *s; + + TAILQ_FOREACH(s, &sessions, entry) { + if (s->id == id) + return (s); + } + return (NULL); +} + +/* + * Tears down a session fully: notifies its store child (if wired) with + * IMSG_STORE_SHUTDOWN, closes both fds, unregisters both events, removes + * from the session table, frees. Closes a small gap surfaced by actually + * writing this: store.c has handled IMSG_STORE_SHUTDOWN since it was + * first written ("arrives directly from listener, no round-trip through + * parent" -- see its header comment), but nothing anywhere ever sent + * one. Best-effort on the imsg send -- if the store child is already + * gone or its channel is backed up, log and move on rather than block + * client teardown on it. + */ +static void +session_teardown(struct session *s) +{ + if (s->store_iev != NULL) { + if (imsg_compose(&s->store_iev->ibuf, IMSG_STORE_SHUTDOWN, + 0, 0, -1, NULL, 0) == -1) + log_warn("session %u: imsg_compose " + "IMSG_STORE_SHUTDOWN", s->id); + else if (imsgbuf_flush(&s->store_iev->ibuf) == -1) + log_warn("session %u: imsgbuf_flush " + "IMSG_STORE_SHUTDOWN", s->id); + event_del(&s->store_iev->ev); + close(s->store_iev->ibuf.fd); + free(s->store_iev); + } + + if (s->client_ev_added) + event_del(&s->client_ev); + + if (s->tls_ctx != NULL) { + int ret = tls_close(s->tls_ctx); + + /* + * tls_close() closes the underlying fd itself -- confirmed + * by reading tls_close()'s real implementation + * (openbsd_source/src/lib/libtls/tls.c) rather than + * assuming; it's a real, easy-to-get-wrong fact (plenty of + * TLS libraries in other languages do NOT close the + * wrapped fd for you). The one case it does NOT close the + * fd is when the close_notify shutdown itself needs another + * network round trip (TLS_WANT_POLLIN/POLLOUT) -- not + * something a forced teardown should wait for, so we close + * the fd ourselves in exactly that case. Any other return + * (0 success, or a hard -1 from a failed shutdown(2)/ + * close(2) syscall) means tls_close() already reached its + * own close(2) call, so closing again here would double- + * close the fd. + */ + if (ret == TLS_WANT_POLLIN || ret == TLS_WANT_POLLOUT) + close(s->client_fd); + else if (ret != 0) + log_warnx("session %u: tls_close: %s", s->id, + tls_error(s->tls_ctx)); + tls_free(s->tls_ctx); + } else { + close(s->client_fd); + } + + free(s->literal_buf); /* NULL-safe if no APPEND literal was ever + * in flight when this session was torn + * down */ + free(s->search_matches); /* NULL-safe if no SEARCH was ever in + * flight when this session was torn down */ + free(s->vanished_ranges); /* NULL-safe -- RFC 7162 QRESYNC resync + * accumulator, see struct session's comment */ + free(s->qresync_fetches); /* NULL-safe, same reason */ + free(s->store_modified); /* NULL-safe -- RFC 7162 STORE UNCHANGEDSINCE + * accumulator, see struct session's comment */ + free(s->pending_header_buf); /* NULL-safe -- see struct session's + * comment; store.c's ordering contract means + * this is normally already NULL by the time a + * FETCH finishes, but a mid-stream teardown + * (client disconnects between the header and + * meta imsgs) could leave it set */ + free(s->pending_body_buf); /* NULL-safe, same reasoning as + * pending_header_buf just above */ + free(s->pending_envelope_buf); /* NULL-safe, same reasoning as + * pending_header_buf just above */ + free(s->pending_bodystructure_buf); /* NULL-safe, same reasoning as + * pending_header_buf just above */ + + TAILQ_REMOVE(&sessions, s, entry); + log_debug("session %u: closed", s->id); + free(s); +} blob - /dev/null blob + 891297a134890265490234e6143f2866aaba7dc8 (mode 644) --- /dev/null +++ src/log.c @@ -0,0 +1,192 @@ +/* + * Copyright (c) 2026 David Williams + * + * Permission to use, copy, modify, and distribute this software for any + * purpose with or without fee is hereby granted, provided that the above + * copyright notice and this permission notice appear in all copies. + * + * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + */ + +#include + +#include +#include +#include +#include +#include +#include +#include +#include + +#include "log.h" + +static int log_foreground = 1; +static int log_verbose = 0; +static char log_procname[32] = "imapd"; + +void +log_init(int foreground, int verbose) +{ + log_foreground = foreground; + log_verbose = verbose; + + if (!log_foreground) + openlog(log_procname, LOG_PID | LOG_NDELAY, LOG_DAEMON); + + tzset(); +} + +void +log_procinit(const char *name) +{ + if (name != NULL) + (void)strlcpy(log_procname, name, sizeof(log_procname)); +} + +void +log_setverbose(int v) +{ + log_verbose = v; +} + +int +log_getverbose(void) +{ + return (log_verbose); +} + +void +vlog(int pri, const char *fmt, va_list ap) +{ + char *nfmt; + int saved_errno = errno; + + if (log_foreground) { + if (asprintf(&nfmt, "%s: %s\n", log_procname, fmt) == -1) { + vfprintf(stderr, fmt, ap); + fprintf(stderr, "\n"); + } else { + vfprintf(stderr, nfmt, ap); + free(nfmt); + } + fflush(stderr); + } else + vsyslog(pri, fmt, ap); + + errno = saved_errno; +} + +void +logit(int pri, const char *fmt, ...) +{ + va_list ap; + + va_start(ap, fmt); + vlog(pri, fmt, ap); + va_end(ap); +} + +void +log_warn(const char *emsg, ...) +{ + char *nfmt; + va_list ap; + + /* best-effort in appending strerror(errno) after the format */ + if (emsg == NULL) + logit(LOG_ERR, "%s", strerror(errno)); + else { + if (asprintf(&nfmt, "%s: %s", emsg, strerror(errno)) == -1) { + va_start(ap, emsg); + vlog(LOG_ERR, emsg, ap); + va_end(ap); + return; + } + va_start(ap, emsg); + vlog(LOG_ERR, nfmt, ap); + va_end(ap); + free(nfmt); + } +} + +void +log_warnx(const char *emsg, ...) +{ + va_list ap; + + va_start(ap, emsg); + vlog(LOG_ERR, emsg, ap); + va_end(ap); +} + +void +log_info(const char *emsg, ...) +{ + va_list ap; + + va_start(ap, emsg); + vlog(LOG_INFO, emsg, ap); + va_end(ap); +} + +void +log_debug(const char *emsg, ...) +{ + va_list ap; + + if (log_verbose == 0) + return; + + va_start(ap, emsg); + vlog(LOG_DEBUG, emsg, ap); + va_end(ap); +} + +static void +vfatalc(int code, const char *emsg, va_list ap) +{ + static char s[BUFSIZ]; + const char *sep; + + if (emsg != NULL) { + (void)vsnprintf(s, sizeof(s), emsg, ap); + sep = ": "; + } else { + s[0] = '\0'; + sep = ""; + } + + if (code != 0) + logit(LOG_CRIT, "fatal: %s%s%s", s, sep, strerror(code)); + else + logit(LOG_CRIT, "fatal: %s", s); +} + +__dead void +fatal(const char *emsg, ...) +{ + va_list ap; + + va_start(ap, emsg); + vfatalc(errno, emsg, ap); + va_end(ap); + exit(1); +} + +__dead void +fatalx(const char *emsg, ...) +{ + va_list ap; + + va_start(ap, emsg); + vfatalc(0, emsg, ap); + va_end(ap); + exit(1); +} blob - /dev/null blob + 7b785502874e0472f92af2fc91ab23b0ad5088de (mode 644) --- /dev/null +++ src/log.h @@ -0,0 +1,53 @@ +/* + * Copyright (c) 2026 David Williams + * + * Permission to use, copy, modify, and distribute this software for any + * purpose with or without fee is hereby granted, provided that the above + * copyright notice and this permission notice appear in all copies. + * + * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + */ + +/* + * Minimal logging, matching the call signatures actually observed in the + * uploaded smtpd.c this session (fatal("pledge"), fatalx("chdir"), + * log_debug("setup_done: %s[%d] done", p->name, p->pid), + * log_procinit(proc_title(smtpd_process))) -- those call sites are real, + * quoted material; this implementation of them is this project's own, not + * copied from smtpd's actual log.c (not among the uploaded files). + */ + +#ifndef OPENIMAP_LOG_H +#define OPENIMAP_LOG_H + +#include +#include + +void log_init(int, int); +void log_procinit(const char *); +void log_setverbose(int); +int log_getverbose(void); + +void log_warn(const char *, ...) + __attribute__((__format__ (printf, 1, 2))); +void log_warnx(const char *, ...) + __attribute__((__format__ (printf, 1, 2))); +void log_info(const char *, ...) + __attribute__((__format__ (printf, 1, 2))); +void log_debug(const char *, ...) + __attribute__((__format__ (printf, 1, 2))); +void logit(int, const char *, ...) + __attribute__((__format__ (printf, 2, 3))); +void vlog(int, const char *, va_list); +__dead void fatal(const char *, ...) + __attribute__((__format__ (printf, 1, 2))); +__dead void fatalx(const char *, ...) + __attribute__((__format__ (printf, 1, 2))); + +#endif /* OPENIMAP_LOG_H */ blob - /dev/null blob + c881c86004f3b0309b567ed7aa864a6e6c5ca0bd (mode 644) --- /dev/null +++ src/main.c @@ -0,0 +1,200 @@ +/* + * Copyright (c) 2026 David Williams + * + * Permission to use, copy, modify, and distribute this software for any + * purpose with or without fee is hereby granted, provided that the above + * copyright notice and this permission notice appear in all copies. + * + * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + */ + +/* + * imapd(8) entry point: argv parsing and the role dispatch described in + * openimap-privsep-design.md's "fork+exec re-invocation" section. + * + * Invoked two different ways: + * + * - No "-x" flag: this is the original, root-owned invocation (from + * rc.d or a shell at boot). Falls through to parent_main(), which + * does the actual fork()+dup2(sp[0], 3)+closefrom(4)+execvp() dance + * for "listener" and "auth" per that document -- parent_main() is + * handed argc/argv so it can re-exec children with the same argv[0] + * plus "-x " appended, exactly as documented. + * + * - "-x ": this is a re-exec'd child. Its control channel to + * parent is already sitting on fd 3 (parent dup2'd it there before + * exec). We skip straight to that role's _main(), which begins by + * reading fd 3 in a setup_proc()-style loop -- see listener.c/auth.c/ + * store.c. + */ + +#include + +#include +#include +#include +#include +#include + +#include "imapd.h" +#include "log.h" + +__dead static void usage(void); + +static const struct { + const char *name; + enum openimap_proc_type type; +} proc_table[] = { + { "listener", PROC_LISTENER }, + { "auth", PROC_AUTH }, + { "store", PROC_STORE }, +}; + +const char * +log_procname(enum openimap_proc_type type) +{ + switch (type) { + case PROC_PARENT: + return "parent"; + case PROC_LISTENER: + return "listener"; + case PROC_AUTH: + return "auth"; + case PROC_STORE: + return "store"; + default: + return "?"; + } +} + +__dead static void +usage(void) +{ + fprintf(stderr, + "usage: %s [-dVv] [-D macro=value] [-f config] [-x role]\n", + getprogname()); + exit(1); +} + +int +main(int argc, char *argv[]) +{ + int ch; + int debug = 0, verbose = 0; + const char *conffile = "/etc/imapd.conf"; + const char *rolearg = NULL; + enum openimap_proc_type role = PROC_PARENT; + size_t i; + struct openimap_config conf; + + memset(&conf, 0, sizeof(conf)); + + while ((ch = getopt(argc, argv, "D:df:Vvx:")) != -1) { + switch (ch) { + case 'V': + /* + * Print version and exit immediately -- before + * log_init()/config_load()/any role dispatch, so + * this has zero side effects (no syslog handle + * opened, no config file touched, no privsep + * children spawned). This is deliberate: it's the + * RELINK smoke-test command in ../Makefile + * (RELINK= "./${PROG} -V > /dev/null"), which bsd. + * prog.mk runs against a freshly-relinked binary + * before installing it -- it has to be safe to run + * as any user, with no config file present, with no + * chroot/privsep setup done, and it has to actually + * exercise "the binary starts and runs its own + * code" rather than just being a static string + * embedded in the executable. + */ + printf("%s %s\n", getprogname(), IMAPD_VERSION); + return (0); + case 'D': + /* + * "-D name=value" config-file macro definitions, + * matching ripd(8)/smtpd(8)'s own "-D" convention -- + * applied to parse.y's symbol table immediately + * (cmdline_symset()), not deferred, so they're + * already in place by the time config_load() below + * calls yyparse(). Only meaningful for role == + * PROC_PARENT (only parent ever parses the config + * file), but harmless to accept unconditionally + * before the role is known -- getopt(3) runs before + * the role switch either way. + */ + if (cmdline_symset(optarg) == -1) + fatalx("could not parse macro definition %s", + optarg); + break; + case 'd': + debug = 1; + break; + case 'f': + conffile = optarg; + break; + case 'v': + verbose = 1; + break; + case 'x': + rolearg = optarg; + break; + default: + usage(); + } + } + + if (rolearg != NULL) { + for (i = 0; i < sizeof(proc_table) / sizeof(proc_table[0]); + i++) { + if (strcmp(rolearg, proc_table[i].name) == 0) { + role = proc_table[i].type; + break; + } + } + if (i == sizeof(proc_table) / sizeof(proc_table[0])) + usage(); + } + + log_init(debug, verbose); + log_procinit(log_procname(role)); + + /* + * Only parent reads imapd.conf. Re-exec'd children (role != + * PROC_PARENT) get their slice of it over fd 3 from parent instead + * -- IMSG_LISTENER_INIT, IMSG_AUTH_INIT, IMSG_STORE_INIT -- not + * from this "conf" local, which is why listener_main()/auth_main()/ + * store_main() don't take it as a parameter at all. This used to be + * a flagged gap (an earlier draft passed this same zeroed, + * never-populated "conf" through to every role, which was silently + * wrong for auth specifically: auth_main() derived its chroot + * directory from conf->cred_file, which was always an empty string + * for a re-exec'd child). Fixed by removing the parameter entirely + * rather than leaving an unused one that looks like it should work. + */ + switch (role) { + case PROC_PARENT: + /* parent alone reads and validates the real config file. */ + if (config_load(conffile, &conf) == -1) + fatalx("config_load: %s", conffile); + parent_main(conffile, argc, argv, &conf); + /* NOTREACHED */ + case PROC_LISTENER: + listener_main(); + /* NOTREACHED */ + case PROC_AUTH: + auth_main(); + /* NOTREACHED */ + case PROC_STORE: + store_main(); + /* NOTREACHED */ + } + + fatalx("unhandled role %d", role); +} blob - /dev/null blob + 0b114fd2ed455f752f3bfed30d1c2dadb7a15023 (mode 644) --- /dev/null +++ src/parent.c @@ -0,0 +1,1346 @@ +/* + * Copyright (c) 2026 David Williams + * + * Permission to use, copy, modify, and distribute this software for any + * purpose with or without fee is hereby granted, provided that the above + * copyright notice and this permission notice appear in all copies. + * + * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + */ + +/* + * parent.c -- privileged supervisor. Implements the mechanisms decided in + * openimap-privsep-design.md: + * + * - fork()+dup2(sp[0], 3)+closefrom(4)+execvp() re-exec for listener and + * auth at boot ("Peer-wiring handshake" section). + * - the SETUP_PEER/SETUP_DONE handshake, sourced against smtpd.c's + * setup_peers()/setup_done()/setup_proc(), wiring listener<->auth at + * boot. + * - per-session store spawn ("Per-session store spawn & peer-wiring"), + * including the deliberate departures from forkmda(): re-exec instead + * of plain fork, IMSG_STORE_INIT carrying runtime uid/gid, and + * session-scoped (not daemon-fatal) handling of a handshake timeout -- + * sourced against setup_done()'s real fatal()-on-timeout behavior, + * which this file deliberately does NOT copy, for the reasons that + * section spells out. + * + * TODO, flagged rather than silently stubbed: TLS cert loading reads the + * files but does no validation. Does not block the process-management + * mechanics this file exists to get right. (config_load() itself used to + * be flagged here too -- it's a real yacc-based imapd.conf parser now, + * living in parse.y, not this file; see that file's header comment.) + * + * API NAMES: checked against the real src/imsg.h ($OpenBSD: imsg.h,v + * 1.24 2025/06/05, Claudio Jeker et al.) this session. Every imsg/imsgbuf + * call used in this skeleton -- imsgbuf_init(), imsgbuf_read(), + * imsgbuf_flush(), imsgbuf_set_maxsize(), imsgbuf_queuelen(), imsg_get(), + * imsg_get_type(), imsg_get_data(), imsg_get_fd(), imsg_compose(), + * imsg_free() -- matches the real header's signature and calling + * convention exactly. imsg_get_ibuf() was guessed with the wrong + * signature (real: `int imsg_get_ibuf(struct imsg *, struct ibuf *)`, + * not what earlier comments assumed) but is never actually called + * anywhere in this codebase, only referenced in stale comments -- no + * runtime bug resulted. One real gap the real header surfaced: + * imsgbuf_allow_fdpass(struct imsgbuf *) exists and was NOT being called + * anywhere, despite every role either sending or receiving fd-passed + * messages (listening sockets, SETUP_PEER peer fds). Added below and in + * imsgev.c/listener.c/auth.c/store.c wherever a struct imsgbuf is + * imsgbuf_init()'d. Same verification applies in listener.c/auth.c/ + * store.c/imsgev.c. + */ + +#include +#include +#include +#include +#include + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "imapd.h" +#include "log.h" + +#define STORE_SETUP_TIMEOUT_SEC 10 /* matches smtpd's setup_done() + * precedent of 10000ms -- see the + * design doc's sourcing note on why + * OpenIMAPD keeps the value but not + * smtpd's fatal()-on-timeout + * behavior. */ + +#define STORE_CHILD_MAX 64 /* F5 fix: cap concurrent per-session + * store children so a fork flood cannot + * exhaust PIDs/fds/memory in the root + * parent. */ + +struct child { + pid_t pid; + enum openimap_proc_type type; + struct imsgev iev; + TAILQ_ENTRY(child) entry; +}; + +/* + * A per-session store child, from fork through teardown. "pending" is + * true from fork until its IMSG_SETUP_DONE ack arrives (or the handshake + * times out); timeout_ev is only armed while pending. Deliberately ONE + * struct for both states rather than two (an earlier draft of this file + * had a separate store_pending vs. store_child pair and "promoted" + * between them by copying struct imsgev -- unsafe, since struct imsgev + * embeds a struct event that's already registered with libevent at its + * original address by the time of that copy; event(3) requires the + * struct not move while registered. One struct, allocated once, closes + * that bug rather than papering over it. + */ +struct store_child { + uint32_t session_id; + pid_t pid; + struct imsgev iev; /* parent<->this store child */ + int pending; + struct event timeout_ev; + struct imsgev *listener_iev; /* who to notify on + * failure -- see + * store_child_fail() */ + TAILQ_ENTRY(store_child) entry; +}; + +static TAILQ_HEAD(, child) children = + TAILQ_HEAD_INITIALIZER(children); +static TAILQ_HEAD(, store_child) store_children = + TAILQ_HEAD_INITIALIZER(store_children); + +static struct imsgev *iev_listener; +static struct imsgev *iev_auth; +static struct openimap_config *gconf; +static char progpath[PATH_MAX]; +static char **saved_argv; +static const char *conf_path; /* set once in parent_main(), read by + * sighup_handler() to re-run + * config_load() against the same + * file main() originally loaded -- + * see imapd.h's parent_main() comment */ + +static struct event ev_sighup, ev_sigterm, ev_sigchld; + +static pid_t fork_child(enum openimap_proc_type, struct imsgev **, + void (*)(int, short, void *)); +static void setup_peer_send(struct imsgev *, struct imsgev *, uint32_t); +static void setup_done_send(struct imsgev *); +static void parent_dispatch_child(int, short, void *); +static void parent_handle_store_fork(uint32_t, uid_t, gid_t, const char *, + struct imsgev *); +static void store_child_dispatch(int, short, void *); +static void store_child_timeout(int, short, void *); +static void store_child_teardown(struct store_child *, int); +static void store_child_fail(struct store_child *); +static int bind_one(int, const struct sockaddr *, socklen_t, uint16_t); +static int bind_listen_socket(const char *, uint16_t, + int[LISTENER_MAX_ADDRS]); +static void send_listener_sockets(struct imsgev *, int *, int, int *, + int); +static void send_tls_certs(struct imsgev *, struct openimap_config *); +static void send_listener_init(struct imsgev *, struct openimap_config *, + int, int); +static void send_auth_init(struct imsgev *, struct openimap_config *); +static void sighup_handler(int, short, void *); +static void sigterm_handler(int, short, void *); +static void sigchld_handler(int, short, void *); +static void reap_child(pid_t, int); + +/* + * config_load() used to live here as a stub that filled in v1 defaults and + * ignored its "path" argument entirely. The real implementation -- a full + * imapd.conf grammar, yacc-based, matching the pattern used throughout + * the OpenBSD base system's own daemons -- now lives in parse.y, per the + * project's own "follow the OpenSMTPD/httpd/ntpd pattern" design + * philosophy. See parse.y's header comment for the grammar's sourcing and + * scope; imapd.h still carries config_load()'s prototype unchanged, so + * this file's only callers (main.c) needed no changes at all. + */ + +__dead void +parent_main(const char *conffile, int argc, char *argv[], + struct openimap_config *conf) +{ + int cleartext_fds[LISTENER_MAX_ADDRS], tls_fds[LISTENER_MAX_ADDRS]; + int n_cleartext, n_tls, i; + + (void)argc; /* saved_argv (argv itself) is what fork_child()/ + * parent_handle_store_fork() actually walk when + * building a child's re-exec argv -- see both for + * why: they scan for a NUL terminator via + * saved_argv[i] != NULL rather than using argc. */ + + if (geteuid() != 0) + fatalx("parent must start as root"); + + gconf = conf; + conf_path = conffile; + saved_argv = argv; + if (realpath(argv[0], progpath) == NULL) + fatal("realpath"); + + /* bind(2) on <1024 needs root -- done here, before any privdrop. */ + n_cleartext = bind_listen_socket(conf->listen_addr, + conf->port_cleartext, cleartext_fds); + n_tls = bind_listen_socket(conf->listen_addr, + conf->port_implicit_tls, tls_fds); + + event_init(); + + /* + * Boot-time children: listener and auth only. store is NOT started + * here -- see the "Per-session store spawn" section of + * openimap-privsep-design.md. This corrects an earlier draft of + * that document, which originally showed store as a fourth + * boot-time process. + */ + fork_child(PROC_LISTENER, &iev_listener, parent_dispatch_child); + fork_child(PROC_AUTH, &iev_auth, parent_dispatch_child); + + /* + * IMSG_AUTH_INIT before the peer handshake, not after (unlike + * listener's INIT below) -- auth_main() derives its chroot + * directory from cred_file, so it needs this message before it can + * even chroot(), let alone finish the handshake. Mirrors + * IMSG_STORE_INIT's precedent of arriving before anything else on + * that child's fd 3. auth.c reads this first, matching the order + * sent here. + */ + send_auth_init(iev_auth, conf); + + /* IMSG_SETUP_PEER / IMSG_SETUP_DONE, sourced against smtpd.c's + * setup_peers()/setup_done() -- see design doc. Only listener<->auth + * is wired at boot. */ + setup_peer_send(iev_listener, iev_auth, 0); + setup_done_send(iev_listener); + setup_done_send(iev_auth); + + /* + * send_listener_sockets() fd-passes cleartext_fds[]/tls_fds[] to + * listener via imsg_compose() (SCM_RIGHTS over the AF_UNIX + * socketpair, not a local dup(2)) -- our own copies are closed right + * after, listener now owns the only live references. + * IMSG_LISTENER_INIT rides along with these post-handshake boot + * extras, carrying n_cleartext/n_tls so listener's synchronous + * drain loop knows how many IMSG_LISTENER_SOCKET_CLEARTEXT/_TLS + * messages to expect (1 each normally, 2 each for "listen on *") -- + * that loop tolerates any arrival order among these messages (see + * listener.c), so sending init first here is for readability, not + * correctness. + */ + send_listener_init(iev_listener, conf, n_cleartext, n_tls); + send_listener_sockets(iev_listener, cleartext_fds, n_cleartext, + tls_fds, n_tls); + send_tls_certs(iev_listener, conf); + for (i = 0; i < n_cleartext; i++) + close(cleartext_fds[i]); + for (i = 0; i < n_tls; i++) + close(tls_fds[i]); + + signal_set(&ev_sighup, SIGHUP, sighup_handler, NULL); + signal_set(&ev_sigterm, SIGTERM, sigterm_handler, NULL); + signal_set(&ev_sigchld, SIGCHLD, sigchld_handler, NULL); + signal_add(&ev_sighup, NULL); + signal_add(&ev_sigterm, NULL); + signal_add(&ev_sigchld, NULL); + signal(SIGPIPE, SIG_IGN); + + /* + * pledge, resolved in openimap-privsep-design.md: proc/exec/sendfd + * stay for the process lifetime (not narrowed post-boot) because + * store is fork-per-session -- parent forks continuously, not just + * at startup. See that document's item 5 for why an earlier draft + * of this decision was wrong. + */ +#ifdef __OpenBSD__ + if (pledge("stdio rpath inet proc exec sendfd", NULL) == -1) + fatal("pledge"); +#endif + + event_dispatch(); + fatalx("parent: exited event loop"); +} + +/* + * Re-exec mechanism, sourced against smtpd.c's start_child(): parent + * creates a socketpair, fork()s, the child dup2()s its end onto fd 3, + * closefrom(4)s, and execvp()s argv[0] plus "-x ". Returns the + * child's pid; *ievp is initialized in place on a freshly allocated + * struct child, never copied afterward -- see the struct store_child + * comment above for why that matters (a struct imsgev must not move once + * imsgev_init() has registered its struct event with libevent). + * + * Only used for the two boot-time children (listener, auth). Per-session + * store children go through parent_handle_store_fork() below instead, + * which duplicates a little of this function's fork/exec body rather + * than threading an opaque "child type" through one shared path -- boot + * children and per-session store children have different enough + * lifecycles (fixed pair-wiring vs. a dynamic session-keyed table with a + * timeout) that forcing one function to cover both got harder to read + * than two similar ones. + */ +static pid_t +fork_child(enum openimap_proc_type type, struct imsgev **ievp, + void (*handler)(int, short, void *)) +{ + int sp[2]; + pid_t pid; + char *nargv[16]; + int i, n; + struct child *c; + + if (socketpair(AF_UNIX, SOCK_STREAM, PF_UNSPEC, sp) == -1) + fatal("socketpair"); + + if ((pid = fork()) == -1) + fatal("fork"); + + if (pid == 0) { + /* child */ + close(sp[0]); + if (dup2(sp[1], 3) == -1) + _exit(1); + closefrom(4); + + n = 0; + nargv[n++] = progpath; + nargv[n++] = "-x"; + nargv[n++] = (char *)log_procname(type); + for (i = 1; saved_argv[i] != NULL && n < 13; i++) { + /* skip a pre-existing -x from our own argv, + * everything else (e.g. -d/-v) passes through */ + if (strcmp(saved_argv[i], "-x") == 0) { + i++; + continue; + } + nargv[n++] = saved_argv[i]; + } + nargv[n] = NULL; + + execvp(nargv[0], nargv); + _exit(1); + } + + /* parent */ + close(sp[1]); + + c = calloc(1, sizeof(*c)); + if (c == NULL) + fatal("calloc"); + c->pid = pid; + c->type = type; + /* + * Real bug caught via gdb backtrace + core dump on premio (first + * real-hardware run to ever exercise this path -- parent_dispatch_ + * child() is only invoked post-boot, and nothing sent listener/auth + * anything post-boot until this session's first successful + * AUTHENTICATE triggered the first-ever IMSG_STORE_FORK): passing + * "c" here instead of NULL meant iev->data (and therefore the arg + * libevent hands back to parent_dispatch_child()) was the enclosing + * struct child*, not &c->iev. Every dispatch handler in this + * codebase -- parent_dispatch_child() included, unlike + * store_child_dispatch(), which deliberately takes a struct + * store_child* and is the one caller that correctly passes its + * enclosing struct -- does `struct imsgev *iev = arg;` and expects + * arg to already be &iev. With arg == c (8 bytes before &c->iev on + * this build), iev->ibuf's fields were read shifted by the width of + * struct child's pid+type header, producing exactly the implausible + * fd/pointer values the core dump showed (confirmed by comparing + * the crashing arg against a live `print iev_listener` in the same + * gdb session: arg was iev_listener - 8, i.e. c itself). NULL here + * takes imsgev_init()'s own fallback (iev->data = data != NULL ? + * data : iev) to self-reference &c->iev instead -- the same pattern + * every other imsgev_init() call site in the tree already uses + * (store.c's, listener.c's x3, auth.c's). + */ + imsgev_init(&c->iev, sp[0], handler, NULL); + TAILQ_INSERT_TAIL(&children, c, entry); + + *ievp = &c->iev; + return (pid); +} + +/* + * IMSG_SETUP_PEER: parent opens a fresh socketpair and fd-passes one end + * to each of "a" and "b" over their existing fd-3 channels. Sourced + * against smtpd.c's setup_peers() (imsg_compose(..., IMSG_SETUP_PEER, + * b->proc, b->pid, sp[0], ...)) -- that quoted call shape is exactly + * where "id" slots into imsg_compose()'s second argument the same way + * "b->proc" does there; this project's "id" is a uint32_t session_id + * instead of smtpd's proc type, but the mechanism (the imsg header's + * generic 32-bit id field, read back via imsg_get_id()) is the same one. + * + * "id" is 0 for the one boot-time call (listener<->auth, no session + * concept yet) and the session_id for every per-session store call -- + * see parent_handle_store_fork() below. This is the fix for the gap + * flagged in an earlier pass: listener previously had no way to tell + * which in-flight store handshake a given IMSG_SETUP_PEER belonged to, + * since this always sent peerid/id 0. listener.c's IMSG_SETUP_PEER + * handler now reads it back with imsg_get_id() to find the right + * session. + */ +static void +setup_peer_send(struct imsgev *a, struct imsgev *b, uint32_t id) +{ + int sp[2]; + + if (socketpair(AF_UNIX, SOCK_STREAM, PF_UNSPEC, sp) == -1) + fatal("socketpair"); + + if (imsg_compose(&a->ibuf, IMSG_SETUP_PEER, id, 0, sp[0], NULL, 0) + == -1) + fatal("imsg_compose IMSG_SETUP_PEER (a)"); + if (imsg_compose(&b->ibuf, IMSG_SETUP_PEER, id, 0, sp[1], NULL, 0) + == -1) + fatal("imsg_compose IMSG_SETUP_PEER (b)"); + + if (imsgbuf_flush(&a->ibuf) == -1) + fatal("imsgbuf_flush"); + if (imsgbuf_flush(&b->ibuf) == -1) + fatal("imsgbuf_flush"); +} + +/* + * IMSG_SETUP_DONE: tell a child no more IMSG_SETUP_PEER messages are + * coming, then block for its ack. Sourced against smtpd.c's setup_done() + * -- including its 10000ms blocking wait -- but see the design doc for + * why this file does NOT copy setup_done()'s fatal()-on-timeout for the + * *per-session* store handshake (parent_handle_store_setup_done() below + * uses an event-driven timeout instead, precisely because that handshake + * happens continuously at runtime, not once at boot like this one). + * This blocking version is only used for the boot-time listener/auth + * wiring, where blocking the whole daemon during startup is fine -- there + * is no other in-flight work yet to protect. + */ +static void +setup_done_send(struct imsgev *iev) +{ + struct imsg imsg; + ssize_t n; + + if (imsg_compose(&iev->ibuf, IMSG_SETUP_DONE, 0, 0, -1, NULL, 0) == -1) + fatal("imsg_compose IMSG_SETUP_DONE"); + if (imsgbuf_flush(&iev->ibuf) == -1) + fatal("imsgbuf_flush"); + + /* + * imsg_get() before imsgbuf_read(): a real deadlock, caught via + * ktrace(1) on the first real-OpenBSD run, not this sandbox -- + * see imsgev.c's setup_recv_one_peer()/setup_recv_done_and_ack() + * header comments for the full citation. setup_peer_send() and + * this function's own IMSG_SETUP_DONE send happen back to back + * on the same fd with no wait in between, so the kernel can (and, + * on the live hang, did) coalesce both into one readable chunk on + * the child's end -- meaning the child's ack can arrive already + * sitting in *our own* ibuf here, buffered from this same + * function's earlier reads in the multi-child boot sequence. + * Blindly calling imsgbuf_read() first would issue a real blocking + * syscall waiting for bytes that were never coming. + */ + for (;;) { + if ((n = imsg_get(&iev->ibuf, &imsg)) == -1) + fatal("imsg_get"); + if (n != 0) + break; + if ((n = imsgbuf_read(&iev->ibuf)) == -1) + fatal("imsgbuf_read"); + if (n == 0) + fatalx("setup_done_send: child closed channel"); + } + + if (imsg_get_type(&imsg) != IMSG_SETUP_DONE) + fatalx("setup_done_send: expected IMSG_SETUP_DONE"); + imsg_free(&imsg); +} + +/* dispatch for the boot-time listener/auth channels once we're in the + * main event loop. TODO: real handling of IMSG_AUTH_* passthrough isn't + * needed here at all -- listener and auth talk to each other directly + * over the peer channel wired above, not through parent. The only + * message parent expects on these channels post-boot is + * IMSG_STORE_FORK, from listener. */ +static void +parent_dispatch_child(int fd, short event, void *arg) +{ + struct imsgev *iev = arg; + struct imsg imsg; + ssize_t n; + + /* + * EV_WRITE: real bug caught on first real-hardware run -- see + * auth.c's auth_dispatch() header comment for the full citation + * against imsg_init(3)'s own EXAMPLES section. imsg_compose() only + * queues; imsgbuf_write() is what actually puts bytes on the wire, + * and nothing here ever called it. + */ + if (event & EV_WRITE) { + if (imsgbuf_write(&iev->ibuf) == -1) + fatal("imsgbuf_write"); + } + + if (event & EV_READ) { + if ((n = imsgbuf_read(&iev->ibuf)) == -1) + fatal("imsgbuf_read"); + if (n == 0) { + /* child's end closed -- reaped via SIGCHLD */ + event_del(&iev->ev); + return; + } + } + + for (;;) { + if ((n = imsg_get(&iev->ibuf, &imsg)) == -1) + fatal("imsg_get"); + if (n == 0) + break; + + switch (imsg_get_type(&imsg)) { + case IMSG_STORE_FORK: { + struct imsg_store_fork req; + + if (imsg_get_data(&imsg, &req, sizeof(req)) == -1) { + log_warnx("bad IMSG_STORE_FORK"); + break; + } + parent_handle_store_fork(req.session_id, req.uid, + req.gid, req.maildir, iev); + break; + } + default: + log_debug("parent_dispatch_child: unhandled %d", + imsg_get_type(&imsg)); + break; + } + imsg_free(&imsg); + } + /* + * Real bug caught on first real-hardware run, same shape and same + * fix as auth.c's auth_dispatch() and store.c's store_dispatch() + * (see auth_dispatch()'s header comment for the full citation + * against imsg_init(3)): nothing re-armed this channel when the + * for loop above found nothing to process, which is exactly what + * happens on a pure EV_WRITE firing now that EV_WRITE is actually + * handled (just above). Unconditional call closes that gap. + */ + imsgev_add(iev); + (void)fd; +} + +/* + * F1 helper: reject a maildir that could escape the intended spool + * subtree. Rejects NULL/empty, absolute paths, and any path containing + * a "." or ".." component. Used before spawning a store child because + * the maildir is relayed through the untrusted, network-facing listener. + */ +static int +maildir_path_is_safe(const char *p) +{ + const char *seg = p; + + if (p == NULL || p[0] == '\0' || p[0] == '/') + return (0); + for (;;) { + const char *slash = strchr(seg, '/'); + size_t len = slash ? (size_t)(slash - seg) : strlen(seg); + + if (len == 1 && seg[0] == '.') + return (0); + if (len == 2 && seg[0] == '.' && seg[1] == '.') + return (0); + if (slash == NULL) + break; + seg = slash + 1; + } + return (1); +} + +/* + * Per-session store spawn, triggered by listener's IMSG_STORE_FORK. + * Mirrors the boot-time re-exec + SETUP_PEER mechanism above, with the + * departures documented in openimap-privsep-design.md: re-exec (not + * forkmda()'s plain fork), an IMSG_STORE_INIT carrying this session's + * uid/gid before the child privilege-drops, and an event-driven timeout + * instead of setup_done()'s blocking-and-fatal() behavior. + */ +static void +parent_handle_store_fork(uint32_t session_id, uid_t uid, gid_t gid, + const char *maildir, struct imsgev *listener_iev) +{ + struct store_child *sc; + int pair[2]; + pid_t pid; + char *nargv[16]; + int i, n; + struct timeval tv; + struct imsg_store_init init_payload; + struct imsg_store_fork fail_payload; + struct store_child *it; + unsigned int nchildren = 0; + + /* + * F1 fix (defense in depth): uid/gid/maildir arrived relayed + * through the untrusted, network-facing listener. Never grant a + * store child uid/gid 0, and never accept an unsafe maildir. The + * complete fix is for the trusted auth process to vend these to the + * parent directly, keyed by session_id; this closes the escalation + * in the meantime. + */ + if (uid == 0 || gid == 0) { + log_warnx("refusing IMSG_STORE_FORK: privileged uid=%u gid=%u " + "for session %u", (unsigned)uid, (unsigned)gid, + session_id); + goto fail; + } + if (!maildir_path_is_safe(maildir)) { + log_warnx("refusing IMSG_STORE_FORK: unsafe maildir for " + "session %u", session_id); + goto fail; + } + + /* F5 fix: cap concurrent store children. */ + TAILQ_FOREACH(it, &store_children, entry) + nchildren++; + if (nchildren >= STORE_CHILD_MAX) { + log_warnx("refusing IMSG_STORE_FORK: %u store children active " + "(session %u)", nchildren, session_id); + goto fail; + } + + if (socketpair(AF_UNIX, SOCK_STREAM, PF_UNSPEC, pair) == -1) { + log_warn("socketpair"); + goto fail; + } + + if ((pid = fork()) == -1) { + log_warn("fork"); + close(pair[0]); + close(pair[1]); + goto fail; + } + + if (pid == 0) { + close(pair[0]); + if (dup2(pair[1], 3) == -1) + _exit(1); + closefrom(4); + + n = 0; + nargv[n++] = progpath; + nargv[n++] = "-x"; + nargv[n++] = "store"; + for (i = 1; saved_argv[i] != NULL && n < 13; i++) { + if (strcmp(saved_argv[i], "-x") == 0) { + i++; + continue; + } + nargv[n++] = saved_argv[i]; + } + nargv[n] = NULL; + + execvp(nargv[0], nargv); + _exit(1); + } + + close(pair[1]); + + /* + * Allocated once, in place -- sc->iev is never copied afterward. + * See the struct store_child comment above for why that matters. + */ + sc = calloc(1, sizeof(*sc)); + if (sc == NULL) + fatal("calloc"); + sc->session_id = session_id; + sc->pid = pid; + sc->pending = 1; + sc->listener_iev = listener_iev; + imsgev_init(&sc->iev, pair[0], store_child_dispatch, sc); + + /* IMSG_STORE_INIT: the one message a per-session store child needs + * that a boot-time child doesn't -- its privilege target is a + * runtime value, not fixed by the role flag. See the design doc's + * "Departure #2". */ + init_payload.session_id = session_id; + init_payload.uid = uid; + init_payload.gid = gid; + (void)strlcpy(init_payload.spool_root, gconf->spool_root, + sizeof(init_payload.spool_root)); + /* maildir came from listener's IMSG_STORE_FORK, which got it from + * auth's IMSG_AUTH_RESULT, which resolved it from the credential + * file's fifth field -- see imapd.h's imsg_store_fork comment. + * Passed through verbatim; parent itself never interprets it. */ + (void)strlcpy(init_payload.maildir, maildir, + sizeof(init_payload.maildir)); + init_payload.bodystructure_read_max = gconf->bodystructure_read_max; + if (imsg_compose(&sc->iev.ibuf, IMSG_STORE_INIT, 0, 0, -1, + &init_payload, sizeof(init_payload)) == -1) { + log_warn("imsg_compose IMSG_STORE_INIT"); + goto fail_kill; + } + + /* session_id threaded through as the imsg "id" field -- see + * setup_peer_send()'s header comment -- so listener.c's + * IMSG_SETUP_PEER handler can tell which in-flight store handshake + * this new peer fd belongs to. */ + setup_peer_send(&sc->iev, listener_iev, session_id); + + if (imsg_compose(&sc->iev.ibuf, IMSG_SETUP_DONE, 0, 0, -1, NULL, 0) + == -1) { + log_warn("imsg_compose IMSG_SETUP_DONE"); + goto fail_kill; + } + imsgev_add(&sc->iev); + + evtimer_set(&sc->timeout_ev, store_child_timeout, sc); + tv.tv_sec = STORE_SETUP_TIMEOUT_SEC; + tv.tv_usec = 0; + evtimer_add(&sc->timeout_ev, &tv); + + TAILQ_INSERT_TAIL(&store_children, sc, entry); + return; + +fail_kill: + kill(pid, SIGKILL); + event_del(&sc->iev.ev); + close(sc->iev.ibuf.fd); /* F11 fix: match store_child_teardown(); + * without this each setup failure leaks the + * socketpair fd in the root parent. */ + free(sc); +fail: + /* Per the design doc's resolution: fail only this one session, not + * the daemon. Reusing IMSG_STORE_FORK as the failure-reply type + * (listener never otherwise receives that type from parent), with + * uid/gid zeroed -- listener must key off "this came from parent" + * plus session_id to recognize a failure reply, not the uid/gid + * fields, which are meaningless here. memset() first now that the + * struct also carries maildir -- otherwise that field would send + * whatever garbage happened to be on the stack rather than an + * empty string; harmless over a local imsg channel either way, but + * sloppy, and listener.c doesn't read it on this path regardless. */ + memset(&fail_payload, 0, sizeof(fail_payload)); + fail_payload.session_id = session_id; + if (imsg_compose(&listener_iev->ibuf, IMSG_STORE_FORK, 0, 0, -1, + &fail_payload, sizeof(fail_payload)) == -1) + log_warn("imsg_compose IMSG_STORE_FORK (failure reply)"); + imsgev_add(listener_iev); +} + +/* + * Real bug caught while wiring listener.c's SELECT round trip (same root + * cause, three places -- see listener_dispatch_auth()'s header comment + * there for the full citation against the real event.h): imsgev_init() + * registers plain EV_READ, not EV_PERSIST, and this function never + * re-armed sc->iev's own read side after servicing it. Not actively + * harmful yet (store.c never sends parent anything after its boot-time + * IMSG_SETUP_DONE), but it's the identical latent gap, so fixed the same + * way: unconditional imsgev_add() at the end, reached only when sc wasn't + * just torn down by store_child_fail() (both call sites return + * immediately after calling it). + */ +static void +store_child_dispatch(int fd, short event, void *arg) +{ + struct store_child *sc = arg; + struct imsg imsg; + ssize_t n; + + /* + * EV_WRITE: same real bug as this file's parent_dispatch_child() -- + * see auth.c's auth_dispatch() header comment for the full citation + * against imsg_init(3). imsg_compose() only queues; without an + * actual imsgbuf_write() on a writable fd, nothing this process + * sends a store child (IMSG_STORE_INIT, IMSG_SETUP_PEER) would ever + * really leave the socket. + */ + if (event & EV_WRITE) { + if (imsgbuf_write(&sc->iev.ibuf) == -1) { + store_child_fail(sc); + return; + } + } + + if (event & EV_READ) { + if ((n = imsgbuf_read(&sc->iev.ibuf)) == -1 || n == 0) { + store_child_fail(sc); + return; + } + } + + for (;;) { + if ((n = imsg_get(&sc->iev.ibuf, &imsg)) == -1) { + store_child_fail(sc); + return; + } + if (n == 0) + break; + + if (sc->pending && imsg_get_type(&imsg) == IMSG_SETUP_DONE) { + evtimer_del(&sc->timeout_ev); + sc->pending = 0; + imsg_free(&imsg); + continue; + } + + log_debug("store_child_dispatch: unhandled %d (session %u)", + imsg_get_type(&imsg), sc->session_id); + imsg_free(&imsg); + } + imsgev_add(&sc->iev); /* re-arm -- see this function's header comment */ + (void)fd; +} + +static void +store_child_timeout(int fd, short event, void *arg) +{ + struct store_child *sc = arg; + + (void)fd; + (void)event; + log_warnx("session %u: store setup timed out after %ds", + sc->session_id, STORE_SETUP_TIMEOUT_SEC); + store_child_fail(sc); +} + +/* + * Shared teardown for a store_child, whether it's still alive (needs + * killing) or was reaped by SIGCHLD already (do not kill(2) a pid that's + * already exited -- harmless on most systems but sloppy, and this project + * would rather be explicit). If it never finished setup (sc->pending), + * tells listener_iev the fork failed, reusing IMSG_STORE_FORK as the + * failure-reply type -- see parent_handle_store_fork()'s "fail:" path for + * the matching send on earlier-failure cases (socketpair/fork failure, + * before a store_child even exists) that never reach this function. + */ +static void +store_child_teardown(struct store_child *sc, int already_dead) +{ + if (sc->pending) { + struct imsg_store_fork fail_payload; + + evtimer_del(&sc->timeout_ev); + memset(&fail_payload, 0, sizeof(fail_payload)); + fail_payload.session_id = sc->session_id; + if (imsg_compose(&sc->listener_iev->ibuf, IMSG_STORE_FORK, + 0, 0, -1, &fail_payload, sizeof(fail_payload)) == -1) + log_warn("imsg_compose IMSG_STORE_FORK (failure reply)"); + imsgev_add(sc->listener_iev); + } + if (!already_dead) + kill(sc->pid, SIGKILL); + event_del(&sc->iev.ev); + close(sc->iev.ibuf.fd); + TAILQ_REMOVE(&store_children, sc, entry); + free(sc); +} + +static void +store_child_fail(struct store_child *sc) +{ + store_child_teardown(sc, 0); +} + +/* + * Binds, sets SO_REUSEADDR, and listen(2)s one socket of the given family + * against sa/salen -- the common tail end of every case in + * bind_listen_socket() below, factored out rather than repeated three + * times. listen(2)'s backlog of 16 is unchanged from before this pass; + * see README.skeleton's own note on that being a known, minor, + * accepted v1 limitation, not something this pass touches. + */ +static int +bind_one(int family, const struct sockaddr *sa, socklen_t salen, + uint16_t port) +{ + int fd, val; + + if ((fd = socket(family, SOCK_STREAM, 0)) == -1) + fatal("socket"); + + val = 1; + if (setsockopt(fd, SOL_SOCKET, SO_REUSEADDR, &val, sizeof(val)) == -1) + fatal("setsockopt"); + + if (bind(fd, sa, salen) == -1) + fatal("bind port %u", port); + if (listen(fd, 16) == -1) + fatal("listen"); + + return (fd); +} + +/* + * Resolves and binds "addr" for "port", filling fds[] and returning how + * many it filled (1, or LISTENER_MAX_ADDRS for "*" -- see that macro's + * comment in imapd.h for the full "why two sockets, not one dual-mapped + * one" sourcing). "addr" is deliberately restricted to "*" or a literal + * IPv4/IPv6 address -- no getaddrinfo(3) hostname resolution -- per this + * pass's scoped decision: a personal single-box mail server has no real + * need for "listen on" to carry a DNS dependency, and parse.y's own + * grammar action rejects anything else before config_load() ever returns, + * so reaching the fatalx() below at runtime would mean parse.y's check and + * this one have drifted out of sync with each other, not a bad config + * file reaching this far. + */ +static int +bind_listen_socket(const char *addr, uint16_t port, int fds[LISTENER_MAX_ADDRS]) +{ + struct sockaddr_in sin; + struct sockaddr_in6 sin6; + struct in_addr ina; + struct in6_addr ina6; + int n = 0; + + if (strcmp(addr, "*") == 0) { + memset(&sin, 0, sizeof(sin)); + sin.sin_family = AF_INET; + sin.sin_addr.s_addr = INADDR_ANY; + sin.sin_port = htons(port); + fds[n++] = bind_one(AF_INET, (struct sockaddr *)&sin, + sizeof(sin), port); + + memset(&sin6, 0, sizeof(sin6)); + sin6.sin6_family = AF_INET6; + sin6.sin6_addr = in6addr_any; + sin6.sin6_port = htons(port); + fds[n++] = bind_one(AF_INET6, (struct sockaddr *)&sin6, + sizeof(sin6), port); + return (n); + } + + memset(&ina, 0, sizeof(ina)); + if (inet_pton(AF_INET, addr, &ina) == 1) { + memset(&sin, 0, sizeof(sin)); + sin.sin_family = AF_INET; + sin.sin_addr = ina; + sin.sin_port = htons(port); + fds[n++] = bind_one(AF_INET, (struct sockaddr *)&sin, + sizeof(sin), port); + return (n); + } + + memset(&ina6, 0, sizeof(ina6)); + if (inet_pton(AF_INET6, addr, &ina6) == 1) { + memset(&sin6, 0, sizeof(sin6)); + sin6.sin6_family = AF_INET6; + sin6.sin6_addr = ina6; + sin6.sin6_port = htons(port); + fds[n++] = bind_one(AF_INET6, (struct sockaddr *)&sin6, + sizeof(sin6), port); + return (n); + } + + fatalx("bind_listen_socket: \"%s\" is not \"*\" or a literal " + "IPv4/IPv6 address", addr); +} + +static void +send_listener_sockets(struct imsgev *iev, int cleartext_fds[], + int n_cleartext, int tls_fds[], int n_tls) +{ + int i; + + for (i = 0; i < n_cleartext; i++) { + if (imsg_compose(&iev->ibuf, IMSG_LISTENER_SOCKET_CLEARTEXT, + 0, 0, cleartext_fds[i], NULL, 0) == -1) + fatal("imsg_compose IMSG_LISTENER_SOCKET_CLEARTEXT"); + } + for (i = 0; i < n_tls; i++) { + if (imsg_compose(&iev->ibuf, IMSG_LISTENER_SOCKET_TLS, 0, 0, + tls_fds[i], NULL, 0) == -1) + fatal("imsg_compose IMSG_LISTENER_SOCKET_TLS"); + } + if (imsgbuf_flush(&iev->ibuf) == -1) + fatal("imsgbuf_flush"); +} + +/* + * Reads and sends both the TLS certificate AND the private key, as two + * separate IMSG_TLS_CERT / IMSG_TLS_KEY messages. Real gap fixed this + * pass: this function used to be named send_tls_cert() (singular) and + * only ever read+sent conf->tls_cert_file -- listener.c's TLS server + * context needs both (tls_config_set_keypair_mem(), sourced against + * httpd's server_tls_init() -- see listener.c), so without the key, real + * TLS could never have worked no matter what listener.c did with the + * cert alone. + * + * Two more real bugs fixed this pass, both found testing against smtpd(8) + * on the same box (premio) rather than guessed at: + * + * 1. The private key was never permission-checked. Found because + * /etc/ssl/private/openimap.key sat at mode 0644 (world-readable) + * this whole time -- this daemon never noticed or complained, while + * smtpd's load_pki_keys() refused to even start with the same file + * ("insecure permissions: must be at most rwxr-----"). Added the + * same check smtpd's ssl_load_key() does (openbsd_source/src/ + * usr.sbin/smtpd/ssl.c:132-145): reject unless owned by uid 0 and + * mode is no more permissive than rwxr----- (0740). The cert file + * gets no such check -- it's public by definition, same as smtpd + * only permission-checks the key, never the cert. + * + * 2. Both the old fopen()-failure paths and (without care) a new + * permission-check failure would have `return`ed early, sending + * neither IMSG_TLS_CERT nor IMSG_TLS_KEY for that failed file. But + * listener.c's boot loop (its "while (!got_sockets || !got_cert || + * !got_key || !got_init)" drain, see listener_main()) blocks until + * it receives exactly one of each message -- parent_main() never + * retries after send_tls_certs() returns, so a missing/unreadable/ + * insecurely-permissioned cert or key used to hang listener at boot + * forever instead of degrading to "no TLS", which is what listener's + * own cert_len==0/key_len==0 handling (line ~1164) is designed to + * do. Fixed by always sending both messages, with a zero-length + * payload standing in for "unusable" on any failure, so listener's + * existing graceful-degrade path is what actually runs. + * + * TODO: reads cert/key files but does no further validation (expiry, + * matching pubkey, etc.) -- libtls's own loading functions should + * probably do this work instead of hand-rolled file reads. TODO: each + * read is capped at sizeof(buf) (8192) and sent as a single imsg, itself + * capped at MAX_IMSGSIZE (16384) -- fine for a single leaf cert/key pair + * (typically 1-2KB each PEM-encoded), but there's no chunking for a + * certificate chain that exceeded either limit. Flagged, not solved -- + * v1 doesn't need chain support per the design docs' current scope. + */ +static void +send_tls_certs(struct imsgev *iev, struct openimap_config *conf) +{ + FILE *fp; + char buf[8192]; + size_t n; + struct stat st; + + /* Cert is public -- existence/readability is all that matters. */ + n = 0; + if ((fp = fopen(conf->tls_cert_file, "r")) == NULL) { + log_warn("fopen %s", conf->tls_cert_file); + } else { + n = fread(buf, 1, sizeof(buf), fp); + fclose(fp); + } + + if (imsg_compose(&iev->ibuf, IMSG_TLS_CERT, 0, 0, -1, buf, n) == -1) + fatal("imsg_compose IMSG_TLS_CERT"); + if (imsgbuf_flush(&iev->ibuf) == -1) + fatal("imsgbuf_flush"); + + /* + * Key is private material: same ownership/mode check smtpd applies + * (see this function's header comment), and n stays 0 -- sending an + * empty IMSG_TLS_KEY -- on any failure rather than returning early. + */ + n = 0; + if ((fp = fopen(conf->tls_key_file, "r")) == NULL) { + log_warn("fopen %s", conf->tls_key_file); + } else if (fstat(fileno(fp), &st) == -1) { + log_warn("fstat %s", conf->tls_key_file); + fclose(fp); + } else if (st.st_uid != 0) { + log_warnx("%s: not owned by uid 0 -- refusing to load " + "(TLS will be disabled)", conf->tls_key_file); + fclose(fp); + } else if (st.st_mode & (S_IRWXU | S_IRWXG | S_IRWXO) & ~0740) { + log_warnx("%s: insecure permissions -- must be at most " + "rwxr----- (TLS will be disabled)", conf->tls_key_file); + fclose(fp); + } else { + n = fread(buf, 1, sizeof(buf), fp); + fclose(fp); + } + + /* + * imsg_compose() copies buf's contents into the imsgbuf's own + * internal queue immediately (see imsg.h -- it takes a `const + * void *`/length pair, not ownership of buf itself), so it's safe + * to scrub our own stack copy of the key material right after this + * call returns, before imsgbuf_flush() actually writes the queued + * bytes to the socket. + */ + if (imsg_compose(&iev->ibuf, IMSG_TLS_KEY, 0, 0, -1, buf, n) == -1) { + explicit_bzero(buf, sizeof(buf)); + fatal("imsg_compose IMSG_TLS_KEY"); + } + explicit_bzero(buf, sizeof(buf)); + if (imsgbuf_flush(&iev->ibuf) == -1) + fatal("imsgbuf_flush"); +} + +/* + * IMSG_LISTENER_INIT: listener's slice of config. Used for a startup log + * line as before, but n_cleartext_addrs/n_tls_addrs are load-bearing now + * -- listener's boot-time drain loop needs them to know how many + * IMSG_LISTENER_SOCKET_CLEARTEXT/_TLS messages to wait for (see that + * struct's comment in imapd.h). The listening sockets themselves still + * always arrive as already-bound fds via send_listener_sockets(), never + * reconstructed from listen_addr here. + */ +static void +send_listener_init(struct imsgev *iev, struct openimap_config *conf, + int n_cleartext, int n_tls) +{ + struct imsg_listener_init init; + + memset(&init, 0, sizeof(init)); + (void)strlcpy(init.listen_addr, conf->listen_addr, + sizeof(init.listen_addr)); + init.port_cleartext = conf->port_cleartext; + init.port_implicit_tls = conf->port_implicit_tls; + init.n_cleartext_addrs = (uint8_t)n_cleartext; + init.n_tls_addrs = (uint8_t)n_tls; + + if (imsg_compose(&iev->ibuf, IMSG_LISTENER_INIT, 0, 0, -1, + &init, sizeof(init)) == -1) + fatal("imsg_compose IMSG_LISTENER_INIT"); + if (imsgbuf_flush(&iev->ibuf) == -1) + fatal("imsgbuf_flush"); +} + +/* + * IMSG_AUTH_INIT: auth's slice of config -- just cred_file, the one + * field auth_main() actually needs (to derive its chroot directory and, + * post-chroot, the unveil() path). Not optional polish -- see this + * struct's comment in imapd.h for the bug this closes. + */ +static void +send_auth_init(struct imsgev *iev, struct openimap_config *conf) +{ + struct imsg_auth_init init; + + memset(&init, 0, sizeof(init)); + (void)strlcpy(init.cred_file, conf->cred_file, sizeof(init.cred_file)); + + if (imsg_compose(&iev->ibuf, IMSG_AUTH_INIT, 0, 0, -1, + &init, sizeof(init)) == -1) + fatal("imsg_compose IMSG_AUTH_INIT"); + if (imsgbuf_flush(&iev->ibuf) == -1) + fatal("imsgbuf_flush"); +} + +/* + * SIGHUP: re-read imapd.conf and adopt what's safely reloadable, matching + * httpd(8)'s own documented behavior (man.openbsd.org/httpd.8: "httpd + * rereads its configuration file when it receives SIGHUP") -- the model + * this project follows throughout for "listen on" syntax, so it's the + * natural precedent for reload semantics too. Unlike httpd, there's no + * SIGUSR1/log-file-reopen counterpart here: this daemon logs to syslog(3) + * only (see log.c), same as smtpd(8) (man.openbsd.org/smtpd.8: no log-file + * FILES entry at all, just stderr under -d or syslogd(8) otherwise) -- + * httpd's SIGUSR1 exists specifically to reopen its access.log/error.log + * after newsyslog(8) rotation, which doesn't apply here. + * + * Not everything in imapd.conf can be safely swapped into a *running* + * daemon: + * + * - listen_addr/port_cleartext/port_implicit_tls: the listening sockets + * are already bound and fd-passed to listener at boot (bind_listen_ + * socket() above, called once from parent_main() before the privilege + * drop). Changing these live would mean rebinding new sockets and + * redoing the whole IMSG_LISTENER_SOCKET_CLEARTEXT/_TLS fd-passing + * handshake listener's boot-time drain loop expects exactly once -- + * not attempted here. A real change to these three fields requires a + * restart, same as smtpd/httpd expect for their own listen directives. + * + * - cred_file: auth_main() chroot(2)s into this file's dirname once, at + * boot (see auth.c), before ever reading IMSG_AUTH_REQUEST. There is no + * way to hand auth a new cred_file living outside that chroot without + * re-chrooting, which is impossible for an already-running process. + * Same "requires a restart" treatment. + * + * Both of the above are detected and logged as a warning -- config_load() + * itself always succeeds against a file that merely changed one of these, + * so silently ignoring the new values (rather than pretending they took + * effect) is what keeps the running daemon's actual behavior truthful. + * + * spool_root and bodystructure_read_max are trivially safe: both are read + * fresh out of *gconf by parent_handle_store_fork() every time a session + * authenticates and a new store child is spawned (see that function's + * imsg_store_init population above) -- nothing caches a copy anywhere else, + * so updating gconf's copy here is the entire fix. + * + * TLS cert/key are re-read and re-pushed to listener unconditionally, + * whether or not the *path* strings changed -- the ordinary reason to + * SIGHUP a running mail server at all is that acme-client(1) (or a cron + * job wrapping it) just renewed the certificate at the same path, not that + * the path moved. listener.c's new IMSG_TLS_CERT/IMSG_TLS_KEY handling in + * listener_dispatch_parent() (added this same pass) rebuilds its TLS + * context from the fresh bytes and only swaps it in on complete success -- + * see listener_reload_tls()'s header comment for why a reload failure must + * never tear down an already-working TLS context. send_tls_certs() already + * degrades gracefully to a zero-length payload on any read/permission + * failure (see that function's own header comment), which listener's + * reload path treats the same way: keep serving with whatever TLS + * configuration already worked. + */ +static void +sighup_handler(int fd, short event, void *arg) +{ + struct openimap_config newconf; + + (void)fd; (void)event; (void)arg; + + log_info("SIGHUP: reloading %s", conf_path); + + memset(&newconf, 0, sizeof(newconf)); + if (config_load(conf_path, &newconf) == -1) { + log_warnx("SIGHUP: %s: reload failed -- keeping the " + "already-running configuration", conf_path); + return; + } + + if (strcmp(newconf.listen_addr, gconf->listen_addr) != 0 || + newconf.port_cleartext != gconf->port_cleartext || + newconf.port_implicit_tls != gconf->port_implicit_tls) { + log_warnx("SIGHUP: %s: \"listen on\" changed but listening " + "sockets cannot be rebound without a restart -- still " + "serving %s:%u / %s:%u", conf_path, gconf->listen_addr, + gconf->port_cleartext, gconf->listen_addr, + gconf->port_implicit_tls); + } + if (strcmp(newconf.cred_file, gconf->cred_file) != 0) { + log_warnx("SIGHUP: %s: \"credentials\" changed but auth is " + "already chrooted for the old path -- a restart is " + "required for this to take effect", conf_path); + } + + /* Safe to adopt immediately -- see this function's header comment. */ + (void)strlcpy(gconf->spool_root, newconf.spool_root, + sizeof(gconf->spool_root)); + gconf->bodystructure_read_max = newconf.bodystructure_read_max; + + (void)strlcpy(gconf->tls_cert_file, newconf.tls_cert_file, + sizeof(gconf->tls_cert_file)); + (void)strlcpy(gconf->tls_key_file, newconf.tls_key_file, + sizeof(gconf->tls_key_file)); + send_tls_certs(iev_listener, gconf); + + log_info("SIGHUP: reload complete"); +} + +static void +sigterm_handler(int fd, short event, void *arg) +{ + struct child *c; + struct store_child *sc; + + (void)fd; (void)event; (void)arg; + log_info("SIGTERM: shutting down"); + + TAILQ_FOREACH(c, &children, entry) + kill(c->pid, SIGTERM); + TAILQ_FOREACH(sc, &store_children, entry) + kill(sc->pid, SIGTERM); + + exit(0); +} + +static void +sigchld_handler(int fd, short event, void *arg) +{ + pid_t pid; + int status; + + (void)fd; (void)event; (void)arg; + while ((pid = waitpid(-1, &status, WNOHANG)) > 0) + reap_child(pid, status); +} + +/* + * Reaching the "children" (listener/auth) branch below always means an + * unexpected exit, never a graceful shutdown's own SIGCHLD: sigterm_ + * handler() (this file) calls exit(0) immediately after kill()ing every + * child, without re-entering event_dispatch(), so this process never + * actually processes the SIGCHLD from children it deliberately killed. + * + * Deliberately NOT auto-restarted -- checked against both reference + * daemons this project follows before deciding, rather than assumed. + * smtpd's parent_sig_handler() SIGCHLD case (smtpd.c) treats its own + * core structural children (queue/control/lka/scheduler/dispatcher/ca -- + * CHILD_DAEMON, the direct equivalent of listener/auth here) the same + * way this function now does: log a warning, keep running degraded, + * never respawn. (A different category, CHILD_PROCESSOR -- dynamically + * loaded table/filter plugins -- does trigger smtpd's own full shutdown + * on failure, but that's an externally-loaded plugin, not a core + * structural process, so it doesn't apply here.) httpd goes further + * still: its running parent (httpd.c) never even registers a SIGCHLD + * handler, so a dead child there goes completely unnoticed until actual + * shutdown (proc_kill(), proc.c). Neither reference daemon auto-restarts + * a crashed core child, so this one doesn't either -- matches this + * project's "smaller feature set" principle throughout, and avoids the + * real complexity (crash-loop protection, re-running the SETUP_PEER + * handshake and config delivery at *runtime* instead of only at boot) a + * real restart would need. Recovery is operator-driven: "rcctl restart + * imapd", same as an admin would do for smtpd or httpd today. + */ +static void +reap_child(pid_t pid, int status) +{ + struct child *c; + struct store_child *sc; + + TAILQ_FOREACH(c, &children, entry) { + if (c->pid == pid) { + const char *what; + + /* + * listener owns every bound socket, so its death + * takes down IMAP service entirely -- no new + * connections can be accepted. auth's death is + * narrower: already-authenticated sessions are + * unaffected (post-login traffic is listener<->store + * only, never listener<->auth -- see imapd.h's imsg + * catalog), but new AUTHENTICATE attempts will fail, + * since listener.c's own auth-channel-closed handling + * (listener_dispatch_auth(), "auth closed channel") + * never sends a client a completing reply either way. + */ + what = c->type == PROC_LISTENER ? + "IMAP service is now unreachable (no listening " + "sockets)" : "new logins will now fail " + "(already-authenticated sessions are unaffected)"; + + if (WIFSIGNALED(status)) + log_warnx("%s[%d] terminated by signal %d -- " + "%s; run \"rcctl restart imapd\" to " + "recover", log_procname(c->type), pid, + WTERMSIG(status), what); + else if (WIFEXITED(status)) + log_warnx("%s[%d] exited unexpectedly, " + "status %d -- %s; run \"rcctl restart " + "imapd\" to recover", + log_procname(c->type), pid, + WEXITSTATUS(status), what); + else + log_warnx("%s[%d] exited unexpectedly -- %s; " + "run \"rcctl restart imapd\" to recover", + log_procname(c->type), pid, what); + TAILQ_REMOVE(&children, c, entry); + free(c); + return; + } + } + TAILQ_FOREACH(sc, &store_children, entry) { + if (sc->pid == pid) { + log_debug("session %u: store child exited " + "(status %d)", sc->session_id, status); + store_child_teardown(sc, 1); + return; + } + } +} blob - /dev/null blob + 0614ff4d02f82aaa32ca6c562e0cd3e269a6126e (mode 644) --- /dev/null +++ src/parse.y @@ -0,0 +1,881 @@ +/* + * Copyright (c) 2026 David Williams + * + * Permission to use, copy, modify, and distribute this software for any + * purpose with or without fee is hereby granted, provided that the above + * copyright notice and this permission notice appear in all copies. + * + * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + */ + +/* + * parse.y -- imapd.conf grammar. + * + * config_load() (the only function this file exposes -- see imapd.h's + * prototype) replaces parent.c's old stub of the same name, which only + * ever filled in v1 defaults and ignored the config file path entirely. + * + * Structure and most of the machinery below (the pushfile()/popfile() + * file stack for "include", the lgetc()/lungetc()/findeol() character- + * pushback pair, the hand-written yylex() built on top of them, the + * symset()/symget()/cmdline_symset() $macro/-D variable table, and + * check_file_secrecy()) are this project's own rewrite of the same + * pattern used throughout the OpenBSD base system's own daemons -- sourced + * directly against ripd's parse.y (src/usr.sbin/ripd/parse.y, read in + * full this pass) for the generic boilerplate, and against smtpd's and + * httpd's parse.y (src/usr.sbin/smtpd/parse.y, src/usr.sbin/httpd/ + * parse.y) for the "include" grammar rule and the "listen on + * [tls] port " grammar shape respectively. None of these daemons' + * domain-specific grammar (RIP redistribution rules, SMTP rulesets, + * HTTP server blocks) is reused here -- only the parser skeleton and the + * listen-line shape. This project's own grammar covers exactly the eight + * fields in struct openimap_config (imapd.h): the two listeners, + * spool_root, cred_file, tls_cert_file, tls_key_file, and (added this + * pass) bodystructure_read_max via the "attachment max " + * directive -- see that grammar rule's own comment below for why this + * one field gets range-validated where the others don't. + * + * Deliberately NOT following ripd/smtpd's convention in one place: those + * daemons' top-level parse function allocates and returns their own + * config struct. config_load()'s signature is already fixed by imapd.h + * (a caller-supplied struct openimap_config * to fill in, matching the + * existing prototype main.c already calls) and by the man page already + * documenting it, so this file fills that struct in place instead of + * allocating its own. + * + * Full-parity scope decision (AskUserQuestion, this pass): rather than a + * trimmed-down parser with just the core grammar, this includes the same + * $macro / -D command-line variable support and "include" file support + * every OpenBSD base daemon's config grammar has, even though a flat + * seven-field config has only modest use for either -- matching upstream + * convention closely was judged more valuable than the modest size + * savings a trimmed version would give, particularly given this project's + * long-term (if distant) OpenBSD base-inclusion aspiration. + */ + +%{ +#include +#include +#include +#include + +#include + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "imapd.h" +#include "log.h" + +TAILQ_HEAD(files, file) files = TAILQ_HEAD_INITIALIZER(files); +static struct file { + TAILQ_ENTRY(file) entry; + FILE *stream; + char *name; + int lineno; + int errors; +} *file, *topfile; +struct file *pushfile(const char *, int); +int popfile(void); +int yyparse(void); +int yylex(void); +int yyerror(const char *, ...) + __attribute__((__format__ (printf, 1, 2))) + __attribute__((__nonnull__ (1))); +int kw_cmp(const void *, const void *); +int lookup(char *); +int lgetc(int); +int lungetc(int); +int findeol(void); + +TAILQ_HEAD(symhead, sym) symhead = TAILQ_HEAD_INITIALIZER(symhead); +struct sym { + TAILQ_ENTRY(sym) entry; + int used; + int persist; + char *nam; + char *val; +}; +int symset(const char *, const char *, int); +char *symget(const char *); + +static struct openimap_config *conf; +static int errors = 0; + +/* + * Tracks whether a "listen on ..." line for each of the two listeners has + * already been seen this parse, both to reject a second one for the same + * listener and to detect a second listener whose address doesn't match + * the first -- struct openimap_config has only one listen_addr shared by + * both ports (bind_listen_socket() is called once per port against that + * same field, parent.c), so two "listen on" lines naming different + * addresses would silently mean "the second one wins" without this check, + * which is far more likely to be a config mistake than an intentional + * per-listener address split this implementation doesn't actually support. + */ +static int have_cleartext = 0; +static int have_tls_listen = 0; + +typedef struct { + union { + int64_t number; + char *string; + } v; + int lineno; +} YYSTYPE; + +%} + +%token LISTEN ON TLS PORT +%token SPOOL CREDENTIALS CERTIFICATE KEY +%token ATTACHMENT MAX +%token INCLUDE +%token ERROR +%token STRING +%token NUMBER +%type opttls + +%% + +grammar : /* empty */ + | grammar '\n' + | grammar include '\n' + | grammar varset '\n' + | grammar main '\n' + | grammar error '\n' { file->errors++; } + ; + +include : INCLUDE STRING { + struct file *nfile; + + if ((nfile = pushfile($2, 0)) == NULL) { + yyerror("failed to include file %s", $2); + free($2); + YYERROR; + } + free($2); + + file = nfile; + lungetc('\n'); + } + ; + +varset : STRING '=' STRING { + char *s = $1; + + while (*s++) { + if (isspace((unsigned char)*s)) { + yyerror("macro name cannot contain " + "whitespace"); + free($1); + free($3); + YYERROR; + } + } + if (symset($1, $3, 0) == -1) + fatal("cannot store variable"); + free($1); + free($3); + } + ; + +opttls : /* empty */ { $$ = 0; } + | TLS { $$ = 1; } + ; + +main : LISTEN ON STRING opttls PORT NUMBER { + if ($6 < 1 || $6 > 65535) { + yyerror("invalid port: %lld", (long long)$6); + free($3); + YYERROR; + } + + /* + * "*" (dual-stack, both IPv4-any and IPv6-any) or a + * literal IPv4/IPv6 address only -- no getaddrinfo(3) + * hostname resolution, a deliberate scope decision + * for this pass (see LISTENER_MAX_ADDRS's comment in + * imapd.h for the full reasoning). Checked here, at + * parse time, so a bad address is rejected with a + * clear config-file error instead of surfacing later + * as parent.c's bind_listen_socket() fatal()ing at + * startup. + */ + if (strcmp($3, "*") != 0) { + struct in_addr ina; + struct in6_addr ina6; + + if (inet_pton(AF_INET, $3, &ina) != 1 && + inet_pton(AF_INET6, $3, &ina6) != 1) { + yyerror("listen address \"%s\" is " + "not \"*\" or a literal IPv4/" + "IPv6 address", $3); + free($3); + YYERROR; + } + } + + if (have_cleartext || have_tls_listen) { + if (strcmp(conf->listen_addr, $3) != 0) { + yyerror("listen address \"%s\" does " + "not match earlier \"listen on " + "%s\" -- imapd binds both " + "listeners to the same address", + $3, conf->listen_addr); + free($3); + YYERROR; + } + } else if (strlcpy(conf->listen_addr, $3, + sizeof(conf->listen_addr)) >= + sizeof(conf->listen_addr)) { + yyerror("listen address too long: %s", $3); + free($3); + YYERROR; + } + free($3); + + if ($4) { + if (have_tls_listen) { + yyerror("tls listener already " + "configured"); + YYERROR; + } + conf->port_implicit_tls = (uint16_t)$6; + have_tls_listen = 1; + } else { + if (have_cleartext) { + yyerror("cleartext listener already " + "configured"); + YYERROR; + } + conf->port_cleartext = (uint16_t)$6; + have_cleartext = 1; + } + } + | SPOOL STRING { + if (strlcpy(conf->spool_root, $2, + sizeof(conf->spool_root)) >= + sizeof(conf->spool_root)) { + yyerror("spool path too long: %s", $2); + free($2); + YYERROR; + } + free($2); + } + | CREDENTIALS STRING { + if (strlcpy(conf->cred_file, $2, + sizeof(conf->cred_file)) >= + sizeof(conf->cred_file)) { + yyerror("credentials path too long: %s", $2); + free($2); + YYERROR; + } + free($2); + } + | TLS CERTIFICATE STRING { + if (strlcpy(conf->tls_cert_file, $3, + sizeof(conf->tls_cert_file)) >= + sizeof(conf->tls_cert_file)) { + yyerror("tls certificate path too long: %s", + $3); + free($3); + YYERROR; + } + free($3); + } + | TLS KEY STRING { + if (strlcpy(conf->tls_key_file, $3, + sizeof(conf->tls_key_file)) >= + sizeof(conf->tls_key_file)) { + yyerror("tls key path too long: %s", $3); + free($3); + YYERROR; + } + free($3); + } + | ATTACHMENT MAX NUMBER { + /* + * Range-validated, unlike the other directives above: + * this value flows straight into a per-FETCH malloc() + * in store.c's read_message_body() (readbuf_size = + * maxlen + 1), once per session-child process, so an + * operator typo here (an extra zero, a value meant as + * megabytes typed as bytes) has a real memory-exhaustion + * consequence that a bad path string or port number + * doesn't. Lower bound matches BODYSTRUCTURE_MAX + * (imapd.h) as a floor -- below that, no message could + * ever produce a usable BODYSTRUCTURE anyway, so a + * smaller value is certainly a mistake, not a real + * intended restriction. Upper bound (1 GiB) is a + * generous, round sanity ceiling, not a protocol limit + * -- personal mail attachments this large are not a + * real-world case this implementation targets (see + * BODYSTRUCTURE_READ_DEFAULT's own Gmail-sourced + * reasoning), and rejecting it here surfaces the + * mistake at config-parse time rather than as a + * confusing malloc failure or slow FETCH later. + */ + if ($3 < BODYSTRUCTURE_MAX || $3 > 1073741824) { + yyerror("attachment max out of range " + "(%d-1073741824 bytes): %lld", + BODYSTRUCTURE_MAX, (long long)$3); + YYERROR; + } + conf->bodystructure_read_max = (uint32_t)$3; + } + ; + +%% + +struct keywords { + const char *k_name; + int k_val; +}; + +int +yyerror(const char *fmt, ...) +{ + va_list ap; + char *msg; + + file->errors++; + va_start(ap, fmt); + if (vasprintf(&msg, fmt, ap) == -1) + fatalx("yyerror vasprintf"); + va_end(ap); + logit(LOG_CRIT, "%s:%d: %s", file->name, yylval.lineno, msg); + free(msg); + return (0); +} + +int +kw_cmp(const void *k, const void *e) +{ + return (strcmp(k, ((const struct keywords *)e)->k_name)); +} + +int +lookup(char *s) +{ + /* this has to be sorted always */ + static const struct keywords keywords[] = { + {"attachment", ATTACHMENT}, + {"certificate", CERTIFICATE}, + {"credentials", CREDENTIALS}, + {"include", INCLUDE}, + {"key", KEY}, + {"listen", LISTEN}, + {"max", MAX}, + {"on", ON}, + {"port", PORT}, + {"spool", SPOOL}, + {"tls", TLS}, + }; + const struct keywords *p; + + p = bsearch(s, keywords, sizeof(keywords) / sizeof(keywords[0]), + sizeof(keywords[0]), kw_cmp); + + if (p) + return (p->k_val); + else + return (STRING); +} + +#define MAXPUSHBACK 128 + +static char *parsebuf; +static int parseindex; +static char pushback_buffer[MAXPUSHBACK]; +static int pushback_index = 0; + +int +lgetc(int quotec) +{ + int c, next; + + if (parsebuf) { + /* Read character from the parsebuffer instead of input. */ + if (parseindex >= 0) { + c = (unsigned char)parsebuf[parseindex++]; + if (c != '\0') + return (c); + parsebuf = NULL; + } else + parseindex++; + } + + if (pushback_index) + return ((unsigned char)pushback_buffer[--pushback_index]); + + if (quotec) { + if ((c = getc(file->stream)) == EOF) { + yyerror("reached end of file while parsing " + "quoted string"); + if (file == topfile || popfile() == EOF) + return (EOF); + return (quotec); + } + return (c); + } + + while ((c = getc(file->stream)) == '\\') { + next = getc(file->stream); + if (next != '\n') { + c = next; + break; + } + yylval.lineno = file->lineno; + file->lineno++; + } + + while (c == EOF) { + if (file == topfile || popfile() == EOF) + return (EOF); + c = getc(file->stream); + } + return (c); +} + +int +lungetc(int c) +{ + if (c == EOF) + return (EOF); + if (parsebuf) { + parseindex--; + if (parseindex >= 0) + return (c); + } + if (pushback_index + 1 >= MAXPUSHBACK) + return (EOF); + pushback_buffer[pushback_index++] = c; + return (c); +} + +int +findeol(void) +{ + int c; + + parsebuf = NULL; + + /* skip to either EOF or the first real EOL */ + while (1) { + if (pushback_index) + c = (unsigned char)pushback_buffer[--pushback_index]; + else + c = lgetc(0); + if (c == '\n') { + file->lineno++; + break; + } + if (c == EOF) + break; + } + return (ERROR); +} + +int +yylex(void) +{ + char buf[8096]; + char *p, *val; + int quotec, next, c; + int token; + +top: + p = buf; + while ((c = lgetc(0)) == ' ' || c == '\t') + ; /* nothing */ + + yylval.lineno = file->lineno; + if (c == '#') + while ((c = lgetc(0)) != '\n' && c != EOF) + ; /* nothing */ + if (c == '$' && parsebuf == NULL) { + while (1) { + if ((c = lgetc(0)) == EOF) + return (0); + + if (p + 1 >= buf + sizeof(buf) - 1) { + yyerror("string too long"); + return (findeol()); + } + if (isalnum(c) || c == '_') { + *p++ = c; + continue; + } + *p = '\0'; + lungetc(c); + break; + } + val = symget(buf); + if (val == NULL) { + yyerror("macro '%s' not defined", buf); + return (findeol()); + } + parsebuf = val; + parseindex = 0; + goto top; + } + + switch (c) { + case '\'': + case '"': + quotec = c; + while (1) { + if ((c = lgetc(quotec)) == EOF) + return (0); + if (c == '\n') { + file->lineno++; + continue; + } else if (c == '\\') { + if ((next = lgetc(quotec)) == EOF) + return (0); + if (next == quotec || next == ' ' || + next == '\t') + c = next; + else if (next == '\n') { + file->lineno++; + continue; + } else + lungetc(next); + } else if (c == quotec) { + *p = '\0'; + break; + } else if (c == '\0') { + yyerror("syntax error"); + return (findeol()); + } + if (p + 1 >= buf + sizeof(buf) - 1) { + yyerror("string too long"); + return (findeol()); + } + *p++ = c; + } + yylval.v.string = strdup(buf); + if (yylval.v.string == NULL) + err(1, "%s", __func__); + return (STRING); + } + +#define allowed_to_end_number(x) \ + (isspace(x) || x == ')' || x == ',' || x == '/' || x == '}' || \ + x == '=') + + if (c == '-' || isdigit(c)) { + do { + *p++ = c; + if ((size_t)(p - buf) >= sizeof(buf)) { + yyerror("string too long"); + return (findeol()); + } + } while ((c = lgetc(0)) != EOF && isdigit(c)); + lungetc(c); + if (p == buf + 1 && buf[0] == '-') + goto nodigits; + if (c == EOF || allowed_to_end_number(c)) { + const char *errstr = NULL; + + *p = '\0'; + yylval.v.number = strtonum(buf, LLONG_MIN, + LLONG_MAX, &errstr); + if (errstr) { + yyerror("\"%s\" invalid number: %s", + buf, errstr); + return (findeol()); + } + return (NUMBER); + } else { +nodigits: + while (p > buf + 1) + lungetc((unsigned char)*--p); + c = (unsigned char)*--p; + if (c == '-') + return (c); + } + } + +#define allowed_in_string(x) \ + (isalnum(x) || (ispunct(x) && x != '(' && x != ')' && \ + x != '{' && x != '}' && \ + x != '!' && x != '=' && x != '#' && \ + x != ',')) + + /* + * '*' added alongside the pre-existing ':' (there for bare IPv6 + * literals like "::") so the dual-stack "listen on *" wildcard from + * the IPv6 pass works unquoted, matching every other address form + * documented in imapd.conf.example -- found as a real bug, not + * designed in up front: an unquoted "listen on * port 143" line + * failed with a bare "syntax error" on real hardware (premio) before + * this fix, since '*' alone satisfied none of the original starting + * characters here and fell through to being returned as a raw, + * unexpected single-character token instead of ever entering this + * bareword-accumulation loop. Quoting it ("listen on \"*\" ...") + * already worked, since the quoted-string branch above this one + * doesn't consult allowed_in_string() at all -- this fix is only + * about making the unquoted form work too. + */ + if (isalnum(c) || c == ':' || c == '_' || c == '*') { + do { + *p++ = c; + if ((size_t)(p - buf) >= sizeof(buf)) { + yyerror("string too long"); + return (findeol()); + } + } while ((c = lgetc(0)) != EOF && (allowed_in_string(c))); + lungetc(c); + *p = '\0'; + if ((token = lookup(buf)) == STRING) + if ((yylval.v.string = strdup(buf)) == NULL) + err(1, "%s", __func__); + return (token); + } + if (c == '\n') { + yylval.lineno = file->lineno; + file->lineno++; + } + if (c == EOF) + return (0); + return (c); +} + +/* + * Same root-owned-or-current-user, not-group-or-world-writable check this + * project already applies to the TLS private key file (parent.c's + * send_tls_certs(), added when that boot-deadlock/permission-check pass + * closed a real gap there) -- applied here to the top-level config file + * itself, since it can name the credentials file path and TLS key path, + * matching every base-system daemon's own parse.y (ripd's is the direct + * source for this function, byte-for-byte apart from the log_warn/ + * log_warnx call signatures matching this project's own log.h instead of + * ripd's warn(3)/warnx(3)-based logit() wrappers). Not applied to + * "include"d files (pushfile()'s second argument is 0 for those) -- + * matching smtpd's own convention of only checking the top-level file. + */ +static int +check_file_secrecy(int fd, const char *fname) +{ + struct stat st; + + if (fstat(fd, &st)) { + log_warn("cannot stat %s", fname); + return (-1); + } + if (st.st_uid != 0 && st.st_uid != getuid()) { + log_warnx("%s: owner not root or current user", fname); + return (-1); + } + if (st.st_mode & (S_IWGRP | S_IXGRP | S_IRWXO)) { + log_warnx("%s: group writable or world read/writable", fname); + return (-1); + } + return (0); +} + +struct file * +pushfile(const char *name, int secret) +{ + struct file *nfile; + + if ((nfile = calloc(1, sizeof(struct file))) == NULL) { + log_warn("%s", __func__); + return (NULL); + } + if ((nfile->name = strdup(name)) == NULL) { + log_warn("%s", __func__); + free(nfile); + return (NULL); + } + if ((nfile->stream = fopen(nfile->name, "r")) == NULL) { + log_warn("%s: %s", __func__, nfile->name); + free(nfile->name); + free(nfile); + return (NULL); + } else if (secret && + check_file_secrecy(fileno(nfile->stream), nfile->name)) { + fclose(nfile->stream); + free(nfile->name); + free(nfile); + return (NULL); + } + nfile->lineno = 1; + TAILQ_INSERT_TAIL(&files, nfile, entry); + return (nfile); +} + +int +popfile(void) +{ + struct file *prev; + + if ((prev = TAILQ_PREV(file, files, entry)) != NULL) + prev->errors += file->errors; + + TAILQ_REMOVE(&files, file, entry); + fclose(file->stream); + free(file->name); + free(file); + file = prev; + return (file ? 0 : EOF); +} + +/* + * config_load(): imapd.h's public entry point (replaces parent.c's old + * config_load() stub of the same name/signature -- see this file's header + * comment). Fills xconf with the v1 defaults first (same values the old + * stub always used unconditionally), then parses path over those defaults + * -- so a config file that sets only, say, "spool" leaves every other + * field at its documented v1 default rather than requiring a client to + * spell out all seven fields every time. Returns 0 on success (xconf + * fully populated) or -1 (a parse error occurred; xconf's contents are + * unspecified) -- same "reject rather than run with a half-parsed config" + * precedent as every other MAX-constant/parse-failure case in this + * codebase, left to the caller (main.c) to treat as fatal. + */ +int +config_load(const char *path, struct openimap_config *xconf) +{ + struct sym *sym, *next; + + conf = xconf; + memset(conf, 0, sizeof(*conf)); + + (void)strlcpy(conf->listen_addr, "0.0.0.0", sizeof(conf->listen_addr)); + conf->port_cleartext = 143; + conf->port_implicit_tls = 993; + (void)strlcpy(conf->spool_root, "/var/mail/imapd", + sizeof(conf->spool_root)); + (void)strlcpy(conf->cred_file, "/etc/imapd/credentials", + sizeof(conf->cred_file)); + (void)strlcpy(conf->tls_cert_file, "/etc/ssl/imapd.crt", + sizeof(conf->tls_cert_file)); + (void)strlcpy(conf->tls_key_file, "/etc/ssl/private/imapd.key", + sizeof(conf->tls_key_file)); + conf->bodystructure_read_max = BODYSTRUCTURE_READ_DEFAULT; + + have_cleartext = 0; + have_tls_listen = 0; + errors = 0; + + if ((file = pushfile(path, 1)) == NULL) + return (-1); + topfile = file; + + yyparse(); + errors = file->errors; + popfile(); + + /* Free macros and warn about any that were never referenced. */ + TAILQ_FOREACH_SAFE(sym, &symhead, entry, next) { + if (!sym->used) + log_debug("config_load: macro '%s' not used", + sym->nam); + if (!sym->persist) { + free(sym->nam); + free(sym->val); + TAILQ_REMOVE(&symhead, sym, entry); + free(sym); + } + } + + if (errors) + return (-1); + + return (0); +} + +int +symset(const char *nam, const char *val, int persist) +{ + struct sym *sym; + + TAILQ_FOREACH(sym, &symhead, entry) { + if (strcmp(nam, sym->nam) == 0) + break; + } + + if (sym != NULL) { + if (sym->persist == 1) + return (0); + else { + free(sym->nam); + free(sym->val); + TAILQ_REMOVE(&symhead, sym, entry); + free(sym); + } + } + if ((sym = calloc(1, sizeof(*sym))) == NULL) + return (-1); + + sym->nam = strdup(nam); + if (sym->nam == NULL) { + free(sym); + return (-1); + } + sym->val = strdup(val); + if (sym->val == NULL) { + free(sym->nam); + free(sym); + return (-1); + } + sym->used = 0; + sym->persist = persist; + TAILQ_INSERT_TAIL(&symhead, sym, entry); + return (0); +} + +/* + * cmdline_symset(): "-D name=value" command-line macro definitions, + * matching ripd/smtpd's own "-D" flag exactly -- called directly from + * main.c's getopt(3) loop, before config_load() runs, so these persist + * (the "1" argument to symset()) across the parse: a "-D" definition + * always wins over the same name defined inside the config file itself. + */ +int +cmdline_symset(char *s) +{ + char *sym, *val; + int ret; + + if ((val = strrchr(s, '=')) == NULL) + return (-1); + sym = strndup(s, val - s); + if (sym == NULL) + fatal("%s: strndup", __func__); + ret = symset(sym, val + 1, 1); + free(sym); + + return (ret); +} + +char * +symget(const char *nam) +{ + struct sym *sym; + + TAILQ_FOREACH(sym, &symhead, entry) { + if (strcmp(nam, sym->nam) == 0) { + sym->used = 1; + return (sym->val); + } + } + return (NULL); +} blob - /dev/null blob + bbbf2d8cb4c55ae27eed5cb70cec9c3ff817bb28 (mode 755) --- /dev/null +++ src/rc.d/imapd @@ -0,0 +1,83 @@ +#!/bin/ksh +# +# $OpenIMAPD$ +# +# rc.d(8) service script for imapd(8). imapd does not detach from +# its controlling terminal or otherwise background itself (see +# imapd.8) -- rc_bg=YES tells rc.subr to run it under "set -o monitor" +# instead (own process group, avoids SIGHUP at boot), which is rc.subr's +# documented idiom for exactly this kind of foreground-only daemon. No +# rc_start override is needed: the default "rc_exec ${daemon} +# ${daemon_flags}" is sufficient with rc_bg=YES alone -- confirmed against +# ypbind's real, shipped rc.d script (/etc/rc.d/ypbind), which uses the +# identical rc_bg=YES-with-default-rc_start pattern for the same stated +# reason ("avoid SIGHUP at boot"). +# +# rc_stop needs no override either: parent.c's sigterm_handler already +# cascades SIGTERM to every child (listener, auth, each per-session store +# child) on receipt of a single SIGTERM to the root parent -- exactly what +# rc.subr's default rc_stop() sends (pkill -TERM against the tracked +# process). See parent.c for that cascade logic. + +daemon="/usr/local/sbin/imapd" + +. /etc/rc.d/rc.subr + +rc_bg=YES + +# --- boot-time relink consumer ------------------------------------------- +# +# "make install" (see ../Makefile's RELINK=) drops a re-link kit at +# ${_relink_tar} if the Makefile's RELINK variable is set, per bsd.prog. +# mk's documented re-link-kit mechanism. Nothing on this system consumes +# it automatically the way base does for libc/libcrypto/ld.so/sshd: +# /etc/rc's reorder_libs() has a hardcoded allowlist (confirmed by reading +# /etc/rc directly) that doesn't and won't include imapd -- extending +# it would mean patching base /etc/rc, which gets clobbered on every base +# upgrade. This hook is the substitute consumer. +# +# It runs every time this service is (re)started -- at boot via +# pkg_scripts, or by hand via "rcctl restart imapd" -- checks for a +# pending tarball, and if present, extracts it and runs the install.sh +# that bsd.prog.mk generated inside it. That install.sh recompiles ${PROG} +# from the tarball's own object files in a freshly randomized link order, +# smoke-tests the result via the Makefile's RELINK command ("imapd -V", +# see main.c -- prints a version string and exits 0 with no other side +# effects), and only then installs the result over the currently running +# binary. +# +# Failure here is deliberately non-fatal to starting the service: if the +# relink or its smoke test fails, install.sh's own "install -c -s ..." +# step never runs, so the binary already at ${daemon} (from the last +# successful relink, or a plain "make install") is untouched. We log a +# warning, leave the tarball in place for investigation, and fall through +# to starting that known-good binary rather than taking the mail server +# down over a relink hiccup -- KARL's own kernel relink (reorder_kernel.sh) +# follows the same non-fatal-on-failure philosophy for the same reason. +_relink_tar="/usr/share/relink/usr/local/sbin/imapd/imapd.tar" + +rc_pre() { + local _tmp + + [[ -f ${_relink_tar} ]] || return 0 + + _tmp=$(mktemp -d /tmp/imapd-relink.XXXXXXXXXX) || { + logger -t imapd -p daemon.err \ + "boot-time relink: mktemp failed, skipping (${_relink_tar} left in place)" + return 0 + } + + if ( cd "${_tmp}" && tar xf "${_relink_tar}" && sh install.sh ) \ + >/tmp/imapd-relink.log 2>&1; then + rm -f "${_relink_tar}" + logger -t imapd -p daemon.info \ + "boot-time relink applied successfully" + else + logger -t imapd -p daemon.err \ + "boot-time relink failed -- starting previously installed binary; see /tmp/imapd-relink.log and ${_relink_tar}" + fi + rm -rf "${_tmp}" + return 0 +} + +rc_cmd $1 blob - /dev/null blob + d53b32ebb2cc3d19a0d7784be92978875707d759 (mode 644) --- /dev/null +++ src/store.c @@ -0,0 +1,7768 @@ +/* + * Copyright (c) 2026 David Williams + * + * Permission to use, copy, modify, and distribute this software for any + * purpose with or without fee is hereby granted, provided that the above + * copyright notice and this permission notice appear in all copies. + * + * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + */ + +/* + * store.c -- mailbox-store, filesystem process. Implements the + * "mailbox-store" section of openimap-privsep-design.md's fork-per-session + * design: a fresh process per authenticated session, privilege-dropped to + * that session's own uid/gid (via IMSG_STORE_INIT, received before any + * chroot/pledge/unveil -- see store_main() below), then wired to listener + * over a dedicated peer channel per the same SETUP_PEER/SETUP_DONE + * mechanism used at boot for listener<->auth. + * + * IMSG_MBOX_SELECT is now real: index_load()/index_append()/index_save() + * implement the line-oriented UIDVALIDITY:UIDNEXT + UID:basename:keywords + * index format resolved in openimap-storage-backend.md (flock(2)-guarded, + * full-rewrite-to-temp-file-then-rename(2) on every mutation), and + * handle_mbox_select() -- v1 is INBOX-only, openimap-v1-dispatch.md -- + * scans new/ for not-yet-indexed messages and assigns them UIDs on every + * SELECT, per that document's own "notice externally-delivered files in + * new/ ... assign UID via index" note. IMSG_MBOX_FETCH (message metadata: + * FLAGS/UID/INTERNALDATE/RFC822.SIZE) and IMSG_MBOX_STORE (set/add/remove + * flags, RFC 9051 SS6.4.6) are real too now -- handle_mbox_fetch() and + * handle_mbox_store(), the latter also responsible for keeping a message's + * maildir flag-suffix letters and cur//new/ placement in sync with its + * flags, since the filename is maildir's sole source of truth for system + * flags (openimap-storage-backend.md's "no standard-flag caching in the + * index" decision). IMSG_MBOX_EXPUNGE (RFC 9051 SS6.4.3 -- also used, with + * silent=1, by CLOSE) is real too -- handle_mbox_expunge() permanently + * unlinks every \Deleted message and compacts the index in a single + * lower-numbered-first pass, which is also what produces SS7.5.1's + * "immediately decremented" per-message sequence numbers for free + * (verified against both of SS6.4.3/SS7.5.1's own worked examples). + * IMSG_MBOX_APPEND (RFC 9051 SS6.3.12) is real too now -- handle_mbox_ + * append() delivers a new message via maildir's tmp/-then-cur/ atomic + * rename(2), assigns it a UID from the index (index_append(), same path + * handle_mbox_select() already uses for externally-delivered mail), and + * writes the index BEFORE the tmp->cur rename so a crash between the two + * leaves an indexed-but-not-yet-visible message rather than a delivered- + * but-unindexed one -- consistent with this file's existing tolerance + * elsewhere for "indexed but missing on disk" over the reverse. The + * message body itself arrives as variable-length trailing data on the + * same imsg as its struct imsg_mbox_append header (see that struct's + * comment in imapd.h for the imsg_get_buf()/imsg_get_len() wire- + * protocol verification this relies on); listener.c caps what it will + * ever send to APPEND_LITERAL_MAX (12000 bytes) to guarantee that fits. + * IMSG_MBOX_SEARCH (RFC 9051 SS6.4.4) is real too now -- handle_mbox_ + * search() reads a flat postfix bytecode (struct search_node array, + * compiled by listener.c's parse_search_key()/parse_search_key_list() + * from the client's search-program, same variable-length-trailing-data + * imsg technique as IMSG_MBOX_APPEND) and evaluates it against every + * message via search_eval()/search_eval_leaf(), streaming matches back + * as IMSG_MBOX_SEARCH_MATCH. Covers flag-based (ANSWERED/DELETED/DRAFT/ + * FLAGGED/SEEN and their UN- forms), KEYWORD/UNKEYWORD, BEFORE/ON/SINCE + * (internal date), LARGER/SMALLER (size), sequence-set and UID-range + * search keys, plus NOT/OR/parenthesized-AND-list combination -- not + * the content-and-header-based keys (BCC/BODY/CC/FROM/HEADER/ + * SENTBEFORE/SENTON/SENTSINCE/SUBJECT/TEXT/TO), which listener.c rejects + * before this imsg is ever built (see imapd.h's imsg_mbox_search + * comment for the full v1-scope reasoning). This paragraph is a snapshot + * of the state as of the SEARCH pass; every other IMSG_MBOX_* case in + * store_dispatch()'s switch below (COPY/MOVE -- including cross-mailbox + * -- STATUS, UID forms, CONDSTORE/QRESYNC, NAMESPACE, CREATE/DELETE/ + * RENAME, EXAMINE, IDLE) has since been implemented in later passes; see + * README.skeleton for the full history rather than trusting this + * comment's age. Privilege drop uses the + * actual uid/gid/spool_root/maildir delivered at runtime (maildir being + * new this pass too -- see imapd.h's imsg_store_init comment for the + * gap that closed), and unveil(2) is now scoped to this session's own + * maildir subdirectory specifically, not the whole shared chroot. + * + * API NAMES: checked against the real src/imsg.h this session -- see + * parent.c's header comment for the full verification note. + */ + +#include +#include +#include + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "imapd.h" +#include "log.h" + +static struct imsgev iev_listener; +static uint32_t session_id; +static uint32_t bodystructure_read_max; /* from IMSG_STORE_INIT -- + * see imapd.h's struct imsg_store_init + * comment and BODYSTRUCTURE_READ_ + * DEFAULT's comment for what this + * gates and where the operator- + * configured value comes from. */ + +/* + * Per-store-child monotonic counter feeding the maildir basename uniquer + * (see handle_mbox_append()'s header comment for the full `. + * _.` scheme). Was originally a function-local + * static inside handle_mbox_append() alone; hoisted to file scope this + * pass so handle_mbox_copy() -- COPY's own message-duplication path, which + * mints a brand new basename per copied message the same way APPEND does + * -- shares the same monotonic sequence rather than starting its own at 0, + * which could otherwise collide with an APPEND's basename minted in the + * same second by the same pid. + */ +static uint32_t append_counter; + +/* + * RFC 9051 SS6.3.4-SS6.3.6/SS6.3.9 addition (flat multi-mailbox support, + * see docs/openimap-storage-backend.md's "Open items" #10): which named + * mailbox (if any) this store child's cwd is currently chdir'd into, + * relative to the session's own maildir root. Empty string means "at the + * root" -- i.e. INBOX, or nothing selected yet -- the same state every + * store child has always implicitly been in before this pass, when INBOX + * was the only mailbox that could ever exist. handle_mbox_select() is the + * only function that ever changes this; every other IMSG_MBOX_* handler + * (FETCH/STORE/EXPUNGE/IDLE-refresh/QRESYNC/...) keeps using bare + * relative paths exactly as before, unmodified, since cwd is always + * "wherever the currently selected mailbox is" by construction. + */ +static char current_mailbox_dir[MBOX_NAME_MAX]; + +static void store_dispatch(int, short, void *); +static void store_shutdown(void); +static void handle_mbox_select(struct imsg_mbox_select *, struct imsgev *); +static void handle_mbox_fetch(struct imsg_mbox_fetch *, struct imsgev *); +static void handle_mbox_store(struct imsg_mbox_store *, struct imsgev *); +static void handle_mbox_expunge(struct imsg_mbox_expunge *, struct imsgev *); +static void handle_mbox_append(struct imsg_mbox_append *, const char *, + size_t, struct imsgev *); +static void handle_mbox_search(struct imsg_mbox_search *, + struct search_node *, uint32_t, struct imsgev *); +static void handle_mbox_status(struct imsg_mbox_status *, struct imsgev *); +static void handle_mbox_copy(struct imsg_mbox_copy *, struct imsgev *); +static void handle_mbox_move(struct imsg_mbox_copy *, struct imsgev *); +static void handle_mbox_create(struct imsg_mbox_create *, struct imsgev *); +static void handle_mbox_delete(struct imsg_mbox_delete *, struct imsgev *); +static void handle_mbox_rename(struct imsg_mbox_rename *, struct imsgev *); +static void handle_mbox_list(struct imsgev *); +static int mailbox_name_valid(const char *); +static int mailbox_name_is_inbox(const char *); +static int select_mailbox_dir(const char *); +static int locate_message_file(const char *, off_t *, char *, size_t); +static int open_message_file(const char *); +static int read_message_header(const char *, char **, uint32_t *); +static int read_message_body(const char *, int, size_t, const char *, + char **, uint32_t *); +static int header_field_name_matches(const char *, size_t, const char *); +static int read_message_header_fields(const char *, const char *, int, + char **, uint32_t *); +static void build_flags_string(const char *, const char *, char *, + size_t); +static int extract_header_field(const char *, size_t, const char *, + char **, size_t *); +static int envbuf_append(char *, size_t, size_t *, const char *, size_t); +static int envbuf_append_str(char *, size_t, size_t *, const char *); +static int envbuf_append_nstring(char *, size_t, size_t *, const char *, + size_t); +static int envbuf_append_one_address(char *, size_t, size_t *, + const char *, size_t); +static int envbuf_append_address_list(char *, size_t, size_t *, + const char *, size_t); +static int append_field_nstring(char *, size_t, size_t *, const char *, + uint32_t, const char *); +static int build_envelope(const char *, char **, uint32_t *); +static int find_header_body_split(const char *, size_t, size_t *); +static int mime_is_tspecial(char); +static int mime_read_token_or_qstring(const char *, size_t, size_t *, + char *, size_t); +static void mime_str_upper(char *); +static int parse_content_type(const char *, size_t, char *, size_t, + char *, size_t, char *, size_t, char *, size_t, int *); +static int split_multipart(const char *, size_t, const char *, + size_t *, size_t *, int *, int); +static int build_body_structure(int, int *, const char *, size_t, + const char *, size_t, char *, size_t, size_t *); +static int build_bodystructure(const char *, char **, uint32_t *); +static int parse_section_part(const char *, int *, int); +static int find_mime_part(int, const char *, size_t, const char *, + size_t, const int *, int, const char **, size_t *); +static int locate_mime_part(const char *, size_t, const char *, size_t, + const int *, int, const char **, size_t *); +static void apply_partial_range(const char *, size_t, int, uint32_t, + uint32_t, const char **, size_t *); +static int extract_mime_part(const char *, const int *, int, int, + uint32_t, uint32_t, char **, uint32_t *); + +/* + * The maildir+index format resolved in openimap-storage-backend.md's + * "Recommendation" section: stock maildir (tmp/new/cur) for message + * bodies, plus one small line-oriented text index per mailbox holding + * UIDVALIDITY/UIDNEXT and the UID<->basename(<->keywords) map. v1 is + * INBOX-only (openimap-v1-dispatch.md), and that index lives directly in + * the maildir root store.c is now chdir'd into -- a sibling of tmp/new/ + * cur, not inside any of them. Name is this implementation's own choice + * (the design doc never picked one): plain, `ls`-visible, matching the + * document's own stated preference for boring/inspectable formats over + * hidden dotfiles. + */ +#define STORE_INDEX_NAME "imapd.index" +#define STORE_INDEX_TMP_NAME "imapd.index.tmp" +#define STORE_INDEX_LINE_MAX 1024 /* matches auth.c's cred_lookup() + * line-buffer precedent for the same + * kind of small, personal-scope flat + * text file */ + +/* + * In-memory copy of one mailbox's index while a single IMSG_MBOX_SELECT + * (or, later, any other mutating mbox op) is being handled -- built fresh + * from the on-disk file, mutated, and rewritten in full per the design + * doc's resolved "whole file rewritten to a temp file and rename(2)'d + * over on every mutation" rule, then discarded. Never kept around between + * imsg messages. + */ +struct mbox_index { + uint32_t uidvalidity; + uint32_t uidnext; + uint64_t highestmodseq; /* RFC 7162 SS3.1: per-mailbox highest + * mod-sequence, persisted as the + * index header's third field (see + * index_load()/index_save()). v1's + * only mailbox always supports this + * (there is no on-disk format that + * predates it in a real deployment of + * this not-yet-released server), so + * the NOMODSEQ response code + * (RFC 7162 SS3.1.2.2) is simply + * unreachable in this implementation. */ + char **lines; /* raw "UID:basename:keywords:MODSEQ" + * lines, no trailing newline, one + * malloc(3) each -- the MODSEQ field + * is this pass's addition; see + * index_parse_line() */ + size_t nlines; + size_t cap; +}; + +/* + * One index message line, parsed. Introduced this pass (RFC 7162) to stop + * handle_mbox_fetch()/handle_mbox_store()/handle_mbox_expunge() from each + * hand-rolling their own "UID:basename:keywords" strchr() chain -- adding + * a fourth colon-delimited field (MODSEQ) to every one of those independently + * would have tripled the size of this change and the chance of one of them + * drifting out of sync with the on-disk format. basename/keywords are + * fixed-size copies (same 512-byte bound index_load()'s own STORE_INDEX_ + * LINE_MAX line buffer already implies is generous for either field), not + * pointers into the original line -- callers are free to mutate or discard + * the line string after parsing. + * + * Backward compatibility: the MODSEQ field is new this pass. A line with + * only three colon-delimited fields (written by a pre-CONDSTORE build of + * this server) parses successfully with modseq defaulted to 1 -- this + * server has never had a real deployment to be compatible *with*, so this + * is a defensive nicety, not a migration guarantee this project is making + * any promise about. + */ +struct index_rec { + uint32_t uid; + char basename[512]; + char keywords[512]; + uint64_t modseq; +}; + +static int index_load(int, struct mbox_index *); +static int index_has_basename(struct mbox_index *, const char *); +static int index_append(struct mbox_index *, uint32_t, const char *); +static int index_save(struct mbox_index *); +static void index_free(struct mbox_index *); +static int index_parse_line(const char *, struct index_rec *); +static uint32_t index_max_uid(struct mbox_index *); +static void send_vanished_range(struct mbox_index *, uint32_t, uint32_t, + struct imsgev *); +static int refresh_index(struct mbox_index *, int); +static void handle_mbox_idle_refresh(struct imsgev *); + +__dead void +store_main(void) +{ + struct imsgbuf ibuf3; + struct imsg imsg; + ssize_t n; + struct imsg_store_init init; + int peer_fd; + gid_t gid; + + /* store children don't use imapd.conf directly -- no struct + * openimap_config * parameter at all, same as listener_main()/ + * auth_main() now (see main.c's comment and imapd.h's prototype + * comment). Everything store needs (uid/gid/spool_root) arrives via + * IMSG_STORE_INIT below instead. */ + + if (imsgbuf_init(&ibuf3, 3) == -1) + fatal("imsgbuf_init"); + imsgbuf_allow_fdpass(&ibuf3); /* receives the fd-passed + * IMSG_SETUP_PEER peer fd below -- see + * imsgev.c's imsgev_init() comment. */ + + /* + * IMSG_STORE_INIT must be the first message read -- this is the + * one departure from the generic setup_recv_*() helpers used by + * listener.c/auth.c: a per-session store child's privilege target + * is a runtime value, not fixed by its role, so it needs this + * extra message before it can even chroot(). See the design doc's + * "Departure #2" for the "why". + */ + /* + * imsg_get() before imsgbuf_read() -- see imsgev.c's + * setup_recv_one_peer() header comment for the real deadlock this + * ordering caused elsewhere (two back-to-back sends on the same + * SOCK_STREAM socketpair coalescing into one recvmsg(), leaving a + * complete message sitting unread in userspace while a later, + * naive imsgbuf_read()-first loop blocks forever for bytes that + * already arrived). Applied here defensively for the same reason + * auth.c's IMSG_AUTH_INIT read was: nothing currently forces + * IMSG_STORE_INIT and this session's IMSG_SETUP_PEER into separate + * reads. + */ + for (;;) { + if ((n = imsg_get(&ibuf3, &imsg)) == -1) + fatal("imsg_get"); + if (n != 0) + break; + if ((n = imsgbuf_read(&ibuf3)) == -1) + fatal("imsgbuf_read"); + if (n == 0) + fatalx("store: parent closed channel before INIT"); + } + if (imsg_get_type(&imsg) != IMSG_STORE_INIT) + fatalx("store: expected IMSG_STORE_INIT, got %d", + imsg_get_type(&imsg)); + if (imsg_get_data(&imsg, &init, sizeof(init)) == -1) + fatalx("store: bad IMSG_STORE_INIT payload"); + imsg_free(&imsg); + + session_id = init.session_id; + bodystructure_read_max = init.bodystructure_read_max; + + if (chroot(init.spool_root) == -1) + fatal("chroot %s", init.spool_root); + if (chdir("/") == -1) + fatal("chdir /"); + + /* + * The actual privilege drop this whole per-session design exists + * for: setresuid/setresgid to the AUTHENTICATED USER's own uid/gid + * (not a fixed service account), so a bug in this process's + * FETCH/STORE handling -- once that exists -- is confined by the + * kernel's own uid permission checks to this one user's files, per + * openimap-privsep-design.md's fork-per-session resolution. + */ + gid = init.gid; + if (setgroups(1, &gid) == -1 || + setresgid(init.gid, init.gid, init.gid) == -1 || + setresuid(init.uid, init.uid, init.uid) == -1) + fatal("session %u: cannot drop privileges to uid %u gid %u", + session_id, init.uid, init.gid); + + /* Now privilege-dropped: finish the handshake exactly like a + * boot-time child (one peer -- listener -- then SETUP_DONE+ack). */ + peer_fd = setup_recv_one_peer(&ibuf3); + setup_recv_done_and_ack(&ibuf3); + + event_init(); + imsgev_init(&iev_listener, peer_fd, store_dispatch, NULL); + + /* + * unveil() scoped to THIS session's own mailbox subdirectory, not + * the whole (shared, multi-user) chroot -- tightened this pass now + * that init.maildir actually arrives here (see imapd.h's + * imsg_store_init comment: it didn't, until now). The chroot alone + * was already shared across every store child regardless of user + * (one global spool_root from imapd.conf); unveil is this + * project's own established tool for narrowing a process's + * filesystem view further than chroot can (auth.c does the same + * thing -- unveils only its one credential-file path, not its + * whole chroot directory) and the fork-per-session privilege drop + * this file's header comment describes is exactly the kind of + * "confine a bug in this one session's handling to this one + * session's own files" reasoning unveil is for. This is this + * implementation's own judgment call, not something openimap- + * privsep-design.md's unveil discussion spells out explicitly -- + * that document argues for keeping unveil() at all (vs. relying on + * chroot alone), not for which specific path to scope it to. + */ + { + char unveil_path[sizeof(init.maildir) + 1]; + + if (snprintf(unveil_path, sizeof(unveil_path), "/%s", + init.maildir) >= (int)sizeof(unveil_path)) + fatalx("session %u: maildir path too long: %s", + session_id, init.maildir); + if (unveil(unveil_path, "rwc") == -1) + fatal("unveil %s", unveil_path); + + /* + * chdir() into the unveiled directory now, once, so every + * IMSG_MBOX_* handler below can use bare relative paths + * ("imapd.index", "new/", ...) instead of + * threading init.maildir through every single one of them. + * unveil(2)'s restriction is on the resolved path, not the + * literal argument a syscall is given, so a relative open() + * against this cwd still resolves inside the path just + * unveiled above -- same reasoning parent.c/auth.c/listener.c + * already rely on after their own chroot()+chdir("/") pairs, + * just one directory deeper here. + */ + if (chdir(unveil_path) == -1) + fatal("chdir %s", unveil_path); + } + if (unveil(NULL, NULL) == -1) + fatal("unveil lock"); + + /* + * Resolved pledge string (openimap-privsep-design.md, checked + * against smtpd's queue.c and the fork-per-session concurrency + * argument in openimap-storage-backend.md): flock for the index + * read-modify-write cycle, rpath/wpath/cpath for maildir's + * rename(2)-based delivery and flag-suffix changes, no fattr + * (nothing here calls chmod/utimes/chflags). + */ +#ifdef __OpenBSD__ + if (pledge("stdio rpath wpath cpath flock recvfd sendfd", NULL) + == -1) + fatal("pledge"); +#endif + + log_debug("session %u: store ready (uid %u, gid %u, spool %s, " + "maildir %s)", session_id, init.uid, init.gid, init.spool_root, + init.maildir); + + event_dispatch(); + fatalx("store: exited event loop"); +} + +/* + * Reads the index at STORE_INDEX_NAME (already open on fd, already + * flock(2)'d LOCK_EX by the caller) into *idx. An empty file (including + * one that was just created by handle_mbox_select()'s O_CREAT, i.e. this + * mailbox has never been indexed before) is not an error -- it means a + * fresh UIDVALIDITY/UIDNEXT get assigned, per RFC 9051 SS2.3.1.1: "A good + * UIDVALIDITY value to use is a 32-bit representation of the current + * date/time when the value is assigned: this ensures that the value is + * unique and always increases." UIDNEXT starts at 1, since UIDs are + * "unsigned non-zero" (same section). + * + * Reads via a dup(2)'d fd wrapped in stdio -- fclose() below closes the + * dup, not the caller's original fd, so the caller's flock(2) (which is + * associated with the open file description, and would be dropped by + * closing every fd referencing it) survives this function returning. + */ +static int +index_load(int fd, struct mbox_index *idx) +{ + FILE *fp; + char line[STORE_INDEX_LINE_MAX]; + int dupfd; + int first = 1; + + memset(idx, 0, sizeof(*idx)); + + if (lseek(fd, 0, SEEK_SET) == -1) { + log_warn("session %u: lseek %s", session_id, STORE_INDEX_NAME); + return (-1); + } + if ((dupfd = dup(fd)) == -1) { + log_warn("session %u: dup %s", session_id, STORE_INDEX_NAME); + return (-1); + } + if ((fp = fdopen(dupfd, "r")) == NULL) { + log_warn("session %u: fdopen %s", session_id, STORE_INDEX_NAME); + close(dupfd); + return (-1); + } + + while (fgets(line, sizeof(line), fp) != NULL) { + line[strcspn(line, "\n")] = '\0'; + if (line[0] == '\0') + continue; + + if (first) { + char *colon, *colon2, *ep; + + first = 0; + if ((colon = strchr(line, ':')) == NULL) { + log_warnx("session %u: malformed index " + "header: %s", session_id, line); + fclose(fp); + return (-1); + } + *colon = '\0'; + errno = 0; + idx->uidvalidity = (uint32_t)strtoul(line, &ep, 10); + if (*ep != '\0' || errno != 0) { + log_warnx("session %u: malformed " + "UIDVALIDITY: %s", session_id, line); + fclose(fp); + return (-1); + } + + /* + * RFC 7162 addition: an optional third header field, + * HIGHESTMODSEQ. colon2 == NULL means a header + * written before this pass (two fields only) -- + * tolerated, not an error, per this function's header + * comment on backward compatibility; defaults to 1, + * the same "start of the world" value a brand-new + * index gets below. + */ + if ((colon2 = strchr(colon + 1, ':')) != NULL) + *colon2 = '\0'; + + errno = 0; + idx->uidnext = (uint32_t)strtoul(colon + 1, &ep, 10); + if (*ep != '\0' || errno != 0) { + log_warnx("session %u: malformed UIDNEXT: %s", + session_id, colon + 1); + fclose(fp); + return (-1); + } + + if (colon2 != NULL) { + errno = 0; + idx->highestmodseq = strtoull(colon2 + 1, &ep, + 10); + if (*ep != '\0' || errno != 0) { + log_warnx("session %u: malformed " + "HIGHESTMODSEQ: %s", session_id, + colon2 + 1); + fclose(fp); + return (-1); + } + } else + idx->highestmodseq = 1; + continue; + } + + if (idx->nlines == idx->cap) { + size_t newcap = (idx->cap == 0) ? 16 : idx->cap * 2; + char **newlines = reallocarray(idx->lines, newcap, + sizeof(*idx->lines)); + + if (newlines == NULL) { + log_warn("session %u: reallocarray index", + session_id); + fclose(fp); + return (-1); + } + idx->lines = newlines; + idx->cap = newcap; + } + if ((idx->lines[idx->nlines] = strdup(line)) == NULL) { + log_warn("session %u: strdup index line", session_id); + fclose(fp); + return (-1); + } + idx->nlines++; + } + if (ferror(fp)) { + log_warn("session %u: fgets %s", session_id, STORE_INDEX_NAME); + fclose(fp); + return (-1); + } + fclose(fp); + + if (first) { + idx->uidvalidity = (uint32_t)time(NULL); + idx->uidnext = 1; + idx->highestmodseq = 1; + } + + return (0); +} + +/* + * Parses one "UID:basename:keywords[:MODSEQ]" index line -- see struct + * index_rec's comment above for why this exists (one shared parser instead + * of three-plus independent strchr() chains). Returns -1 (logged) on a + * corrupt line, matching the existing "log and skip/keep, don't fail the + * whole operation over one bad line" tolerance every caller below already + * had before this pass. + */ +static int +index_parse_line(const char *line, struct index_rec *rec) +{ + const char *p, *q, *r; + char *ep; + + memset(rec, 0, sizeof(*rec)); + + errno = 0; + rec->uid = (uint32_t)strtoul(line, &ep, 10); + if (*ep != ':') { + log_warnx("session %u: corrupt index line: %s", session_id, + line); + return (-1); + } + p = ep + 1; + if ((q = strchr(p, ':')) == NULL) { + log_warnx("session %u: corrupt index line: %s", session_id, + line); + return (-1); + } + if ((size_t)(q - p) >= sizeof(rec->basename)) { + log_warnx("session %u: basename too long in index line", + session_id); + return (-1); + } + memcpy(rec->basename, p, (size_t)(q - p)); + rec->basename[q - p] = '\0'; + + p = q + 1; + if ((r = strchr(p, ':')) != NULL) { + /* keywords field ends at the MODSEQ separator */ + if ((size_t)(r - p) >= sizeof(rec->keywords)) { + log_warnx("session %u: keywords too long in index " + "line", session_id); + return (-1); + } + memcpy(rec->keywords, p, (size_t)(r - p)); + rec->keywords[r - p] = '\0'; + + errno = 0; + rec->modseq = strtoull(r + 1, &ep, 10); + if (*ep != '\0' || errno != 0) { + log_warnx("session %u: malformed per-message MODSEQ " + "in index line: %s", session_id, line); + return (-1); + } + } else { + /* no MODSEQ field -- pre-CONDSTORE line, see struct + * index_rec's backward-compatibility comment */ + strlcpy(rec->keywords, p, sizeof(rec->keywords)); + rec->modseq = 1; + } + + return (0); +} + +/* + * Highest UID currently assigned to a *present* message in idx, or 0 if + * the mailbox has never held one -- idx->lines is maintained in ascending + * UID order (see struct mbox_index's own comment), so the last line has + * the highest UID. This is "*" for UID SEARCH (SS9's `seq-number = nz- + * number / "*"`, "*" meaning "the largest number in use") and, this pass, + * for UID FETCH/UID STORE/UID EXPUNGE too -- deliberately NOT idx->uidnext + * - 1, which is the highest UID *ever assigned*, not the highest UID of a + * message that still exists (those differ whenever the highest-UID + * message has since been expunged). Factored out of what was previously + * inline duplicated logic in handle_mbox_search() alone; now shared by + * every UID-space "*" resolution in this file. + */ +static uint32_t +index_max_uid(struct mbox_index *idx) +{ + const char *line; + char *ep; + uint32_t v; + + if (idx->nlines == 0) + return (0); + + line = idx->lines[idx->nlines - 1]; + errno = 0; + v = (uint32_t)strtoul(line, &ep, 10); + if (*ep != ':') + return (0); /* corrupt last line -- treat as "no UIDs in + * use" rather than guessing; a hi_is_star + * range simply won't match anything in that + * case, which is safer than an arbitrary + * wrong bound */ + return (v); +} + +/* + * Reports every UID in [lo, hi] not currently present in idx as one or + * more IMSG_MBOX_SELECT_VANISHED ranges -- the same "walk the present- + * message list once, report gaps" computation qresync_send_resync() uses + * for QRESYNC SELECT resync (see that function's own comment and + * imsg_mbox_select_vanished's SS5.1 minimal-state comment in imapd.h), + * factored out as an independent, simpler helper (no interleaved FETCH- + * resync emission) for RFC 7162 SS3.2.6's VANISHED UID FETCH modifier, + * which just needs the gap-reporting half against a UID FETCH's own + * seq_lo/seq_hi range rather than QRESYNC's known-uids range. Kept + * separate from qresync_send_resync() rather than sharing code with it, + * to avoid risking a regression in that already-working, RFC-example- + * verified path for the sake of a few dozen shared lines. + */ +static void +send_vanished_range(struct mbox_index *idx, uint32_t lo, uint32_t hi, + struct imsgev *iev) +{ + uint32_t want, i; + + if (hi < lo) + return; + + want = lo; + for (i = 0; i < idx->nlines && want <= hi; i++) { + struct index_rec rec; + + if (index_parse_line(idx->lines[i], &rec) == -1) + continue; + if (rec.uid < want) + continue; + if (rec.uid > hi) + break; + + if (rec.uid > want) { + struct imsg_mbox_select_vanished van; + + memset(&van, 0, sizeof(van)); + van.uid_lo = want; + van.uid_hi = rec.uid - 1; + if (imsg_compose(&iev->ibuf, IMSG_MBOX_SELECT_VANISHED, + 0, 0, -1, &van, sizeof(van)) == -1) + log_warn("session %u: imsg_compose " + "IMSG_MBOX_SELECT_VANISHED", session_id); + } + + want = rec.uid + 1; + } + + if (want <= hi) { + struct imsg_mbox_select_vanished van; + + memset(&van, 0, sizeof(van)); + van.uid_lo = want; + van.uid_hi = hi; + if (imsg_compose(&iev->ibuf, IMSG_MBOX_SELECT_VANISHED, 0, 0, + -1, &van, sizeof(van)) == -1) + log_warn("session %u: imsg_compose " + "IMSG_MBOX_SELECT_VANISHED", session_id); + } +} + +/* + * Linear scan for a basename already present in the index's UID<->basename + * map -- O(n) per lookup, O(n^2) total when called once per new/ entry + * during a scan, same "fine for v1's personal-use, modest-mailbox-size + * scope" reasoning openimap-storage-backend.md already applies to the + * full-file-rewrite cost of index_save() below; revisit only if that + * assumption stops holding. + */ +static int +index_has_basename(struct mbox_index *idx, const char *basename) +{ + size_t i; + + for (i = 0; i < idx->nlines; i++) { + const char *p = idx->lines[i]; + const char *basefield, *end; + size_t len; + + if ((basefield = strchr(p, ':')) == NULL) + continue; + basefield++; + end = strchr(basefield, ':'); + len = (end != NULL) ? (size_t)(end - basefield) : + strlen(basefield); + if (strlen(basename) == len && + strncmp(basefield, basename, len) == 0) + return (1); + } + return (0); +} + +/* + * Appends one new "UID:basename::MODSEQ" record (no keywords -- a freshly + * noticed message has none yet) to the in-memory index. Does not touch + * idx->uidnext itself; the caller owns incrementing it, since the caller + * (handle_mbox_select()) is the one that knows whether this was the last + * new/ entry or more remain. + * + * RFC 7162 addition: bumps idx->highestmodseq and assigns the new value to + * this message, per SS3.1's "When a message is appended to a mailbox (via + * the IMAP APPEND command, COPY to the mailbox, or using an external + * mechanism), the server generates a new modification sequence that is + * higher than the highest modification sequence of all messages in the + * mailbox and assigns it to the appended message." Both of this function's + * two callers (handle_mbox_select()'s new/-scan and handle_mbox_append()) + * are exactly "external mechanism" and "APPEND command" respectively, so + * this single shared bump covers both per-message, even when a SELECT + * discovers several externally-delivered messages in the same scan (each + * gets its own distinct value, not one shared across the batch -- v1's own + * choice where the RFC doesn't say either way for multiple simultaneous + * arrivals; see this function's caller-side comments for why a STORE or + * EXPUNGE affecting several messages at once is different and shares one + * value instead). + */ +static int +index_append(struct mbox_index *idx, uint32_t uid, const char *basename) +{ + char line[STORE_INDEX_LINE_MAX]; + int len; + + /* + * F10 fix (defense in depth): refuse a basename containing the + * index field separator ':' or a newline. refresh_index() already + * pre-skips these when scanning new/; server-generated basenames + * (APPEND/COPY) never contain them. + */ + if (strpbrk(basename, ":\r\n") != NULL) { + log_warnx("session %u: refusing index entry with unsafe " + "basename: %s", session_id, basename); + return (-1); + } + + idx->highestmodseq++; + + len = snprintf(line, sizeof(line), "%u:%s::%llu", uid, basename, + (unsigned long long)idx->highestmodseq); + if (len < 0 || (size_t)len >= sizeof(line)) { + log_warnx("session %u: index line too long for %s", + session_id, basename); + return (-1); + } + + if (idx->nlines == idx->cap) { + size_t newcap = (idx->cap == 0) ? 16 : idx->cap * 2; + char **newlines = reallocarray(idx->lines, newcap, + sizeof(*idx->lines)); + + if (newlines == NULL) { + log_warn("session %u: reallocarray index", session_id); + return (-1); + } + idx->lines = newlines; + idx->cap = newcap; + } + if ((idx->lines[idx->nlines] = strdup(line)) == NULL) { + log_warn("session %u: strdup index line", session_id); + return (-1); + } + idx->nlines++; + return (0); +} + +/* + * Rewrites the whole index to STORE_INDEX_TMP_NAME and rename(2)s it over + * STORE_INDEX_NAME -- openimap-storage-backend.md's resolved mutation + * rule, and the same atomicity guarantee (confirmed against the real + * rename(2) man page during that design pass: "an instance of the + * destination name will always exist even if the system crashes + * mid-rename") already relied on for maildir delivery and flag-suffix + * renames elsewhere in this design -- so a reader can never observe a + * torn or partially-written index, only the old version or the new one. + */ +static int +index_save(struct mbox_index *idx) +{ + FILE *fp; + int fd; + size_t i; + + /* + * F9 fix: create the temp exclusively so a pre-planted symlink at + * this fixed path cannot be followed on open. Unlink any stale temp + * left by a previous crash first (it is ours to replace), ignoring + * ENOENT. + */ + if (unlink(STORE_INDEX_TMP_NAME) == -1 && errno != ENOENT) { + log_warn("session %u: unlink %s", session_id, + STORE_INDEX_TMP_NAME); + return (-1); + } + if ((fd = open(STORE_INDEX_TMP_NAME, O_WRONLY | O_CREAT | O_EXCL, + 0600)) == -1) { + log_warn("session %u: open %s", session_id, + STORE_INDEX_TMP_NAME); + return (-1); + } + if ((fp = fdopen(fd, "w")) == NULL) { + log_warn("session %u: fdopen %s", session_id, + STORE_INDEX_TMP_NAME); + close(fd); + return (-1); + } + + if (fprintf(fp, "%u:%u:%llu\n", idx->uidvalidity, idx->uidnext, + (unsigned long long)idx->highestmodseq) < 0) { + log_warnx("session %u: write index header failed", + session_id); + fclose(fp); + return (-1); + } + for (i = 0; i < idx->nlines; i++) { + if (fprintf(fp, "%s\n", idx->lines[i]) < 0) { + log_warnx("session %u: write index line failed", + session_id); + fclose(fp); + return (-1); + } + } + if (fclose(fp) != 0) { + log_warn("session %u: fclose %s", session_id, + STORE_INDEX_TMP_NAME); + return (-1); + } + + if (rename(STORE_INDEX_TMP_NAME, STORE_INDEX_NAME) == -1) { + log_warn("session %u: rename %s -> %s", session_id, + STORE_INDEX_TMP_NAME, STORE_INDEX_NAME); + return (-1); + } + return (0); +} + +static void +index_free(struct mbox_index *idx) +{ + size_t i; + + for (i = 0; i < idx->nlines; i++) + free(idx->lines[i]); + free(idx->lines); + memset(idx, 0, sizeof(*idx)); +} + +/* + * IMSG_MBOX_SELECT handling. v1 scope: INBOX is the only mailbox that + * exists at all (openimap-v1-dispatch.md's SELECT row; see listener.c's + * cmd_namespace() comment for why a real multi-mailbox hierarchy isn't + * designed yet) -- anything else is answered ok=0 (listener.c turns that + * into a tagged NO, RFC 9051 SS6.3.2's own "no such mailbox" result), + * matched case-insensitively per RFC 9051 SS5.1 ("The special name INBOX + * is case-insensitive"). + * + * On a real INBOX select: open/flock/parse the index, scan new/ for + * basenames the index doesn't know about yet and assign each one the + * next UID (openimap-storage-backend.md: "A store child does still need + * to notice externally-delivered files in new/ ... and assign them a UID + * via the index"), rewrite the index, and report back exists/uidvalidity/ + * uidnext. Deliberately does NOT move anything from new/ to cur/ -- RFC + * 9051 deprecated \Recent (message flag), the untagged RECENT response, + * and RECENT STATUS entirely (SS8 erratum note 12), and this is a pure + * IMAP4rev2 server (no IMAP4rev1 back-compat), so there is no protocol + * reason left to track "recent" status via a new/->cur/ move at all -- + * that migration is purely a maildir-hygiene/interop concern, tied to + * \Seen handling once FETCH/STORE exist, not something SELECT needs to + * do itself. + * + * reply->exists is currently exactly "every basename ever indexed" -- + * correct for now since nothing in this codebase can remove an index + * entry yet (EXPUNGE is still a stub_not_implemented() in listener.c); + * will need to become "indexed minus expunged" once that changes. + * + * RFC 7162 additions this pass: reply.highestmodseq is always populated + * (see imsg_mbox_selected's comment in imapd.h). If req->qresync is set + * and req->qresync_uidvalidity matches the mailbox's real UIDVALIDITY, + * performs the QRESYNC resync described in RFC 7162 SS3.2.5.1 -- streams + * IMSG_MBOX_SELECT_VANISHED range(s) for the requested known-uids range (or + * the default 1:) followed by IMSG_MBOX_FETCH_META for every + * still-present message in that range whose mod-sequence exceeds req-> + * qresync_modseq -- via qresync_send_resync(), before composing the + * terminal IMSG_MBOX_SELECTED. A UIDVALIDITY mismatch is not an error + * (SS3.2.5: "the server MUST ignore the remaining parameters and behave as + * if no dynamic message data changed") -- resync is simply skipped and a + * normal SELECT reply goes out. + */ +static void +qresync_send_resync(struct imsg_mbox_select *req, struct mbox_index *idx, + struct imsgev *iev) +{ + uint32_t uid_lo, uid_hi, want, i; + + if (req->qresync_has_uids) { + uid_lo = req->qresync_uid_lo; + uid_hi = req->qresync_uid_hi; + } else { + /* SS3.2.5.1: "the server acts as if the client has specified + * '1:' ... If the mailbox is empty and never had any + * messages in it, then lack of the list of UIDs is + * interpreted as an empty set of UIDs." maxuid is uidnext-1; + * uidnext == 1 means no message has ever been assigned a UID + * (index_load()'s fresh-file default), i.e. exactly that + * empty case. */ + if (idx->uidnext <= 1) + return; + uid_lo = 1; + uid_hi = idx->uidnext - 1; + } + if (uid_hi < uid_lo) + return; /* empty requested range -- nothing to do */ + + /* + * Single forward pass over idx->lines (bounded by mailbox size, not + * by the requested UID range's numeric span -- see imsg_mbox_ + * select_vanished's comment in imapd.h for why iterating the + * range itself, one UID at a time, isn't safe). want tracks the + * lowest UID not yet accounted for; any present-message UID greater + * than want means everything in [want, that UID - 1] vanished. + */ + want = uid_lo; + for (i = 0; i < idx->nlines && want <= uid_hi; i++) { + struct index_rec rec; + + if (index_parse_line(idx->lines[i], &rec) == -1) + continue; /* corrupt line, already logged -- + * conservatively not reported either + * way, same tolerance handle_mbox_ + * fetch() etc. already apply */ + if (rec.uid < want) + continue; /* below the requested range, or + * already accounted for */ + if (rec.uid > uid_hi) + break; + + if (rec.uid > want) { + struct imsg_mbox_select_vanished van; + + memset(&van, 0, sizeof(van)); + van.uid_lo = want; + van.uid_hi = rec.uid - 1; + if (imsg_compose(&iev->ibuf, IMSG_MBOX_SELECT_VANISHED, + 0, 0, -1, &van, sizeof(van)) == -1) + log_warn("session %u: imsg_compose " + "IMSG_MBOX_SELECT_VANISHED", session_id); + } + + if (rec.modseq > req->qresync_modseq) { + struct imsg_mbox_fetch_meta meta; + char suffix[64]; + off_t size; + + memset(&meta, 0, sizeof(meta)); + meta.seqno = i + 1; + meta.uid = rec.uid; + meta.modseq = rec.modseq; + if (locate_message_file(rec.basename, &size, suffix, + sizeof(suffix)) == 0) { + build_flags_string(suffix, rec.keywords, + meta.flags, sizeof(meta.flags)); + } + if (imsg_compose(&iev->ibuf, IMSG_MBOX_FETCH_META, 0, + 0, -1, &meta, sizeof(meta)) == -1) + log_warn("session %u: imsg_compose " + "IMSG_MBOX_FETCH_META (qresync)", + session_id); + } + + want = rec.uid + 1; + } + + if (want <= uid_hi) { + struct imsg_mbox_select_vanished van; + + memset(&van, 0, sizeof(van)); + van.uid_lo = want; + van.uid_hi = uid_hi; + if (imsg_compose(&iev->ibuf, IMSG_MBOX_SELECT_VANISHED, 0, 0, + -1, &van, sizeof(van)) == -1) + log_warn("session %u: imsg_compose " + "IMSG_MBOX_SELECT_VANISHED", session_id); + } +} + +/* + * Loads the mailbox index (STORE_INDEX_NAME, already open on fd and + * already flock(2)'d LOCK_EX by the caller) and discovers any message + * files sitting in new/ that the index doesn't know about yet -- mail + * delivered directly into the maildir by something other than this + * daemon's own APPEND (migrating new/ to cur/ isn't done here or anywhere + * else in this codebase yet), appending each with the next UID and + * re-saving the index in full. Originally handle_mbox_select()'s own + * inline logic; factored out this pass so handle_mbox_idle_refresh() + * (RFC 9051 SS6.3.13) can be exactly as authoritative as a fresh SELECT + * would be, rather than just re-reading whatever the index file happened + * to say on disk before this call -- including picking up mail placed + * directly in new/ since the last time anything looked, regardless of + * which of the two callers triggered this particular refresh. + * + * On success, idx is populated and the caller owns it (index_free() when + * done, same as index_load() itself). On failure, idx has already been + * index_free()'d by this function -- the caller must not call index_free() + * itself in that case, matching every existing "index_free() already ran + * on this error path" precedent elsewhere in this file. + */ +static int +refresh_index(struct mbox_index *idx, int fd) +{ + DIR *dp; + struct dirent *de; + + if (index_load(fd, idx) == -1) + return (-1); + + dp = opendir("new"); + if (dp == NULL) { + if (errno == ENOENT) + return (0); /* no new/ yet on a never-used + * mailbox -- not an error, just + * nothing to discover */ + log_warn("session %u: opendir new", session_id); + index_free(idx); + return (-1); + } + while ((de = readdir(dp)) != NULL) { + if (de->d_name[0] == '.') + continue; /* ".", "..", and dotfiles -- maildir + * delivery never creates the latter + * in new/ */ + /* + * F10 fix: never index a new/ filename containing the index + * field separator ':' or a newline -- it would corrupt the + * "uid:basename:keywords:modseq" line (splitting the basename + * field, or a newline injecting a spurious record). A well- + * formed maildir new/ name never contains either. + */ + if (strpbrk(de->d_name, ":\r\n") != NULL) { + log_warnx("session %u: skipping new/ file with unsafe " + "name (contains ':' or newline): %s", session_id, + de->d_name); + continue; + } + if (index_has_basename(idx, de->d_name)) + continue; + if (index_append(idx, idx->uidnext, de->d_name) == -1) { + closedir(dp); + index_free(idx); + return (-1); + } + idx->uidnext++; + } + closedir(dp); + + if (index_save(idx) == -1) { + index_free(idx); + return (-1); + } + return (0); +} + +/* + * RFC 9051 SS6.3.4-SS6.3.6/SS6.3.9 additions (flat multi-mailbox support, + * see docs/openimap-storage-backend.md's "Open items" #10). Re-validated + * here independently of listener.c's own client-side check on the same + * rules -- defense in depth across the privsep boundary, the same "don't + * trust the other side of an imsg channel" posture every other mailbox- + * name-carrying request in this file already gets. + */ +static int +mailbox_name_valid(const char *name) +{ + size_t i, len; + + len = strlen(name); + if (len == 0 || len >= MBOX_NAME_MAX) + return (0); + for (i = 0; i < len; i++) { + unsigned char c = (unsigned char)name[i]; + + /* + * RFC 9051 SS5.1.1: hierarchy levels are separated by a + * single reserved delimiter character ("/" -- see docs/ + * openimap-storage-backend.md item 9). v1 is flat, so a + * name containing it can never be created or addressed -- + * not a hard spec violation to refuse, since SS5.1.1's + * hierarchy support is itself conditional ("if it is + * desired to export hierarchical mailbox names"). + */ + if (c == '/') + return (0); + /* + * SS5.1 point 2: "CTL and other non-graphic characters... + * Servers MAY refuse to create mailbox names containing + * Unicode CTL characters." Taking that MAY for the ASCII + * C0/DEL range; full Net-Unicode validation is an + * explicitly accepted gap (see the design doc). + */ + if (c < 0x20 || c == 0x7f) + return (0); + } + + /* + * Reserved: "tmp"/"new"/"cur" are INBOX's own maildir internals, + * living as siblings of any named-mailbox subdirectory at the same + * level (the session's maildir root). Refusing these as mailbox + * names outright, at validation time, closes a real hazard rather + * than just working around it at LIST-enumeration time: without + * this, CREATE "tmp" issued before INBOX's own tmp/ has ever been + * lazily created (ensure_maildir_dirs()) would succeed, and a + * later ensure_maildir_dirs("") for INBOX itself would then treat + * that user-created directory as INBOX's own tmp/ (EEXIST is + * tolerated there by design) -- silent data corruption across two + * unrelated mailboxes. Case-sensitive, matching maildir's own + * lowercase convention exactly (Courier maildir(5)). + */ + if (strcmp(name, "tmp") == 0 || strcmp(name, "new") == 0 || + strcmp(name, "cur") == 0) + return (0); + + /* + * F3/F4 fix: also reject "." and ".." -- otherwise DELETE "." + * resolves to the maildir root and destroys INBOX (unveil does not + * contain ".", which stays inside the maildir) -- and reject the + * on-disk index filenames, since a mailbox directory colliding with + * them makes INBOX unusable. + */ + if (strcmp(name, ".") == 0 || strcmp(name, "..") == 0) + return (0); + if (strcmp(name, STORE_INDEX_NAME) == 0 || + strcmp(name, STORE_INDEX_TMP_NAME) == 0) + return (0); + + return (1); +} + +static int +mailbox_name_is_inbox(const char *name) +{ + return (strcasecmp(name, "INBOX") == 0); +} + +/* + * Changes this store child's cwd to match `target` (the empty string for + * INBOX/the session's own maildir root, or an already mailbox_name_valid() + * name), tracking the transition in current_mailbox_dir so every other + * IMSG_MBOX_* handler can keep using bare relative paths unmodified. A + * no-op if `target` is already the currently selected mailbox (RFC 9051 + * has no prohibition against re-SELECTing the same mailbox). On failure + * (target doesn't exist, or a chdir(2) itself fails), restores cwd to + * wherever it was before this call, so a failed SELECT never leaves this + * store child sitting somewhere unexpected for whatever command the + * client sends next. + */ +static int +select_mailbox_dir(const char *target) +{ + if (strcmp(current_mailbox_dir, target) == 0) + return (0); + + if (current_mailbox_dir[0] != '\0' && chdir("..") == -1) { + log_warn("session %u: chdir .. (leaving %s)", session_id, + current_mailbox_dir); + return (-1); + } + + if (target[0] != '\0') { + struct stat st; + int exists; + + exists = (stat(target, &st) == 0 && S_ISDIR(st.st_mode)); + if (!exists || chdir(target) == -1) { + /* + * A missing/non-directory target is the common, + * expected "no such mailbox" case -- not warning- + * level. A chdir(2) failure on a target that does + * exist (permissions, ENOTDIR race, ...) is + * genuinely unexpected and worth a real log line. + */ + if (exists) + log_warn("session %u: chdir %s", session_id, + target); + if (current_mailbox_dir[0] != '\0' && + chdir(current_mailbox_dir) == -1) + log_warn("session %u: chdir %s (restoring " + "after failed select)", session_id, + current_mailbox_dir); + return (-1); + } + } + + strlcpy(current_mailbox_dir, target, sizeof(current_mailbox_dir)); + return (0); +} + +static void +handle_mbox_select(struct imsg_mbox_select *req, struct imsgev *iev) +{ + struct mbox_index idx; + struct imsg_mbox_selected reply; + int fd; + const char *target; + + memset(&reply, 0, sizeof(reply)); + + if (mailbox_name_is_inbox(req->mailbox)) + target = ""; + else if (mailbox_name_valid(req->mailbox)) + target = req->mailbox; + else { + log_debug("session %u: SELECT %s: invalid mailbox name", + session_id, req->mailbox); + reply.ok = 0; + goto send; + } + + if (select_mailbox_dir(target) == -1) { + log_debug("session %u: SELECT %s: no such mailbox", + session_id, req->mailbox); + reply.ok = 0; + goto send; + } + + if ((fd = open(STORE_INDEX_NAME, O_RDWR | O_CREAT, 0600)) == -1) { + log_warn("session %u: open %s", session_id, STORE_INDEX_NAME); + reply.ok = 0; + goto send; + } + /* + * flock(2) -- already in store.c's pledge string specifically for + * this: more than one store child can run for the same user at + * once (fork-per-session, not fork-per-user), so this read-modify- + * write cycle needs mutual exclusion across processes, not just + * within this one. + */ + if (flock(fd, LOCK_EX) == -1) { + log_warn("session %u: flock %s", session_id, STORE_INDEX_NAME); + close(fd); + reply.ok = 0; + goto send; + } + + if (refresh_index(&idx, fd) == -1) { + flock(fd, LOCK_UN); + close(fd); + reply.ok = 0; + goto send; + } + + reply.ok = 1; + reply.exists = (uint32_t)idx.nlines; + reply.uidvalidity = idx.uidvalidity; + reply.uidnext = idx.uidnext; + reply.highestmodseq = idx.highestmodseq; + + if (req->qresync && req->qresync_uidvalidity == idx.uidvalidity) + qresync_send_resync(req, &idx, iev); + + index_free(&idx); + flock(fd, LOCK_UN); + close(fd); + +send: + if (imsg_compose(&iev->ibuf, IMSG_MBOX_SELECTED, 0, 0, -1, &reply, + sizeof(reply)) == -1) + log_warn("session %u: imsg_compose IMSG_MBOX_SELECTED", + session_id); + imsgev_add(iev); +} + +/* + * IMSG_MBOX_IDLE_REFRESH (RFC 9051 SS6.3.13, IDLE): no request payload -- + * listener.c already knows which session's mailbox to check purely from + * which store child this arrived on. Streams one IMSG_MBOX_IDLE_UID per + * currently-existing message (ascending UID order, straight from refresh_ + * index()'s idx.lines, same order handle_mbox_select()/handle_mbox_fetch() + * already rely on), then one terminal IMSG_MBOX_IDLE_REFRESHED. + * + * v1 scope (confirmed with the user): EXISTS/EXPUNGE only, not per-message + * FETCH -- so unlike handle_mbox_fetch()/handle_mbox_search(), this has no + * per-message flags/modseq to report, just the bare UID each message + * currently has. listener.c does its own diffing against whatever UID list + * it cached from this session's last SELECT or last idle-refresh -- this + * function has no idea whether that's the first refresh of a new IDLE or a + * change-triggered one, and doesn't need to: it just reports current, + * authoritative state, identically either way. + * + * ok=0 on any failure (index open/lock/load error, same failure modes as + * handle_mbox_select()) -- listener.c treats that as "nothing to report + * this round" rather than tearing anything down, same leniency reasoning + * as a single skipped message elsewhere in this file: a mailbox that's + * momentarily locked by another one of this same user's sessions shouldn't + * abort the IDLE session over it. + */ +static void +handle_mbox_idle_refresh(struct imsgev *iev) +{ + struct mbox_index idx; + struct imsg_mbox_idle_refreshed reply; + struct imsg_mbox_idle_uid item; + int fd; + size_t i; + struct index_rec rec; + + memset(&reply, 0, sizeof(reply)); + + if ((fd = open(STORE_INDEX_NAME, O_RDWR | O_CREAT, 0600)) == -1) { + log_warn("session %u: open %s", session_id, STORE_INDEX_NAME); + goto send; + } + if (flock(fd, LOCK_EX) == -1) { + log_warn("session %u: flock %s", session_id, STORE_INDEX_NAME); + close(fd); + goto send; + } + if (refresh_index(&idx, fd) == -1) { + flock(fd, LOCK_UN); + close(fd); + goto send; + } + + for (i = 0; i < idx.nlines; i++) { + if (index_parse_line(idx.lines[i], &rec) == -1) + continue; /* same "skip a malformed line rather + * than fail the whole request" + * leniency index_parse_line()'s own + * callers already use elsewhere */ + memset(&item, 0, sizeof(item)); + item.uid = rec.uid; + if (imsg_compose(&iev->ibuf, IMSG_MBOX_IDLE_UID, 0, 0, -1, + &item, sizeof(item)) == -1) + log_warn("session %u: imsg_compose " + "IMSG_MBOX_IDLE_UID", session_id); + } + + reply.ok = 1; + reply.exists = (uint32_t)idx.nlines; + reply.uidvalidity = idx.uidvalidity; + reply.uidnext = idx.uidnext; + reply.highestmodseq = idx.highestmodseq; + + index_free(&idx); + flock(fd, LOCK_UN); + close(fd); + +send: + if (imsg_compose(&iev->ibuf, IMSG_MBOX_IDLE_REFRESHED, 0, 0, -1, + &reply, sizeof(reply)) == -1) + log_warn("session %u: imsg_compose IMSG_MBOX_IDLE_REFRESHED", + session_id); + imsgev_add(iev); +} + +/* + * Finds basename's on-disk message file: new/ exactly (the only + * place anything in this codebase currently puts a message -- see + * handle_mbox_select()'s comment on why new/->cur/ migration isn't done + * here), falling back to a ":2,*" prefix scan of cur/ for + * robustness against a message an external tool or administrator moved + * there by hand. Returns 0 and fills size_out and suffix_out (the matched + * file's maildir flag-suffix, e.g. ":2,FS", or "" if found in new/ -- + * unflagged, per maildir convention) on success, -1 if the message can't + * be found in either place (logged, not fatal -- handle_mbox_fetch() + * skips just that one message rather than failing the whole FETCH). + */ +static int +locate_message_file(const char *basename, off_t *size_out, char *suffix_out, + size_t suffix_out_size) +{ + struct stat st; + char path[600]; + + suffix_out[0] = '\0'; + + if (snprintf(path, sizeof(path), "new/%s", basename) >= + (int)sizeof(path)) { + log_warnx("session %u: basename too long: %s", session_id, + basename); + return (-1); + } + if (stat(path, &st) == 0) { + *size_out = st.st_size; + return (0); + } + if (errno != ENOENT) { + log_warn("session %u: stat %s", session_id, path); + return (-1); + } + + { + DIR *dp; + struct dirent *de; + size_t baselen = strlen(basename); + int rv = -1; + + if ((dp = opendir("cur")) == NULL) { + if (errno != ENOENT) + log_warn("session %u: opendir cur", session_id); + return (-1); + } + while ((de = readdir(dp)) != NULL) { + if (strncmp(de->d_name, basename, baselen) != 0 || + de->d_name[baselen] != ':') + continue; + if (snprintf(path, sizeof(path), "cur/%s", de->d_name) + >= (int)sizeof(path)) + break; + if (stat(path, &st) == 0) { + *size_out = st.st_size; + strlcpy(suffix_out, de->d_name + baselen, + suffix_out_size); + rv = 0; + } + break; + } + closedir(dp); + return (rv); + } +} + +/* + * Same new/->cur/ fallback lookup as locate_message_file() (Courier + * maildir(5): a message lives in new/ until some client/MDA moves it to + * cur/ and appends its flag suffix), but opens the file and returns a + * readable fd instead of just stat()'ing it -- locate_message_file() + * itself isn't changed to also return a path/fd, to avoid touching its + * eight existing call sites (RFC822.SIZE/FLAGS lookups across FETCH, + * STORE, COPY, MOVE, SEARCH) for a capability only BODY.PEEK[HEADER] + * needs. The duplication is small and self-contained; see read_message_ + * header() below for the one caller. + */ +static int +open_message_file(const char *basename) +{ + char path[600]; + int fd; + + if (snprintf(path, sizeof(path), "new/%s", basename) >= + (int)sizeof(path)) { + log_warnx("session %u: basename too long: %s", session_id, + basename); + return (-1); + } + if ((fd = open(path, O_RDONLY)) != -1) + return (fd); + if (errno != ENOENT) { + log_warn("session %u: open %s", session_id, path); + return (-1); + } + + { + DIR *dp; + struct dirent *de; + size_t baselen = strlen(basename); + int rv = -1; + + if ((dp = opendir("cur")) == NULL) { + if (errno != ENOENT) + log_warn("session %u: opendir cur", session_id); + return (-1); + } + while ((de = readdir(dp)) != NULL) { + if (strncmp(de->d_name, basename, baselen) != 0 || + de->d_name[baselen] != ':') + continue; + if (snprintf(path, sizeof(path), "cur/%s", de->d_name) + >= (int)sizeof(path)) + break; + rv = open(path, O_RDONLY); + break; + } + closedir(dp); + return (rv); + } +} + +/* + * Reads basename's on-disk message file (open_message_file() above) and + * returns just its raw RFC 5322 header block: everything from the start of + * the file through and including the blank-line separator between headers + * and body. Including the separator itself in the returned bytes is this + * implementation's own judgment call -- RFC 9051 SS6.4.5 only says HEADER + * means "the [RFC5322] header of the message", without spelling out the + * exact byte boundary -- but it matches that section's own SS8 worked + * example (a blank line appears inside the {342}-octet literal, right + * before the response's closing paren) and is what lets BODY[HEADER] + + * BODY[TEXT] concatenate back into the original message with nothing + * missing, which is the common expectation among real IMAP servers. + * Accepts either "\r\n\r\n" or bare "\n\n" as the separator, since this + * implementation's APPEND stores literal bytes exactly as the client sent + * them, unnormalized -- a maildir message here isn't guaranteed to be + * strictly CRLF-terminated. + * + * Returns 0 and a malloc(3)'d *buf_out (caller frees) + *len_out on + * success. Returns -1 (nothing left to free) if: the message file can't be + * located; no blank-line separator is found within the first + * FETCH_HEADER_MAX+1 bytes read (treated as "can't confidently say where + * the header ends" rather than guessing -- the same "give up rather than + * guess" principle this file's CRLF-literal parsing already follows); the + * separator itself lands past FETCH_HEADER_MAX (the header is simply too + * big for this pass's inline-imsg design, same size-cap precedent as + * APPEND_LITERAL_MAX); or a NUL byte appears within the header region + * (defensive -- nothing downstream expects message content read this way + * to flow through a NUL-terminated C string, so this rejects the unusual + * case outright rather than risk truncating it somewhere later). + */ +static int +read_message_header(const char *basename, char **buf_out, uint32_t *len_out) +{ + char readbuf[FETCH_HEADER_MAX + 1]; + int fd; + ssize_t n, total = 0; + size_t i; + int sepindex = -1; + + if ((fd = open_message_file(basename)) == -1) + return (-1); + + while (total < (ssize_t)sizeof(readbuf)) { + n = read(fd, readbuf + total, sizeof(readbuf) - total); + if (n == -1) { + log_warn("session %u: read message header (%s)", + session_id, basename); + close(fd); + return (-1); + } + if (n == 0) + break; + total += n; + } + close(fd); + + for (i = 0; i + 1 < (size_t)total; i++) { + if (readbuf[i] == '\0') { + log_warnx("session %u: message %s has a NUL byte in " + "its header -- BODY.PEEK[HEADER] skipped", + session_id, basename); + return (-1); + } + if (i + 3 < (size_t)total && readbuf[i] == '\r' && + readbuf[i + 1] == '\n' && readbuf[i + 2] == '\r' && + readbuf[i + 3] == '\n') { + sepindex = (int)i + 4; + break; + } + if (readbuf[i] == '\n' && readbuf[i + 1] == '\n') { + sepindex = (int)i + 2; + break; + } + } + + if (sepindex == -1 || sepindex > FETCH_HEADER_MAX) { + log_warnx("session %u: message %s: no header/body separator " + "found within %d bytes -- BODY.PEEK[HEADER] skipped", + session_id, basename, FETCH_HEADER_MAX); + return (-1); + } + + if ((*buf_out = malloc((size_t)sepindex)) == NULL) { + log_warn("session %u: malloc message header (%s)", + session_id, basename); + return (-1); + } + memcpy(*buf_out, readbuf, (size_t)sepindex); + *len_out = (uint32_t)sepindex; + return (0); +} + +/* + * Reads basename's on-disk message file (open_message_file() above) and + * returns either the entire raw RFC 5322 message (text_only == 0) or + * everything after the same header/body blank-line separator read_ + * message_header() scans for (text_only == 1) -- SS6.4.5.1: "The TEXT part + * specifier refers to the text body of the message, omitting the + * [RFC5322] header." Unlike read_message_header(), which only needs to + * read far enough to find that separator, this has to read the whole + * file: BODY.PEEK[] wants every byte, BODY.PEEK[TEXT] can't know how much + * to return without first knowing the total length, and build_ + * bodystructure() needs the whole file to find every MIME part's own + * boundary, even though its own *output* stays small. + * + * maxlen and label are supplied per call site rather than hardcoded, + * because this function's two callers have genuinely different size + * realities: BODY.PEEK[]/BODY.PEEK[TEXT] (handle_mbox_fetch()) pass + * APPEND_LITERAL_MAX, since whatever comes out of this function still has + * to fit whole on a single imsg to reach listener.c (struct imsg_mbox_ + * fetch_body's own bodylen comment, imapd.h) -- raising that would need + * real multi-imsg streaming, out of scope this pass. build_bodystructure() + * passes the much larger bodystructure_read_max instead (operator- + * configurable via imapd.conf's "attachment max" directive, received + * from parent over IMSG_STORE_INIT -- see BODYSTRUCTURE_READ_DEFAULT's + * comment in imapd.h for the default and full rationale), since it only + * derives a small, MIME_MAX_PARTS/ + * MIME_MAX_DEPTH/BODYSTRUCTURE_MAX-bounded structure summary from these + * bytes -- it never sends the raw bytes themselves over the wire, so it + * isn't limited by imsg's own size ceiling the way BODY.PEEK[]/[TEXT] is. + * (This split was added after real-hardware testing showed a genuine + * Apple Mail message with a small image attachment -- unsurprisingly, + * larger than 12000 bytes once base64-encoded -- silently failed + * BODYSTRUCTURE entirely, because this function used to share one + * hardcoded APPEND_LITERAL_MAX-sized buffer across both callers.) label + * (e.g. "BODY[]", "BODY[TEXT]", "BODYSTRUCTURE") is used only so a + * failure here is logged accurately regardless of which caller hit it, + * rather than this function unconditionally assuming it's always being + * called for a BODY.PEEK[...] request. + * + * readbuf is heap-allocated (maxlen + 1 bytes) rather than a fixed-size + * stack array, sized per-call to the caller's own maxlen -- necessary now + * that the two callers want very different sizes, and BODYSTRUCTURE_READ_ + * MAX is far too large for a stack allocation anyway. + * + * Returns 0 and a malloc(3)'d *buf_out (caller frees; left NULL if + * *len_out comes back 0 -- an empty body is legitimately different from + * "not found", same distinction APPEND already makes for a zero-length + * message) + *len_out on success. Returns -1 (nothing left to free) if: + * the message file can't be located; the file is larger than maxlen (same + * "reject rather than truncate" precedent as read_message_header()/APPEND + * itself); the content contains a NUL byte (same defensive reasoning as + * read_message_header()); or (text_only only) no header/body separator + * can be found anywhere in the file, which this implementation treats as + * "nothing to give" rather than guessing that an entire header-less file + * is all body -- a deliberate simplification, not a claim that's the only + * reasonable reading of SS6.4.5.1 for a malformed or genuinely header-less + * message. + */ +static int +read_message_body(const char *basename, int text_only, size_t maxlen, + const char *label, char **buf_out, uint32_t *len_out) +{ + char *readbuf; + size_t readbuf_size = maxlen + 1; + int fd; + ssize_t n, total = 0; + size_t i; + int sepindex = -1; + + *buf_out = NULL; + *len_out = 0; + + if ((readbuf = malloc(readbuf_size)) == NULL) { + log_warn("session %u: malloc message body readbuf (%s)", + session_id, basename); + return (-1); + } + + if ((fd = open_message_file(basename)) == -1) { + free(readbuf); + return (-1); + } + + while (total < (ssize_t)readbuf_size) { + n = read(fd, readbuf + total, readbuf_size - total); + if (n == -1) { + log_warn("session %u: read message body (%s)", + session_id, basename); + close(fd); + free(readbuf); + return (-1); + } + if (n == 0) + break; + total += n; + } + close(fd); + + if (total > (ssize_t)maxlen) { + log_warnx("session %u: message %s exceeds %zu bytes -- " + "%s skipped", session_id, basename, maxlen, label); + free(readbuf); + return (-1); + } + + for (i = 0; i < (size_t)total; i++) { + if (readbuf[i] == '\0') { + log_warnx("session %u: message %s has a NUL byte -- " + "%s skipped", session_id, basename, label); + free(readbuf); + return (-1); + } + } + + if (text_only) { + for (i = 0; i + 1 < (size_t)total; i++) { + if (i + 3 < (size_t)total && readbuf[i] == '\r' && + readbuf[i + 1] == '\n' && readbuf[i + 2] == '\r' && + readbuf[i + 3] == '\n') { + sepindex = (int)i + 4; + break; + } + if (readbuf[i] == '\n' && readbuf[i + 1] == '\n') { + sepindex = (int)i + 2; + break; + } + } + if (sepindex == -1) { + log_warnx("session %u: message %s: no header/body " + "separator found -- %s skipped", + session_id, basename, label); + free(readbuf); + return (-1); + } + } else { + sepindex = 0; + } + + *len_out = (uint32_t)((size_t)total - (size_t)sepindex); + if (*len_out > 0) { + if ((*buf_out = malloc(*len_out)) == NULL) { + log_warn("session %u: malloc message body (%s)", + session_id, basename); + *len_out = 0; + free(readbuf); + return (-1); + } + memcpy(*buf_out, readbuf + sepindex, *len_out); + } + free(readbuf); + return (0); +} + +/* + * True if the header field name spanning name[0..namelen) case- + * insensitively (ASCII-range, per RFC 9051 SS6.4.5.1: "The field-matching + * is ASCII-range case insensitive but is otherwise exact") matches one of + * the space-separated names in list. Re-copies and re-tokenizes list on + * every call -- called once per header field in a message, and real + * messages have at most a few dozen header fields, so the clarity of a + * fresh strtok_r() pass each time outweighs the cost of not caching a + * pre-split array. + */ +static int +header_field_name_matches(const char *name, size_t namelen, const char *list) +{ + char listcopy[HEADER_FIELDS_MAX]; + char *tok, *save; + + if (namelen == 0 || namelen >= sizeof(listcopy)) + return (0); + + strlcpy(listcopy, list, sizeof(listcopy)); + for (tok = strtok_r(listcopy, " ", &save); tok != NULL; + tok = strtok_r(NULL, " ", &save)) { + if (strlen(tok) == namelen && + strncasecmp(tok, name, namelen) == 0) + return (1); + } + return (0); +} + +/* + * BODY.PEEK[HEADER.FIELDS (fields_spec)] (want_not == 0) or BODY.PEEK + * [HEADER.FIELDS.NOT (fields_spec)] (want_not == 1) -- RFC 9051 SS6.4.5.1. + * fields_spec is listener.c's already-grammar-validated, space-joined + * field-name list (see struct imsg_mbox_fetch's header_fields comment in + * imapd.h) -- this function trusts it's well-formed rather than + * re-validating. + * + * Builds on read_message_header() rather than re-deriving the header + * block from disk: gets the same whole raw header (through and including + * the trailing blank-line separator) that BODY.PEEK[HEADER] already + * returns, then splits *that* into individual RFC 5322 fields itself, + * respecting obs-fold continuation lines (RFC 5322 SS2.2.3: a field's + * value may continue onto following lines, each of which "begins with a + * space or tab" per section 3.2.2's WSP-based folding) -- a continuation + * line is included as part of whichever field precedes it, never treated + * as its own field. Each field's *complete* raw byte span (its own + * "Name:" line through the last byte of its final continuation line) is + * copied verbatim into the output if its name matches fields_spec (for + * HEADER.FIELDS) or doesn't (for HEADER.FIELDS.NOT); order and every + * repeated occurrence of a field name (e.g. multiple "Received:" lines) + * are both preserved exactly as they appear in the source, not + * deduplicated or reordered. The trailing blank line itself is always + * copied unconditionally, matching SS6.4.5.1's "Subsetting does not + * exclude the [RFC5322] delimiting blank line... the blank line is + * included in all header fetches" -- regardless of whether any field + * matched at all (an empty-but-valid result is legitimate: HEADER.FIELDS + * with a field-name list that matches nothing in this particular message + * still returns just the blank line, found == 1, not "not found"). + * + * Returns 0 and a malloc(3)'d *buf_out (caller frees) + *len_out (always + * at least 1, the blank line's own terminator) on success. Returns -1 + * (nothing left to free) if read_message_header() itself fails (message + * not found, oversized, NUL byte, no separator -- see that function's + * comment) or if the header block it returns is somehow malformed enough + * that this function's own line-walk never reaches a blank line before + * running off the end -- shouldn't happen, since read_message_header() + * only ever returns a block that already ends at one, but checked + * defensively rather than assumed. + */ +static int +read_message_header_fields(const char *basename, const char *fields_spec, + int want_not, char **buf_out, uint32_t *len_out) +{ + char *hdrbuf = NULL; + uint32_t hdrlen = 0; + char *out; + size_t outlen = 0; + size_t off = 0; + int rc = -1; + + *buf_out = NULL; + *len_out = 0; + + if (read_message_header(basename, &hdrbuf, &hdrlen) == -1) + return (-1); + + /* Filtered output can never exceed the unfiltered header's own + * size -- every byte copied below comes verbatim from hdrbuf. */ + if ((out = malloc(hdrlen)) == NULL) { + log_warn("session %u: malloc HEADER.FIELDS buffer (%s)", + session_id, basename); + free(hdrbuf); + return (-1); + } + + while (off < hdrlen) { + size_t field_start = off, line_end, i; + int is_blank, matched, include; + + i = off; + while (i < hdrlen && hdrbuf[i] != '\n') + i++; + if (i >= hdrlen) + break; /* malformed -- see this function's comment */ + line_end = (i > field_start && hdrbuf[i - 1] == '\r') ? + i - 1 : i; + off = i + 1; + + is_blank = (line_end == field_start); + if (is_blank) { + memcpy(out + outlen, hdrbuf + field_start, + off - field_start); + outlen += off - field_start; + rc = 0; + break; + } + + /* consume obs-fold continuation lines (start with SP/HTAB) + * belonging to this same field */ + while (off < hdrlen && + (hdrbuf[off] == ' ' || hdrbuf[off] == '\t')) { + size_t j = off; + + while (j < hdrlen && hdrbuf[j] != '\n') + j++; + if (j >= hdrlen) { + off = hdrlen; + break; + } + off = j + 1; + } + + { + size_t namelen = 0, k; + + for (k = field_start; k < line_end; k++) { + if (hdrbuf[k] == ':') + break; + } + namelen = k - field_start; + + matched = header_field_name_matches(hdrbuf + + field_start, namelen, fields_spec); + include = want_not ? !matched : matched; + + if (include) { + memcpy(out + outlen, hdrbuf + field_start, + off - field_start); + outlen += off - field_start; + } + } + } + + free(hdrbuf); + + if (rc == -1) { + free(out); + return (-1); + } + + *buf_out = out; + *len_out = (uint32_t)outlen; + return (0); +} + +/* + * Finds the first header field named `name` (ASCII-range case-insensitive, + * same match rule as header_field_name_matches() above) in the raw header + * block hdr[0..hdrlen), and returns its *unfolded* value: RFC 5322 SS2.2.3 + * "Unfolding is accomplished by simply removing any CRLF that is + * immediately followed by WSP" -- the WSP itself is retained, so "Subject: + * foo\r\n bar" unfolds to "foo bar", not "foobar". Every fold point inside + * the value is guaranteed to be followed by WSP by construction: this + * function walks continuation lines with the exact same "starts with SP or + * HTAB" loop read_message_header_fields() already uses, so every internal + * line break in the span it collects is, by definition, one of those + * fold points. Leading FWS immediately after the colon is trimmed (SS3.6.5's + * conventional single-space separator is not itself part of the field + * body); the field's own final line terminator is trimmed from the end. + * + * Only the header block itself needs to be well-formed for this to work -- + * hdr is trusted to already be NUL-free (read_message_header() guarantees + * that, rejecting any header containing one) and to end at a header/body + * blank-line separator, same preconditions read_message_header_fields() + * already relies on. + * + * Returns 0 and a malloc(3)'d *val_out (caller frees; may be a valid + * zero-length allocation for a present-but-empty field, e.g. "Subject:\r\n" + * -- RFC 9051 SS7.5.2 distinguishes "absent" (NIL) from "present but + * empty" (empty string) for exactly this reason) + *vallen_out on success. + * Returns -1 (nothing to free) if the field is not present in the header + * at all. + */ +static int +extract_header_field(const char *hdr, size_t hdrlen, const char *name, + char **val_out, size_t *vallen_out) +{ + size_t namelen = strlen(name); + size_t off = 0; + + *val_out = NULL; + *vallen_out = 0; + + while (off < hdrlen) { + size_t field_start = off, line_end, i, k; + + i = off; + while (i < hdrlen && hdr[i] != '\n') + i++; + if (i >= hdrlen) + break; /* malformed -- no trailing blank line found */ + line_end = (i > field_start && hdr[i - 1] == '\r') ? i - 1 : i; + off = i + 1; + + if (line_end == field_start) + break; /* blank line -- end of header, not found */ + + for (k = field_start; k < line_end; k++) { + if (hdr[k] == ':') + break; + } + + /* Consume this field's obs-fold continuation lines + * regardless of whether its name matches below -- off has + * to land past them either way to keep scanning correctly + * positioned for the next field. */ + while (off < hdrlen && (hdr[off] == ' ' || hdr[off] == '\t')) { + size_t j = off; + + while (j < hdrlen && hdr[j] != '\n') + j++; + if (j >= hdrlen) { + off = hdrlen; + break; + } + off = j + 1; + } + + if (k - field_start != namelen || + strncasecmp(hdr + field_start, name, namelen) != 0) + continue; + + { + size_t vstart = k + 1; + size_t vend = off; + size_t p; + char *out; + size_t outlen = 0; + + if (vend > vstart && hdr[vend - 1] == '\n') + vend--; + if (vend > vstart && hdr[vend - 1] == '\r') + vend--; + while (vstart < vend && + (hdr[vstart] == ' ' || hdr[vstart] == '\t')) + vstart++; + + if ((out = malloc(vend - vstart + 1)) == NULL) + return (-1); + + for (p = vstart; p < vend; ) { + if (hdr[p] == '\r' && p + 1 < vend && + hdr[p + 1] == '\n') { + p += 2; + continue; + } + if (hdr[p] == '\n') { + p += 1; + continue; + } + out[outlen++] = hdr[p]; + p++; + } + + *val_out = out; + *vallen_out = outlen; + return (0); + } + } + + return (-1); +} + +/* + * envbuf_append()/envbuf_append_str(): bounded-buffer append primitives + * shared by every ENVELOPE formatting function below. Unlike strlcat(3), + * these reject (return -1, buffer left as it was before the call) rather + * than truncate on overflow -- matching this project's usual "reject rather + * than silently do something the client didn't ask for" precedent (see + * HEADER_FIELDS_MAX's comment in imapd.h for the same reasoning applied + * elsewhere), and letting build_envelope() propagate that as ENVELOPE_MAX + * exceeded (found = 0) rather than ever emitting truncated, syntactically- + * broken envelope text. + */ +static int +envbuf_append(char *buf, size_t bufsize, size_t *outlen, const char *data, + size_t datalen) +{ + if (*outlen + datalen > bufsize) + return (-1); + memcpy(buf + *outlen, data, datalen); + *outlen += datalen; + return (0); +} + +static int +envbuf_append_str(char *buf, size_t bufsize, size_t *outlen, const char *s) +{ + return (envbuf_append(buf, bufsize, outlen, s, strlen(s))); +} + +/* + * Appends one RFC 9051 nstring: `NIL` if val is NULL, else an IMAP quoted + * string with backslash and double-quote escaped (SS9's `quoted-specials = + * DQUOTE / "\"` -- a quoted string's QUOTED-CHAR is "any TEXT-CHAR except + * quoted-specials" or "\" followed by a quoted-special, so exactly those + * two bytes need escaping, nothing else). Deliberately does not decode RFC + * 2047 encoded-words (e.g. "=?UTF-8?B?...?=") in val -- RFC 9051 SS7.5.2 + * doesn't require it (ENVELOPE fields are described as extracted from the + * RFC 5322 header, not MIME-decoded), and skipping it keeps this pass + * scoped to RFC 5322 header parsing rather than also pulling in RFC 2047 + * decoding; a client sees the raw encoded-word text verbatim, same as it + * would from the header itself, and can decode it exactly the same way it + * always has to for extension fields. 8-bit bytes (raw unencoded UTF-8 in a + * technically-non-conformant header) are passed through as-is rather than + * rejected -- this server has no CHARSET negotiation for ENVELOPE and RFC + * 9051 does not define an error path for it here, so passing the bytes + * through unmodified (same as most real-world server implementations do) + * is the more useful behavior than refusing the whole field. + */ +static int +envbuf_append_nstring(char *buf, size_t bufsize, size_t *outlen, + const char *val, size_t vallen) +{ + size_t i; + + if (val == NULL) + return (envbuf_append_str(buf, bufsize, outlen, "NIL")); + + if (envbuf_append(buf, bufsize, outlen, "\"", 1) == -1) + return (-1); + for (i = 0; i < vallen; i++) { + if ((val[i] == '"' || val[i] == '\\') && + envbuf_append(buf, bufsize, outlen, "\\", 1) == -1) + return (-1); + if (envbuf_append(buf, bufsize, outlen, &val[i], 1) == -1) + return (-1); + } + return (envbuf_append(buf, bufsize, outlen, "\"", 1)); +} + +/* + * Parses and formats a single RFC 5322 mailbox (one entry from an address- + * list field, already isolated by envbuf_append_address_list()'s top-level- + * comma split below) into one IMAP `address` tuple, `"(" addr-name SP + * addr-adl SP addr-mailbox SP addr-host ")"` (RFC 9051 SS9). This is a + * deliberately scoped-down RFC 5322 address parser, not a complete one -- + * consistent with this project's smaller-feature-set-over-completeness + * philosophy (see openimap-privsep-design.md's design philosophy section) + * and the explicit v1 ENVELOPE-only scope decision (see this file's header + * comment and imapd.h's MBOX_FETCH_ENVELOPE comment). Specifically NOT + * supported, by design: + * + * - RFC 5322 `group` syntax ("Undisclosed-recipients:;") -- rare in + * modern mail; addr-host's NIL-marks-a-group convention (SS7.5.2's + * "If the mailbox name field is also NIL, this is an end-of-group + * marker") is simply never produced by this implementation. + * - obs-route / addr-adl -- obsolete since RFC 2822 (2001); always + * formatted as NIL, which addr-adl's own ABNF comment in imapd.h + * confirms is spec-legal ("Holds route from RFC5322 obs-route if + * non-NIL"). + * - A quoted local-part containing an unescaped "@" (e.g. + * `"foo@bar"@host.example`) -- this function finds the mailbox/host + * split at the *last* unquoted "@", which is correct for the + * overwhelming majority of real addresses but not that specific edge + * case. + * - RFC 5322 `comment` ("(...)") stripping -- comments are not + * specially recognized or removed; since this parser never uses "("/ + * ")" as a structural delimiter anywhere, an address containing a + * comment doesn't break parsing, it's just included verbatim as part + * of whichever component (display name or mailbox/host) it falls + * within, which may look odd but is not a correctness hazard. + * + * A mailbox this function can't make sense of (no "@" found in the + * addr-spec portion, or a `<...>` with no matching closing angle bracket) + * is reported via -1 -- the caller skips it and continues with the rest of + * the list, rather than failing the whole ENVELOPE for one malformed + * address (see envbuf_append_address_list()'s comment). + * + * Writes into a fixed local stack buffer sized generously for any + * realistic single address; a single address whose formatted form + * (including escaping) would exceed that buffer is also treated as -1 -- + * an even more defensible simplification than the ones above, since a + * multi-kilobyte single address is already well outside anything a real + * mail client would ever produce. + */ +static int +envbuf_append_one_address(char *buf, size_t bufsize, size_t *outlen, + const char *tok, size_t toklen) +{ + char addrbuf[1024]; + size_t addrlen = 0; + const char *name = NULL; + size_t namelen = 0; + const char *spec; + size_t speclen; + const char *mailbox, *host; + size_t mailboxlen, hostlen; + size_t at; + int found_at; + size_t lt; + + while (toklen > 0 && (tok[0] == ' ' || tok[0] == '\t')) { + tok++; + toklen--; + } + while (toklen > 0 && (tok[toklen - 1] == ' ' || tok[toklen - 1] == '\t')) + toklen--; + if (toklen == 0) + return (-1); + + /* Find an unquoted '<' -- if present, everything before it is the + * display-name, and the addr-spec is the (still unquoted-tracked) + * span up to the matching unquoted '>'. */ + lt = toklen; + { + size_t i; + int q = 0; + + for (i = 0; i < toklen; i++) { + if (tok[i] == '"') + q = !q; + else if (!q && tok[i] == '<') { + lt = i; + break; + } + } + } + + if (lt < toklen) { + size_t gt = toklen, i; + int q = 0; + + for (i = lt + 1; i < toklen; i++) { + if (tok[i] == '"') + q = !q; + else if (!q && tok[i] == '>') { + gt = i; + break; + } + } + if (gt >= toklen) + return (-1); /* unmatched '<' -- malformed, skip */ + + { + const char *disp = tok; + size_t displen = lt; + + while (displen > 0 && (disp[0] == ' ' || disp[0] == '\t')) { + disp++; + displen--; + } + while (displen > 0 && + (disp[displen - 1] == ' ' || disp[displen - 1] == '\t')) + displen--; + + if (displen >= 2 && disp[0] == '"' && + disp[displen - 1] == '"') { + /* quoted-string display-name: strip the + * surrounding quotes; the emission loop + * below (the "if (name != NULL)" block) + * undoes RFC 5322 quoted-pair escaping + * ("\" + escaped octet) as it walks name[], + * then re-escapes for the IMAP wire + * independently, so this round-trips + * correctly without a separate unescape pass + * here. Applying that same backslash-aware + * walk to an *unquoted* phrase (the else + * branch below) is harmless -- RFC 5322's + * `atext`/`word` grammar for an unquoted + * phrase has no quoted-pair mechanism, so a + * literal backslash there would already be + * non-conformant input, not a real case this + * needs to get right. */ + disp++; + displen -= 2; + } + if (displen > 0) { + name = disp; + namelen = displen; + } + } + + spec = tok + lt + 1; + speclen = gt - (lt + 1); + } else { + spec = tok; + speclen = toklen; + } + + while (speclen > 0 && (spec[0] == ' ' || spec[0] == '\t')) { + spec++; + speclen--; + } + while (speclen > 0 && (spec[speclen - 1] == ' ' || spec[speclen - 1] == '\t')) + speclen--; + + found_at = 0; + at = 0; + { + size_t i; + int q = 0; + + for (i = 0; i < speclen; i++) { + if (spec[i] == '"') + q = !q; + else if (!q && spec[i] == '@') { + at = i; + found_at = 1; + } + } + } + if (!found_at || at == 0 || at + 1 >= speclen) + return (-1); /* no usable local-part@domain split */ + + mailbox = spec; + mailboxlen = at; + host = spec + at + 1; + hostlen = speclen - at - 1; + + /* strip surrounding quotes from a quoted local-part, same + * unescaping as the display-name case above -- deliberately not + * handling an unescaped "@" inside a quoted local-part, see this + * function's header comment */ + if (mailboxlen >= 2 && mailbox[0] == '"' && mailbox[mailboxlen - 1] == '"') { + mailbox++; + mailboxlen -= 2; + } + + if (name != NULL) { + size_t j; + + if (envbuf_append(addrbuf, sizeof(addrbuf), &addrlen, "\"", 1) == -1) + return (-1); + for (j = 0; j < namelen; j++) { + char c = name[j]; + + if (c == '\\' && j + 1 < namelen) { + j++; + c = name[j]; + } + if ((c == '"' || c == '\\') && + envbuf_append(addrbuf, sizeof(addrbuf), &addrlen, + "\\", 1) == -1) + return (-1); + if (envbuf_append(addrbuf, sizeof(addrbuf), &addrlen, + &c, 1) == -1) + return (-1); + } + if (envbuf_append(addrbuf, sizeof(addrbuf), &addrlen, "\"", 1) == -1) + return (-1); + } else { + if (envbuf_append_str(addrbuf, sizeof(addrbuf), &addrlen, "NIL") == -1) + return (-1); + } + + if (envbuf_append_str(addrbuf, sizeof(addrbuf), &addrlen, " NIL ") == -1) + return (-1); + if (envbuf_append_nstring(addrbuf, sizeof(addrbuf), &addrlen, mailbox, + mailboxlen) == -1) + return (-1); + if (envbuf_append(addrbuf, sizeof(addrbuf), &addrlen, " ", 1) == -1) + return (-1); + if (envbuf_append_nstring(addrbuf, sizeof(addrbuf), &addrlen, host, + hostlen) == -1) + return (-1); + + if (envbuf_append(buf, bufsize, outlen, "(", 1) == -1) + return (-1); + if (envbuf_append(buf, bufsize, outlen, addrbuf, addrlen) == -1) + return (-1); + return (envbuf_append(buf, bufsize, outlen, ")", 1)); +} + +/* + * Formats an RFC 5322 address-list field value (From/Sender/Reply-To/To/ + * Cc/Bcc's raw, already-unfolded value) as an IMAP `env-from`/`env-to`/etc. + * production: `"(" 1*address ")"` if at least one address parses, `NIL` + * otherwise (RFC 9051 SS7.5.2: "If the ... header fields are absent ... or + * are present but empty, the corresponding member of the envelope is + * NIL" -- this implementation extends that to "or contains nothing this + * parser could make an address out of", rather than distinguishing + * "empty" from "unparseable" at the wire level, since RFC 9051 gives no + * separate response for the latter). + * + * Splits val on top-level commas -- respecting RFC 5322 quoted-string and + * angle-addr nesting, so a quoted display name containing a literal comma + * ("Doe, John" ) is not mistaken for a list separator. + * Adjacent `address` tuples are written back-to-back with no separator + * between them (`"(" 1*address ")"`'s own ABNF has none -- each `address` + * is already self-delimiting via its own parens, confirmed against RFC + * 9051 SS7.5.2's own worked example, which shows + * `((NIL NIL "minutes" "..." )("John Klensin" NIL "KLENSIN" "MIT.EDU"))` + * with no space or comma between the two address tuples). + */ +static int +envbuf_append_address_list(char *buf, size_t bufsize, size_t *outlen, + const char *val, size_t vallen) +{ + size_t save = *outlen; + size_t i = 0; + int any = 0; + + while (vallen > 0 && (val[0] == ' ' || val[0] == '\t')) { + val++; + vallen--; + } + while (vallen > 0 && (val[vallen - 1] == ' ' || val[vallen - 1] == '\t')) + vallen--; + + if (vallen == 0) + return (envbuf_append_str(buf, bufsize, outlen, "NIL")); + + if (envbuf_append(buf, bufsize, outlen, "(", 1) == -1) + return (-1); + + while (i < vallen) { + size_t tok_start; + size_t tok_len; + int in_quotes = 0, in_angle = 0; + + while (i < vallen && (val[i] == ' ' || val[i] == '\t' || + val[i] == ',')) + i++; + tok_start = i; + while (i < vallen) { + char c = val[i]; + + if (c == '"') + in_quotes = !in_quotes; + else if (!in_quotes && c == '<') + in_angle = 1; + else if (!in_quotes && c == '>') + in_angle = 0; + else if (!in_quotes && !in_angle && c == ',') + break; + i++; + } + tok_len = i - tok_start; + while (tok_len > 0 && (val[tok_start + tok_len - 1] == ' ' || + val[tok_start + tok_len - 1] == '\t')) + tok_len--; + + if (tok_len > 0) { + if (envbuf_append_one_address(buf, bufsize, outlen, + val + tok_start, tok_len) == 0) + any = 1; + else if (*outlen > bufsize) { + /* can't actually happen -- envbuf_append() + * never leaves *outlen past bufsize -- but + * checked defensively rather than assumed */ + return (-1); + } + /* malformed single address: envbuf_append_one_ + * address() left the buffer exactly as it found it + * on failure (see that function's own local addrbuf + * staging, which is only ever flushed to buf/outlen + * as one atomic append), so skipping it here is + * safe -- not a hard failure for the whole list */ + } + } + + if (!any) { + *outlen = save; + return (envbuf_append_str(buf, bufsize, outlen, "NIL")); + } + return (envbuf_append(buf, bufsize, outlen, ")", 1)); +} + +/* + * Looks up header field `name`, appends its nstring form (see envbuf_ + * append_nstring()) -- NIL if the field is absent, an escaped quoted + * string (possibly empty, `""`) if present. Shared by ENVELOPE's four + * plain-string members (date, subject, in-reply-to, message-id). + */ +static int +append_field_nstring(char *out, size_t outsize, size_t *outlen, + const char *hdrbuf, uint32_t hdrlen, const char *name) +{ + char *val; + size_t vallen; + int rc; + + if (extract_header_field(hdrbuf, hdrlen, name, &val, &vallen) == 0) { + rc = envbuf_append_nstring(out, outsize, outlen, val, vallen); + free(val); + } else { + rc = envbuf_append_nstring(out, outsize, outlen, NULL, 0); + } + return (rc); +} + +/* + * Builds the complete RFC 9051 SS7.5.2 ENVELOPE parenthesized-list text for + * one message: `"(" env-date SP env-subject SP env-from SP env-sender SP + * env-reply-to SP env-to SP env-cc SP env-bcc SP env-in-reply-to SP + * env-message-id ")"`, field order taken directly from SS7.5.2 ("The + * fields of the envelope structure are in the following order: date, + * subject, from, sender, reply-to, to, cc, bcc, in-reply-to, and + * message-id"). Reads the header via read_message_header() independently + * of any other requested FETCH content item -- same "each content item's + * store.c handler does its own read_message_header()/read_message_body() + * call" pattern already used for BODY.PEEK[HEADER] vs. BODY.PEEK[HEADER. + * FIELDS...] vs. BODY.PEEK[]/[TEXT] (see handle_mbox_fetch()'s comment); + * not sharing one read across FETCH items in the same request is a known, + * accepted minor inefficiency, not a new one introduced here. + * + * Sender/Reply-To default to the already-formatted From value when their + * own header field is absent or present-but-completely-empty (RFC 9051 + * SS7.5.2: "If the Sender or Reply-To header fields are absent ..., or are + * present but empty, the server sets the corresponding member of the + * envelope to be the same value as the from member"). "Present but empty" + * here specifically means the raw header value is zero bytes after + * unfolding (e.g. "Sender:\r\n") -- a value that's present but entirely + * whitespace is treated as a normal (if unusual) address-list value that + * happens to parse to no addresses, which independently produces the same + * NIL rather than the from-value fallback; a narrow, documented difference + * from a fully literal reading of "empty", not expected to matter for any + * real message. + * + * Returns 0 and a malloc(3)'d *buf_out (caller frees) + *len_out on + * success. Returns -1 (nothing to free) if the message's header can't be + * read at all, or if the fully-formatted envelope text would exceed + * ENVELOPE_MAX -- both cases are "ENVELOPE not found" for this message, + * same as every other content item's own not-found handling. + */ +static int +build_envelope(const char *basename, char **buf_out, uint32_t *len_out) +{ + char *hdrbuf = NULL; + uint32_t hdrlen = 0; + char out[ENVELOPE_MAX]; + size_t outlen = 0; + char from_formatted[ENVELOPE_MAX]; + size_t from_len = 0; + + *buf_out = NULL; + *len_out = 0; + + if (read_message_header(basename, &hdrbuf, &hdrlen) == -1) + return (-1); + + if (envbuf_append(out, sizeof(out), &outlen, "(", 1) == -1) + goto fail; + + if (append_field_nstring(out, sizeof(out), &outlen, hdrbuf, hdrlen, + "Date") == -1) + goto fail; + if (envbuf_append(out, sizeof(out), &outlen, " ", 1) == -1) + goto fail; + if (append_field_nstring(out, sizeof(out), &outlen, hdrbuf, hdrlen, + "Subject") == -1) + goto fail; + if (envbuf_append(out, sizeof(out), &outlen, " ", 1) == -1) + goto fail; + + /* from */ + { + char *val; + size_t vallen; + + if (extract_header_field(hdrbuf, hdrlen, "From", &val, + &vallen) == 0) { + int rc = envbuf_append_address_list(from_formatted, + sizeof(from_formatted), &from_len, val, vallen); + free(val); + if (rc == -1) + goto fail; + } else { + if (envbuf_append_str(from_formatted, + sizeof(from_formatted), &from_len, "NIL") == -1) + goto fail; + } + } + if (envbuf_append(out, sizeof(out), &outlen, from_formatted, + from_len) == -1) + goto fail; + if (envbuf_append(out, sizeof(out), &outlen, " ", 1) == -1) + goto fail; + + /* sender, reply-to: default to from_formatted per SS7.5.2 */ + { + static const char *const fallback_fields[] = + { "Sender", "Reply-To" }; + size_t fi; + + for (fi = 0; fi < 2; fi++) { + char *val; + size_t vallen; + int used_value = 0; + + if (extract_header_field(hdrbuf, hdrlen, + fallback_fields[fi], &val, &vallen) == 0) { + if (vallen > 0) { + int rc = envbuf_append_address_list( + out, sizeof(out), &outlen, val, + vallen); + used_value = 1; + free(val); + if (rc == -1) + goto fail; + } else + free(val); + } + if (!used_value && + envbuf_append(out, sizeof(out), &outlen, + from_formatted, from_len) == -1) + goto fail; + if (envbuf_append(out, sizeof(out), &outlen, " ", 1) == -1) + goto fail; + } + } + + /* to, cc, bcc */ + { + static const char *const addr_fields[] = { "To", "Cc", "Bcc" }; + size_t fi; + + for (fi = 0; fi < 3; fi++) { + char *val; + size_t vallen; + + if (extract_header_field(hdrbuf, hdrlen, + addr_fields[fi], &val, &vallen) == 0) { + int rc = envbuf_append_address_list(out, + sizeof(out), &outlen, val, vallen); + free(val); + if (rc == -1) + goto fail; + } else { + if (envbuf_append_str(out, sizeof(out), + &outlen, "NIL") == -1) + goto fail; + } + if (envbuf_append(out, sizeof(out), &outlen, " ", 1) == -1) + goto fail; + } + } + + if (append_field_nstring(out, sizeof(out), &outlen, hdrbuf, hdrlen, + "In-Reply-To") == -1) + goto fail; + if (envbuf_append(out, sizeof(out), &outlen, " ", 1) == -1) + goto fail; + if (append_field_nstring(out, sizeof(out), &outlen, hdrbuf, hdrlen, + "Message-Id") == -1) + goto fail; + + if (envbuf_append(out, sizeof(out), &outlen, ")", 1) == -1) + goto fail; + + free(hdrbuf); + + if ((*buf_out = malloc(outlen)) == NULL) { + log_warn("session %u: malloc ENVELOPE buffer (%s)", + session_id, basename); + return (-1); + } + memcpy(*buf_out, out, outlen); + *len_out = (uint32_t)outlen; + return (0); + +fail: + log_warnx("session %u: message %s: formatted ENVELOPE exceeds " + "ENVELOPE_MAX -- ENVELOPE skipped", session_id, basename); + free(hdrbuf); + return (-1); +} + +/* + * --------------------------------------------------------------------- + * BODYSTRUCTURE (RFC 9051 SS7.5.2) -- recursive MIME structure parsing. + * Scope, sourced from RFC 2045/RFC 2046 (fetched and read directly this + * pass, saved under research/) and an explicit user choice (AskUserQuestion, + * "full recursive, depth-capped" over "single-part only" or "one level of + * multipart, no nesting"): parses Content-Type/Content-Transfer-Encoding/ + * Content-Id/Content-Description and recurses into multipart bodies via + * RFC 2046 SS5.1.1's boundary-delimited body-part grammar, bounded by + * MIME_MAX_DEPTH/MIME_MAX_PARTS (imapd.h). Deliberately does NOT + * implement: RFC 9051's optional extension data (body MD5/disposition/ + * language/location -- SS7.5.2 says this "can be returned... if present" + * with BODYSTRUCTURE, not that it MUST be, so omitting it entirely means + * BODYSTRUCTURE and the non-extensible "BODY" macro produce identical + * output here, which is spec-legal); message/rfc822 and message/global + * parts (body-type-msg needs a full nested ENVELOPE plus a nested BODY + * structure of the *embedded* message, i.e. recursively re-deriving + * everything this file already does for a top-level message, just against + * an inner message extracted from a part's own body -- scoped out as a + * separate, larger unit of work, not attempted this pass; a message + * containing one anywhere in its structure gets found=0 for its whole + * BODYSTRUCTURE, not a partially-correct structure); and RFC 2231 + * parameter-value continuations/charset encoding (SS7.5.2's own "Servers + * SHOULD decode parameter-value continuations" language is a SHOULD, not a + * MUST -- unhandled continuation parameters like "name*0"/"name*1" are + * passed through as literal, un-joined attribute names instead, a cosmetic + * rather than structural gap). + * --------------------------------------------------------------------- + */ + +/* + * Same header/body blank-line-separator scan as read_message_header()'s own + * (see that function's comment), just operating on a buffer already fully + * in memory rather than reading fresh from a file descriptor -- needed + * because BODYSTRUCTURE's recursive walk works entirely off of one whole- + * message read (build_bodystructure()'s read_message_body() call) and has + * to re-locate this same separator both for the top-level message and for + * every multipart sub-part carved out of it along the way. + */ +static int +find_header_body_split(const char *buf, size_t len, size_t *hdrend_out) +{ + size_t i; + + for (i = 0; i + 1 < len; i++) { + if (i + 3 < len && buf[i] == '\r' && buf[i + 1] == '\n' && + buf[i + 2] == '\r' && buf[i + 3] == '\n') { + *hdrend_out = i + 4; + return (0); + } + if (buf[i] == '\n' && buf[i + 1] == '\n') { + *hdrend_out = i + 2; + return (0); + } + } + return (-1); +} + +/* + * RFC 2045 SS5.1: `tspecials := "(" / ")" / "<" / ">" / "@" / "," / ";" / + * ":" / "\" / <"> / "/" / "[" / "]" / "?" / "="` -- the RFC 822 `specials` + * set plus "/", "?", "=", minus ".". A `token` is any US-ASCII CHAR except + * SPACE, CTLs, or one of these. + */ +static int +mime_is_tspecial(char c) +{ + return (strchr("()<>@,;:\\\"/[]?=", c) != NULL); +} + +/* + * Reads one RFC 2045 `token` or `quoted-string` starting at s[*pos], + * advancing *pos past it, into a NUL-terminated out[]. A leading '"' + * switches to quoted-string mode: reads until the matching unescaped '"', + * undoing RFC 822 quoted-pair escaping ("\" + one CHAR) as it goes -- + * needed in practice because most real message generators quote the + * multipart "boundary" parameter's value even though a bare token would be + * legal, precisely because boundary strings very often contain characters + * (like "=", part of base64-derived boundary strings such as + * "----=_NextPart_...") that are tspecials and thus illegal in a bare, + * unquoted token. Shared by every Content-Type token/value this file reads + * (type, subtype, attribute names, and parameter values) -- type/subtype/ + * attribute are always plain `token`s in practice (RFC 2045's grammar + * never allows them to be quoted-strings in the first place, so a leading + * '"' at those positions would itself be a tspecial and simply fail to + * read as a valid bare token, correctly rejected). + * + * Returns 0 on success (out[] is NUL-terminated, possibly the empty string + * only in the quoted-string case). Returns -1 or a zero-length result for + * a bare token (RFC 2045 tokens are 1*<...>, never zero-length) if nothing + * valid could be read, an unterminated quoted-string, or a value too long + * for outsize -- callers treat any of these as "stop parsing further + * parameters, keep what's already been parsed" rather than failing the + * whole Content-Type field over one malformed trailing parameter. + */ +static int +mime_read_token_or_qstring(const char *s, size_t len, size_t *pos, + char *out, size_t outsize) +{ + size_t outlen = 0; + + if (outsize == 0) + return (-1); + + if (*pos < len && s[*pos] == '"') { + (*pos)++; + while (*pos < len && s[*pos] != '"') { + char c = s[*pos]; + + if (c == '\\' && *pos + 1 < len) { + (*pos)++; + c = s[*pos]; + } + if (outlen + 1 >= outsize) + return (-1); + out[outlen++] = c; + (*pos)++; + } + if (*pos >= len) + return (-1); /* unterminated quoted-string */ + (*pos)++; + out[outlen] = '\0'; + return (0); + } + + while (*pos < len && !mime_is_tspecial(s[*pos]) && + s[*pos] != ' ' && s[*pos] != '\t' && + (unsigned char)s[*pos] >= 0x20 && (unsigned char)s[*pos] != 0x7f) { + if (outlen + 1 >= outsize) + return (-1); + out[outlen++] = s[*pos]; + (*pos)++; + } + out[outlen] = '\0'; + if (outlen == 0) + return (-1); + return (0); +} + +/* In-place ASCII-range uppercase -- used to canonicalize MIME type/ + * subtype/attribute names for output, matching RFC 9051 SS7.5.2's own + * worked examples ("TEXT" "PLAIN" ("CHARSET" ...)). RFC 2045 SS5.1 says + * type/subtype/attribute matching is "ALWAYS case-insensitive", so this is + * a display-canonicalization choice, not a correctness requirement -- + * parameter *values* (e.g. a filename) are deliberately left as-is, + * un-uppercased, since those are often case-sensitive in practice + * (filenames, charset aliases some clients treat case-sensitively, etc). */ +static void +mime_str_upper(char *s) +{ + for (; *s != '\0'; s++) { + if (*s >= 'a' && *s <= 'z') + *s -= ('a' - 'A'); + } +} + +/* + * RFC 2045 SS5.1: `content := "Content-Type" ":" type "/" subtype + * *(";" parameter)`, `parameter := attribute "=" value`. Parses the + * Content-Type header field of hdr[0..hdrlen) (a message or MIME body- + * part's own header block) into type_out/subtype_out (both uppercased -- + * see mime_str_upper()'s comment) and params_fmt_out (the RFC 9051 + * body-fld-param list, already formatted as IMAP wire text: `("CHARSET" + * "US-ASCII")`-style, or "NIL" if there were no parameters). Separately + * captures the "boundary" parameter's raw (unescaped) value in + * boundary_out / has_boundary_out if present, needed by split_multipart() + * for multipart types -- extracted in the same single pass rather than + * re-parsing the field twice. + * + * RFC 2045 SS5.2: "Default RFC 822 messages without a MIME Content-Type + * header are taken by this protocol to be plain text in the US-ASCII + * character set... It is also recommended that this default be assumed + * when a syntactically invalid Content-Type header field is encountered." + * -- both the absent case and the type/subtype-unparseable case fall back + * to this exact default. A malformed *parameter* partway through an + * otherwise-valid "type/subtype" (e.g. a parameter with no "=", or an + * unterminated quoted-string value) does NOT trigger this fallback -- + * parameter parsing simply stops there, keeping type/subtype and whatever + * parameters were already successfully parsed, rather than discarding a + * perfectly good type/subtype over one bad trailing parameter. + * + * Returns 0 on success (always -- the RFC 2045 SS5.2 default means there's + * always *something* valid to report) or -1 only if writing params_fmt_out + * itself overflows params_fmt_outsize (propagated from envbuf_append*()), + * which the caller treats as a hard BODYSTRUCTURE_MAX-exceeded-style + * failure for this part. + */ +static int +parse_content_type(const char *hdr, size_t hdrlen, char *type_out, + size_t typesize, char *subtype_out, size_t subtypesize, + char *params_fmt_out, size_t params_fmt_outsize, char *boundary_out, + size_t boundary_outsize, int *has_boundary_out) +{ + char *val = NULL; + size_t vallen = 0; + size_t pos = 0; + size_t plen = 0; + int nparams = 0; + int use_default = 0; + /* + * envbuf_append()/envbuf_append_str()/envbuf_append_nstring() track + * length purely via the outlen pointer and never write a + * terminating NUL -- correct for their original ENVELOPE callers, + * which only ever consume the tracked length, never strlen(). This + * function's caller (build_body_structure()) does call + * strlen(params_fmt) on the finished buffer, so one byte of + * params_fmt_out's capacity is reserved here for an explicit NUL + * written at every return point below, rather than leaving that + * byte to whatever uninitialized stack content the caller's buffer + * happened to start with. (Found via real-hardware testing: a + * genuine multipart message's BODYSTRUCTURE came back with a run of + * stray bytes -- leftover stack content from the message's own + * base64 attachment data sitting in scope earlier -- spliced in + * right after the params list, because strlen() ran past the + * intended content looking for a NUL that was never written.) + */ + size_t pfsize = (params_fmt_outsize > 0) ? params_fmt_outsize - 1 : 0; + + *has_boundary_out = 0; + boundary_out[0] = '\0'; + + if (pfsize == 0) + return (-1); + + if (extract_header_field(hdr, hdrlen, "Content-Type", &val, + &vallen) == -1 || vallen == 0) + use_default = 1; + + if (!use_default && (mime_read_token_or_qstring(val, vallen, &pos, + type_out, typesize) == -1 || pos >= vallen || val[pos] != '/')) + use_default = 1; + if (!use_default) { + pos++; + if (mime_read_token_or_qstring(val, vallen, &pos, subtype_out, + subtypesize) == -1) + use_default = 1; + } + + if (use_default) { + free(val); + strlcpy(type_out, "TEXT", typesize); + strlcpy(subtype_out, "PLAIN", subtypesize); + if (envbuf_append_str(params_fmt_out, pfsize, &plen, + "(\"CHARSET\" \"US-ASCII\")") == -1) + return (-1); + params_fmt_out[plen] = '\0'; + return (0); + } + + mime_str_upper(type_out); + mime_str_upper(subtype_out); + + for (;;) { + char attr[64], value[256]; + + while (pos < vallen && (val[pos] == ' ' || val[pos] == '\t')) + pos++; + if (pos >= vallen || val[pos] != ';') + break; + pos++; + while (pos < vallen && (val[pos] == ' ' || val[pos] == '\t')) + pos++; + if (mime_read_token_or_qstring(val, vallen, &pos, attr, + sizeof(attr)) == -1) + break; + while (pos < vallen && (val[pos] == ' ' || val[pos] == '\t')) + pos++; + if (pos >= vallen || val[pos] != '=') + break; + pos++; + while (pos < vallen && (val[pos] == ' ' || val[pos] == '\t')) + pos++; + if (mime_read_token_or_qstring(val, vallen, &pos, value, + sizeof(value)) == -1) + break; + + mime_str_upper(attr); + if (envbuf_append_str(params_fmt_out, pfsize, + &plen, nparams == 0 ? "(" : " ") == -1 || + envbuf_append_nstring(params_fmt_out, pfsize, + &plen, attr, strlen(attr)) == -1 || + envbuf_append(params_fmt_out, pfsize, &plen, + " ", 1) == -1 || + envbuf_append_nstring(params_fmt_out, pfsize, + &plen, value, strlen(value)) == -1) { + free(val); + return (-1); + } + nparams++; + + if (strcasecmp(attr, "BOUNDARY") == 0) { + strlcpy(boundary_out, value, boundary_outsize); + *has_boundary_out = 1; + } + } + + free(val); + if (nparams > 0) { + if (envbuf_append_str(params_fmt_out, pfsize, &plen, + ")") == -1) + return (-1); + params_fmt_out[plen] = '\0'; + return (0); + } + if (envbuf_append_str(params_fmt_out, pfsize, &plen, "NIL") == -1) + return (-1); + params_fmt_out[plen] = '\0'; + return (0); +} + +/* + * RFC 2046 SS5.1.1: splits body[0..bodylen) -- a multipart Content-Type's + * own body, i.e. everything after that part/message's header/body blank + * line -- into its `body-part`s, per `multipart-body := [preamble CRLF] + * dash-boundary transport-padding CRLF body-part *encapsulation close- + * delimiter transport-padding [CRLF epilogue]`. Each returned span + * part_starts[i]..part_ends[i] is one body-part's raw bytes (its own + * MIME-part-headers through the end of its content, not yet split into + * header/body -- the caller does that separately per sub-part, via + * find_header_body_split(), since a sub-part's header/body separator is + * unrelated to this function's own boundary-scanning). + * + * A `dash-boundary` ("--" + the Content-Type's boundary parameter value) + * only counts as a real delimiter if it appears at the start of a line + * (position 0, or immediately after a CRLF or bare LF -- RFC 2046's own + * requirement that "Lines in a body-part must not start with the + * specified dash-boundary" means a conformant generator guarantees this + * check is sufficient) and is immediately followed -- after optional + * `transport-padding` (spaces/tabs) -- by either a CRLF/LF (a normal + * delimiter, more parts follow) or "--" then CRLF/LF/end-of-body (the + * close-delimiter, no more parts). Anything else at a boundary-shaped + * line start is treated as a coincidental non-match, not a delimiter. + * + * Returns 0 and *nparts_out >= 1 on success (a valid multipart body has at + * least the preamble's dash-boundary and a close-delimiter bracketing at + * least one body-part -- RFC 2046's own `1*body-part` isn't quite right + * here since this implementation doesn't distinguish "zero-body-part + * multipart" as separately illegal from "no close-delimiter found", both + * simply fail). Returns -1 (nothing found or usable) if the boundary + * parameter is malformed/oversized, no close-delimiter is ever found, or + * more than maxparts body-parts would result -- all treated by the caller + * as "can't build a BODYSTRUCTURE for this multipart part", not a partial + * result. + */ +static int +split_multipart(const char *body, size_t bodylen, const char *boundary, + size_t *part_starts, size_t *part_ends, int *nparts_out, int maxparts) +{ + char needle[2 + 70 + 1]; /* "--" + boundary; RFC 2046 SS5.1.1 + * caps boundary at 70 characters */ + size_t needlelen; + size_t pos = 0; + int found_first = 0; + int n = 0; + int closed = 0; + + strlcpy(needle, "--", sizeof(needle)); + needlelen = strlcat(needle, boundary, sizeof(needle)); + if (needlelen >= sizeof(needle)) + return (-1); /* boundary too long -- non-conformant */ + + *nparts_out = 0; + + while (pos < bodylen) { + int at_bol = (pos == 0) || + (pos >= 2 && body[pos - 2] == '\r' && + body[pos - 1] == '\n') || + (pos >= 1 && body[pos - 1] == '\n'); + size_t after; + int is_close = 0; + + if (!at_bol || pos + needlelen > bodylen || + memcmp(body + pos, needle, needlelen) != 0) { + pos++; + continue; + } + + after = pos + needlelen; + if (after + 1 < bodylen && body[after] == '-' && + body[after + 1] == '-') { + is_close = 1; + after += 2; + } + while (after < bodylen && + (body[after] == ' ' || body[after] == '\t')) + after++; + if (after < bodylen && body[after] != '\r' && + body[after] != '\n') { + /* not actually followed by CRLF/LF/end-of-body -- + * coincidental match inside real content, not a + * delimiter */ + pos++; + continue; + } + + if (found_first) { + size_t content_end = pos; + size_t part_start = part_starts[n - 1]; + + /* + * Bug found during manual security review: an empty + * body-part (this boundary line immediately following + * the previous one, with no content line between them) + * previously let this strip read/subtract bytes that + * actually belong to the *previous* boundary's own + * line-ending, not this (empty) part's content -- + * driving content_end below part_start. Every caller + * computes part_ends[i] - part_starts[i] as unsigned + * size_t arithmetic, so that underflowed to a + * near-SIZE_MAX length, later used as a memcpy/scan + * bound over attacker-controlled MIME content. + * Confirmed empirically with a standalone harness + * against body "--X\r\n--X--\r\n": content_end came out + * to pos-2 = 3, less than part_start = 5. Bounding the + * strip to never go below part_start keeps an empty + * part's length at a legitimate 0 instead. + */ + if (content_end >= part_start + 2 && + body[content_end - 2] == '\r' && + body[content_end - 1] == '\n') + content_end -= 2; + else if (content_end >= part_start + 1 && + body[content_end - 1] == '\n') + content_end -= 1; + part_ends[n - 1] = content_end; + } + + if (is_close) { + closed = 1; + break; + } + + if (after < bodylen && body[after] == '\r' && + after + 1 < bodylen && body[after + 1] == '\n') + after += 2; + else if (after < bodylen && body[after] == '\n') + after += 1; + else + break; /* delimiter runs to end of body with no + * CRLF -- no body-part can follow */ + + if (n >= maxparts) + return (-1); + part_starts[n] = after; + n++; + found_first = 1; + pos = after; + } + + if (!closed || !found_first) + return (-1); + + *nparts_out = n; + return (0); +} + +/* + * Recursively builds one RFC 9051 SS7.5.2 `body` -- `"(" (body-type-1part / + * body-type-mpart) ")"` -- for the MIME entity whose header is hdr[0.. + * hdrlen) and whose (still-encoded) content is body[0..bodylen), appending + * the formatted text directly onto out/outlen (the same shared buffer the + * whole recursive walk writes into, so a deeply nested structure is built + * with a single top-level BODYSTRUCTURE_MAX size check rather than N + * separate per-part buffers). + * + * depth/nparts_used together enforce MIME_MAX_DEPTH/MIME_MAX_PARTS + * (imapd.h's comment on those constants has the full "message content + * is attacker/sender-controlled, both need a hard ceiling" reasoning): + * depth increases by exactly one per multipart nesting level (checked + * before doing any work at this level); nparts_used is a single counter + * threaded through the *entire* recursive call tree by pointer (not a + * per-multipart-parent count), incremented once per call regardless of + * whether this call turns out to be a leaf or another multipart container, + * so it bounds total part count across the whole structure, not just + * immediate siblings. + * + * Three shapes, matching RFC 9051's body-type-1part/body-type-mpart + * grammar: + * + * - MULTIPART: requires a "boundary" Content-Type parameter (RFC 2046 + * SS5.1.1: "The only mandatory global parameter for the multipart + * media type is the boundary parameter"); split_multipart() carves + * body into its body-parts, each of which is itself split into + * header/body (find_header_body_split()) and recursed into. Formatted + * as `"(" body body ... SP media-subtype ")"` -- RFC 9051's own + * SS7.5.2 example shows adjacent address/body tuples with no + * separator between them, since each is already self-delimiting via + * its own parens. + * - message/rfc822 or message/global: scoped out entirely (see this + * section's header comment) -- returns -1, which propagates as + * "BODYSTRUCTURE not found" for the whole top-level message, not a + * partially-correct structure with a fake or missing part. + * - Everything else (body-type-basic / body-type-text): a single, + * non-multipart part. body-fields (RFC 9051 SS9: `body-fld-param SP + * body-fld-id SP body-fld-desc SP body-fld-enc SP body-fld-octets`) + * are always emitted in full -- these are required basic fields, not + * the optional extension data this implementation omits (see this + * section's header comment) -- followed by body-fld-lines only for + * "TEXT" parts (body-type-text's own extra field). body-fld-octets is + * simply bodylen: RFC 9051 SS7.5.2 "the size in its transfer encoding + * and not the resulting size after any decoding" means the *encoded* + * byte count is exactly right, with no need to ever actually decode + * base64/quoted-printable content. body-fld-lines similarly counts + * raw '\n' occurrences in the still-encoded body bytes, not decoded + * text lines -- same "size in its content transfer encoding" framing, + * and the counting convention real servers commonly use. + * + * Returns 0 on success, -1 (propagated all the way up to build_ + * bodystructure()'s caller as "not found") on any depth/part-count/ + * malformed-multipart/message-rfc822/output-overflow failure. + */ +static int +build_body_structure(int depth, int *nparts_used, const char *hdr, + size_t hdrlen, const char *body, size_t bodylen, char *out, + size_t outsize, size_t *outlen) +{ + char type[64], subtype[64]; + char params_fmt[600]; + char boundary[70 + 1]; /* RFC 2046 SS5.1.1 caps boundary at + * 70 characters, +1 for NUL */ + int has_boundary; + + if (depth > MIME_MAX_DEPTH) + return (-1); + if (++*nparts_used > MIME_MAX_PARTS) + return (-1); + + params_fmt[0] = '\0'; + if (parse_content_type(hdr, hdrlen, type, sizeof(type), subtype, + sizeof(subtype), params_fmt, sizeof(params_fmt), boundary, + sizeof(boundary), &has_boundary) == -1) + return (-1); + + if (strcasecmp(type, "MULTIPART") == 0) { + size_t part_starts[MIME_MAX_PARTS], part_ends[MIME_MAX_PARTS]; + int n, i; + + if (!has_boundary) + return (-1); + if (split_multipart(body, bodylen, boundary, part_starts, + part_ends, &n, MIME_MAX_PARTS) == -1) + return (-1); + + if (envbuf_append(out, outsize, outlen, "(", 1) == -1) + return (-1); + for (i = 0; i < n; i++) { + const char *pbuf = body + part_starts[i]; + size_t plen = part_ends[i] - part_starts[i]; + size_t phdrend; + + /* + * A genuinely empty body-part (RFC 2046 SS5.1.1's + * body-part := MIME-part-headers [CRLF *OCTET] -- + * the CRLF-and-octets half is itself optional, so + * zero bytes between two boundary delimiters is + * spec-legal, not malformed) has no header/body + * separator to find at all, by construction. Found + * via real-hardware testing once the split_multipart() + * empty-part fix (above) started correctly producing + * plen == 0 for a part like this, instead of the + * huge underflowed length that used to reach here + * first: find_header_body_split() correctly reports + * "not found" for a zero-length buffer (there IS no + * separator in zero bytes), but treating that as this + * whole part's failure -- and this loop returning -1 + * for the *entire* multipart structure the moment any + * single part fails -- meant one legitimately empty + * part broke BODYSTRUCTURE for the whole message. + * Treat plen == 0 as hdrlen == 0 / bodylen == 0 + * directly instead: parse_content_type() with + * hdrlen == 0 already falls through its own + * "Content-Type field absent" path to the RFC 2045 + * SS5.2 default (text/plain; charset=us-ascii), so + * this recurses into an ordinary empty leaf part + * rather than aborting. + */ + if (plen == 0) + phdrend = 0; + else if (find_header_body_split(pbuf, plen, + &phdrend) == -1) + return (-1); + if (build_body_structure(depth + 1, nparts_used, pbuf, + phdrend, pbuf + phdrend, plen - phdrend, out, + outsize, outlen) == -1) + return (-1); + } + if (envbuf_append(out, outsize, outlen, " ", 1) == -1) + return (-1); + if (envbuf_append_nstring(out, outsize, outlen, subtype, + strlen(subtype)) == -1) + return (-1); + return (envbuf_append(out, outsize, outlen, ")", 1)); + } + + if (strcasecmp(type, "MESSAGE") == 0 && + (strcasecmp(subtype, "RFC822") == 0 || + strcasecmp(subtype, "GLOBAL") == 0)) + return (-1); /* scoped out -- see this section's header + * comment */ + + { + char *idval = NULL, *descval = NULL, *encval = NULL; + size_t idlen = 0, desclen = 0, enclen = 0; + char encstr[40]; + int is_text = (strcasecmp(type, "TEXT") == 0); + int rc = 0; + + if (envbuf_append(out, outsize, outlen, "(", 1) == -1) + return (-1); + if (envbuf_append_nstring(out, outsize, outlen, type, + strlen(type)) == -1) + return (-1); + if (envbuf_append(out, outsize, outlen, " ", 1) == -1) + return (-1); + if (envbuf_append_nstring(out, outsize, outlen, subtype, + strlen(subtype)) == -1) + return (-1); + if (envbuf_append(out, outsize, outlen, " ", 1) == -1) + return (-1); + if (envbuf_append(out, outsize, outlen, params_fmt, + strlen(params_fmt)) == -1) + return (-1); + if (envbuf_append(out, outsize, outlen, " ", 1) == -1) + return (-1); + + if (extract_header_field(hdr, hdrlen, "Content-Id", &idval, + &idlen) == 0) + rc = envbuf_append_nstring(out, outsize, outlen, idval, + idlen); + else + rc = envbuf_append_nstring(out, outsize, outlen, NULL, 0); + free(idval); + if (rc == -1) + return (-1); + if (envbuf_append(out, outsize, outlen, " ", 1) == -1) + return (-1); + + if (extract_header_field(hdr, hdrlen, "Content-Description", + &descval, &desclen) == 0) + rc = envbuf_append_nstring(out, outsize, outlen, + descval, desclen); + else + rc = envbuf_append_nstring(out, outsize, outlen, NULL, 0); + free(descval); + if (rc == -1) + return (-1); + if (envbuf_append(out, outsize, outlen, " ", 1) == -1) + return (-1); + + if (extract_header_field(hdr, hdrlen, + "Content-Transfer-Encoding", &encval, &enclen) == 0 && + enclen > 0 && enclen < sizeof(encstr)) { + memcpy(encstr, encval, enclen); + encstr[enclen] = '\0'; + } else { + strlcpy(encstr, "7BIT", sizeof(encstr)); + } + free(encval); + { + static const char *const known[] = { "7BIT", "8BIT", + "BINARY", "BASE64", "QUOTED-PRINTABLE" }; + size_t ki; + + for (ki = 0; ki < 5; ki++) { + if (strcasecmp(encstr, known[ki]) == 0) { + strlcpy(encstr, known[ki], + sizeof(encstr)); + break; + } + } + } + if (envbuf_append_nstring(out, outsize, outlen, encstr, + strlen(encstr)) == -1) + return (-1); + if (envbuf_append(out, outsize, outlen, " ", 1) == -1) + return (-1); + + { + char numbuf[32]; + + snprintf(numbuf, sizeof(numbuf), "%zu", bodylen); + if (envbuf_append_str(out, outsize, outlen, numbuf) == -1) + return (-1); + } + + if (is_text) { + size_t lines = 0, li; + char numbuf2[32]; + + for (li = 0; li < bodylen; li++) { + if (body[li] == '\n') + lines++; + } + snprintf(numbuf2, sizeof(numbuf2), " %zu", lines); + if (envbuf_append_str(out, outsize, outlen, numbuf2) == -1) + return (-1); + } + + return (envbuf_append(out, outsize, outlen, ")", 1)); + } +} + +/* + * Top-level entry point: reads the whole message once (read_message_body(), + * text_only=0 -- same underlying whole-message reader BODY.PEEK[] uses, + * but capped at the much larger bodystructure_read_max rather than + * APPEND_LITERAL_MAX -- see read_message_body()'s own comment for why + * these two callers need different caps), locates its own header/body + * split, and kicks off the recursive walk at depth 0. Returns 0 and a + * malloc(3)'d *buf_out (caller frees) + *len_out on success; -1 (nothing + * to free) if the message can't be read at all, has no header/body + * separator, or build_body_structure() failed for any of its own + * documented reasons (depth/part-count exceeded, malformed multipart, a + * message/rfc822 part, or BODYSTRUCTURE_MAX exceeded) -- all "not found" + * for this message's BODYSTRUCTURE, same failure handling as every other + * content item in this file. + */ +static int +build_bodystructure(const char *basename, char **buf_out, uint32_t *len_out) +{ + char *wholebuf = NULL; + uint32_t wholelen = 0; + size_t hdrend; + char out[BODYSTRUCTURE_MAX]; + size_t outlen = 0; + int nparts_used = 0; + + *buf_out = NULL; + *len_out = 0; + + if (read_message_body(basename, 0, bodystructure_read_max, + "BODYSTRUCTURE", &wholebuf, &wholelen) == -1) + return (-1); + if (wholelen == 0 || + find_header_body_split(wholebuf, wholelen, &hdrend) == -1) { + free(wholebuf); + return (-1); + } + + if (build_body_structure(0, &nparts_used, wholebuf, hdrend, + wholebuf + hdrend, wholelen - hdrend, out, sizeof(out), + &outlen) == -1) { + free(wholebuf); + return (-1); + } + free(wholebuf); + + if ((*buf_out = malloc(outlen)) == NULL) { + log_warn("session %u: malloc BODYSTRUCTURE buffer (%s)", + session_id, basename); + return (-1); + } + memcpy(*buf_out, out, outlen); + *len_out = (uint32_t)outlen; + return (0); +} + +/* + * Parses a dotted-numeric section-part string (RFC 9051 SS6.4.5.1's + * section-part = nz-number *("." nz-number), e.g. "3.1") into an array of + * 1-based part numbers. listener.c's tokenizer has already validated this + * exact grammar before ever setting struct imsg_mbox_fetch's section_part + * (see that field's comment in imapd.h) -- this function re-validates + * anyway, cheap insurance rather than trusting a cross-process field + * unconditionally, matching how every other store.c parser in this file + * (parse_content_type(), the index line parser, etc.) treats its own input + * as untrusted regardless of what already checked it upstream. + * + * Returns the number of parts on success (>= 1), or -1 if s is empty, + * contains anything but digits and '.', has a leading-zero or otherwise + * non-nz-number component, or has more components than maxpath (the + * caller passes MIME_MAX_DEPTH, the same bound build_body_structure()'s + * own recursion enforces -- a path deeper than that could never match + * anything build_bodystructure() would have described in the first + * place). + */ +static int +parse_section_part(const char *s, int *path, int maxpath) +{ + int n = 0; + + if (s == NULL || *s == '\0') + return (-1); + + while (*s != '\0') { + long val; + char *end; + + if (*s < '1' || *s > '9') /* nz-number: digit-nz first */ + return (-1); + if (n >= maxpath) + return (-1); + + errno = 0; + val = strtol(s, &end, 10); + if (val <= 0 || val > INT_MAX || errno != 0) + return (-1); + path[n++] = (int)val; + + s = end; + if (*s == '\0') + break; + if (*s != '.') + return (-1); + s++; + if (*s == '\0') + return (-1); /* trailing dot */ + } + return (n); +} + +/* + * Recursive descent used only once path[0] has already been established to + * select among hdr/body's own children -- i.e. hdr/body is known-MULTIPART + * before this is ever called (see locate_mime_part() below, the only + * caller). Mirrors build_body_structure()'s own MULTIPART branch almost + * exactly (same parse_content_type()/split_multipart()/find_header_body_ + * split() calls, same depth cap), but walks down exactly one child per + * level -- the one path[0] names -- instead of visiting every child to + * format output text. + * + * path[0..pathlen) is remaining, relative to hdr/body: path[0] selects + * which child of *this* multipart to descend into; the rest resolves + * relative to that child. pathlen is always >= 1 on entry (locate_mime_ + * part() guarantees this before the first call, and this function only + * ever recurses with pathlen - 1 after confirming the next level is + * itself MULTIPART, so a pathlen of 0 here would be a caller bug, not a + * legal "arrived" state -- unlike build_body_structure(), there is no + * separate depth-0-only entry point to special-case, since that's + * locate_mime_part()'s job). + * + * Returns 0 and sets *part_out / *partlen_out to the target leaf part's raw + * (still transfer-encoded) body bytes on success. Returns -1 if: depth + * exceeds MIME_MAX_DEPTH; path[0] names a child that doesn't exist at + * this level; the named child fails to parse as a header/body pair; or + * (recursing further) the child path continues into isn't itself + * MULTIPART where more path remains, or is a MULTIPART/MESSAGE-typed part + * where no path remains (a container, not a leaf -- see MBOX_FETCH_BODY_ + * PART's imapd.h comment for why only leaf parts are returned) -- + * detected next call in either case, since every call re-parses its own + * hdr/body's Content-Type first. + */ +static int +find_mime_part(int depth, const char *hdr, size_t hdrlen, const char *body, + size_t bodylen, const int *path, int pathlen, const char **part_out, + size_t *partlen_out) +{ + char type[64], subtype[64]; + char params_fmt[600]; + char boundary[70 + 1]; + int has_boundary; + size_t part_starts[MIME_MAX_PARTS], part_ends[MIME_MAX_PARTS]; + int n, want; + const char *pbuf; + size_t plen, phdrend; + + if (depth > MIME_MAX_DEPTH) + return (-1); + + params_fmt[0] = '\0'; + if (parse_content_type(hdr, hdrlen, type, sizeof(type), subtype, + sizeof(subtype), params_fmt, sizeof(params_fmt), boundary, + sizeof(boundary), &has_boundary) == -1) + return (-1); + if (strcasecmp(type, "MULTIPART") != 0 || !has_boundary) + return (-1); + if (split_multipart(body, bodylen, boundary, part_starts, part_ends, + &n, MIME_MAX_PARTS) == -1) + return (-1); + + want = path[0]; + if (want < 1 || want > n) + return (-1); /* no such part at this level */ + + pbuf = body + part_starts[want - 1]; + plen = part_ends[want - 1] - part_starts[want - 1]; + /* Same empty-part handling as build_body_structure() above -- see + * its comment. */ + if (plen == 0) + phdrend = 0; + else if (find_header_body_split(pbuf, plen, &phdrend) == -1) + return (-1); + + if (pathlen == 1) { + /* + * path fully consumed by selecting this child. It must + * itself be a genuine leaf (not MULTIPART, not MESSAGE/ + * RFC822|GLOBAL) to be returnable -- both remain out of + * v1 scope, same cut build_body_structure() already makes + * for MESSAGE/RFC822|GLOBAL, extended here to MULTIPART + * containers too, since there is no "combined raw bytes of + * all my children" concept to return for one. + */ + char ctype[64], csub[64], cparams[600]; + char cboundary[70 + 1]; + int chb; + + cparams[0] = '\0'; + if (parse_content_type(pbuf, phdrend, ctype, sizeof(ctype), + csub, sizeof(csub), cparams, sizeof(cparams), cboundary, + sizeof(cboundary), &chb) == -1) + return (-1); + if (strcasecmp(ctype, "MULTIPART") == 0 || + (strcasecmp(ctype, "MESSAGE") == 0 && + (strcasecmp(csub, "RFC822") == 0 || + strcasecmp(csub, "GLOBAL") == 0))) + return (-1); + + *part_out = pbuf + phdrend; + *partlen_out = plen - phdrend; + return (0); + } + + return (find_mime_part(depth + 1, pbuf, phdrend, pbuf + phdrend, + plen - phdrend, path + 1, pathlen - 1, part_out, partlen_out)); +} + +/* + * Top-level entry point for locating one leaf MIME part's raw body bytes + * by dotted-numeric path (path[0..pathlen), from parse_section_part()). + * Handles a case find_mime_part() itself deliberately doesn't: RFC 9051 + * SS6.4.5.1's "Every message has at least one part number... Messages + * that do not use MIME, ... only have a part 1" -- i.e. a non-multipart + * top-level message's *own* body is addressed as section-part "1", with + * nothing to descend into, which is a different rule than any nested + * level (where path[0] always selects a real child of a real MULTIPART + * container). Folding that special case into find_mime_part() itself + * would make its own contract asymmetric between depth 0 and every other + * depth; splitting it out here keeps both functions' contracts simple. + * + * Returns 0 and sets *part_out / *partlen_out on success. Returns -1 if: + * pathlen is 0; the top-level message is non-multipart (or itself + * MESSAGE/RFC822|GLOBAL, also scoped out -- see find_mime_part()'s + * comment) and path isn't exactly {1}; or find_mime_part() failed for + * any of its own documented reasons. + */ +static int +locate_mime_part(const char *hdr, size_t hdrlen, const char *body, + size_t bodylen, const int *path, int pathlen, const char **part_out, + size_t *partlen_out) +{ + char type[64], subtype[64]; + char params_fmt[600]; + char boundary[70 + 1]; + int has_boundary; + + if (pathlen < 1) + return (-1); + + params_fmt[0] = '\0'; + if (parse_content_type(hdr, hdrlen, type, sizeof(type), subtype, + sizeof(subtype), params_fmt, sizeof(params_fmt), boundary, + sizeof(boundary), &has_boundary) == -1) + return (-1); + + if (strcasecmp(type, "MULTIPART") == 0) + return (find_mime_part(1, hdr, hdrlen, body, bodylen, path, + pathlen, part_out, partlen_out)); + + if (pathlen != 1 || path[0] != 1) + return (-1); + *part_out = body; + *partlen_out = bodylen; + return (0); +} + +/* + * Applies a <> range (RFC 9051 SS6.4.5's "") to + * content[0..contentlen), used uniformly by handle_mbox_fetch() for + * BODY.PEEK[]/BODY.PEEK[TEXT]/BODY.PEEK[] alike -- content + * itself is never copied here, only *out / *outlen (a subrange of content) + * computed; the caller does the one real memcpy(3), same "compute + * offsets, let the caller own the copy" shape as every other length- + * tracking helper in this file. + * + * !has_partial (no <<...>> suffix in the request) means "the whole + * content, subject only to FETCH_PART_MAX" -- *outlen is silently + * clamped down to that cap here (empty-string special case aside, RFC + * 9051 doesn't distinguish a server-imposed cap from any other partial + * response; a client that wants guaranteed-complete large content is + * expected to use its own <> ranging, same real-world pattern + * observed from Apple Mail's own "BODY.PEEK[TEXT]<0.16384>" traffic). + * has_partial clamps the requested partial_count down to FETCH_PART_MAX + * too, and separately clamps partial_start/partial_count against + * contentlen itself per SS6.4.5: "If the starting octet is beyond the + * end of the text, an empty string is returned... Any partial fetch that + * attempts to read beyond the end of the text is truncated as + * appropriate." + */ +static void +apply_partial_range(const char *content, size_t contentlen, int has_partial, + uint32_t partial_start, uint32_t partial_count, const char **out, + size_t *outlen) +{ + if (!has_partial) { + *out = content; + *outlen = contentlen; + if (*outlen > FETCH_PART_MAX) + *outlen = FETCH_PART_MAX; + return; + } + + if (partial_start >= contentlen) { + *out = content; + *outlen = 0; + return; + } + + *out = content + partial_start; + *outlen = contentlen - partial_start; + if (*outlen > partial_count) + *outlen = partial_count; + if (*outlen > FETCH_PART_MAX) + *outlen = FETCH_PART_MAX; +} + +/* + * Top-level entry point for BODY.PEEK[]: reads the whole + * message once (read_message_body(), text_only=0, bodystructure_read_max + * -- same large cap build_bodystructure() uses, and for the same reason: + * a message can legitimately be far larger than APPEND_LITERAL_MAX once + * it carries an attachment, even though what's ultimately returned here + * -- one leaf part, sliced -- is small), locates its own header/body + * split, resolves path[0..pathlen) via locate_mime_part(), then applies + * the caller's partial range via apply_partial_range(). Returns 0 and a + * malloc(3)'d *buf_out (caller frees; left NULL if *len_out comes back 0) + * + *len_out on success. Returns -1 (nothing to free) if the message + * can't be read, has no header/body separator, or locate_mime_part() + * failed for any of its own documented reasons -- all "not found" for + * this content item, same failure handling as every other one in this + * file. + */ +static int +extract_mime_part(const char *basename, const int *path, int pathlen, + int has_partial, uint32_t partial_start, uint32_t partial_count, + char **buf_out, uint32_t *len_out) +{ + char *wholebuf = NULL; + uint32_t wholelen = 0; + size_t hdrend; + const char *part = NULL; + size_t partlen = 0; + const char *out; + size_t outlen; + + *buf_out = NULL; + *len_out = 0; + + if (read_message_body(basename, 0, bodystructure_read_max, + "BODY[]", &wholebuf, &wholelen) == -1) + return (-1); + if (wholelen == 0 || + find_header_body_split(wholebuf, wholelen, &hdrend) == -1) { + free(wholebuf); + return (-1); + } + + if (locate_mime_part(wholebuf, hdrend, wholebuf + hdrend, + wholelen - hdrend, path, pathlen, &part, &partlen) == -1) { + free(wholebuf); + return (-1); + } + + apply_partial_range(part, partlen, has_partial, partial_start, + partial_count, &out, &outlen); + + if (outlen > 0) { + if ((*buf_out = malloc(outlen)) == NULL) { + log_warn("session %u: malloc BODY[] buffer (%s)", + session_id, basename); + free(wholebuf); + return (-1); + } + memcpy(*buf_out, out, outlen); + } + *len_out = (uint32_t)outlen; + free(wholebuf); + return (0); +} + +/* + * Builds the space-separated IMAP flag-atom list for one message from two + * sources: the maildir flag-suffix letters (from locate_message_file()'s + * suffix_out, e.g. ":2,FS" -- "" if the message is still in new/, + * unflagged) and the index's own comma-separated keyword field. Letter -> + * flag mapping is Courier's maildir(5) (already sourced in openimap- + * storage-backend.md): D=Draft, F=Flagged, R=Answered (historically + * "Replied"), S=Seen, T=Deleted (historically "Trashed"). + */ +static void +build_flags_string(const char *maildir_suffix, const char *keywords, + char *out, size_t outsize) +{ + static const struct { + char letter; + const char *flag; + } stdflags[] = { + { 'D', "\\Draft" }, + { 'F', "\\Flagged" }, + { 'R', "\\Answered" }, + { 'S', "\\Seen" }, + { 'T', "\\Deleted" }, + }; + const char *letters; + size_t i; + int first = 1; + + out[0] = '\0'; + + letters = strstr(maildir_suffix, "2,"); + letters = (letters != NULL) ? letters + 2 : ""; + + for (i = 0; i < sizeof(stdflags) / sizeof(stdflags[0]); i++) { + if (strchr(letters, stdflags[i].letter) == NULL) + continue; + if (!first) + strlcat(out, " ", outsize); + strlcat(out, stdflags[i].flag, outsize); + first = 0; + } + + if (keywords != NULL && keywords[0] != '\0') { + char kwbuf[512]; + char *kw, *save; + + strlcpy(kwbuf, keywords, sizeof(kwbuf)); + for (kw = strtok_r(kwbuf, ",", &save); kw != NULL; + kw = strtok_r(NULL, ",", &save)) { + if (!first) + strlcat(out, " ", outsize); + strlcat(out, kw, outsize); + first = 0; + } + } +} + +/* + * RFC 9051 SS2.3.1.1 INTERNALDATE is "the internal date of the message" -- + * this implementation's own choice (not something the design docs + * resolve) is the delivery timestamp already encoded in the maildir + * basename's own leading field (Courier maildir(5)'s unique-name format, + * `..`, already sourced in openimap- + * storage-backend.md), not the file's mtime -- more portable across + * backups/copies that can touch mtime without touching content. Falls + * back to the current time for a foreign/hand-placed file that doesn't + * follow that convention, rather than failing the whole FETCH over one + * oddly-named message. + */ +static int64_t +parse_maildir_timestamp(const char *basename) +{ + char *ep; + long long ts; + + errno = 0; + ts = strtoll(basename, &ep, 10); + if (ep == basename || *ep != '.' || errno != 0 || ts < 0) + return ((int64_t)time(NULL)); + return ((int64_t)ts); +} + +/* + * IMSG_MBOX_FETCH handling -- v1 scope is message metadata only (FLAGS, + * UID, INTERNALDATE, RFC822.SIZE), one sequence-set range per request; see + * imapd.h's imsg_mbox_fetch comment and listener.c's cmd_fetch() for + * the full reasoning. Sends one IMSG_MBOX_FETCH_META per matching message, + * in ascending sequence order (the index is UID-ordered; meta.seqno is + * always this message's live 1-based position within the index freshly + * loaded a few lines below, i.e. i itself in the loop below -- never a + * separately cached number, so it's automatically correct even though + * EXPUNGE now exists and can renumber later messages between one FETCH + * and the next; a correction to this comment's own earlier claim, from + * before EXPUNGE was implemented, that index order and sequence-number + * order were "always identical" specifically because EXPUNGE didn't exist + * yet -- the real reason is simpler and still holds regardless: there was + * never any separate bookkeeping to go stale in the first place), then + * exactly one terminal IMSG_MBOX_RESULT. + * + * Uses LOCK_SH (not handle_mbox_select()'s LOCK_EX): FETCH only reads the + * index, never mutates it, so multiple concurrent FETCHes (from different + * store children for the same user -- fork-per-session, not + * fork-per-user) can proceed together; a shared lock still blocks against + * a concurrent SELECT's exclusive lock, so a FETCH can never observe an + * index mid-rewrite (belt-and-suspenders on top of the rename(2)-based + * atomicity that already prevents a torn read either way). + * + * RFC 7162 additions this pass: meta.modseq is always populated from the + * index's per-message field (index_parse_line()) -- cheap, already parsed + * -- and req->has_changedsince (CHANGEDSINCE fetch-modifier, SS3.1.4.1) + * skips any message whose mod-sequence is not strictly greater than the + * given value, exactly the "only returned for messages that have a + * mod-sequence bigger than " rule. req->attrs & MBOX_FETCH_ + * MODSEQ is not consulted here at all -- meta.modseq is unconditionally + * computed regardless (matching every other field in this struct), and it + * is listener.c's session_send_fetch_response() that decides whether to + * print it, same split as FLAGS/UID/INTERNALDATE/RFC822.SIZE already use. + * + * RFC 9051 SS6.4.9 addition (UID command): req->by_uid switches lo/hi (and + * "*") to UID-space resolution instead of position-space, and req->want_ + * vanished (RFC 7162 SS3.2.6) reports gaps in that range as VANISHED + * (EARLIER) via send_vanished_range() before any FETCH response -- see + * both fields' comments in imapd.h and this function's own lo/hi- + * resolution comment below. index order and sequence-number order are + * still always identical regardless of by_uid (v1 has EXPUNGE now, but + * meta.seqno = i is still each message's live, correctly-numbered + * position in the just-loaded idx -- EXPUNGE only ever runs under its own + * exclusive lock, never concurrently with this shared-locked read). + */ +static void +handle_mbox_fetch(struct imsg_mbox_fetch *req, struct imsgev *iev) +{ + struct mbox_index idx; + struct imsg_mbox_result result; + int fd; + uint32_t lo, hi, i, sent = 0; + int ok = 1; + + memset(&idx, 0, sizeof(idx)); + + if ((fd = open(STORE_INDEX_NAME, O_RDWR | O_CREAT, 0600)) == -1) { + log_warn("session %u: open %s", session_id, STORE_INDEX_NAME); + ok = 0; + goto done; + } + if (flock(fd, LOCK_SH) == -1) { + log_warn("session %u: flock %s", session_id, STORE_INDEX_NAME); + close(fd); + ok = 0; + goto done; + } + if (index_load(fd, &idx) == -1) { + flock(fd, LOCK_UN); + close(fd); + ok = 0; + goto done; + } + flock(fd, LOCK_UN); + close(fd); + + /* + * RFC 9051 SS6.4.9: UID FETCH's sequence-set argument is UIDs, not + * positions -- lo/hi (and "*") resolve against index_max_uid(), the + * same UID-space upper bound SEARCH's UIDSET criterion already uses, + * not idx.nlines. The hi-clamp-to-nlines below only makes sense in + * position space, so it's skipped for by_uid -- the unified loop + * below bounds itself against rec.uid instead. + */ + if (req->by_uid) { + uint32_t max_uid = index_max_uid(&idx); + + lo = req->lo_is_star ? max_uid : req->seq_lo; + hi = req->hi_is_star ? max_uid : req->seq_hi; + } else { + lo = req->lo_is_star ? (uint32_t)idx.nlines : req->seq_lo; + hi = req->hi_is_star ? (uint32_t)idx.nlines : req->seq_hi; + } + if (lo < 1) + lo = 1; + if (!req->by_uid && hi > (uint32_t)idx.nlines) + hi = (uint32_t)idx.nlines; + + /* + * RFC 7162 SS3.2.6 VANISHED UID FETCH modifier: report gaps in + * [lo, hi] as VANISHED (EARLIER) *before* any FETCH response below + * ("Any VANISHED (EARLIER) responses MUST be returned before any + * FETCH responses") -- guaranteed here purely by send order, since + * imsg messages on one channel are read by listener.c in the order + * they're sent, and this call is textually before the loop that + * sends IMSG_MBOX_FETCH_META below. + */ + if (req->by_uid && req->want_vanished) + send_vanished_range(&idx, lo, hi, iev); + + for (i = 1; i <= (uint32_t)idx.nlines; i++) { + struct imsg_mbox_fetch_meta meta; + struct index_rec rec; + off_t size = 0; + char suffix[64]; + int have_file = 0; + + if (index_parse_line(idx.lines[i - 1], &rec) == -1) + continue; + + /* + * Position-space (i) or UID-space (rec.uid) range check + * depending on req->by_uid -- see the lo/hi resolution + * above. Both sides are ascending (i by loop construction, + * rec.uid because the index itself is UID-ordered), so a + * plain "below range, skip; above range, stop" test is + * correct and only ever makes one pass over idx.lines. + */ + if (req->by_uid) { + if (rec.uid < lo) + continue; + if (rec.uid > hi) + break; + } else { + if (i < lo) + continue; + if (i > hi) + break; + } + + if (req->has_changedsince && rec.modseq <= req->changedsince) + continue; + + memset(&meta, 0, sizeof(meta)); + meta.seqno = i; + meta.uid = rec.uid; + meta.modseq = rec.modseq; + + if (req->attrs & (MBOX_FETCH_RFC822_SIZE | MBOX_FETCH_FLAGS)) { + if (locate_message_file(rec.basename, &size, suffix, + sizeof(suffix)) == -1) { + log_warnx("session %u: message %s (uid %u) " + "indexed but missing on disk -- skipped", + session_id, rec.basename, meta.uid); + continue; + } + have_file = 1; + } + + if (req->attrs & MBOX_FETCH_RFC822_SIZE) + meta.size = (uint64_t)size; /* F13 fix: was (uint32_t) */ + if (req->attrs & MBOX_FETCH_FLAGS) + build_flags_string(have_file ? suffix : "", + rec.keywords, meta.flags, sizeof(meta.flags)); + if (req->attrs & MBOX_FETCH_INTERNALDATE) + meta.internaldate = parse_maildir_timestamp( + rec.basename); + + /* + * IMSG_MBOX_FETCH_HEADER, sent before this message's own + * IMSG_MBOX_FETCH_META -- see that struct's comment in + * imapd.h for why listener.c depends on this exact + * ordering. Always sent (with found=0 on any failure) when + * either MBOX_FETCH_BODY_HEADER or MBOX_FETCH_HEADER_FIELDS + * was requested, rather than only sent on success, so + * listener.c never has to distinguish "no header message + * arrived for this seqno" from "one legitimately hasn't been + * processed yet" -- it just checks found on whatever it + * gets. The two requested-item kinds share this one imsg/ + * struct wholesale (see MBOX_FETCH_HEADER_FIELDS's comment + * in imapd.h for why) -- listener.c's parse_fetch_atts() + * already made HEADER win if a client somehow requested both + * in the same FETCH, so at most one of the two bits is set + * here in practice, but the check below still prefers plain + * HEADER defensively either way. + */ + if (req->attrs & (MBOX_FETCH_BODY_HEADER | + MBOX_FETCH_HEADER_FIELDS)) { + struct imsg_mbox_fetch_header hdrmeta; + char *hdrbuf = NULL; + uint32_t hdrlen = 0; + char *combined; + size_t combined_len; + int rc; + + memset(&hdrmeta, 0, sizeof(hdrmeta)); + hdrmeta.seqno = i; + hdrmeta.uid = rec.uid; + if (req->attrs & MBOX_FETCH_BODY_HEADER) + rc = read_message_header(rec.basename, + &hdrbuf, &hdrlen); + else + rc = read_message_header_fields(rec.basename, + req->header_fields, + req->header_fields_not, &hdrbuf, &hdrlen); + if (rc == 0) { + hdrmeta.found = 1; + hdrmeta.hdrlen = hdrlen; + } + + combined_len = sizeof(hdrmeta) + + (hdrmeta.found ? hdrlen : 0); + if ((combined = malloc(combined_len)) == NULL) { + log_warn("session %u: malloc " + "IMSG_MBOX_FETCH_HEADER buffer", + session_id); + } else { + memcpy(combined, &hdrmeta, sizeof(hdrmeta)); + if (hdrmeta.found) + memcpy(combined + sizeof(hdrmeta), + hdrbuf, hdrlen); + if (imsg_compose(&iev->ibuf, + IMSG_MBOX_FETCH_HEADER, 0, 0, -1, combined, + combined_len) == -1) + log_warn("session %u: imsg_compose " + "IMSG_MBOX_FETCH_HEADER", + session_id); + free(combined); + } + free(hdrbuf); + } + + /* + * IMSG_MBOX_FETCH_BODY, same "always sent when requested, + * found=0 on any failure, ordered right before this + * message's own IMSG_MBOX_FETCH_META" contract as the + * IMSG_MBOX_FETCH_HEADER block just above -- see struct + * imsg_mbox_fetch_body's comment in imapd.h. WHOLE wins + * over TEXT if a client somehow requested both (see MBOX_ + * FETCH_BODY_TEXT's comment in imapd.h for why that's a + * deliberate, not accidental, choice); listener.c's tokenizer + * never sets more than one of WHOLE/TEXT/PART together (each + * comes from a distinct, mutually exclusive section-spec), so + * PART is simply a third case here, not a fourth combination + * to disambiguate. + * + * Partial-range handling (<>, req->has_partial/ + * partial_start/partial_count) applies uniformly across all + * three: for PART, extract_mime_part() does its own read + + * locate + range-slice internally (it needs the large + * bodystructure_read_max read cap regardless of whether a + * range was requested, since the target part's *offset* + * within the message isn't known until the whole message is + * read and walked). For WHOLE/TEXT, read_cap switches to that + * same large cap only when a partial range was requested -- + * see apply_partial_range()'s own comment for why a *ranged* + * request on an over-APPEND_LITERAL_MAX message should still + * succeed (this is exactly the gap real Apple Mail traffic + * hit: "BODY.PEEK[TEXT]<0.16384>" against a message that only + * exceeds 12000 bytes because of its own attachment) while a + * *whole*-content request keeps the tighter reject-not- + * truncate behavior it always had. + */ + if (req->attrs & (MBOX_FETCH_BODY_WHOLE | MBOX_FETCH_BODY_TEXT | + MBOX_FETCH_BODY_PART)) { + struct imsg_mbox_fetch_body bodymeta; + char *bodybuf = NULL; + uint32_t bodylen = 0; + char *combined; + size_t combined_len; + int want_whole = + (req->attrs & MBOX_FETCH_BODY_WHOLE) != 0; + int want_part = !want_whole && + (req->attrs & MBOX_FETCH_BODY_PART) != 0 && + !(req->attrs & MBOX_FETCH_BODY_TEXT); + int want_text = !want_whole && + !want_part; + + memset(&bodymeta, 0, sizeof(bodymeta)); + bodymeta.seqno = i; + bodymeta.uid = rec.uid; + bodymeta.is_text = want_text; + + if (want_part) { + int path[MIME_MAX_DEPTH]; + int pathlen; + + pathlen = parse_section_part(req->section_part, + path, MIME_MAX_DEPTH); + if (pathlen != -1 && + extract_mime_part(rec.basename, path, + pathlen, req->has_partial, + req->partial_start, req->partial_count, + &bodybuf, &bodylen) == 0) + bodymeta.found = 1; + } else { + size_t read_cap = req->has_partial ? + bodystructure_read_max : APPEND_LITERAL_MAX; + char *wholebuf = NULL; + uint32_t wholelen = 0; + + if (read_message_body(rec.basename, want_text, + read_cap, want_text ? "BODY[TEXT]" : + "BODY[]", &wholebuf, &wholelen) == 0) { + const char *out; + size_t outlen; + + apply_partial_range(wholebuf, wholelen, + req->has_partial, + req->partial_start, + req->partial_count, &out, &outlen); + if (outlen == 0) { + bodymeta.found = 1; + } else if ((bodybuf = malloc(outlen)) == + NULL) { + log_warn("session %u: malloc " + "IMSG_MBOX_FETCH_BODY " + "slice (%s)", session_id, + rec.basename); + } else { + memcpy(bodybuf, out, outlen); + bodylen = (uint32_t)outlen; + bodymeta.found = 1; + } + } + free(wholebuf); + } + bodymeta.bodylen = bodylen; + + combined_len = sizeof(bodymeta) + + (bodymeta.found ? bodylen : 0); + if ((combined = malloc(combined_len)) == NULL) { + log_warn("session %u: malloc " + "IMSG_MBOX_FETCH_BODY buffer", session_id); + } else { + memcpy(combined, &bodymeta, sizeof(bodymeta)); + if (bodymeta.found && bodylen > 0) + memcpy(combined + sizeof(bodymeta), + bodybuf, bodylen); + if (imsg_compose(&iev->ibuf, + IMSG_MBOX_FETCH_BODY, 0, 0, -1, combined, + combined_len) == -1) + log_warn("session %u: imsg_compose " + "IMSG_MBOX_FETCH_BODY", session_id); + free(combined); + } + free(bodybuf); + } + + /* + * IMSG_MBOX_FETCH_ENVELOPE, same "always sent when + * requested, found=0 on any failure, ordered right before + * this message's own IMSG_MBOX_FETCH_META" contract as the + * IMSG_MBOX_FETCH_HEADER/IMSG_MBOX_FETCH_BODY blocks above -- + * see struct imsg_mbox_fetch_envelope's comment in imapd.h. + * Unlike those two, the trailing bytes here are build_ + * envelope()'s already-formatted response text, not raw + * message bytes -- see that function's own comment. + */ + if (req->attrs & MBOX_FETCH_ENVELOPE) { + struct imsg_mbox_fetch_envelope envmeta; + char *envbuf = NULL; + uint32_t envlen = 0; + char *combined; + size_t combined_len; + + memset(&envmeta, 0, sizeof(envmeta)); + envmeta.seqno = i; + envmeta.uid = rec.uid; + if (build_envelope(rec.basename, &envbuf, &envlen) == 0) { + envmeta.found = 1; + envmeta.envlen = envlen; + } + + combined_len = sizeof(envmeta) + + (envmeta.found ? envlen : 0); + if ((combined = malloc(combined_len)) == NULL) { + log_warn("session %u: malloc " + "IMSG_MBOX_FETCH_ENVELOPE buffer", session_id); + } else { + memcpy(combined, &envmeta, sizeof(envmeta)); + if (envmeta.found && envlen > 0) + memcpy(combined + sizeof(envmeta), + envbuf, envlen); + if (imsg_compose(&iev->ibuf, + IMSG_MBOX_FETCH_ENVELOPE, 0, 0, -1, combined, + combined_len) == -1) + log_warn("session %u: imsg_compose " + "IMSG_MBOX_FETCH_ENVELOPE", session_id); + free(combined); + } + free(envbuf); + } + + /* + * IMSG_MBOX_FETCH_BODYSTRUCTURE, same "always sent when + * requested, found=0 on any failure, ordered right before + * this message's own IMSG_MBOX_FETCH_META" contract, and same + * "already-formatted response text" shape, as IMSG_MBOX_ + * FETCH_ENVELOPE just above -- see struct imsg_mbox_fetch_ + * bodystructure's comment in imapd.h. + */ + if (req->attrs & MBOX_FETCH_BODYSTRUCTURE) { + struct imsg_mbox_fetch_bodystructure bsmeta; + char *bsbuf = NULL; + uint32_t bslen = 0; + char *combined; + size_t combined_len; + + memset(&bsmeta, 0, sizeof(bsmeta)); + bsmeta.seqno = i; + bsmeta.uid = rec.uid; + if (build_bodystructure(rec.basename, &bsbuf, &bslen) == 0) { + bsmeta.found = 1; + bsmeta.bslen = bslen; + } + + combined_len = sizeof(bsmeta) + + (bsmeta.found ? bslen : 0); + if ((combined = malloc(combined_len)) == NULL) { + log_warn("session %u: malloc " + "IMSG_MBOX_FETCH_BODYSTRUCTURE buffer", + session_id); + } else { + memcpy(combined, &bsmeta, sizeof(bsmeta)); + if (bsmeta.found && bslen > 0) + memcpy(combined + sizeof(bsmeta), + bsbuf, bslen); + if (imsg_compose(&iev->ibuf, + IMSG_MBOX_FETCH_BODYSTRUCTURE, 0, 0, -1, + combined, combined_len) == -1) + log_warn("session %u: imsg_compose " + "IMSG_MBOX_FETCH_BODYSTRUCTURE", + session_id); + free(combined); + } + free(bsbuf); + } + + if (imsg_compose(&iev->ibuf, IMSG_MBOX_FETCH_META, 0, 0, -1, + &meta, sizeof(meta)) == -1) + log_warn("session %u: imsg_compose " + "IMSG_MBOX_FETCH_META", session_id); + else + sent++; + } + +done: + index_free(&idx); + + memset(&result, 0, sizeof(result)); + result.ok = ok; + result.count = sent; + if (imsg_compose(&iev->ibuf, IMSG_MBOX_RESULT, 0, 0, -1, &result, + sizeof(result)) == -1) + log_warn("session %u: imsg_compose IMSG_MBOX_RESULT", + session_id); + imsgev_add(iev); +} + +/* + * Inverse of build_flags_string()'s letter table: maps a maildir + * flag-suffix's letters (the part after "2,", e.g. "FS") back to the + * MBOX_FLAG_* bitmask. Needed only here -- FETCH only ever renders flags + * outward and never needed to merge new ones in, but STORE has to know a + * message's *current* system flags before it can compute what ADD/REMOVE + * (as opposed to SET) leaves it with. + */ +static uint32_t +letters_to_sysflags(const char *letters) +{ + uint32_t f = 0; + + if (strchr(letters, 'D') != NULL) + f |= MBOX_FLAG_DRAFT; + if (strchr(letters, 'F') != NULL) + f |= MBOX_FLAG_FLAGGED; + if (strchr(letters, 'R') != NULL) + f |= MBOX_FLAG_ANSWERED; + if (strchr(letters, 'S') != NULL) + f |= MBOX_FLAG_SEEN; + if (strchr(letters, 'T') != NULL) + f |= MBOX_FLAG_DELETED; + return (f); +} + +/* + * Renders sysflags back into the maildir suffix's required letter order -- + * Courier maildir(5): "The letters must be in ASCII order" -- D, F, R, S, + * T, the same order build_flags_string()'s table already uses. + */ +static void +sysflags_to_letters(uint32_t sysflags, char *out, size_t outsize) +{ + size_t i = 0; + + if ((sysflags & MBOX_FLAG_DRAFT) && i + 1 < outsize) + out[i++] = 'D'; + if ((sysflags & MBOX_FLAG_FLAGGED) && i + 1 < outsize) + out[i++] = 'F'; + if ((sysflags & MBOX_FLAG_ANSWERED) && i + 1 < outsize) + out[i++] = 'R'; + if ((sysflags & MBOX_FLAG_SEEN) && i + 1 < outsize) + out[i++] = 'S'; + if ((sysflags & MBOX_FLAG_DELETED) && i + 1 < outsize) + out[i++] = 'T'; + out[i] = '\0'; +} + +/* + * RFC 9051 SS6.3.11 STATUS. Modeled directly on handle_mbox_select()'s + * open/flock/index_load/new-mail-scan/index_save sequence -- STATUS "does + * not change the currently selected mailbox, nor does it affect the state + * of any messages" (SS6.3.11), so unlike a real SELECT this never touches + * s->state or holds the fd open past this one call, but it deliberately + * reuses the *same* new/ scan SELECT does (picking up messages delivered + * since the index was last written and appending them with fresh UIDs) + * rather than answering from a possibly-stale on-disk index: MESSAGES/ + * UIDNEXT/UNSEEN would otherwise be able to under-report mail that already + * arrived. This is a judgment call, not something the RFC mandates -- + * SS6.3.11 only says STATUS "MUST NOT be used as a check for new messages" + * (i.e. clients shouldn't rely on it *instead of* EXISTS/RECENT/IDLE), which + * is a client-behavior note, not a server prohibition on noticing new mail + * while it's already got the index open. + * + * MESSAGES/UIDNEXT/UIDVALIDITY/HIGHESTMODSEQ are always computed (free -- + * index header fields once index_load() has run), matching imsg_mbox_ + * selected's own "always compute" precedent. UNSEEN/DELETED/SIZE are the + * deliberate exception: computing any of them requires calling locate_ + * message_file() once per message, which RFC 9051 SS6.3.11 explicitly + * flags as potentially expensive ("the STATUS command SIZE...can take a + * significant amount of time...clients should use STATUS SIZE cautiously") + * -- so that scan runs only when req->attrs asks for at least one of the + * three, and once it runs, all three are computed together regardless of + * which subset was actually requested (locate_message_file() already + * returns both the flag suffix and the size in a single call, so there is + * no marginal cost to computing all three vs. one). + */ +static void +handle_mbox_status(struct imsg_mbox_status *req, struct imsgev *iev) +{ + struct mbox_index idx; + struct imsg_mbox_status_result reply; + int fd; + DIR *dp; + struct dirent *de; + char saved[MBOX_NAME_MAX]; + const char *target; + int switched = 0; + + memset(&reply, 0, sizeof(reply)); + + /* + * RFC 9051 SS6.3.11: STATUS targets a mailbox independent of + * whatever this session currently has selected -- "without... + * opening a mailbox" -- so this can't just answer for cwd the way + * FETCH/STORE/EXPUNGE correctly do. select_mailbox_dir() is reused + * here as a temporary visit-and-restore primitive rather than a + * lasting selection change: resolve the target, remember whatever + * was selected before, and switch back to it before replying + * regardless of how this function exits (see the "send:" label). + */ + strlcpy(saved, current_mailbox_dir, sizeof(saved)); + if (mailbox_name_is_inbox(req->mailbox)) + target = ""; + else if (mailbox_name_valid(req->mailbox)) + target = req->mailbox; + else { + log_debug("session %u: STATUS %s: invalid mailbox name", + session_id, req->mailbox); + reply.ok = 0; + goto send; + } + if (select_mailbox_dir(target) == -1) { + log_debug("session %u: STATUS %s: no such mailbox", + session_id, req->mailbox); + reply.ok = 0; + goto send; + } + switched = 1; + + if ((fd = open(STORE_INDEX_NAME, O_RDWR | O_CREAT, 0600)) == -1) { + log_warn("session %u: open %s", session_id, STORE_INDEX_NAME); + reply.ok = 0; + goto send; + } + if (flock(fd, LOCK_EX) == -1) { + log_warn("session %u: flock %s", session_id, STORE_INDEX_NAME); + close(fd); + reply.ok = 0; + goto send; + } + + if (index_load(fd, &idx) == -1) { + flock(fd, LOCK_UN); + close(fd); + reply.ok = 0; + goto send; + } + + dp = opendir("new"); + if (dp == NULL) { + if (errno != ENOENT) + log_warn("session %u: opendir new", session_id); + } else { + while ((de = readdir(dp)) != NULL) { + if (de->d_name[0] == '.') + continue; + if (index_has_basename(&idx, de->d_name)) + continue; + if (index_append(&idx, idx.uidnext, de->d_name) + == -1) { + closedir(dp); + index_free(&idx); + flock(fd, LOCK_UN); + close(fd); + reply.ok = 0; + goto send; + } + idx.uidnext++; + } + closedir(dp); + } + + if (index_save(&idx) == -1) { + index_free(&idx); + flock(fd, LOCK_UN); + close(fd); + reply.ok = 0; + goto send; + } + + reply.ok = 1; + reply.messages = (uint32_t)idx.nlines; + reply.uidnext = idx.uidnext; + reply.uidvalidity = idx.uidvalidity; + reply.highestmodseq = idx.highestmodseq; + + if (req->attrs & + (STATUS_ATT_UNSEEN | STATUS_ATT_DELETED | STATUS_ATT_SIZE)) { + size_t i; + + for (i = 0; i < idx.nlines; i++) { + struct index_rec rec; + const char *lp; + uint32_t sysflags; + char suffix[64]; + off_t size; + + if (index_parse_line(idx.lines[i], &rec) == -1) + continue; + if (locate_message_file(rec.basename, &size, suffix, + sizeof(suffix)) == -1) { + log_warnx("session %u: message %s indexed but " + "missing on disk -- skipped for STATUS " + "UNSEEN/DELETED/SIZE", session_id, + rec.basename); + continue; + } + + lp = strstr(suffix, "2,"); + sysflags = letters_to_sysflags(lp != NULL ? lp + 2 : ""); + if (!(sysflags & MBOX_FLAG_SEEN)) + reply.unseen++; + if (sysflags & MBOX_FLAG_DELETED) + reply.deleted++; + reply.size += (uint64_t)size; + } + } + + index_free(&idx); + flock(fd, LOCK_UN); + close(fd); + +send: + /* Restore whatever this session actually had selected before this + * STATUS call -- select_mailbox_dir() is a no-op if that's already + * where we are (e.g. STATUS on the currently selected mailbox, or + * the "switched" gate below wasn't even reached). */ + if (switched && select_mailbox_dir(saved) == -1) + log_warnx("session %u: STATUS: failed to restore selection " + "to %s -- session may be left in an inconsistent state", + session_id, saved[0] != '\0' ? saved : "INBOX"); + if (imsg_compose(&iev->ibuf, IMSG_MBOX_STATUS_RESULT, 0, 0, -1, &reply, + sizeof(reply)) == -1) + log_warn("session %u: imsg_compose IMSG_MBOX_STATUS_RESULT", + session_id); + imsgev_add(iev); +} + +/* + * True if comma-separated list contains kw as a whole token (not a + * substring match) -- used by merge_keywords() below to dedupe and to + * implement REMOVE. O(n) per lookup, same "fine at v1's scale" reasoning + * index_has_basename() already documents for the index itself: per-message + * keyword lists are short (a handful of entries at most), not the mailbox + * as a whole. + */ +static int +kw_list_contains(const char *list, const char *kw) +{ + char tmp[MBOX_FLAGS_MAX]; + char *tok, *save; + + strlcpy(tmp, list, sizeof(tmp)); + for (tok = strtok_r(tmp, ",", &save); tok != NULL; + tok = strtok_r(NULL, ",", &save)) { + if (strcmp(tok, kw) == 0) + return (1); + } + return (0); +} + +/* + * Computes *out, a comma-separated keyword list (matching the index's own + * on-disk delimiter -- see imapd.h's imsg_mbox_store comment), as the + * result of applying mode/new_kws to a message's current old_kws. RFC 9051 + * SS6.4.6: "FLAGS Replace the flags for the message with the + * argument" -- keywords are flags too (SS2.3.2), so an unqualified FLAGS + * STORE really does replace the entire keyword set, not just the five + * system flags; that's why MBOX_STORE_SET ignores old_kws entirely below, + * unlike ADD/REMOVE. Deduplicates in every mode -- a client naming a + * keyword twice, or one already present, shouldn't produce a doubled + * index entry. + */ +static void +merge_keywords(int mode, const char *old_kws, const char *new_kws, + char *out, size_t outsize) +{ + char tmp[MBOX_FLAGS_MAX]; + char *tok, *save; + int first = 1; + + out[0] = '\0'; + + if (mode == MBOX_STORE_REMOVE) { + strlcpy(tmp, old_kws, sizeof(tmp)); + for (tok = strtok_r(tmp, ",", &save); tok != NULL; + tok = strtok_r(NULL, ",", &save)) { + if (kw_list_contains(new_kws, tok)) + continue; + if (!first) + strlcat(out, ",", outsize); + strlcat(out, tok, outsize); + first = 0; + } + return; + } + + if (mode == MBOX_STORE_ADD) { + strlcpy(tmp, old_kws, sizeof(tmp)); + for (tok = strtok_r(tmp, ",", &save); tok != NULL; + tok = strtok_r(NULL, ",", &save)) { + if (!first) + strlcat(out, ",", outsize); + strlcat(out, tok, outsize); + first = 0; + } + } + + /* SET starts from nothing (old_kws is discarded); ADD continues + * from the copy of old_kws just built above. Either way, append + * anything in new_kws not already present. */ + strlcpy(tmp, new_kws, sizeof(tmp)); + for (tok = strtok_r(tmp, ",", &save); tok != NULL; + tok = strtok_r(NULL, ",", &save)) { + if (kw_list_contains(out, tok)) + continue; + if (!first) + strlcat(out, ",", outsize); + strlcat(out, tok, outsize); + first = 0; + } +} + +/* + * Per-message context handed to search_eval()/search_eval_leaf() during + * handle_mbox_search()'s scan below -- one message's worth of the fields + * any in-scope search key (SEARCH_OP_*, imapd.h) might need to test. + */ +struct search_msg_ctx { + uint32_t seqno; + uint32_t uid; + uint32_t sysflags; + const char *keywords; /* comma-separated, same convention as + * everywhere else in this file */ + int64_t internaldate; + uint64_t size; /* F13 fix: 64-bit for SEARCH LARGER/SMALLER */ + uint64_t modseq; /* RFC 7162 SS3.1.5 MODSEQ search key */ +}; + +/* + * One postfix node's worth of evaluation against a single message. + * SEARCH_OP_AND/OR/NOT never reach here -- search_eval()'s stack machine + * below handles those three directly, since they combine *other* nodes' + * results rather than testing the message themselves. + */ +static int +search_eval_leaf(const struct search_node *n, const struct search_msg_ctx *m) +{ + switch (n->op) { + case SEARCH_OP_ALL: + return (1); + case SEARCH_OP_ANSWERED: + return ((m->sysflags & MBOX_FLAG_ANSWERED) != 0); + case SEARCH_OP_UNANSWERED: + return ((m->sysflags & MBOX_FLAG_ANSWERED) == 0); + case SEARCH_OP_DELETED: + return ((m->sysflags & MBOX_FLAG_DELETED) != 0); + case SEARCH_OP_UNDELETED: + return ((m->sysflags & MBOX_FLAG_DELETED) == 0); + case SEARCH_OP_DRAFT: + return ((m->sysflags & MBOX_FLAG_DRAFT) != 0); + case SEARCH_OP_UNDRAFT: + return ((m->sysflags & MBOX_FLAG_DRAFT) == 0); + case SEARCH_OP_FLAGGED: + return ((m->sysflags & MBOX_FLAG_FLAGGED) != 0); + case SEARCH_OP_UNFLAGGED: + return ((m->sysflags & MBOX_FLAG_FLAGGED) == 0); + case SEARCH_OP_SEEN: + return ((m->sysflags & MBOX_FLAG_SEEN) != 0); + case SEARCH_OP_UNSEEN: + return ((m->sysflags & MBOX_FLAG_SEEN) == 0); + case SEARCH_OP_KEYWORD: + return (kw_list_contains(m->keywords, n->keyword)); + case SEARCH_OP_UNKEYWORD: + return (!kw_list_contains(m->keywords, n->keyword)); + case SEARCH_OP_BEFORE: + return (m->internaldate < n->num); + case SEARCH_OP_ON: + return (m->internaldate >= n->num && + m->internaldate < n->num + 86400); + case SEARCH_OP_SINCE: + return (m->internaldate >= n->num); + case SEARCH_OP_LARGER: + return ((int64_t)m->size > n->num); + case SEARCH_OP_SMALLER: + return ((int64_t)m->size < n->num); + case SEARCH_OP_SEQSET: + return (m->seqno >= n->seq_lo && m->seqno <= n->seq_hi); + case SEARCH_OP_UIDSET: + return (m->uid >= n->seq_lo && m->uid <= n->seq_hi); + case SEARCH_OP_MODSEQ: + return (m->modseq >= (uint64_t)n->num); + default: + return (0); + } +} + +/* + * Evaluates nodes[0..nnodes) -- a postfix (reverse Polish) boolean + * expression compiled by listener.c's parse_search_key()/parse_search_ + * key_list() -- against one message. Bounded by SEARCH_PROGRAM_MAX_NODES + * (imapd.h): listener.c never emits a program longer than that, and + * never emits a malformed one (AND/OR/NOT with too few operands already + * on the stack) -- the bounds checks below are defensive against a + * build-time struct-layout skew between listener and store, the same + * posture store_dispatch()'s IMSG_MBOX_SEARCH case already takes on a + * raw length mismatch, not something a client can trigger through normal + * protocol use. + */ +static int +search_eval(const struct search_node *nodes, uint32_t nnodes, + const struct search_msg_ctx *m) +{ + int stack[SEARCH_PROGRAM_MAX_NODES]; + uint32_t sp = 0, i; + + for (i = 0; i < nnodes; i++) { + const struct search_node *n = &nodes[i]; + + switch (n->op) { + case SEARCH_OP_AND: + if (sp < 2) + return (0); + sp--; + stack[sp - 1] = stack[sp - 1] && stack[sp]; + break; + case SEARCH_OP_OR: + if (sp < 2) + return (0); + sp--; + stack[sp - 1] = stack[sp - 1] || stack[sp]; + break; + case SEARCH_OP_NOT: + if (sp < 1) + return (0); + stack[sp - 1] = !stack[sp - 1]; + break; + default: + if (sp >= SEARCH_PROGRAM_MAX_NODES) + return (0); + stack[sp++] = search_eval_leaf(n, m); + break; + } + } + + return (sp == 1 ? stack[0] : 0); +} + +/* + * IMSG_MBOX_SEARCH handling. Same index-loading/locking shape as handle_ + * mbox_fetch() (LOCK_SH -- SEARCH only reads), but iterates every message + * in the mailbox (a search key can be anything, so there's no shortcut + * range the way FETCH's own sequence-set bounds the scan) and, for each, + * evaluates the compiled postfix program via search_eval(). Matches are + * streamed as IMSG_MBOX_SEARCH_MATCH in ascending sequence order (same + * order handle_mbox_fetch() already streams in, for the same "index + * order == sequence-number order, v1 has no reordering operation" + * reason), then one terminal IMSG_MBOX_RESULT. + * + * SEQSET/UIDSET nodes' "*" is resolved once, up front, against this + * scan's own idx.nlines (sequence numbers) or highest in-use UID (UID + * ranges) -- not per message, since neither changes mid-scan (LOCK_SH is + * held only long enough to read the index into memory, same as FETCH; + * nothing else in this process mutates it afterward). "Largest UID in + * use" is taken from the last index line (index_append() always appends + * in increasing-UID order, and v1 has no operation that reorders or + * renumbers existing entries), not from idx.uidnext - 1, so this stays + * correct even if a future pass ever introduces UID gaps. + * + * A message that's indexed but missing on disk is skipped, same + * tolerance handle_mbox_fetch()/handle_mbox_store()/handle_mbox_ + * expunge() already document -- unlike those, SEARCH always needs the + * on-disk file (locate_message_file()) for both its flag-suffix letters + * and its size, since almost any real query touches at least one of + * ANSWERED/DELETED/DRAFT/FLAGGED/SEEN/KEYWORD/LARGER/SMALLER; there's no + * cheaper partial path worth special-casing here the way FETCH's attrs + * bitmask lets it skip the stat(2) entirely for a FLAGS-less request. + */ +static void +handle_mbox_search(struct imsg_mbox_search *req, struct search_node *nodes, + uint32_t nnodes, struct imsgev *iev) +{ + struct mbox_index idx; + struct imsg_mbox_result result; + int fd; + int ok = 1; + uint32_t sent = 0; + uint32_t max_uid = 0; + uint32_t i; + + (void)req; /* nnodes (passed separately, already used by the + * caller to size the node array) is currently its + * only field -- see imapd.h's imsg_mbox_search + * comment */ + + memset(&idx, 0, sizeof(idx)); + + if ((fd = open(STORE_INDEX_NAME, O_RDWR | O_CREAT, 0600)) == -1) { + log_warn("session %u: open %s", session_id, STORE_INDEX_NAME); + ok = 0; + goto done; + } + if (flock(fd, LOCK_SH) == -1) { + log_warn("session %u: flock %s", session_id, STORE_INDEX_NAME); + close(fd); + ok = 0; + goto done; + } + if (index_load(fd, &idx) == -1) { + flock(fd, LOCK_UN); + close(fd); + ok = 0; + goto done; + } + flock(fd, LOCK_UN); + close(fd); + + max_uid = index_max_uid(&idx); /* factored out this pass -- UID + * FETCH/UID STORE/UID EXPUNGE's own + * "*" resolution now shares this same + * helper, see its comment */ + + for (i = 0; i < nnodes; i++) { + struct search_node *n = &nodes[i]; + + if (n->op != SEARCH_OP_SEQSET && n->op != SEARCH_OP_UIDSET) + continue; + + if (n->lo_is_star) + n->seq_lo = (n->op == SEARCH_OP_SEQSET) ? + (uint32_t)idx.nlines : max_uid; + if (n->hi_is_star) + n->seq_hi = (n->op == SEARCH_OP_SEQSET) ? + (uint32_t)idx.nlines : max_uid; + } + + for (i = 1; i <= (uint32_t)idx.nlines; i++) { + struct search_msg_ctx m; + struct index_rec rec; + const char *letters; + off_t size = 0; + char suffix[64]; + + if (index_parse_line(idx.lines[i - 1], &rec) == -1) + continue; + + memset(&m, 0, sizeof(m)); + m.seqno = i; + m.uid = rec.uid; + m.modseq = rec.modseq; + + if (locate_message_file(rec.basename, &size, suffix, + sizeof(suffix)) == -1) { + log_warnx("session %u: message %s (uid %u) indexed " + "but missing on disk -- skipped", session_id, + rec.basename, m.uid); + continue; + } + + letters = strstr(suffix, "2,"); + letters = (letters != NULL) ? letters + 2 : ""; + m.sysflags = letters_to_sysflags(letters); + m.keywords = rec.keywords; + m.internaldate = parse_maildir_timestamp(rec.basename); + m.size = (uint64_t)size; /* F13 fix: was (uint32_t) */ + + if (search_eval(nodes, nnodes, &m)) { + struct imsg_mbox_search_match match; + + memset(&match, 0, sizeof(match)); + match.seqno = i; + match.uid = m.uid; + match.modseq = m.modseq; + if (imsg_compose(&iev->ibuf, IMSG_MBOX_SEARCH_MATCH, + 0, 0, -1, &match, sizeof(match)) == -1) + log_warn("session %u: imsg_compose " + "IMSG_MBOX_SEARCH_MATCH", session_id); + else + sent++; + } + } + +done: + index_free(&idx); + + memset(&result, 0, sizeof(result)); + result.ok = ok; + result.count = sent; + if (imsg_compose(&iev->ibuf, IMSG_MBOX_RESULT, 0, 0, -1, &result, + sizeof(result)) == -1) + log_warn("session %u: imsg_compose IMSG_MBOX_RESULT", + session_id); + imsgev_add(iev); +} + +/* + * IMSG_MBOX_STORE handling -- see imapd.h's imsg_mbox_store comment for + * the wire shape and cmd_store_cmd() in listener.c for the RFC 9051 + * SS6.4.6 parsing this responds to. Uses LOCK_EX (like handle_mbox_ + * select(), unlike handle_mbox_fetch()'s LOCK_SH): STORE mutates both the + * index (keywords) and, for any message whose system flags actually + * change, the maildir filename itself, so it needs the same mutual- + * exclusion-across-processes guarantee SELECT's read-modify-write cycle + * already established, for the same fork-per-session (not fork-per-user) + * reason. + * + * Sends one IMSG_MBOX_FETCH_META per modified message (skipped if + * req->silent) and exactly one terminal IMSG_MBOX_RESULT, matching + * handle_mbox_fetch()'s own reply shape -- see imapd.h's comment on + * why STORE reuses FETCH's reply types rather than inventing new ones. + * + * RFC 7162 additions this pass: + * + * - req->has_unchangedsince (UNCHANGEDSINCE store-modifier, SS3.1.3): a + * message whose current mod-sequence exceeds req->unchangedsince fails + * the conditional test -- the requested flag operation is skipped for + * it entirely, and its seqno/uid are reported via one IMSG_MBOX_STORE_ + * MODIFIED (listener.c folds these into the tagged response's MODIFIED + * response code). This overrides req->silent for messages that *pass* + * the test: SS3.1.3 "An untagged FETCH response MUST be sent, even if + * the .SILENT suffix is specified, and the response MUST include the + * MODSEQ message data item" -- see send_fetch below. + * + * - One shared mod-sequence bump per STORE command, applied to every + * message this command actually changes -- not a fresh bump per + * message. Sourced from RFC 7162 SS3.1.3's own worked examples (9 and + * 10 especially): a range STORE that changes many messages shows the + * *identical* MODSEQ value on every changed message's FETCH echo, and + * SS3.1's guarantee is phrased per *command* ("each STORE command... + * will get a different mod-sequence value"), not per message. new_ + * modseq below is computed once, up front, and only actually committed + * to idx.highestmodseq (via the `changed` flag) if at least one message + * ends up using it -- see also SS3.1.11/SS3.1.12: adding an + * already-set flag (or removing an already-unset one) SHOULD NOT bump + * the mod-sequence at all, so a message whose resulting flags/keywords + * come out identical to what it already had keeps its *old* mod- + * sequence rather than taking new_modseq. + * + * RFC 9051 SS6.4.9 addition (UID command): req->by_uid switches lo/hi (and + * "*") to UID-space resolution -- see handle_mbox_fetch()'s identical + * mechanism and comment. meta.uid is already unconditionally populated in + * the FETCH echo regardless of by_uid; only listener.c's decision to print + * it changes. + */ +static void +handle_mbox_store(struct imsg_mbox_store *req, struct imsgev *iev) +{ + struct mbox_index idx; + struct imsg_mbox_result result; + int fd = -1; + uint32_t lo, hi, i, sent = 0; + uint64_t new_modseq; + int ok = 1, changed = 0, locked = 0; + + memset(&idx, 0, sizeof(idx)); + + if ((fd = open(STORE_INDEX_NAME, O_RDWR | O_CREAT, 0600)) == -1) { + log_warn("session %u: open %s", session_id, STORE_INDEX_NAME); + ok = 0; + goto done; + } + if (flock(fd, LOCK_EX) == -1) { + log_warn("session %u: flock %s", session_id, STORE_INDEX_NAME); + ok = 0; + goto done; + } + locked = 1; + + if (index_load(fd, &idx) == -1) { + ok = 0; + goto done; + } + + new_modseq = idx.highestmodseq + 1; + + /* RFC 9051 SS6.4.9: UID STORE's sequence-set is UID-space -- same + * lo/hi resolution switch as handle_mbox_fetch(), see that + * function's comment for the full reasoning. */ + if (req->by_uid) { + uint32_t max_uid = index_max_uid(&idx); + + lo = req->lo_is_star ? max_uid : req->seq_lo; + hi = req->hi_is_star ? max_uid : req->seq_hi; + } else { + lo = req->lo_is_star ? (uint32_t)idx.nlines : req->seq_lo; + hi = req->hi_is_star ? (uint32_t)idx.nlines : req->seq_hi; + } + if (lo < 1) + lo = 1; + if (!req->by_uid && hi > (uint32_t)idx.nlines) + hi = (uint32_t)idx.nlines; + + for (i = 1; i <= (uint32_t)idx.nlines; i++) { + struct imsg_mbox_fetch_meta meta; + struct index_rec rec; + const char *lp; + uint32_t old_sysflags, new_sysflags; + char suffix[64], newletters[8]; + char newkeywords[MBOX_FLAGS_MAX]; + char newline[STORE_INDEX_LINE_MAX]; + char oldpath[600], newpath[600]; + off_t size; + uint64_t this_modseq; + int in_new, len, real_change, send_fetch; + + if (index_parse_line(idx.lines[i - 1], &rec) == -1) + continue; + + /* Same position-space/UID-space range check as handle_mbox_ + * fetch() -- see that function's comment. */ + if (req->by_uid) { + if (rec.uid < lo) + continue; + if (rec.uid > hi) + break; + } else { + if (i < lo) + continue; + if (i > hi) + break; + } + + if (req->has_unchangedsince && + rec.modseq > req->unchangedsince) { + struct imsg_mbox_store_modified mod; + + memset(&mod, 0, sizeof(mod)); + mod.seqno = i; + mod.uid = rec.uid; + if (imsg_compose(&iev->ibuf, IMSG_MBOX_STORE_MODIFIED, + 0, 0, -1, &mod, sizeof(mod)) == -1) + log_warn("session %u: imsg_compose " + "IMSG_MBOX_STORE_MODIFIED", session_id); + continue; + } + + if (locate_message_file(rec.basename, &size, suffix, + sizeof(suffix)) == -1) { + log_warnx("session %u: message %s (uid %u) indexed " + "but missing on disk -- skipped", session_id, + rec.basename, rec.uid); + continue; + } + in_new = (suffix[0] == '\0'); + + lp = strstr(suffix, "2,"); + old_sysflags = letters_to_sysflags(lp != NULL ? lp + 2 : ""); + + switch (req->mode) { + case MBOX_STORE_SET: + new_sysflags = req->sysflags; + break; + case MBOX_STORE_ADD: + new_sysflags = old_sysflags | req->sysflags; + break; + case MBOX_STORE_REMOVE: + default: + new_sysflags = old_sysflags & ~req->sysflags; + break; + } + merge_keywords(req->mode, rec.keywords, req->keywords, + newkeywords, sizeof(newkeywords)); + sysflags_to_letters(new_sysflags, newletters, + sizeof(newletters)); + + real_change = (new_sysflags != old_sysflags) || + (strcmp(newkeywords, rec.keywords) != 0); + this_modseq = real_change ? new_modseq : rec.modseq; + + /* + * Once a message is targeted by STORE at all, it moves (if + * not already there) from new/ to cur/ with an explicit + * ":2," suffix, even if ends up empty -- + * maildir's new/ specifically means "not yet seen by any + * client" (Courier maildir(5)), and a STORE is unambiguous + * evidence the client has now processed the message, the + * same "no protocol reason left to leave it in new/ once + * touched" reasoning handle_mbox_select()'s header comment + * already applies to \Recent. Never moves cur/ -> new/ -- + * no maildir tool does; new/ is one-directional. Skipped + * entirely if the message is already in cur/ with exactly + * this letter set, to avoid a no-op rename on every STORE + * that only touches keywords. + */ + if (in_new || strcmp(lp != NULL ? lp + 2 : "", + newletters) != 0) { + if (snprintf(oldpath, sizeof(oldpath), "%s/%s%s", + in_new ? "new" : "cur", rec.basename, suffix) >= + (int)sizeof(oldpath) || + snprintf(newpath, sizeof(newpath), "cur/%s:2,%s", + rec.basename, newletters) >= (int)sizeof(newpath)) { + log_warnx("session %u: path too long for %s", + session_id, rec.basename); + continue; + } + if (rename(oldpath, newpath) == -1) { + log_warn("session %u: rename %s -> %s", + session_id, oldpath, newpath); + continue; + } + } + + len = snprintf(newline, sizeof(newline), "%u:%s:%s:%llu", + rec.uid, rec.basename, newkeywords, + (unsigned long long)this_modseq); + if (len < 0 || (size_t)len >= sizeof(newline)) { + log_warnx("session %u: new index line too long for " + "%s", session_id, rec.basename); + continue; + } + free(idx.lines[i - 1]); + if ((idx.lines[i - 1] = strdup(newline)) == NULL) { + log_warn("session %u: strdup index line", session_id); + ok = 0; + goto done; + } + if (real_change) + changed = 1; + + /* SS3.1.3: UNCHANGEDSINCE forces the FETCH echo (with + * MODSEQ) even under .SILENT, for every message that passed + * the conditional test -- independent of req->silent. */ + send_fetch = !req->silent || req->has_unchangedsince; + + if (send_fetch) { + char newsuffix[16]; + + memset(&meta, 0, sizeof(meta)); + meta.seqno = i; + meta.uid = rec.uid; + meta.modseq = this_modseq; + snprintf(newsuffix, sizeof(newsuffix), "2,%s", + newletters); + build_flags_string(newsuffix, newkeywords, meta.flags, + sizeof(meta.flags)); + + if (imsg_compose(&iev->ibuf, IMSG_MBOX_FETCH_META, 0, + 0, -1, &meta, sizeof(meta)) == -1) + log_warn("session %u: imsg_compose " + "IMSG_MBOX_FETCH_META", session_id); + else + sent++; + } + } + + if (changed) { + idx.highestmodseq = new_modseq; + if (index_save(&idx) == -1) + ok = 0; + } + +done: + if (locked) + flock(fd, LOCK_UN); + if (fd != -1) + close(fd); + + memset(&result, 0, sizeof(result)); + result.ok = ok; + result.count = sent; + result.highestmodseq = idx.highestmodseq; + index_free(&idx); + if (imsg_compose(&iev->ibuf, IMSG_MBOX_RESULT, 0, 0, -1, &result, + sizeof(result)) == -1) + log_warn("session %u: imsg_compose IMSG_MBOX_RESULT", + session_id); + imsgev_add(iev); +} + +/* + * IMSG_MBOX_EXPUNGE handling -- see imapd.h's imsg_mbox_expunge comment + * for the wire shape (also used, with silent=1, by CLOSE). Uses LOCK_EX, + * same reason as handle_mbox_store(): this mutates the index structurally + * (removes lines, not just their content) and unlinks message files. + * + * Compacts idx.lines[] in place with a classic remove_if two-pointer scan + * (in reads every line, out is where the next *kept* line goes) rather + * than building a second array -- removed lines are simply never copied + * to out and their storage is free()d immediately; kept lines are moved + * down by however many removals preceded them. This single pass is also + * what produces RFC 9051 SS7.5.1's "immediately decremented" sequence + * numbers for free: at the moment a message at idx.lines[in] is found to + * be \Deleted, out already equals the count of kept messages before it, + * which -- because every earlier removal already shifted everything after + * it down by one -- is exactly that message's current sequence number + * minus one. See imapd.h's imsg_mbox_expunge comment for the worked + * check against SS6.4.3's own example (3, 3, 5, 8). + * + * A message that can't be classified for some reason (corrupt index + * line, missing on-disk file, path too long, or a failed unlink(2)) is + * conservatively kept in the index rather than dropped -- losing track of + * a message store.c couldn't actually verify as removed would be worse + * than a missed expunge of it. + * + * UIDVALIDITY and UIDNEXT are both left untouched: removing messages must + * never cause a UID to be reused (RFC 9051 SS2.3.1.1's whole point), and + * since this function only removes index entries -- never renumbers or + * reassigns the UIDs of messages that remain -- there's no reason either + * value would need to change. + * + * RFC 7162 additions this pass: exp.uid is now populated too (see imapd.h's + * imsg_mbox_expunged comment -- needed once QRESYNC is enabled, since + * listener.c then reports VANISHED, which is UID-based, instead of EXPUNGE). + * One shared mod-sequence bump for the whole EXPUNGE/UID EXPUNGE/CLOSE + * command, applied (conceptually -- this implementation doesn't persist a + * per-expunge-event value at all, see imsg_mbox_select_vanished's SS5.1 + * minimal-state comment) to every message removed by it, same "one bump + * per command, not per message" reasoning as handle_mbox_store() -- RFC + * 7162 SS3.2.7's own example shows a single HIGHESTMODSEQ value covering + * an entire multi-message EXPUNGE. Applies regardless of req->silent + * (CLOSE): SS3.2.8 requires the mod-sequence bump for CLOSE too, it just + * additionally forbids CLOSE's tagged OK from *reporting* the new value + * (a listener.c-side choice, not something this function needs to know + * about). + * + * RFC 9051 SS6.4.9 addition (UID command): req->by_uid, when set, restricts + * removal to \Deleted messages whose UID also falls in [seq_lo, seq_hi] + * (resolved once, up front, via index_max_uid() for "*" -- same mechanism + * FETCH/STORE now share) -- "If a message... has a UID that is not + * included in the specified sequence set, it is not affected." A message + * excluded this way is kept in the index exactly like the "not \Deleted at + * all" case, so it still gets its own turn at a later EXPUNGE/UID EXPUNGE + * that does include it. + */ +static void +handle_mbox_expunge(struct imsg_mbox_expunge *req, struct imsgev *iev) +{ + struct mbox_index idx; + struct imsg_mbox_result result; + int fd = -1; + size_t in, out; + uint32_t sent = 0; + uint32_t uid_lo, uid_hi; + int ok = 1, changed = 0, locked = 0; + + memset(&idx, 0, sizeof(idx)); + + if ((fd = open(STORE_INDEX_NAME, O_RDWR | O_CREAT, 0600)) == -1) { + log_warn("session %u: open %s", session_id, STORE_INDEX_NAME); + ok = 0; + goto done; + } + if (flock(fd, LOCK_EX) == -1) { + log_warn("session %u: flock %s", session_id, STORE_INDEX_NAME); + ok = 0; + goto done; + } + locked = 1; + + if (index_load(fd, &idx) == -1) { + ok = 0; + goto done; + } + + /* + * RFC 9051 SS6.4.9's second UID command form (UID EXPUNGE): resolve + * the required UID range once, up front, against the mailbox as + * loaded -- same index_max_uid() "*" resolution as UID FETCH/UID + * STORE. Meaningless (and unused below) when !req->by_uid, since a + * plain EXPUNGE takes no arguments at all. + */ + uid_lo = uid_hi = 0; + if (req->by_uid) { + uint32_t max_uid = index_max_uid(&idx); + + uid_lo = req->lo_is_star ? max_uid : req->seq_lo; + uid_hi = req->hi_is_star ? max_uid : req->seq_hi; + if (uid_lo < 1) + uid_lo = 1; + } + + out = 0; + for (in = 0; in < idx.nlines; in++) { + struct index_rec rec; + const char *lp; + uint32_t sysflags; + char suffix[64], path[600]; + off_t size; + + if (index_parse_line(idx.lines[in], &rec) == -1) { + idx.lines[out++] = idx.lines[in]; + continue; + } + + if (locate_message_file(rec.basename, &size, suffix, + sizeof(suffix)) == -1) { + log_warnx("session %u: message %s indexed but missing " + "on disk -- kept in index, not counted as " + "expunged", session_id, rec.basename); + idx.lines[out++] = idx.lines[in]; + continue; + } + + lp = strstr(suffix, "2,"); + sysflags = letters_to_sysflags(lp != NULL ? lp + 2 : ""); + if (!(sysflags & MBOX_FLAG_DELETED)) { + idx.lines[out++] = idx.lines[in]; + continue; + } + + /* + * RFC 9051 SS6.4.9: "If a message either does not have the + * \Deleted flag set or has a UID that is not included in the + * specified sequence set, it is not affected" -- a \Deleted + * message outside the UID EXPUNGE range is kept, exactly + * like the "not \Deleted at all" case just above. + */ + if (req->by_uid && (rec.uid < uid_lo || rec.uid > uid_hi)) { + idx.lines[out++] = idx.lines[in]; + continue; + } + + if (snprintf(path, sizeof(path), "%s/%s%s", + suffix[0] == '\0' ? "new" : "cur", rec.basename, suffix) >= + (int)sizeof(path)) { + log_warnx("session %u: path too long for %s -- kept " + "in index", session_id, rec.basename); + idx.lines[out++] = idx.lines[in]; + continue; + } + if (unlink(path) == -1 && errno != ENOENT) { + log_warn("session %u: unlink %s", session_id, path); + idx.lines[out++] = idx.lines[in]; + continue; + } + + if (!req->silent) { + struct imsg_mbox_expunged exp; + + memset(&exp, 0, sizeof(exp)); + exp.seqno = (uint32_t)(out + 1); + exp.uid = rec.uid; + if (imsg_compose(&iev->ibuf, IMSG_MBOX_EXPUNGED, 0, 0, + -1, &exp, sizeof(exp)) == -1) + log_warn("session %u: imsg_compose " + "IMSG_MBOX_EXPUNGED", session_id); + else + sent++; + } + + free(idx.lines[in]); + changed = 1; + } + idx.nlines = out; + + if (changed) { + idx.highestmodseq++; + if (index_save(&idx) == -1) + ok = 0; + } + +done: + if (locked) + flock(fd, LOCK_UN); + if (fd != -1) + close(fd); + + memset(&result, 0, sizeof(result)); + result.ok = ok; + result.count = sent; + result.highestmodseq = idx.highestmodseq; + index_free(&idx); + if (imsg_compose(&iev->ibuf, IMSG_MBOX_RESULT, 0, 0, -1, &result, + sizeof(result)) == -1) + log_warn("session %u: imsg_compose IMSG_MBOX_RESULT", + session_id); + imsgev_add(iev); +} + +/* + * Ensures tmp/, new/, and cur/ all exist -- APPEND is the first thing in + * this codebase that ever needs to *deliver* a message itself (see + * handle_mbox_append()'s header comment): every other IMSG_MBOX_* handler + * only ever reads directories assumed to already exist, populated by + * something external (smtpd's native maildir delivery action, per + * openimap-storage-backend.md). A mailbox that has only ever received + * mail that way could plausibly have new/ and cur/ but genuinely lack + * tmp/ (nothing external needs it -- smtpd's own maildir action manages + * its own tmp/ usage internally and doesn't leave evidence of it here). + * mkdir(2) with EEXIST tolerated is the standard idempotent "ensure + * exists" idiom; 0700 matches the store child's own already-narrow + * privilege scope (privilege-dropped to the session's uid/gid, chroot'd, + * unveiled to just this maildir subdirectory). + * + * Generalized this pass (flat multi-mailbox support) to take a path + * prefix rather than always assuming cwd is the mailbox root: "" for + * INBOX (every existing caller, unchanged behavior) or "name/" for a + * named mailbox, e.g. CREATE or an APPEND targeting a mailbox other than + * the one currently selected. Every existing call site already passes + * "" explicitly, so this is a signature change, not a behavior change, + * for anything that isn't new this pass. + */ +static int +ensure_maildir_dirs(const char *prefix) +{ + static const char *dirs[] = { "tmp", "new", "cur" }; + char path[MBOX_NAME_MAX + 8]; + size_t i; + + for (i = 0; i < sizeof(dirs) / sizeof(dirs[0]); i++) { + if (snprintf(path, sizeof(path), "%s%s", prefix, dirs[i]) >= + (int)sizeof(path)) { + log_warnx("session %u: mailbox path too long", + session_id); + return (-1); + } + if (mkdir(path, 0700) == -1 && errno != EEXIST) { + log_warn("session %u: mkdir %s", session_id, path); + return (-1); + } + } + return (0); +} + +/* + * IMSG_MBOX_CREATE (RFC 9051 SS6.3.4). See docs/openimap-storage-backend.md + * item 10 for the full design: CREATE needs no index-initialization code + * of its own at all -- index_load() already default-initializes a fresh + * UIDVALIDITY/UIDNEXT when handed an empty index fd, so the first command + * that actually opens this mailbox (SELECT, APPEND, ...) does that work, + * exactly as it always has for INBOX. This handler is therefore just: + * mkdir the mailbox directory itself (EEXIST here means "already exists", + * SS6.3.4's required refusal, unlike ensure_maildir_dirs()'s idempotent + * EEXIST-tolerant use elsewhere), then the same tmp/new/cur scaffolding + * APPEND already needs. + */ +static void +handle_mbox_create(struct imsg_mbox_create *req, struct imsgev *iev) +{ + struct imsg_mbox_result result; + char prefix[MBOX_NAME_MAX + 1]; + char saved[MBOX_NAME_MAX]; + int switched = 0; + + memset(&result, 0, sizeof(result)); + + if (mailbox_name_is_inbox(req->mailbox) || + !mailbox_name_valid(req->mailbox)) { + log_debug("session %u: CREATE %s: invalid name", session_id, + req->mailbox); + result.ok = 0; + goto send; + } + + /* + * mkdir(2)/ensure_maildir_dirs() below take req->mailbox as a bare + * path relative to cwd -- correct only when cwd is this session's + * maildir root, which is true by default but no longer guaranteed: + * if this session currently has some *other* named mailbox SELECTed + * (select_mailbox_dir() having already chdir'd into it), cwd sits + * one level below root. Temporarily visiting root first -- the same + * select_mailbox_dir()-based "visit and restore" pattern handle_ + * mbox_status() already established for this identical problem -- + * makes every relative path below correct regardless of what's + * currently selected. Found by real-hardware testing on premio: + * CREATE from a session with nothing else selected happened to work + * (cwd was already root by luck), but a RENAME issued against the + * mailbox that was itself currently SELECTed failed with a spurious + * "no such mailbox" -- same root cause; see handle_mbox_rename()'s + * identical fix, and handle_mbox_delete()'s/handle_mbox_list()'s. + */ + strlcpy(saved, current_mailbox_dir, sizeof(saved)); + if (select_mailbox_dir("") == -1) { + log_warnx("session %u: CREATE %s: couldn't reach maildir " + "root", session_id, req->mailbox); + result.ok = 0; + goto send; + } + switched = 1; + + if (mkdir(req->mailbox, 0700) == -1) { + if (errno != EEXIST) + log_warn("session %u: CREATE: mkdir %s", session_id, + req->mailbox); + else + log_debug("session %u: CREATE %s: already exists", + session_id, req->mailbox); + result.ok = 0; + goto send; + } + + if (snprintf(prefix, sizeof(prefix), "%s/", req->mailbox) >= + (int)sizeof(prefix) || ensure_maildir_dirs(prefix) == -1) { + /* Best-effort cleanup: a half-initialized mailbox (directory + * exists, tmp/new/cur don't) would otherwise be stuck in a + * state CREATE can never retry (mkdir would now see EEXIST) + * but that can't actually be SELECTed usefully either. */ + rmdir(req->mailbox); + result.ok = 0; + goto send; + } + + result.ok = 1; + +send: + if (switched && select_mailbox_dir(saved) == -1) + log_warnx("session %u: CREATE %s: couldn't restore " + "previously selected mailbox %s", session_id, + req->mailbox, saved); + if (imsg_compose(&iev->ibuf, IMSG_MBOX_RESULT, 0, 0, -1, &result, + sizeof(result)) == -1) + log_warn("session %u: imsg_compose IMSG_MBOX_RESULT", + session_id); + imsgev_add(iev); +} + +/* + * Bounded removal of one flat mailbox's own tmp/new/cur contents plus its + * index file -- deliberately NOT a general-purpose recursive delete. + * Refuses (logs and stops, leaving whatever was already removed rather + * than guessing further) if tmp/, new/, or cur/ contains anything that + * isn't a regular file: this codebase only ever creates that exact + * three-subdirectory-of-plain-files shape, so anything else there means + * something unexpected is going on and blind recursion would be unsafe. + * Does not remove the mailbox directory itself -- the caller (handle_ + * mbox_delete()) does that last, once this returns success. + */ +static int +remove_maildir_subtree(const char *prefix) +{ + static const char *dirs[] = { "tmp", "new", "cur" }; + /* + * F14 fix: buffer was MBOX_NAME_MAX + 8, too small to append + * "imapd.index" (STORE_INDEX_NAME, 11 bytes) to a near-maximal + * mailbox name. DELETE of such a mailbox failed with "index path + * too long" *after* already removing tmp/new/cur -- a partial + * delete that left the mailbox undeletable. + 32 covers the + * longest suffix this function appends. + */ + char path[MBOX_NAME_MAX + 32]; + size_t i; + + for (i = 0; i < sizeof(dirs) / sizeof(dirs[0]); i++) { + DIR *dp; + struct dirent *de; + int ok = 1; + + if (snprintf(path, sizeof(path), "%s%s", prefix, dirs[i]) >= + (int)sizeof(path)) { + log_warnx("session %u: DELETE: path too long", + session_id); + return (-1); + } + if ((dp = opendir(path)) == NULL) { + if (errno == ENOENT) + continue; /* tmp/ in particular may never + * have been created -- see + * ensure_maildir_dirs()'s own + * comment */ + log_warn("session %u: DELETE: opendir %s", + session_id, path); + return (-1); + } + while ((de = readdir(dp)) != NULL) { + char entpath[sizeof(path) + 300]; + struct stat st; + + if (strcmp(de->d_name, ".") == 0 || + strcmp(de->d_name, "..") == 0) + continue; + if (snprintf(entpath, sizeof(entpath), "%s/%s", path, + de->d_name) >= (int)sizeof(entpath)) { + log_warnx("session %u: DELETE: entry path " + "too long", session_id); + ok = 0; + continue; + } + if (lstat(entpath, &st) == -1) { + log_warn("session %u: DELETE: lstat %s", + session_id, entpath); + ok = 0; + continue; + } + if (!S_ISREG(st.st_mode)) { + log_warnx("session %u: DELETE: refusing -- " + "%s is not a regular file", session_id, + entpath); + ok = 0; + continue; + } + if (unlink(entpath) == -1) { + log_warn("session %u: DELETE: unlink %s", + session_id, entpath); + ok = 0; + } + } + closedir(dp); + if (!ok) + return (-1); + if (rmdir(path) == -1 && errno != ENOENT) { + log_warn("session %u: DELETE: rmdir %s", session_id, + path); + return (-1); + } + } + + if (snprintf(path, sizeof(path), "%s%s", prefix, STORE_INDEX_NAME) >= + (int)sizeof(path)) { + log_warnx("session %u: DELETE: index path too long", + session_id); + return (-1); + } + if (unlink(path) == -1 && errno != ENOENT) { + log_warn("session %u: DELETE: unlink %s", session_id, path); + return (-1); + } + + return (0); +} + +/* + * IMSG_MBOX_DELETE (RFC 9051 SS6.3.5). v1 is flat (docs/openimap-storage- + * backend.md item 10), so no mailbox can ever have children -- SS6.3.5's + * "inferior hierarchical names"/HASCHILDREN carve-out never applies here; + * deletion is unconditional once the name resolves to a real, non-INBOX + * mailbox. The UID-preservation requirement ("value of the highest-used + * unique identifier... MUST be preserved... unless the new incarnation + * has a different unique identifier validity value") is satisfied for + * free: removing the directory outright and letting a later CREATE of the + * same name start from a brand-new time(NULL)-based UIDVALIDITY (see + * handle_mbox_create()) makes the "different UIDVALIDITY" escape clause + * trivially true. + * + * Known, accepted limitation, same category as handle_mbox_rename()'s own + * documented one below: no special check exists for "the mailbox being + * deleted is this session's own currently SELECTed mailbox" (RFC 9051 + * doesn't forbid DELETE of a selected mailbox outright, and real servers + * differ on how they handle it). If that happens, the send: label's + * restore-to-saved-selection select_mailbox_dir() call will itself fail + * (the directory it's trying to chdir back into no longer exists), which + * is logged but otherwise silently leaves this store child sitting at + * root while its own current_mailbox_dir bookkeeping and listener.c's + * s->state/s->selected_mailbox still believe the deleted mailbox is + * selected -- until that session's next SELECT/EXAMINE/CLOSE/UNSELECT + * naturally resolves the mismatch. Not solved here; flagged, not silent. + */ +static void +handle_mbox_delete(struct imsg_mbox_delete *req, struct imsgev *iev) +{ + struct imsg_mbox_result result; + struct stat st; + char prefix[MBOX_NAME_MAX + 1]; + char saved[MBOX_NAME_MAX]; + int switched = 0; + + memset(&result, 0, sizeof(result)); + + if (mailbox_name_is_inbox(req->mailbox) || + !mailbox_name_valid(req->mailbox)) { + log_debug("session %u: DELETE %s: invalid name", session_id, + req->mailbox); + result.ok = 0; + goto send; + } + + /* Same "req->mailbox is a bare path relative to cwd, which is only + * guaranteed to be root by default" fix as handle_mbox_create()'s + * identical comment -- see that function for the real-hardware bug + * this closes. */ + strlcpy(saved, current_mailbox_dir, sizeof(saved)); + if (select_mailbox_dir("") == -1) { + log_warnx("session %u: DELETE %s: couldn't reach maildir " + "root", session_id, req->mailbox); + result.ok = 0; + goto send; + } + switched = 1; + + if (stat(req->mailbox, &st) == -1 || !S_ISDIR(st.st_mode)) { + log_debug("session %u: DELETE %s: no such mailbox", + session_id, req->mailbox); + result.ok = 0; + goto send; + } + + if (snprintf(prefix, sizeof(prefix), "%s/", req->mailbox) >= + (int)sizeof(prefix)) { + result.ok = 0; + goto send; + } + + if (remove_maildir_subtree(prefix) == -1) { + result.ok = 0; + goto send; + } + if (rmdir(req->mailbox) == -1 && errno != ENOENT) { + log_warn("session %u: DELETE: rmdir %s", session_id, + req->mailbox); + result.ok = 0; + goto send; + } + + result.ok = 1; + +send: + if (switched && select_mailbox_dir(saved) == -1) + log_warnx("session %u: DELETE %s: couldn't restore " + "previously selected mailbox %s", session_id, + req->mailbox, saved); + if (imsg_compose(&iev->ibuf, IMSG_MBOX_RESULT, 0, 0, -1, &result, + sizeof(result)) == -1) + log_warn("session %u: imsg_compose IMSG_MBOX_RESULT", + session_id); + imsgev_add(iev); +} + +/* + * IMSG_MBOX_RENAME (RFC 9051 SS6.3.6). Source and destination are always + * direct siblings under the same already-unveiled, single-filesystem + * maildir root (docs/openimap-storage-backend.md item 6's EXDEV note), so + * this is one rename(2) call after existence checks. + * + * Refusing INBOX specifically as a RENAME *source* is a v1 judgment call, + * not a hard spec requirement -- SS6.3.6 permits renaming INBOX (with + * special content-move semantics: INBOX ends up empty, its messages move + * to the new name) but also explicitly sanctions refusal: "some servers + * disallow renaming INBOX... clients need to be able to handle the + * failure." INBOX's directory can't simply be renamed away (store.c's + * unveil(2)/chroot(2) structure assumes it's always the session's own + * root), and actually moving message content instead is real, separate + * complexity not taken on here. No special-case is needed to refuse a + * RENAME *to* "INBOX" -- it always exists, so that's already caught by + * the ordinary destination-already-exists check below. + * + * Known, accepted limitation (not solved here), revised now that this + * handler visits root rather than trusting cwd (see the real-hardware bug + * note below): if some *other* session belonging to the same user + * currently has the mailbox being renamed SELECTed, that other store + * child's own cwd remains validly referencing the same directory by inode + * after the rename (POSIX cwd tracking survives a rename of the directory + * it points into), so its own FETCH/STORE/EXPUNGE keep working -- but its + * current_mailbox_dir string cache goes stale (still holds the old name) + * until that session's next SELECT. If instead it's *this* session's own + * currently-selected mailbox being renamed, the send: label's restore-to- + * saved-selection call will itself fail the same way handle_mbox_delete()'s + * identical restore can (the old name no longer resolves), silently + * leaving this store child's cwd at root while listener.c's own + * s->state/s->selected_mailbox still believe the old name is selected -- + * until that session's next SELECT/EXAMINE/CLOSE/UNSELECT naturally + * resolves it. Solving either properly would need the same kind of + * cross-session coordination the deferred COPY/MOVE-to-other-mailbox work + * (see the design doc) does, and is out of scope for this pass. + * + * Real-hardware bug found testing this on premio: req->oldname/req->newname + * are bare paths relative to cwd, which stat(2)/rename(2) below assumed was + * always this session's maildir root -- true by default, but not once a + * *different* named mailbox is the one currently SELECTed (cwd sits one + * level below root). Renaming "Drafts" while "Drafts" itself was the + * currently selected mailbox failed with a spurious "no such mailbox" + * (looking for maildir-root/Drafts/Drafts, which naturally doesn't exist). + * Fixed the same way handle_mbox_create()/handle_mbox_delete() are: visit + * root via select_mailbox_dir("") first, do the real work, restore + * whatever was selected before on the way out. + */ +static void +handle_mbox_rename(struct imsg_mbox_rename *req, struct imsgev *iev) +{ + struct imsg_mbox_result result; + struct stat st; + char saved[MBOX_NAME_MAX]; + int switched = 0; + + memset(&result, 0, sizeof(result)); + + if (mailbox_name_is_inbox(req->oldname) || + !mailbox_name_valid(req->oldname) || + !mailbox_name_valid(req->newname)) { + log_debug("session %u: RENAME %s -> %s: invalid name(s)", + session_id, req->oldname, req->newname); + result.ok = 0; + goto send; + } + + strlcpy(saved, current_mailbox_dir, sizeof(saved)); + if (select_mailbox_dir("") == -1) { + log_warnx("session %u: RENAME %s -> %s: couldn't reach " + "maildir root", session_id, req->oldname, req->newname); + result.ok = 0; + goto send; + } + switched = 1; + + if (stat(req->oldname, &st) == -1 || !S_ISDIR(st.st_mode)) { + log_debug("session %u: RENAME %s: no such mailbox", + session_id, req->oldname); + result.ok = 0; + goto send; + } + + /* + * SS6.3.6: "error to... rename to a mailbox name that already + * exists." Checked explicitly rather than relying on rename(2)'s + * own directory-replace-if-empty semantics, which would otherwise + * let RENAME silently succeed against a stray empty directory that + * was never really a mailbox at all. + */ + if (stat(req->newname, &st) == 0 || errno != ENOENT) { + log_debug("session %u: RENAME %s -> %s: destination exists", + session_id, req->oldname, req->newname); + result.ok = 0; + goto send; + } + + if (rename(req->oldname, req->newname) == -1) { + log_warn("session %u: RENAME: rename %s -> %s", session_id, + req->oldname, req->newname); + result.ok = 0; + goto send; + } + + result.ok = 1; + +send: + if (switched) { + const char *restore = saved; + + /* + * If this session's own currently-selected mailbox is the + * one that just got renamed, follow it to its new name + * rather than trying (and failing) to restore a name that + * no longer exists -- keeps this store child's own cwd/ + * current_mailbox_dir internally consistent even though + * listener.c's separate s->selected_mailbox bookkeeping + * still needs its own fix for the same case (see cmd_ + * rename()/session_finish_mbox_op() in listener.c). + */ + if (result.ok && strcmp(saved, req->oldname) == 0) + restore = req->newname; + if (select_mailbox_dir(restore) == -1) + log_warnx("session %u: RENAME %s -> %s: couldn't " + "restore previously selected mailbox (as %s)", + session_id, req->oldname, req->newname, restore); + } + if (imsg_compose(&iev->ibuf, IMSG_MBOX_RESULT, 0, 0, -1, &result, + sizeof(result)) == -1) + log_warn("session %u: imsg_compose IMSG_MBOX_RESULT", + session_id); + imsgev_add(iev); +} + +/* + * IMSG_MBOX_LIST (RFC 9051 SS6.3.9). Streams one IMSG_MBOX_LIST_ITEM per + * real, on-disk mailbox subdirectory found at the session's maildir root, + * then a terminal IMSG_MBOX_RESULT (count = number streamed, ok = 0 only + * on a real I/O error opening the root itself). INBOX is never included + * here -- see the IMSG_MBOX_LIST_ITEM enum comment in imapd.h -- + * listener.c already handles it locally and unconditionally. "tmp"/"new"/ + * "cur" (INBOX's own maildir internals, living as siblings of any named + * mailbox at this same level) and the index file are skipped, along with + * anything that isn't a directory or fails mailbox_name_valid() (should + * never happen for anything this codebase itself created, but a stray + * unexpected entry shouldn't crash or misreport LIST -- it's just + * silently omitted, the same "ignore what doesn't fit" posture SS6.3.9 + * itself takes for unaccepted patterns). + * + * opendir(".") is only correct when cwd is this session's maildir root -- + * true by default, but not once a named mailbox is the one currently + * SELECTed (cwd sits one level below root). Real-hardware bug found + * testing this on premio: with "Drafts" selected, LIST enumerated Drafts' + * own tmp/new/cur instead of the maildir root's mailboxes, silently + * making every named mailbox but the selected one invisible. Fixed the + * same visit-root-then-restore way handle_mbox_create()/handle_mbox_ + * delete()/handle_mbox_rename() are. + */ +static void +handle_mbox_list(struct imsgev *iev) +{ + struct imsg_mbox_result result; + struct imsg_mbox_list_item item; + DIR *dp; + struct dirent *de; + char saved[MBOX_NAME_MAX]; + int switched = 0; + + memset(&result, 0, sizeof(result)); + + strlcpy(saved, current_mailbox_dir, sizeof(saved)); + if (select_mailbox_dir("") == -1) { + log_warnx("session %u: LIST: couldn't reach maildir root", + session_id); + result.ok = 0; + goto send; + } + switched = 1; + + if ((dp = opendir(".")) == NULL) { + log_warn("session %u: LIST: opendir .", session_id); + result.ok = 0; + goto send; + } + + while ((de = readdir(dp)) != NULL) { + struct stat st; + + if (strcmp(de->d_name, ".") == 0 || + strcmp(de->d_name, "..") == 0) + continue; + if (strcmp(de->d_name, "tmp") == 0 || + strcmp(de->d_name, "new") == 0 || + strcmp(de->d_name, "cur") == 0 || + strcmp(de->d_name, STORE_INDEX_NAME) == 0) + continue; + if (!mailbox_name_valid(de->d_name)) + continue; + if (stat(de->d_name, &st) == -1 || !S_ISDIR(st.st_mode)) + continue; + + memset(&item, 0, sizeof(item)); + strlcpy(item.mailbox, de->d_name, sizeof(item.mailbox)); + if (imsg_compose(&iev->ibuf, IMSG_MBOX_LIST_ITEM, 0, 0, -1, + &item, sizeof(item)) == -1) { + log_warn("session %u: imsg_compose " + "IMSG_MBOX_LIST_ITEM", session_id); + continue; + } + result.count++; + } + closedir(dp); + result.ok = 1; + +send: + if (switched && select_mailbox_dir(saved) == -1) + log_warnx("session %u: LIST: couldn't restore previously " + "selected mailbox %s", session_id, saved); + if (imsg_compose(&iev->ibuf, IMSG_MBOX_RESULT, 0, 0, -1, &result, + sizeof(result)) == -1) + log_warn("session %u: imsg_compose IMSG_MBOX_RESULT", + session_id); + imsgev_add(iev); +} + +/* + * IMSG_MBOX_APPEND handling -- see imapd.h's imsg_mbox_append comment + * for the wire shape (header struct plus variable-length trailing message + * bytes in the same imsg). v1 is INBOX-only, checked the same case- + * insensitive way handle_mbox_select() already does. + * + * Delivery follows maildir's own atomic contract (openimap-storage- + * backend.md: "write to tmp/, rename(2) into new/"): write the full + * message to a freshly-named tmp/ file, fsync, then rename directly into + * cur/ with whatever flags the client requested (or none) encoded in the + * maildir suffix immediately -- not into new/. Unlike externally + * (MTA-)delivered mail, which genuinely hasn't been seen by any client + * yet, an APPENDed message is by construction something the appending + * client already knows about (importing old mail, saving a draft or sent + * copy, etc.) -- there's no "unseen" period to represent, so there's no + * reason to place it in new/ the way SELECT's new/ scan handles + * externally-delivered mail. Not directly sourced -- openimap-storage- + * backend.md doesn't discuss APPEND at all -- this is this + * implementation's own inference from what new/ vs. cur/ actually mean, + * consistent with the "touched by any client action -> cur/, with an + * explicit (possibly empty) suffix" precedent handle_mbox_store() already + * established. + * + * The basename's uniquer (`._.`) is + * this implementation's own scheme, not a reproduction of Courier's exact + * algorithm (not directly sourced this session) -- it only has to satisfy + * maildir's actual requirement (a basename is never reused within this + * mailbox), which timestamp + pid + a per-process counter already does on + * its own; the trailing hostname field is included anyway to match + * Courier's own `..` shape (used there to + * disambiguate multiple physical delivery hosts sharing one NFS-mounted + * maildir -- a deployment v1 doesn't support, but there's no reason not + * to carry the real value once it's available for free). append_ + * hostname() below calls the real gethostname(2) -- confirmed safe under + * pledge(2) this session by reading the real pledge_sysctl() in + * openbsd_source/sys/kern/kern_pledge.c directly: the KERN_HOSTNAME case + * (what gethostname(2) resolves to internally) returns success + * unconditionally, with no `pledge &` gate on any specific promise the + * way most of that function's other cases have -- so store's minimal + * "stdio" pledge already covers it. Not sanitized against unusual + * characters: ':' is the one character that would actually be dangerous + * here, since it also introduces the maildir flag-suffix in the final + * on-disk filename (locate_message_file()'s own basename-prefix-plus- + * ':' matching). Standard DNS hostnames don't contain ':' -- this is + * general knowledge, not something re-verified against an RFC this + * session -- so this is treated as safe in practice rather than + * defended against explicitly; a hostname from a source that could + * return one (not gethostname(2) on a normally configured system) would + * need this revisited. + * + * Ordering matters for failure-mode safety: the index is updated (UID + * assigned, index_save()'d) *before* the tmp/ -> cur/ rename, not after. + * If the rename then fails, the result is an index entry pointing at a + * basename that was never actually created in cur/ -- exactly the + * "message indexed but missing on disk" case handle_mbox_fetch()/ + * handle_mbox_store()/handle_mbox_expunge() already handle gracefully + * (log, skip). The other ordering (rename first, index second) would + * instead risk a file sitting in cur/ with no index entry at all if the + * index step failed afterward -- invisible forever, since (unlike new/) + * nothing ever scans cur/ for unindexed basenames. Per RFC 9051 SS6.3.12, + * "the mailbox MUST be restored to its state before the APPEND attempt + * (other than possibly keeping the changed mailbox's UIDNEXT value)" on + * failure -- UIDNEXT is explicitly allowed to stay bumped; this + * implementation additionally, in the specific rename-fails-after-index- + * save case, leaves behind the phantom index entry rather than reopening + * the index a second time to roll it back -- a small, deliberately + * accepted imperfection in an already-rare failure path, not a silent + * one. + */ + +/* + * Returns this host's name for the maildir basename uniquer above, + * calling the real gethostname(2) once per store child and caching the + * result (a session's hostname can't change mid-process, so there's no + * reason to re-enter the kernel on every APPEND). Falls back to the + * literal string "imapd" -- the same placeholder this code used + * unconditionally before gethostname(2)'s pledge(2) coverage was + * confirmed -- if the call itself ever fails; genuinely unlikely on a + * normally configured system, but a failure here has never been fatal + * to the basename's own uniqueness guarantee (which only ever depended + * on timestamp+pid+counter, see this function's header comment), so + * there's no reason to treat it as fatal to the whole APPEND either. + */ +static const char * +append_hostname(void) +{ + static char hostbuf[256]; + static int resolved; + + if (!resolved) { + if (gethostname(hostbuf, sizeof(hostbuf)) == -1) { + log_warn("session %u: gethostname", session_id); + strlcpy(hostbuf, "imapd", sizeof(hostbuf)); + } + resolved = 1; + } + return (hostbuf); +} + +static void +handle_mbox_append(struct imsg_mbox_append *req, const char *msgbody, + size_t msglen, struct imsgev *iev) +{ + struct mbox_index idx; + struct imsg_mbox_appended reply; + int fd = -1, tmpfd = -1, locked = 0; + char basename[256]; + char tmppath[300], curpath[320]; + char target[MBOX_NAME_MAX]; + char letters[8], line[STORE_INDEX_LINE_MAX]; + int64_t delivery_ts; + char saved[MBOX_NAME_MAX]; + int switched = 0; + + memset(&idx, 0, sizeof(idx)); + memset(&reply, 0, sizeof(reply)); + + /* + * RFC 9051 SS6.3.4-SS6.3.6/SS6.3.9 addition: APPEND's destination is + * independent of whatever this session currently has SELECTed (RFC + * 9051 SS6.3.12: valid in the authenticated state with nothing + * selected at all), so -- unlike FETCH/STORE/EXPUNGE, which stay + * cwd-relative and unchanged because they never need to go anywhere + * else -- this resolves and *moves to* its own target directory via + * select_mailbox_dir(), then uses bare, cwd-relative paths for + * everything from here down, exactly the way handle_mbox_status() + * and (for its destination side) handle_mbox_copy()/handle_mbox_ + * move()'s commit_copy_messages() already do. + * + * This function used to do it differently -- stay at the maildir + * root and build every path with an explicit "name/" prefix string, + * the same shape CREATE/DELETE/RENAME/LIST use (see handle_mbox_ + * create()'s comment for that pattern's own real-hardware bug and + * fix). That approach worked for every path this function *directly* + * builds (the tmp/ write, the cur/ rename, and the initial indexed + * open via a prefixed idxpath) -- but it silently broke on the final + * index_save() call, which is a shared helper that has no prefix + * parameter at all and unconditionally operates on bare "imapd. + * index"/"imapd.index.tmp" relative to cwd. Since this function + * never actually left the maildir root, every APPEND to a *named* + * mailbox correctly read that mailbox's own real index (via the + * prefixed open+fd) but then wrote the updated result to the + * session's INBOX index instead -- overwriting whatever was really + * there, once per APPEND, with a fresh one-line index for whichever + * message happened to be appended last. The actual message file + * still landed in the right place (cur/ under the correct mailbox, + * via the prefixed tmp/cur paths, which don't go through index_ + * save()) -- only the index bookkeeping went to the wrong file, so + * this was a real hazard to a mailbox's *index* integrity, not to + * message data itself. + * + * Found on premio testing real Apple Mail draft autosaves against a + * named Drafts mailbox: four rapid APPENDs each correctly wrote + * their message into Drafts/cur/, but Drafts/openimap.index stayed + * at UIDNEXT 1 (never advanced), while the session's real INBOX + * index -- untouched by anything else in the same window -- ended + * up replaced by a single line pointing at the fourth APPEND's own + * basename. INBOX's original message files were never touched + * (still present in its own cur/), so nothing was lost, but INBOX + * became briefly unreadable through IMAP until the index was + * repaired. Switching this function to actually chdir into the + * target mailbox before doing any of this work -- rather than + * merely building path strings that describe where it is -- closes + * the whole class: index_save() (and everything else below) is now + * cwd-relative and correct by the same construction FETCH/STORE/ + * EXPUNGE/STATUS already rely on, with nothing left that needs its + * own prefix parameter to get right. + */ + strlcpy(saved, current_mailbox_dir, sizeof(saved)); + + if (mailbox_name_is_inbox(req->mailbox)) { + target[0] = '\0'; + } else if (mailbox_name_valid(req->mailbox)) { + strlcpy(target, req->mailbox, sizeof(target)); + } else { + log_debug("session %u: APPEND %s: invalid mailbox name", + session_id, req->mailbox); + reply.no_such_mailbox = 1; + goto done_reply; + } + + if (select_mailbox_dir(target) == -1) { + log_debug("session %u: APPEND %s: no such mailbox", + session_id, req->mailbox); + reply.no_such_mailbox = 1; + goto done_reply; + } + switched = 1; + + if (ensure_maildir_dirs("") == -1) + goto done_reply; + + delivery_ts = req->has_date ? req->date : (int64_t)time(NULL); + + if (snprintf(basename, sizeof(basename), "%lld.%d_%u.%s", + (long long)delivery_ts, (int)getpid(), append_counter++, + append_hostname()) >= (int)sizeof(basename)) { + log_warnx("session %u: generated basename too long", + session_id); + goto done_reply; + } + if (snprintf(tmppath, sizeof(tmppath), "tmp/%s", basename) >= + (int)sizeof(tmppath)) { + log_warnx("session %u: tmp path too long", session_id); + goto done_reply; + } + + if ((tmpfd = open(tmppath, O_WRONLY | O_CREAT | O_EXCL, 0600)) == -1) { + log_warn("session %u: open %s", session_id, tmppath); + goto done_reply; + } + { + size_t written = 0; + + while (written < msglen) { + ssize_t n; + + n = write(tmpfd, msgbody + written, msglen - written); + if (n == -1) { + if (errno == EINTR) + continue; + log_warn("session %u: write %s", session_id, + tmppath); + close(tmpfd); + tmpfd = -1; + unlink(tmppath); + goto done_reply; + } + written += (size_t)n; + } + } + if (fsync(tmpfd) == -1) + log_warn("session %u: fsync %s (continuing)", session_id, + tmppath); + close(tmpfd); + tmpfd = -1; + + if ((fd = open(STORE_INDEX_NAME, O_RDWR | O_CREAT, 0600)) == -1) { + log_warn("session %u: open %s", session_id, STORE_INDEX_NAME); + goto done_unlink; + } + if (flock(fd, LOCK_EX) == -1) { + log_warn("session %u: flock %s", session_id, STORE_INDEX_NAME); + goto done_unlink; + } + locked = 1; + if (index_load(fd, &idx) == -1) + goto done_unlink; + + reply.uid = idx.uidnext; + if (index_append(&idx, reply.uid, basename) == -1) + goto done_unlink; + idx.uidnext++; + + if (req->keywords[0] != '\0') { + int n; + + /* + * index_append() just assigned this message idx.highestmodseq + * (its own bump, above) and wrote an empty-keywords line -- + * this rewrite has to carry that same value forward, not drop + * it, or an APPEND with an initial flag-list would silently + * lose its per-message mod-sequence (RFC 7162 field added this + * pass; see index_append()'s comment). + */ + n = snprintf(line, sizeof(line), "%u:%s:%s:%llu", reply.uid, + basename, req->keywords, + (unsigned long long)idx.highestmodseq); + if (n < 0 || (size_t)n >= sizeof(line)) { + log_warnx("session %u: index line too long for %s", + session_id, basename); + goto done_unlink; + } + free(idx.lines[idx.nlines - 1]); + if ((idx.lines[idx.nlines - 1] = strdup(line)) == NULL) { + log_warn("session %u: strdup index line", session_id); + goto done_unlink; + } + } + + if (index_save(&idx) == -1) + goto done_unlink; + + reply.uidvalidity = idx.uidvalidity; + reply.exists = (uint32_t)idx.nlines; + + flock(fd, LOCK_UN); + close(fd); + fd = -1; + locked = 0; + index_free(&idx); + + /* Index committed -- now the rename; see this function's header + * comment for why this order (index-then-rename, not rename-then- + * index) is the safer one to fail partway through. */ + sysflags_to_letters(req->sysflags, letters, sizeof(letters)); + if (snprintf(curpath, sizeof(curpath), "cur/%s:2,%s", basename, + letters) >= (int)sizeof(curpath)) { + log_warnx("session %u: cur path too long for %s", session_id, + basename); + unlink(tmppath); + goto done_reply; + } + if (rename(tmppath, curpath) == -1) { + log_warn("session %u: rename %s -> %s", session_id, tmppath, + curpath); + unlink(tmppath); + goto done_reply; + } + + reply.ok = 1; + goto done_reply; + +done_unlink: + unlink(tmppath); + if (locked) + flock(fd, LOCK_UN); + if (fd != -1) + close(fd); + index_free(&idx); + +done_reply: + if (switched && select_mailbox_dir(saved) == -1) + log_warnx("session %u: APPEND %s: couldn't restore " + "previously selected mailbox %s", session_id, + req->mailbox, saved); + if (imsg_compose(&iev->ibuf, IMSG_MBOX_APPENDED, 0, 0, -1, &reply, + sizeof(reply)) == -1) + log_warn("session %u: imsg_compose IMSG_MBOX_APPENDED", + session_id); + imsgev_add(iev); +} + +/* + * One message staged by stage_copy_messages() for handle_mbox_copy()/ + * handle_mbox_move() -- read fully into memory from the source mailbox, + * not yet written anywhere. File-scope (not local to either function) + * since move_cross_mailbox() below needs the same shape too. + */ + +/* + * F6 fix: bound how much message data COPY/MOVE holds in memory at once. + * stage_copy_messages() reads every message in the range fully into RAM + * before commit (required for RFC 9051 6.4.7 all-or-nothing COPY), so an + * unbounded range of large, externally-delivered messages could exhaust + * the store child's address space. Exceeding either limit fails the whole + * COPY/MOVE cleanly (no partial copy). Tunable for larger deployments. + */ +#define COPY_STAGE_MSG_MAX ((uint64_t)64 * 1024 * 1024) /* per message */ +#define COPY_STAGE_TOTAL_MAX ((uint64_t)512 * 1024 * 1024) /* per operation */ + +struct copy_staged { + uint32_t src_uid; + uint32_t dest_uid; + char basename[256]; + char keywords[512]; + uint32_t sysflags; + void *body; + size_t bodylen; +}; + +/* + * Resolves a client-supplied mailbox name to the bare "target" string + * select_mailbox_dir() expects: "" for INBOX, or the name itself once + * mailbox_name_valid() has passed syntax. Returns 0 and fills *target on + * success, -1 (target left untouched) if the name is syntactically + * invalid -- this only checks syntax, not existence; select_mailbox_dir()'s + * own stat(2) call is still what determines whether the mailbox genuinely + * exists on disk. Added this pass (RFC 9051 SS6.4.7/SS6.4.8 cross-mailbox + * COPY/MOVE, docs/openimap-storage-backend.md item 10's own follow-up) so + * handle_mbox_copy()/handle_mbox_move() can resolve a destination name the + * identical way handle_mbox_select()/handle_mbox_create() already resolve + * theirs, rather than re-deriving the same mailbox_name_is_inbox()/ + * mailbox_name_valid()/else three-way split a third and fourth time. + */ +static int +resolve_mailbox_target(const char *name, char *target, size_t targetlen) +{ + if (mailbox_name_is_inbox(name)) { + target[0] = '\0'; + return (0); + } + if (mailbox_name_valid(name)) { + if (strlcpy(target, name, targetlen) >= targetlen) + return (-1); + return (0); + } + return (-1); +} + +/* + * Pass 1 of COPY/MOVE: reads every message in the requested sequence/UID + * range (by_uid-or-sequence interpreted the same way handle_mbox_search()/ + * handle_mbox_fetch() already do) from whichever mailbox is currently + * cwd-resident into memory, read-only -- no disk writes, no index + * mutation. Shared by handle_mbox_copy() and move_cross_mailbox() (a + * cross-mailbox MOVE is, per RFC 9051 SS6.4.8's own "COPY... followed by + * removal" equivalence, structurally a COPY that additionally removes the + * originals afterward -- see move_cross_mailbox()'s own header comment), + * and by both of COPY's same-mailbox/cross-mailbox code paths, since + * staging never depends on where the *destination* is, only on where the + * source already is (cwd, by the time this is called). + * + * Returns 1 and fills *staged_out and *nstaged_out on success (caller frees + * each entry's .body plus the array itself), 0 if any message failed to + * stage -- per COPY's own all-or-nothing requirement (SS6.4.7 -- "server + * implementations MUST restore the destination mailbox to its state + * before the COPY attempt... i.e., partial copy MUST NOT be done"), + * nothing has been written anywhere yet either way. + * + * New basenames reuse parse_maildir_timestamp() against the *source* + * basename (not the current time) so the copy's own INTERNALDATE -- + * derived from that same leading field, see that function's comment -- + * matches the original's, per SS6.4.7: "The flags and internal date of + * the message(s) SHOULD be preserved in the copy." append_counter/ + * append_hostname() (file-scope, see append_counter's own comment) mint + * the rest of the basename exactly like APPEND's own basenames. + */ +static int +stage_copy_messages(struct mbox_index *idx, struct imsg_mbox_copy *req, + struct copy_staged **staged_out, size_t *nstaged_out) +{ + struct copy_staged *staged = NULL; + size_t nstaged = 0, stagedcap = 0, i; + uint64_t staged_total = 0; /* F6: bytes staged so far */ + uint32_t lo, hi; + + lo = hi = 0; + if (req->by_uid) { + uint32_t max_uid = index_max_uid(idx); + + lo = req->lo_is_star ? max_uid : req->seq_lo; + hi = req->hi_is_star ? max_uid : req->seq_hi; + if (lo < 1) + lo = 1; + } else { + lo = req->seq_lo; + hi = req->hi_is_star ? (uint32_t)idx->nlines : req->seq_hi; + } + + for (i = 0; i < idx->nlines; i++) { + struct index_rec rec; + const char *lp; + char suffix[64]; + off_t size; + char path[600]; + int srcfd; + struct copy_staged cs; + + if (index_parse_line(idx->lines[i], &rec) == -1) + continue; + + if (req->by_uid) { + if (rec.uid < lo) + continue; + if (rec.uid > hi) + break; + } else { + if (i + 1 < lo) + continue; + if (i + 1 > hi) + break; + } + + if (locate_message_file(rec.basename, &size, suffix, + sizeof(suffix)) == -1) { + log_warnx("session %u: COPY: message %s indexed but " + "missing on disk -- failing whole COPY (partial " + "copy not permitted, RFC 9051 SS6.4.7)", + session_id, rec.basename); + goto fail; + } + + /* F6 fix: refuse before allocating if this message, or the + * running total, would exceed the staging limits. */ + if ((uint64_t)size > COPY_STAGE_MSG_MAX) { + log_warnx("session %u: COPY: message %s is %lld bytes, " + "over the per-message staging limit -- failing COPY", + session_id, rec.basename, (long long)size); + goto fail; + } + if ((uint64_t)size > COPY_STAGE_TOTAL_MAX - staged_total) { + log_warnx("session %u: COPY: staged data would exceed the " + "total staging limit -- failing COPY", session_id); + goto fail; + } + staged_total += (uint64_t)size; + + memset(&cs, 0, sizeof(cs)); + cs.src_uid = rec.uid; + strlcpy(cs.keywords, rec.keywords, sizeof(cs.keywords)); + lp = strstr(suffix, "2,"); + cs.sysflags = letters_to_sysflags(lp != NULL ? lp + 2 : ""); + + if (snprintf(cs.basename, sizeof(cs.basename), + "%lld.%d_%u.%s", + (long long)parse_maildir_timestamp(rec.basename), + (int)getpid(), append_counter++, append_hostname()) >= + (int)sizeof(cs.basename)) { + log_warnx("session %u: COPY: generated basename too " + "long", session_id); + goto fail; + } + + if (snprintf(path, sizeof(path), "%s/%s%s", + suffix[0] == '\0' ? "new" : "cur", rec.basename, suffix) + >= (int)sizeof(path)) { + log_warnx("session %u: COPY: source path too long " + "for %s", session_id, rec.basename); + goto fail; + } + if ((srcfd = open(path, O_RDONLY)) == -1) { + log_warn("session %u: COPY: open %s", session_id, + path); + goto fail; + } + cs.bodylen = (size_t)size; + if (cs.bodylen > 0 && (cs.body = malloc(cs.bodylen)) == NULL) { + log_warn("session %u: COPY: malloc %zu bytes", + session_id, cs.bodylen); + close(srcfd); + goto fail; + } + { + size_t rd = 0; + + while (rd < cs.bodylen) { + ssize_t n = read(srcfd, + (char *)cs.body + rd, cs.bodylen - rd); + if (n == -1) { + if (errno == EINTR) + continue; + log_warn("session %u: COPY: read %s", + session_id, path); + close(srcfd); + free(cs.body); + goto fail; + } + if (n == 0) + break; /* short file -- copy what's + * actually there rather than + * fail */ + rd += (size_t)n; + } + cs.bodylen = rd; + } + close(srcfd); + + if (nstaged == stagedcap) { + size_t newcap = (stagedcap == 0) ? 8 : + stagedcap * 2; + struct copy_staged *newstaged = reallocarray(staged, + newcap, sizeof(*staged)); + + if (newstaged == NULL) { + log_warn("session %u: COPY: reallocarray " + "staged", session_id); + free(cs.body); + goto fail; + } + staged = newstaged; + stagedcap = newcap; + } + staged[nstaged++] = cs; + } + + *staged_out = staged; + *nstaged_out = nstaged; + return (1); + +fail: + for (i = 0; i < nstaged; i++) + free(staged[i].body); + free(staged); + *staged_out = NULL; + *nstaged_out = 0; + return (0); +} + +/* + * Pass 2+3 of COPY/MOVE: commits every staged message (see stage_copy_ + * messages()) to whichever mailbox is currently cwd-resident -- tmp/ + * write + rename into cur/ per message, then one index_append() per + * message into destidx and a single index_save() once every corresponding + * cur/ file is already durably in place, mirroring APPEND's own rename- + * then-index ordering. Streams one IMSG_MBOX_COPY_MAPPING per committed + * message as it's indexed, ready for session_finish_copy_or_move() in + * listener.c to compact into COPYUID's two UID sets. + * + * Caller must already have select_mailbox_dir()'d to the destination and + * ensure_maildir_dirs("")'d it. Returns 1 if every message committed, 0 on + * the first failure -- a mid-pass-2 failure (much rarer than a pass-1 read + * failure, since the source files are already known to exist) leaves + * destidx's UIDNEXT completely untouched, better than RFC 9051 SS6.4.7's + * own minimum bar, which explicitly allows UIDNEXT to have moved -- at the + * cost of a small, deliberately accepted imperfection: any cur/ files + * already renamed earlier in pass 2 before the failure are left behind + * rather than transactionally rolled back, the same category of rare- + * failure-path tradeoff APPEND's own header comment already flags for its + * own rename-ordering choice. + */ +static int +commit_copy_messages(struct mbox_index *destidx, struct copy_staged *staged, + size_t nstaged, struct imsgev *iev) +{ + size_t i; + + for (i = 0; i < nstaged; i++) { + char tmppath[300], curpath[320]; + char letters[8]; + int tmpfd; + + if (snprintf(tmppath, sizeof(tmppath), "tmp/%s", + staged[i].basename) >= (int)sizeof(tmppath)) { + log_warnx("session %u: COPY: tmp path too long", + session_id); + return (0); + } + if ((tmpfd = open(tmppath, O_WRONLY | O_CREAT | O_EXCL, + 0600)) == -1) { + log_warn("session %u: COPY: open %s", session_id, + tmppath); + return (0); + } + { + size_t written = 0; + + while (written < staged[i].bodylen) { + ssize_t n = write(tmpfd, + (char *)staged[i].body + written, + staged[i].bodylen - written); + if (n == -1) { + if (errno == EINTR) + continue; + log_warn("session %u: COPY: write %s", + session_id, tmppath); + close(tmpfd); + unlink(tmppath); + return (0); + } + written += (size_t)n; + } + } + if (fsync(tmpfd) == -1) + log_warn("session %u: COPY: fsync %s (continuing)", + session_id, tmppath); + close(tmpfd); + + sysflags_to_letters(staged[i].sysflags, letters, + sizeof(letters)); + if (snprintf(curpath, sizeof(curpath), "cur/%s:2,%s", + staged[i].basename, letters) >= (int)sizeof(curpath)) { + log_warnx("session %u: COPY: cur path too long", + session_id); + unlink(tmppath); + return (0); + } + if (rename(tmppath, curpath) == -1) { + log_warn("session %u: COPY: rename %s -> %s", + session_id, tmppath, curpath); + unlink(tmppath); + return (0); + } + } + + for (i = 0; i < nstaged; i++) { + uint32_t dest_uid = destidx->uidnext; + + if (index_append(destidx, dest_uid, staged[i].basename) == + -1) + return (0); + destidx->uidnext++; + staged[i].dest_uid = dest_uid; + + if (staged[i].keywords[0] != '\0') { + char line[STORE_INDEX_LINE_MAX]; + int n; + + n = snprintf(line, sizeof(line), "%u:%s:%s:%llu", + dest_uid, staged[i].basename, staged[i].keywords, + (unsigned long long)destidx->highestmodseq); + if (n < 0 || (size_t)n >= sizeof(line)) { + log_warnx("session %u: COPY: index line too " + "long", session_id); + return (0); + } + free(destidx->lines[destidx->nlines - 1]); + if ((destidx->lines[destidx->nlines - 1] = + strdup(line)) == NULL) { + log_warn("session %u: COPY: strdup index " + "line", session_id); + return (0); + } + } + } + + if (index_save(destidx) == -1) + return (0); + + for (i = 0; i < nstaged; i++) { + struct imsg_mbox_copy_mapping mapping; + + memset(&mapping, 0, sizeof(mapping)); + mapping.src_uid = staged[i].src_uid; + mapping.dest_uid = staged[i].dest_uid; + if (imsg_compose(&iev->ibuf, IMSG_MBOX_COPY_MAPPING, 0, 0, -1, + &mapping, sizeof(mapping)) == -1) + log_warn("session %u: imsg_compose " + "IMSG_MBOX_COPY_MAPPING", session_id); + } + + return (1); +} + +/* + * RFC 9051 SS6.4.7 COPY (and, via handle_mbox_move() just below, half of + * SS6.4.8 MOVE too). Destination resolution (RFC 9051 SS6.3.4-SS6.3.6 flat + * multi-mailbox support, docs/openimap-storage-backend.md item 10's own + * follow-up): req->destname names any real, existing mailbox, not just + * the one already SELECTed the way the original v1 code required. Two + * structurally different paths follow, chosen once near the top: + * + * - destname resolves to the mailbox already SELECTed: a single index, + * opened and flock(2)'d exactly once, cwd never moves -- the original + * v1 code, unchanged in spirit. This isn't just simpler than the + * cross-mailbox path below -- opening and flock(2)ing the *same* path + * a second time from this same process would self-deadlock (BSD + * flock(2) locks belong to the open file description, not the + * process, so a second independent open()+flock(LOCK_EX) here would + * block forever on the lock this same process already holds on the + * first). + * + * - destname resolves to a genuinely different mailbox: both mailboxes' + * index files are opened and flock(2)'d for the duration. Two + * sessions issuing opposite-direction copies between the same two + * mailboxes (session A: source X, dest Y; session B: source Y, dest + * X) must not each acquire one mailbox's lock and then block waiting + * for the other -- classic AB-BA cross-session deadlock. Locking in a + * fixed order derived only from the two mailbox names themselves, + * identical for both sessions regardless of which one is "source" and + * which is "destination" for *that* session, closes it: strcmp() + * between the two target strings (INBOX's "" sorts first against any + * named mailbox for free, no special-casing needed) picks the same + * first-to-lock mailbox both sessions above contend for, so they + * queue on it in the same order every time rather than in mirror- + * image order. + * + * See stage_copy_messages()/commit_copy_messages() above for the actual + * staging/commit work and their own all-or-nothing guarantees, shared + * verbatim between both paths. + */ +static void +handle_mbox_copy(struct imsg_mbox_copy *req, struct imsgev *iev) +{ + struct mbox_index idx_a, idx_b; + struct mbox_index *srcidx = NULL, *destidx = NULL; + struct imsg_mbox_result result; + struct copy_staged *staged = NULL; + size_t nstaged = 0, i; + int fd_a = -1, fd_a_locked = 0; + int fd_b = -1, fd_b_locked = 0; + int ok = 1; + char saved[MBOX_NAME_MAX]; + char desttarget[MBOX_NAME_MAX]; + int cross_mailbox; + + memset(&idx_a, 0, sizeof(idx_a)); + memset(&idx_b, 0, sizeof(idx_b)); + memset(&result, 0, sizeof(result)); + + strlcpy(saved, current_mailbox_dir, sizeof(saved)); + + if (resolve_mailbox_target(req->destname, desttarget, + sizeof(desttarget)) == -1) { + log_debug("session %u: COPY %s: invalid destination mailbox " + "name", session_id, req->destname); + result.no_such_mailbox = 1; + ok = 0; + goto done; + } + cross_mailbox = (strcmp(saved, desttarget) != 0); + + if (!cross_mailbox) { + if ((fd_a = open(STORE_INDEX_NAME, O_RDWR | O_CREAT, 0600)) == + -1) { + log_warn("session %u: open %s", session_id, + STORE_INDEX_NAME); + ok = 0; + goto done; + } + if (flock(fd_a, LOCK_EX) == -1) { + log_warn("session %u: flock %s", session_id, + STORE_INDEX_NAME); + ok = 0; + goto done; + } + fd_a_locked = 1; + if (index_load(fd_a, &idx_a) == -1) { + ok = 0; + goto done; + } + srcidx = &idx_a; + destidx = &idx_a; + } else { + char first[MBOX_NAME_MAX], second[MBOX_NAME_MAX]; + int first_is_dest; + + if (strcmp(saved, desttarget) <= 0) { + strlcpy(first, saved, sizeof(first)); + strlcpy(second, desttarget, sizeof(second)); + first_is_dest = 0; + } else { + strlcpy(first, desttarget, sizeof(first)); + strlcpy(second, saved, sizeof(second)); + first_is_dest = 1; + } + + if (select_mailbox_dir(first) == -1) { + if (first_is_dest) + result.no_such_mailbox = 1; + else + log_warnx("session %u: COPY: couldn't reach " + "%s", session_id, first); + ok = 0; + goto done; + } + if ((fd_a = open(STORE_INDEX_NAME, O_RDWR | O_CREAT, 0600)) == + -1) { + log_warn("session %u: open %s", session_id, + STORE_INDEX_NAME); + ok = 0; + goto restore_saved; + } + if (flock(fd_a, LOCK_EX) == -1) { + log_warn("session %u: flock %s", session_id, + STORE_INDEX_NAME); + ok = 0; + goto restore_saved; + } + fd_a_locked = 1; + if (index_load(fd_a, &idx_a) == -1) { + ok = 0; + goto restore_saved; + } + + if (select_mailbox_dir(second) == -1) { + if (!first_is_dest) + result.no_such_mailbox = 1; + else + log_warnx("session %u: COPY: couldn't reach " + "%s", session_id, second); + ok = 0; + goto restore_saved; + } + if ((fd_b = open(STORE_INDEX_NAME, O_RDWR | O_CREAT, 0600)) == + -1) { + log_warn("session %u: open %s", session_id, + STORE_INDEX_NAME); + ok = 0; + goto restore_saved; + } + if (flock(fd_b, LOCK_EX) == -1) { + log_warn("session %u: flock %s", session_id, + STORE_INDEX_NAME); + ok = 0; + goto restore_saved; + } + fd_b_locked = 1; + if (index_load(fd_b, &idx_b) == -1) { + ok = 0; + goto restore_saved; + } + + srcidx = first_is_dest ? &idx_b : &idx_a; + destidx = first_is_dest ? &idx_a : &idx_b; + + /* Back to source -- staging below reads relative to cwd. */ + if (select_mailbox_dir(saved) == -1) { + log_warnx("session %u: COPY: couldn't return to %s " + "to stage messages", session_id, saved); + ok = 0; + goto restore_saved; + } + } + + if (!stage_copy_messages(srcidx, req, &staged, &nstaged)) { + ok = 0; + goto restore_saved; + } + + if (nstaged > 0) { + if (cross_mailbox && select_mailbox_dir(desttarget) == -1) { + log_warnx("session %u: COPY: couldn't reach " + "destination %s", session_id, desttarget); + ok = 0; + goto restore_saved; + } + if (ensure_maildir_dirs("") == -1) { + ok = 0; + goto restore_saved; + } + if (!commit_copy_messages(destidx, staged, nstaged, iev)) + ok = 0; + } + + if (ok) { + result.count = (uint32_t)nstaged; + result.uidvalidity = destidx->uidvalidity; + result.highestmodseq = destidx->highestmodseq; + } + +restore_saved: + if (cross_mailbox && select_mailbox_dir(saved) == -1) + log_warnx("session %u: COPY: couldn't restore previously " + "selected mailbox %s", session_id, saved); + +done: + for (i = 0; i < nstaged; i++) + free(staged[i].body); + free(staged); + + if (fd_a_locked) + flock(fd_a, LOCK_UN); + if (fd_a != -1) + close(fd_a); + if (fd_b_locked) + flock(fd_b, LOCK_UN); + if (fd_b != -1) + close(fd_b); + + result.ok = ok; + index_free(&idx_a); + index_free(&idx_b); + + if (imsg_compose(&iev->ibuf, IMSG_MBOX_RESULT, 0, 0, -1, &result, + sizeof(result)) == -1) + log_warn("session %u: imsg_compose IMSG_MBOX_RESULT", + session_id); + imsgev_add(iev); +} + +/* + * MOVE when the destination is the mailbox already SELECTed -- the + * original v1 algorithm, unchanged: moving a message to the mailbox it's + * already in needs no file I/O at all, the same physical cur/ file just + * gets reindexed under a fresh UID. This is "MOVE = COPY + STORE + * +FLAGS.SILENT \Deleted + UID EXPUNGE" (SS6.4.8's own equivalence) + * collapsed to its net effect once source and destination coincide: the + * old index line disappears, a new one (same basename, new UID) appears + * at the tail, and nothing on disk changes at all. See handle_mbox_ + * move()'s own header comment for why this stays a distinct, cheaper path + * from move_cross_mailbox() below rather than being folded into the + * general "copy then remove" shape a genuinely different destination + * requires. + * + * Unlike COPY, MOVE is explicitly allowed to fail partway through its set + * ("Regardless of whether the command is successful in moving the entire + * set, each individual message MUST be either moved or unaffected" -- + * SS6.4.8), so this doesn't need COPY's stage-then-commit split -- given + * the zero-copy design, the only failure modes left are index_append()'s + * own malloc/line-length checks and the final index_save(), both of which + * fail before any disk state changes, so each message really is + * atomically either moved or unaffected in this implementation. + * + * Two passes over idx, the first shaped exactly like handle_mbox_ + * expunge()'s own compaction algorithm: pass 1 walks and compacts, + * removing each matched message's old line and recording (old UID, old + * seqno, basename, keywords) for pass 2, which re-appends each recorded + * message at the tail with a fresh UID via index_append() (same + * per-message HIGHESTMODSEQ bump as an APPEND or COPY arrival -- see that + * function's own RFC 7162 SS3.1 citation: "the server generates a new + * modification sequence... when a message is appended... via APPEND, + * COPY to the mailbox, or using an external mechanism"; a MOVE's + * destination-side effect is exactly this same kind of arrival). + * old_seqno is computed the identical way handle_mbox_expunge() already + * does ("immediately decremented" as earlier matches are removed), not + * recomputed afterward. + * + * Per SS6.4.8, "servers are also REQUIRED to send the COPYUID response + * code in an untagged OK before sending EXPUNGE" -- this function sends + * every IMSG_MBOX_COPY_MAPPING first (naturally, since pass 2 runs before + * the final notification loop below), then every IMSG_MBOX_EXPUNGED, + * matching send order to the required response order; listener.c + * additionally buffers and re-flushes in this same fixed order regardless + * (see imapd.h's imsg_mbox_copy_mapping comment), so this function's + * own ordering isn't the only thing enforcing it, just a natural match. + * + * Real bug caught testing MOVE live on premio, before flat multi-mailbox + * support existed: "MOVE 2 INBOX" against a 3-message mailbox moved + * messages 2 *and* 3, not just 2. Root cause was using "out" (this loop's + * compaction *write* index, which stalls -- doesn't increment -- every + * time a message matches and gets removed) instead of "in" (the original + * read position) to test sequence-range membership. "in" is the correct + * original 1-based sequence number for the line currently being examined; + * old_seqno just below is a *different*, intentionally correct use of + * "out + 1" -- it reports each EXPUNGE's sequence number relative to the + * mailbox state after earlier removals in the same command, which is + * exactly what the "immediately decremented" value is for. + */ +static int +move_same_mailbox(struct imsg_mbox_copy *req, struct mbox_index *idx, + uint32_t *nmoved_out, struct imsgev *iev) +{ + struct moved { + uint32_t old_uid; + uint32_t old_seqno; + uint32_t dest_uid; + char basename[512]; + char keywords[512]; + }; + + struct moved *moved = NULL; + size_t nmoved = 0, movedcap = 0, in, out, i; + uint32_t lo, hi; + int ok = 1; + + *nmoved_out = 0; + + lo = hi = 0; + if (req->by_uid) { + uint32_t max_uid = index_max_uid(idx); + + lo = req->lo_is_star ? max_uid : req->seq_lo; + hi = req->hi_is_star ? max_uid : req->seq_hi; + if (lo < 1) + lo = 1; + } else { + lo = req->seq_lo; + hi = req->hi_is_star ? (uint32_t)idx->nlines : req->seq_hi; + } + + out = 0; + for (in = 0; in < idx->nlines; in++) { + struct index_rec rec; + int matched; + + if (index_parse_line(idx->lines[in], &rec) == -1) { + idx->lines[out++] = idx->lines[in]; + continue; + } + + if (req->by_uid) + matched = (rec.uid >= lo && rec.uid <= hi); + else + matched = ((uint32_t)(in + 1) >= lo && + (uint32_t)(in + 1) <= hi); + + if (!matched) { + idx->lines[out++] = idx->lines[in]; + continue; + } + + if (nmoved == movedcap) { + size_t newcap = (movedcap == 0) ? 8 : + movedcap * 2; + struct moved *newmoved = reallocarray(moved, newcap, + sizeof(*moved)); + + if (newmoved == NULL) { + log_warn("session %u: MOVE: reallocarray " + "moved", session_id); + free(idx->lines[in]); + ok = 0; + goto done; + } + moved = newmoved; + movedcap = newcap; + } + moved[nmoved].old_uid = rec.uid; + moved[nmoved].old_seqno = (uint32_t)(out + 1); + strlcpy(moved[nmoved].basename, rec.basename, + sizeof(moved[nmoved].basename)); + strlcpy(moved[nmoved].keywords, rec.keywords, + sizeof(moved[nmoved].keywords)); + nmoved++; + + free(idx->lines[in]); + } + idx->nlines = out; + + for (i = 0; i < nmoved; i++) { + uint32_t dest_uid = idx->uidnext; + + if (index_append(idx, dest_uid, moved[i].basename) == -1) { + ok = 0; + goto done; + } + idx->uidnext++; + moved[i].dest_uid = dest_uid; + + if (moved[i].keywords[0] != '\0') { + char line[STORE_INDEX_LINE_MAX]; + int n; + + n = snprintf(line, sizeof(line), "%u:%s:%s:%llu", + dest_uid, moved[i].basename, moved[i].keywords, + (unsigned long long)idx->highestmodseq); + if (n < 0 || (size_t)n >= sizeof(line)) { + log_warnx("session %u: MOVE: index line too " + "long", session_id); + ok = 0; + goto done; + } + free(idx->lines[idx->nlines - 1]); + if ((idx->lines[idx->nlines - 1] = strdup(line)) == + NULL) { + log_warn("session %u: MOVE: strdup index " + "line", session_id); + ok = 0; + goto done; + } + } + } + + if (index_save(idx) == -1) { + ok = 0; + goto done; + } + + /* COPYUID mapping data first... */ + for (i = 0; i < nmoved; i++) { + struct imsg_mbox_copy_mapping mapping; + + memset(&mapping, 0, sizeof(mapping)); + mapping.src_uid = moved[i].old_uid; + mapping.dest_uid = moved[i].dest_uid; + if (imsg_compose(&iev->ibuf, IMSG_MBOX_COPY_MAPPING, 0, 0, -1, + &mapping, sizeof(mapping)) == -1) + log_warn("session %u: imsg_compose " + "IMSG_MBOX_COPY_MAPPING", session_id); + } + /* ...then EXPUNGE notices for the old UIDs/seqnos, per SS6.4.8's + * required ordering. */ + for (i = 0; i < nmoved; i++) { + struct imsg_mbox_expunged exp; + + memset(&exp, 0, sizeof(exp)); + exp.seqno = moved[i].old_seqno; + exp.uid = moved[i].old_uid; + if (imsg_compose(&iev->ibuf, IMSG_MBOX_EXPUNGED, 0, 0, -1, + &exp, sizeof(exp)) == -1) + log_warn("session %u: imsg_compose IMSG_MBOX_EXPUNGED", + session_id); + } + +done: + *nmoved_out = (uint32_t)nmoved; + free(moved); + return (ok); +} + +/* + * MOVE when the destination genuinely differs from the mailbox already + * SELECTed. Per RFC 9051 SS6.4.8's own definition ("MOVE... has the same + * properties as COPY, followed by removal of the copied messages"), this + * is exactly that: stage_copy_messages()/commit_copy_messages() (shared + * with handle_mbox_copy() above) do the "COPY" half -- reusing all of + * COPY's own all-or-nothing staging/commit guarantees for messages that + * cross-mailbox MOVE now needs too, since (unlike move_same_mailbox()'s + * zero-copy design above) real file I/O against two different + * directories is unavoidable here -- and this function's own second half + * then removes each successfully-committed message from the source: + * unlink(2) the original cur/ or new/ file and compact it out of srcidx. + * + * RFC 9051 SS6.4.8 only requires "each individual message MUST be either + * moved or unaffected", which permits a partial MOVE across a large set; + * this implementation is deliberately stricter, matching COPY's own + * all-or-nothing choice (stage_copy_messages()/commit_copy_messages()'s + * own comments already flag that as a safe superset of the RFC's minimum + * bar) -- simpler to reason about, and the risk of a large cross-mailbox + * MOVE landing half-done is the same category of surprise a client would + * get from a half-done COPY, which SS6.4.7 already forbids outright. + * + * One accepted asymmetry remains, and is called out explicitly rather + * than silently: once commit_copy_messages() has durably rename(2)d every + * copy into the destination's cur/ and index_save()'d there, this + * function still has to unlink(2) each source file and index_save() the + * source's own compaction separately -- a real (if narrow) window where a + * crash could leave a message duplicated in both mailboxes rather than + * moved. Preferred over the alternative of removing from source *before* + * the destination commit is confirmed, which could instead lose the + * message outright on the same kind of crash -- duplication is the + * recoverable failure mode, data loss is not. + * + * Messages are matched for removal by the UID stage_copy_messages() + * already recorded (staged[i].src_uid), not by re-deriving sequence-range + * membership a second time -- sidesteps entirely the by-sequence + * "in"-vs-"out" bug class move_same_mailbox()'s own header comment + * documents, since there's no ambiguity left to get wrong once the exact + * UID set is already known. + */ +static int +move_cross_mailbox(struct imsg_mbox_copy *req, struct mbox_index *srcidx, + struct mbox_index *destidx, const char *desttarget, const char *saved, + uint32_t *nmoved_out, struct imsgev *iev) +{ + struct copy_staged *staged = NULL; + size_t nstaged = 0, i; + int ok = 1, any_removed = 0; + + *nmoved_out = 0; + + if (!stage_copy_messages(srcidx, req, &staged, &nstaged)) + return (0); + + if (nstaged == 0) + return (1); + + if (select_mailbox_dir(desttarget) == -1) { + log_warnx("session %u: MOVE: couldn't reach destination %s", + session_id, desttarget); + ok = 0; + goto cleanup; + } + if (ensure_maildir_dirs("") == -1) { + ok = 0; + goto cleanup; + } + if (!commit_copy_messages(destidx, staged, nstaged, iev)) { + ok = 0; + goto cleanup; + } + + /* + * Destination commit is durable -- now remove each original from + * the source. Back to source's own directory first (cwd is + * currently sitting at the destination, from the commit above). + */ + if (select_mailbox_dir(saved) == -1) { + log_warnx("session %u: MOVE: committed to destination but " + "couldn't return to source %s to remove the originals " + "-- message(s) now duplicated in both mailboxes rather " + "than moved", session_id, saved); + ok = 0; + goto cleanup; + } + + for (i = 0; i < nstaged; i++) { + struct index_rec rec; + size_t j, out; + uint32_t old_seqno = 0; + int found = 0; + off_t size; + char suffix[64], path[600]; + + /* + * Re-locate under its *original* basename (staged[i]'s own + * .basename is the freshly minted destination name) via a + * fresh scan of srcidx -- deliberately not a single + * combined compaction pass across every staged[] entry at + * once, so each removal's old_seqno reflects the mailbox + * state after every earlier removal in this same command, + * matching move_same_mailbox()'s "immediately decremented" + * semantics exactly rather than approximating it. + */ + for (j = 0, out = 0; j < srcidx->nlines; j++) { + if (!found && index_parse_line(srcidx->lines[j], + &rec) == 0 && rec.uid == staged[i].src_uid) { + old_seqno = (uint32_t)(out + 1); + found = 1; + free(srcidx->lines[j]); + continue; + } + srcidx->lines[out++] = srcidx->lines[j]; + } + srcidx->nlines = out; + + if (!found) { + /* + * Already gone from the source index -- another of + * this same user's sessions raced an EXPUNGE/STORE/ + * MOVE of the same message between staging and + * here. The copy at the destination is still valid + * and durable; nothing left to remove. + */ + continue; + } + any_removed = 1; + + if (locate_message_file(rec.basename, &size, suffix, + sizeof(suffix)) == 0) { + if (snprintf(path, sizeof(path), "%s/%s%s", + suffix[0] == '\0' ? "new" : "cur", rec.basename, + suffix) < (int)sizeof(path)) + unlink(path); + } + + { + struct imsg_mbox_expunged exp; + + memset(&exp, 0, sizeof(exp)); + exp.seqno = old_seqno; + exp.uid = staged[i].src_uid; + if (imsg_compose(&iev->ibuf, IMSG_MBOX_EXPUNGED, 0, 0, + -1, &exp, sizeof(exp)) == -1) + log_warn("session %u: imsg_compose " + "IMSG_MBOX_EXPUNGED", session_id); + } + } + + /* RFC 7162 SS3.1: same single per-operation HIGHESTMODSEQ bump + * handle_mbox_expunge() already uses for its own compaction. */ + if (any_removed) + srcidx->highestmodseq++; + + if (index_save(srcidx) == -1) { + log_warnx("session %u: MOVE: destination commit succeeded " + "but saving the source's compacted index failed -- " + "message(s) may remain duplicated", session_id); + ok = 0; + goto cleanup; + } + + *nmoved_out = (uint32_t)nstaged; + +cleanup: + for (i = 0; i < nstaged; i++) + free(staged[i].body); + free(staged); + return (ok); +} + +/* + * RFC 9051 SS6.4.8 MOVE, dispatcher half: resolves the destination and + * acquires whichever index file(s) are needed exactly the way handle_ + * mbox_copy() does (see that function's header comment for the same- + * mailbox-vs-cross-mailbox split and the lock-ordering reasoning -- not + * re-explained here), then hands off to move_same_mailbox() or move_ + * cross_mailbox() above for the actual work. + */ +static void +handle_mbox_move(struct imsg_mbox_copy *req, struct imsgev *iev) +{ + struct mbox_index idx_a, idx_b; + struct mbox_index *srcidx = NULL, *destidx = NULL; + struct imsg_mbox_result result; + uint32_t nmoved = 0; + int fd_a = -1, fd_a_locked = 0; + int fd_b = -1, fd_b_locked = 0; + int ok = 1; + char saved[MBOX_NAME_MAX]; + char desttarget[MBOX_NAME_MAX]; + int cross_mailbox; + + memset(&idx_a, 0, sizeof(idx_a)); + memset(&idx_b, 0, sizeof(idx_b)); + memset(&result, 0, sizeof(result)); + + strlcpy(saved, current_mailbox_dir, sizeof(saved)); + + if (resolve_mailbox_target(req->destname, desttarget, + sizeof(desttarget)) == -1) { + log_debug("session %u: MOVE %s: invalid destination mailbox " + "name", session_id, req->destname); + result.no_such_mailbox = 1; + ok = 0; + goto done; + } + cross_mailbox = (strcmp(saved, desttarget) != 0); + + if (!cross_mailbox) { + if ((fd_a = open(STORE_INDEX_NAME, O_RDWR | O_CREAT, 0600)) == + -1) { + log_warn("session %u: open %s", session_id, + STORE_INDEX_NAME); + ok = 0; + goto done; + } + if (flock(fd_a, LOCK_EX) == -1) { + log_warn("session %u: flock %s", session_id, + STORE_INDEX_NAME); + ok = 0; + goto done; + } + fd_a_locked = 1; + if (index_load(fd_a, &idx_a) == -1) { + ok = 0; + goto done; + } + srcidx = &idx_a; + destidx = &idx_a; + } else { + char first[MBOX_NAME_MAX], second[MBOX_NAME_MAX]; + int first_is_dest; + + if (strcmp(saved, desttarget) <= 0) { + strlcpy(first, saved, sizeof(first)); + strlcpy(second, desttarget, sizeof(second)); + first_is_dest = 0; + } else { + strlcpy(first, desttarget, sizeof(first)); + strlcpy(second, saved, sizeof(second)); + first_is_dest = 1; + } + + if (select_mailbox_dir(first) == -1) { + if (first_is_dest) + result.no_such_mailbox = 1; + else + log_warnx("session %u: MOVE: couldn't reach " + "%s", session_id, first); + ok = 0; + goto done; + } + if ((fd_a = open(STORE_INDEX_NAME, O_RDWR | O_CREAT, 0600)) == + -1) { + log_warn("session %u: open %s", session_id, + STORE_INDEX_NAME); + ok = 0; + goto restore_saved; + } + if (flock(fd_a, LOCK_EX) == -1) { + log_warn("session %u: flock %s", session_id, + STORE_INDEX_NAME); + ok = 0; + goto restore_saved; + } + fd_a_locked = 1; + if (index_load(fd_a, &idx_a) == -1) { + ok = 0; + goto restore_saved; + } + + if (select_mailbox_dir(second) == -1) { + if (!first_is_dest) + result.no_such_mailbox = 1; + else + log_warnx("session %u: MOVE: couldn't reach " + "%s", session_id, second); + ok = 0; + goto restore_saved; + } + if ((fd_b = open(STORE_INDEX_NAME, O_RDWR | O_CREAT, 0600)) == + -1) { + log_warn("session %u: open %s", session_id, + STORE_INDEX_NAME); + ok = 0; + goto restore_saved; + } + if (flock(fd_b, LOCK_EX) == -1) { + log_warn("session %u: flock %s", session_id, + STORE_INDEX_NAME); + ok = 0; + goto restore_saved; + } + fd_b_locked = 1; + if (index_load(fd_b, &idx_b) == -1) { + ok = 0; + goto restore_saved; + } + + srcidx = first_is_dest ? &idx_b : &idx_a; + destidx = first_is_dest ? &idx_a : &idx_b; + + if (select_mailbox_dir(saved) == -1) { + log_warnx("session %u: MOVE: couldn't return to %s", + session_id, saved); + ok = 0; + goto restore_saved; + } + } + + if (!cross_mailbox) { + if (!move_same_mailbox(req, srcidx, &nmoved, iev)) + ok = 0; + } else { + if (!move_cross_mailbox(req, srcidx, destidx, desttarget, + saved, &nmoved, iev)) + ok = 0; + } + + if (ok) { + result.count = nmoved; + result.uidvalidity = destidx->uidvalidity; + result.highestmodseq = destidx->highestmodseq; + } + +restore_saved: + if (cross_mailbox && select_mailbox_dir(saved) == -1) + log_warnx("session %u: MOVE: couldn't restore previously " + "selected mailbox %s", session_id, saved); + +done: + if (fd_a_locked) + flock(fd_a, LOCK_UN); + if (fd_a != -1) + close(fd_a); + if (fd_b_locked) + flock(fd_b, LOCK_UN); + if (fd_b != -1) + close(fd_b); + + result.ok = ok; + index_free(&idx_a); + index_free(&idx_b); + + if (imsg_compose(&iev->ibuf, IMSG_MBOX_RESULT, 0, 0, -1, &result, + sizeof(result)) == -1) + log_warn("session %u: imsg_compose IMSG_MBOX_RESULT", + session_id); + imsgev_add(iev); +} + +static void +store_dispatch(int fd, short event, void *arg) +{ + struct imsgev *iev = arg; + struct imsg imsg; + ssize_t n; + + /* + * EV_WRITE: same real bug as listener.c's listener_dispatch_auth()/ + * auth.c's auth_dispatch() -- see those header comments for the + * full citation against imsg_init(3). Every IMSG_MBOX_*_RESULT this + * process sends back is queued via imsg_compose() and needs an + * actual imsgbuf_write() once the fd is writable. + */ + if (event & EV_WRITE) { + if (imsgbuf_write(&iev->ibuf) == -1) + fatal("imsgbuf_write"); + } + + if (event & EV_READ) { + if ((n = imsgbuf_read(&iev->ibuf)) == -1) + fatal("imsgbuf_read"); + if (n == 0) { + /* listener's end closed -- treat like + * IMSG_STORE_SHUTDOWN: nothing left to serve. */ + store_shutdown(); + /* NOTREACHED */ + } + } + + for (;;) { + if ((n = imsg_get(&iev->ibuf, &imsg)) == -1) + fatal("imsg_get"); + if (n == 0) + break; + + switch (imsg_get_type(&imsg)) { + case IMSG_STORE_SHUTDOWN: + imsg_free(&imsg); + store_shutdown(); + /* NOTREACHED */ + break; + case IMSG_MBOX_SELECT: { + struct imsg_mbox_select req; + + if (imsg_get_data(&imsg, &req, sizeof(req)) == -1) { + log_warnx("bad IMSG_MBOX_SELECT"); + break; + } + /* handle_mbox_select() composes its own replies + * (IMSG_MBOX_SELECT_VANISHED / IMSG_MBOX_FETCH_META x N + * for a QRESYNC resync, then one IMSG_MBOX_SELECTED) + * and calls imsgev_add(iev) itself -- same streaming + * pattern as handle_mbox_fetch()/handle_mbox_store()/ + * handle_mbox_expunge()/handle_mbox_search() below. */ + handle_mbox_select(&req, iev); + break; + } + case IMSG_MBOX_FETCH: { + struct imsg_mbox_fetch req; + + if (imsg_get_data(&imsg, &req, sizeof(req)) == -1) { + log_warnx("bad IMSG_MBOX_FETCH"); + break; + } + /* handle_mbox_fetch() composes its own replies + * (IMSG_MBOX_FETCH_META x N, then one + * IMSG_MBOX_RESULT) and calls imsgev_add(iev) itself + * once at the end -- unlike IMSG_MBOX_SELECT above, + * which sends exactly one reply and re-arms right + * after composing it, a variable-length stream of + * replies is cleaner re-armed once, after the whole + * batch is queued, than after each individual + * imsg_compose(). */ + handle_mbox_fetch(&req, iev); + break; + } + case IMSG_MBOX_STORE: { + struct imsg_mbox_store req; + + if (imsg_get_data(&imsg, &req, sizeof(req)) == -1) { + log_warnx("bad IMSG_MBOX_STORE"); + break; + } + /* Same reply-composition shape as IMSG_MBOX_FETCH + * above -- handle_mbox_store() composes its own + * IMSG_MBOX_FETCH_META x N / IMSG_MBOX_RESULT stream + * and calls imsgev_add(iev) itself once at the end. */ + handle_mbox_store(&req, iev); + break; + } + case IMSG_MBOX_EXPUNGE: { + struct imsg_mbox_expunge req; + + if (imsg_get_data(&imsg, &req, sizeof(req)) == -1) { + log_warnx("bad IMSG_MBOX_EXPUNGE"); + break; + } + /* Same reply-composition shape as IMSG_MBOX_FETCH/ + * IMSG_MBOX_STORE above -- handle_mbox_expunge() + * composes its own IMSG_MBOX_EXPUNGED x N / + * IMSG_MBOX_RESULT stream and calls imsgev_add(iev) + * itself once at the end. Also the request CLOSE + * (listener.c's cmd_close()) sends, with silent=1. */ + handle_mbox_expunge(&req, iev); + break; + } + case IMSG_MBOX_IDLE_REFRESH: + /* No request payload -- see imapd.h's imsg_mbox_ + * idle_uid/imsg_mbox_idle_refreshed comment. handle_ + * mbox_idle_refresh() composes its own IMSG_MBOX_ + * IDLE_UID x N / IMSG_MBOX_IDLE_REFRESHED stream and + * calls imsgev_add(iev) itself, same shape as every + * other streaming handler above. */ + handle_mbox_idle_refresh(iev); + break; + case IMSG_MBOX_APPEND: { + struct imsg_mbox_append req; + size_t bodylen; + char *body = NULL; + + /* + * imsg_get_data() (used by every other case here) + * requires an *exact* length match and can't be used + * for a header-plus-variable-body imsg -- imsg_get_ + * buf() (sequential, no length check) plus imsg_get_ + * len() (bytes *remaining*, since it's ibuf_size() = + * wpos - rpos) is the verified-against-real-imsg- + * buffer.c pattern; see imapd.h's imsg_mbox_append + * comment for the full citation. + * + * Any failure here (bad header, a msglen mismatch, a + * malloc failure) is treated the same as every other + * "bad IMSG_MBOX_*" case in this switch: logged and + * dropped, no reply sent. That's a pre-existing + * pattern, not a new gap -- every other case's + * imsg_get_data() failure path does the same, on the + * premise that a malformed imsg between listener and + * store (as opposed to a client protocol error, which + * never reaches here) means a build-time struct- + * layout skew between the two binaries, not something + * a client action can trigger. + */ + if (imsg_get_buf(&imsg, &req, sizeof(req)) == -1) { + log_warnx("bad IMSG_MBOX_APPEND (header)"); + break; + } + bodylen = imsg_get_len(&imsg); + if (bodylen != req.msglen) { + log_warnx("session %u: IMSG_MBOX_APPEND " + "length mismatch (header says %u, imsg " + "has %zu)", session_id, req.msglen, + bodylen); + break; + } + if (bodylen > 0) { + if ((body = malloc(bodylen)) == NULL) { + log_warn("session %u: malloc APPEND " + "body", session_id); + break; + } + if (imsg_get_buf(&imsg, body, bodylen) == -1) { + log_warnx("bad IMSG_MBOX_APPEND " + "(body)"); + free(body); + break; + } + } + handle_mbox_append(&req, body, bodylen, iev); + free(body); + break; + } + case IMSG_MBOX_SEARCH: { + struct imsg_mbox_search req; + size_t bodylen; + struct search_node *nodes = NULL; + + /* Same header-plus-variable-body shape and the same + * imsg_get_buf()/imsg_get_len() technique as + * IMSG_MBOX_APPEND just above -- see imapd.h's + * imsg_mbox_search comment. Trailing data here is an + * array of struct search_node, not raw message bytes, + * but the wire mechanics are identical. */ + if (imsg_get_buf(&imsg, &req, sizeof(req)) == -1) { + log_warnx("bad IMSG_MBOX_SEARCH (header)"); + break; + } + bodylen = imsg_get_len(&imsg); + if (bodylen != (size_t)req.nnodes * + sizeof(struct search_node)) { + log_warnx("session %u: IMSG_MBOX_SEARCH length " + "mismatch (header says %u nodes, imsg has " + "%zu bytes)", session_id, req.nnodes, + bodylen); + break; + } + if (bodylen > 0) { + if ((nodes = malloc(bodylen)) == NULL) { + log_warn("session %u: malloc SEARCH " + "nodes", session_id); + break; + } + if (imsg_get_buf(&imsg, nodes, bodylen) == -1) { + log_warnx("bad IMSG_MBOX_SEARCH (nodes)"); + free(nodes); + break; + } + } + handle_mbox_search(&req, nodes, req.nnodes, iev); + free(nodes); + break; + } + case IMSG_MBOX_STATUS: { + struct imsg_mbox_status req; + + if (imsg_get_data(&imsg, &req, sizeof(req)) == -1) { + log_warnx("bad IMSG_MBOX_STATUS"); + break; + } + handle_mbox_status(&req, iev); + break; + } + case IMSG_MBOX_COPY: { + struct imsg_mbox_copy req; + + if (imsg_get_data(&imsg, &req, sizeof(req)) == -1) { + log_warnx("bad IMSG_MBOX_COPY"); + break; + } + handle_mbox_copy(&req, iev); + break; + } + case IMSG_MBOX_MOVE: { + struct imsg_mbox_copy req; + + if (imsg_get_data(&imsg, &req, sizeof(req)) == -1) { + log_warnx("bad IMSG_MBOX_MOVE"); + break; + } + handle_mbox_move(&req, iev); + break; + } + case IMSG_MBOX_EXAMINE: + /* + * Permanently unreachable: EXAMINE reuses IMSG_MBOX_ + * SELECT wholesale (struct imsg_mbox_select's own + * "readonly" field distinguishes them) -- see + * select_or_examine() in listener.c. This enum value + * predates that design decision and was never + * removed. + */ + log_debug("session %u: unimplemented mbox op %d", + session_id, imsg_get_type(&imsg)); + break; + case IMSG_MBOX_LIST: + handle_mbox_list(iev); + break; + case IMSG_MBOX_CREATE: { + struct imsg_mbox_create req; + + if (imsg_get_data(&imsg, &req, sizeof(req)) == -1) { + log_warnx("bad IMSG_MBOX_CREATE"); + break; + } + handle_mbox_create(&req, iev); + break; + } + case IMSG_MBOX_DELETE: { + struct imsg_mbox_delete req; + + if (imsg_get_data(&imsg, &req, sizeof(req)) == -1) { + log_warnx("bad IMSG_MBOX_DELETE"); + break; + } + handle_mbox_delete(&req, iev); + break; + } + case IMSG_MBOX_RENAME: { + struct imsg_mbox_rename req; + + if (imsg_get_data(&imsg, &req, sizeof(req)) == -1) { + log_warnx("bad IMSG_MBOX_RENAME"); + break; + } + handle_mbox_rename(&req, iev); + break; + } + default: + log_debug("store_dispatch: unhandled %d (session %u)", + imsg_get_type(&imsg), session_id); + break; + } + imsg_free(&imsg); + } + /* + * Real bug caught on first real-hardware run, same shape and same + * fix as auth.c's auth_dispatch() (see its header comment for the + * full citation against imsg_init(3)): every case above delegates + * its own re-arm to whichever handle_mbox_*() function it calls + * (or, for a couple of cases, none at all yet -- see the TODO + * cases above), on the assumption this function always gets called + * with something new to process. That assumption breaks once + * EV_WRITE is actually handled (just above): a pure EV_WRITE + * firing with nothing new to read runs no case at all, so nothing + * re-arms this channel -- and since imsgev_init() registers plain + * EV_READ, not EV_PERSIST, that silently drops store's only + * connection to listener for good. Unconditional call here closes + * that gap regardless of which path (or no path) was taken above. + */ + imsgev_add(iev); + (void)fd; +} + +/* + * Per the design doc: IMSG_STORE_SHUTDOWN arrives directly from listener + * over the peer channel, no round-trip through parent -- "parent isn't on + * this path once wiring completes." Flush-then-exit; there's currently + * nothing to flush (no open index/message fds are held across dispatch + * calls in this skeleton), but the function exists as the one place + * that'll need to change once there is. + */ +static __dead void +store_shutdown(void) +{ + log_debug("session %u: store shutting down", session_id); + exit(0); +}