Message interface

[EtienneBruines]

I would like to discuss https://github.com/alienscience/imapsrv/blob/fetch/mailstore.go#L64-L74 :

// An IMAP message
type Message interface {
    // Get the message flags
    Flags() (uint8, error)
    // Get the date of the message as known by the server
    InternalDate() (time.Time, error)
    // Get the size of the message in bytes
    Size() (uint32, error)
    // Get a reader to access the message content
    Reader() (MessageReader, error)
}

The last one, Reader(), reads the entire message (including attachments). I can imagine one storing the headers separately from the actual body. Or in case of a multipart message, the attachments (because they are often much larger than the textual message) at another server (i.e. OpenStack Swift). Then it would be nice to be able to skip those (since most DBMS use transactions, having an io.Seeker for the entire message, isn't always an option - it's either loading it in the memory all at once, or starting another transaction to load something else). The current implementation (branch fetch), allows only for the first scenario.

As I read in https://tools.ietf.org/html/rfc3501#section-6.4.5, there are multiple attributes that can be fetched:

• BODY
• BODY[
  ]<>
• BODY.PEEK[
  ]<>
• BODYSTRUCTURE
• ENVELOPE
• FLAGS
• INTERNALDATE
• RFC822
• RFC822.HEADER
• RFC822.SIZE
• RFC822.TEXT
• UID

Would it be doable if we made the interface use these instead? That way the separate parts of the message could be stored anywhere the user pleases.

type Message interface {
    GetBody() (io.Reader, error)
    GetPartialBody(section uint8, setSeen bool) (io.Reader, error)
    GetBodyStructure() (io.Reader, error)
    GetEnvelope() (io.Reader, error)
    GetFlags() ([]uint8, error)
    GetInternalDate (time.Time, error)
    GetRFC822Body() (io.Reader, error)
    GetRFC822Header() (io.Reader, error)
    GetRFC822Size() (uint64, error)
    GetRFC822Text() (io.Reader, error)
    GetUID() (int32, error)
}

[alienscience]

The problem with this larger interface is that the mailstore must understand the IMAP RFC and be able to parse a MIME message structure.

With the original interface the mailstore only needs to store messages. The IMAP server library will parse and seek through the messages.

One solution would be to have layers. The top layer is the interface you suggest - a mailstore optimised for performance could use this. Underneath this is another layer that uses the original interface and is suitable for things as such maildir format.

[EtienneBruines]

Interesting point you're making (and a valid one). I guess it'd be easiest if we leave this as they are now. Perhaps, some day in the future, we can do some benchmarks and fine-tuning.

(I just merged the fetch branch into the lmtp branch, so I'll be using the same interfaces you defined there. In a week or so, I should be able to have a fully functioning LMTP-endpoint working, which saves the e-mails which then can be parsed by the Mailstore. Perhaps that would be of help when testing the FETCH command).

We could overcome this by by having some kind of ParseMessage which parses the raw data once, resulting in all those different values - but I guess you're right and we should leave it like this.