cyrus.index format history.

We only track Cyrus version 2 index formats here, so starting with version
3 as shipped with 2.0.0.

NOTE: offsets in the code were sometimes defined in terms of the previous
field rather than exact offsets, which is bogus since the format on disk
will always be exact offsets, so all offsets here are specified exactly.

Version 3: 2.0.0    2000-04-25     H:56  R:52
=============================================

/* Offsets of index header fields */
#define OFFSET_GENERATION_NO 0
#define OFFSET_FORMAT 4
#define OFFSET_MINOR_VERSION 8
#define OFFSET_START_OFFSET 12
#define OFFSET_RECORD_SIZE 16
#define OFFSET_EXISTS 20
#define OFFSET_LAST_APPENDDATE 24
#define OFFSET_LAST_UID 28
#define OFFSET_QUOTA_MAILBOX_USED 32
#define OFFSET_POP3_LAST_LOGIN 36
#define OFFSET_UIDVALIDITY 40
#define OFFSET_DELETED 44>------/* added for ACAP */
#define OFFSET_ANSWERED 48
#define OFFSET_FLAGGED 52

/* Offsets of index_record fields in index file */
#define OFFSET_UID 0
#define OFFSET_INTERNALDATE 4
#define OFFSET_SENTDATE 8
#define OFFSET_SIZE 12
#define OFFSET_HEADER_SIZE 16
#define OFFSET_CONTENT_OFFSET 20
#define OFFSET_CACHE_OFFSET 24
#define OFFSET_LAST_UPDATED 28
#define OFFSET_SYSTEM_FLAGS 32
#define OFFSET_USER_FLAGS 36


Version 4: 2.1.4    2002-04-30     H:76  R:52
=============================================

The UIDL format was changed to include the mailbox UIDVALIDITY, but this
breaks existing caching when upgrading, so a header field was added to
track when an existing mailbox ever got empty, so that the UIDL format
could be changed from then.

Also defined some spares so new fields could be added to the header
without a format change.

#define OFFSET_POP3_NEW_UIDL 56>/* added for Outlook stupidity */
#define OFFSET_SPARE0 60
#define OFFSET_SPARE1 64
#define OFFSET_SPARE2 68
#define OFFSET_SPARE3 72


Version 5: 2.2.1    2003-07-16     H:76  R:52
=============================================

To save repacking the cache file every single EXPUNGE, a new header field
was added to track when the cache had enough dirty data to justify a repack.
Strictly this didn't need a version bump.

#define OFFSET_LEAKED_CACHE 60 /* Number of leaked records in cache file */

Version 6: 2.2.2    2003-10-28     H:76  R:60
=============================================
#define OFFSET_POP3_NEW_UIDL 56>/* added for Outlook stupidity */


Version 7: 2.3.0    2005-12-05     H:76  R:72
=============================================

Added replication support, and the UUID field to allow efficient replication
of copies.  The UUID is 12 bytes long.

#define OFFSET_MESSAGE_UUID 60


Version 8: 2.3.4    2006-05-23     H:92  R:80
=============================================

Added support for CONDSTORE (RFC4551) in the header, including renaming
the POP3_NEW_UIDL to a more general bitmap of options (to which
CONDSTORE_ENABLED was added)

#define OFFSET_MAILBOX_OPTIONS 60

#define OFFSET_HIGHESTMODSEQ_64 68 /* CONDSTORE (64-bit modseq) */
#define OFFSET_HIGHESTMODSEQ 72    /* CONDSTORE (32-bit modseq) */
#define OFFSET_SPARE0 76 /* Spares - only use these if the index */
#define OFFSET_SPARE1 80 /*  record size remains the same */
#define OFFSET_SPARE2 84 /*  (see note above about spares) */
#define OFFSET_SPARE3 88

And the record:

#define OFFSET_MODSEQ_64 72 /* CONDSTORE (64-bit modseq) */
#define OFFSET_MODSEQ 76 /* CONDSTORE (32-bit modseq) */


Version 9: 2.3.7    2006-07-10     H:96  R:80
=============================================

Due to the header not being divisible by 8 bytes, the 64 bit reads for
modseq values were inefficient - so this change just added extra padding
for alignment.

#define OFFSET_SPARE4 92


Version 10: 2.3.10  2007-10-24     H:96  R:88
=============================================

Switch UUID to GUID, which is a 20 byte sha1 instead of a 12 byte
calculated value.

#define OFFSET_MESSAGE_GUID 60
#define OFFSET_MODSEQ_64 80 /* CONDSTORE (64-bit modseq) */
#define OFFSET_MODSEQ 84 /* CONDSTORE (32-bit modseq) */


Version 11 never existed, it was used internally at Fastmail but never
released in a public Cyrus release.

Version 12: 2.4.0   2010-11-10     H:128 R:96
=============================================

Major rewrite.  Renamed exists to NUM_RECORDS and added a separate
new record for EXISTS (number of unexpunged records)

#define OFFSET_NUM_RECORDS 20

Also DELETEDMODSEQ for QRESYNC support, CRCs throughout, fields for
supporting per-user SEEN storage in the cyrus.index file (RECENTUID
and RECENTTIME) and fields to support expunge cleanup.

#define OFFSET_DELETEDMODSEQ_64 76 /* CONDSTORE (64-bit modseq) */
#define OFFSET_DELETEDMODSEQ 80    /* CONDSTORE (32-bit modseq) */
#define OFFSET_EXISTS 84           /* Non-expunged records */
#define OFFSET_FIRST_EXPUNGED 88   /* last_updated of oldest expunged message */
#define OFFSET_LAST_REPACK_TIME 92 /* time of last expunged cleanup  */
#define OFFSET_HEADER_FILE_CRC 96  /* CRC32 of the index header file */
#define OFFSET_SYNC_CRC 100        /* XOR of SYNC CRCs of unexpunged records */
#define OFFSET_RECENTUID 104       /* last UID the owner was told about */
#define OFFSET_RECENTTIME 108      /* last timestamp for seen data */
#define OFFSET_SPARE0 112 /* Spares - only use these if the index */
#define OFFSET_SPARE1 116 /*  record size remains the same */
#define OFFSET_SPARE2 120 /*  (see note above about spares) */
#define OFFSET_HEADER_CRC 124 /* includes all zero for the spares! */

And the record had some changes too.  Header size was always identical to
content offset, so replace that with a GMTIME field for efficient sort by
REVERSE ARRIVAL.

#define OFFSET_GMTIME 20

Also, two new CRC fields for integrity checks on both the cache record and
the index record itself

#define OFFSET_CACHE_CRC 88 /* CRC32 of cache record */
#define OFFSET_RECORD_CRC 92