Kolab 3.0 Storage Format

From Kolab Wiki

Jump to: navigation, search
This page applies to Kolab 3.
This page is a draft only
It is still under construction and content may change. Do not rely on the information on this page.

This page lines out how Kolab Objects are stored on a Kolab IMAP Server (Kolab Version 3.x).

Contents

General

Each Kolab Object is stored as XML object according to #KEP 17 in a Mime-Message which itself is stored in the appropriate folder on the IMAP server.

The following types of Kolab Objects exist:

  • calendar
  • task
  • journal
  • contact
  • note
  • configuration
  • freebusy
  • file

Note that distribution lists are also stored in the contact folder.

Libraries

Two libraries currently exist implementing the Kolab 3.0 Storage Format:

  • libkolabxml: Implements the XML storage format [libkolabxml]
  • libkolab: Implements the MIME storage format [libkolab]

Helper Tools

Both libkolabxml and libkolab contain a validator allowing to validate single xml files (libkolabxml) respectively MIME messages (libkolab). Building of the tools is controlled by the BUILD_UTILS cmake switch and the code is in the utils/ directory.

Kolab Folders

Annotations

  • The difference between ANNOTATEMORE (the draft) and METADATA (the RFC) doesn't matter for the verbiage of this part of the format definition.
  • The shared scope for annotations SHALL be /vendor/kolab/ value.shared in ANNOTATEMORE, /shared/vendor/kolab/ in METADATA
  • The private scope for annotation SHALL be /vendor/kolab/ value.private in ANNOTATEMORE, /private/vendor/kolab/ in METADATA
  • All folders with Kolab Groupware content MUST be annotated with their respective groupware type.
    • The shared scope SHALL be used for groupware types:
      • "event"
      • "journal"
      • "task"
      • "note"
      • "contact"
      • "configuration"
      • "freebusy"
      • "file"
    • The groupware type MUST not be altered at any time.
    • The groupware type SHALL be stored using the "/vendor/kolab/folder-type" metadata location when using METADATA.
    • One groupware folder per groupware type MAY be marked as default location for new kolab-objects by adding a private annotation with the suffix ".default".
  • Folders with mail content MAY exist without metadata.
  • Folders without any metadata MUST be treated as mail folders.
  • Folders with an unrecognized folder-type annotation SHALL NOT be displayed to the user as containing normal mail data.
  • Mail Folders MAY be annotated using the value "mail".
  • Mail Folders with a special purpose SHOULD be annotated in the private scope using the value "mail" with one of the following suffixes:
    • ".inbox"
    • ".drafts"
    • ".sentitems"
    • ".outbox"
    • ".wastebasket"
    • ".junkemail"

There SHOULD always exist only one "default" folder per groupware type.

  • A "default" folder in the private IMAP namespace takes precedence over any other.
  • In case of multiple "default" folders within the private IMAP namespace or none in the private but multiple in the shared IMAP namespace clients SHOULD pretend no folder has been marked as "default".

Structure

  • Kolab Folders MAY be located anywhere in the IMAP account.
    • The MAY be toplevel, in INBOX, in the shared namespace, nested within non-kolab folders.
  • If there are multiple folders of the same type:
    • Each folder SHALL be interpreted as separate group, allowing the user to select where to store a new object (if the client supports that). Examples for interpretations are:
      • calendars
      • tasklists
      • notebook/folder
      • addressbook
    • subfolders SHOULD be interpreted as such (subcalendar, sub-addressbook, ....)
  • TODO: we probably should have a bit a firmer specification how multiple folders SHOULD be interpreted for each type to get a consistent user experience.

Mime Message Format

Each Kolab object MUST be stored in its own MIME Message. The email uses a multipart/mixed structure, with the following parts:

  • A text body part with a fixed text telling about Kolab.
  • The XML describing the object.
  • Other attachments, like a contact’s picture or the attachments associated with e.g. a note or event.

The following mime-headers are used:

  • Subject: MUST be set to the UID of the object (The primary purpose for this is searching).
  • Date: SHALL be updated every time the MIME message is written, with the current timestamp in UTC.
  • From: MAY contain the organizer of the incidence as encoded address (John Doe <doe@kolab.org>).
  • User-Agent: SHALL identify the software used to generate the MIME message
  • Content-Type: MUST be "multipart/mixed"
  • X-Kolab-Type Header: Identifies the Kolab Object Type. This header is REQUIRED. Possible values are:
    • application/x-vnd.kolab.event
    • application/x-vnd.kolab.task
    • application/x-vnd.kolab.journal
    • application/x-vnd.kolab.contact
    • application/x-vnd.kolab.distribution-list
    • application/x-vnd.kolab.note
    • application/x-vnd.kolab.configuration
    • application/x-vnd.kolab.file
  • X-Kolab-Mime-Version: Contains the Kolab MIME Format Version. This header is REQUIRED.
  • X-Kolab-Conflict: Holds the CID of a previous conflicting version, which has been resolved to this current version. This header is OPTIONAL. See Conflict Resolution.

X-Kolab-Mime-Version

This version number specifies the format version of the MIME-Message and not the embedded XML object.

  • The version number MUST be compared using stringcomparison and MUST NOT be converted to a number.
  • The only currently valid version number is "3.0".
  • Future version may introduce a teeny version number in the form of "x.y.z",

Text Body Part

The text body part is there so non-kolab clients which display the message as normal email show a message that this is a Kolab Object. The text should ideally indicate where to get a Kolab Client. The following text is proposed:

This is a Kolab Groupware object.
To view this object you will need a Kolab Groupware Client.
For a list of Kolab Groupware Clients please visit:
http://www.kolab.org/get-kolab
  • The text body part MUST always come first.
  • The following headers/options MAY be used :
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7Bit
Charset/Encoding
This part is purely informal, and a client may change it to whatever is deemed most suitable (different encodings/charsets/text)


XML Object Part

This part contains the Kolab XML Object.

  • This part MUST always come second (directly after the Text Body Part).
  • The following headers/options MUST be used:
Content-Type: MIMETYPE; name="FILENAME"
Content-Transfer-Encoding: ENCODING
Content-Disposition: attachment; filename="FILENAME"
  • MIMETYPE MUST be one of (as specified in KEP #17):
    • application/calendar+xml
    • application/vcard+xml
    • application/vnd.kolab+xml
  • FILENAME SHALL be any name for the kolab xml object. "kolab.xml" is suggested.
  • ENCODING SHALL be "quoted-printable"
    • Other encodings MAY be chosen, if there is a good reason to do so
    • quoted-printable SHALL be used to ensure maximum interoperability (all clients need to be able to deal with the encoding)

Attachment Parts

Large binary objects (e.g. pictures or attachments) SHOULD be stored as separate MIME-Part, and only be referenced from the XML object using its cid (Content-ID). This is to allow clients to fetch those (potentially large) objects on demand only.

  • The attachment-part MUST be referenced and retrieved by it's CID.
    • using the name-header or alike is explicitly not supported.
  • Attachment-parts which are not referenced (from the Kolab XML Object or a header) SHALL be removed.
  • The following headers/options MUST be used:
Content-ID: <CID>
Content-Type: MIMETYPE; name=FILENAME
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename=FILENAME; size=SIZE
  • The FILENAME is optional for all object types except the "file" objecttype, but often used by email clients to display the attachment to the user (This helps to extract parts of a Kolab Object using a normal email client)..
    • This property should typically be the same as the label property stored in the Kolab Format attachment property
    • For attachments which are just kolab object properties, such as a contact logo, a descriptive name such as logo.png SHOULD be chosen.
    • See http://www.imc.org/ietf-smtp/mail-archive/msg05023.html for more info.
  • The CID MUST be a (usually random) string according to the provisions set forth in http://tools.ietf.org/html/rfc1521.
  • The CID SHALL be preserved if an attachment has not changed, this allows clients to not redownload attachments that have not changed.
    • The CID MUST change if the attachment has been changed/modified.
  • The SIZE is optional for all object types except the "file" objecttype. It contains the filesize of the unencoded (extracted) attachment in octets (bytes), and is used to be displayed to the user. (See http://www.ietf.org/rfc/rfc2183.txt for more information on the size parameter)
  • filename and size properties are required for the file objecttype

Example

 Date: Mon, 23 Apr 2012 12:37:59 +0200
 X-Kolab-Type: application/x-vnd.kolab.event
 X-Kolab-Mime-Version: 3.0
 User-Agent: Libkolab-0.1.0
 Content-Type: multipart/mixed; boundary="nextPart1929983.SbWkbbbi0G"
 Subject: KOrganizer-1687167952.818
 MIME-Version: 1.0
 
 
 --nextPart1929983.SbWkbbbi0G
 Content-Type: text/plain; charset="us-ascii"
 Content-Transfer-Encoding: 7Bit
 
 This is a Kolab Groupware object.
 To view this object you will need an email client that can understand the Kolab Groupware format.
 For a list of such email clients please visit
 http://www.kolab.org/get-kolab
 
 --nextPart1929983.SbWkbbbi0G
 Content-Type: application/calendar+xml; name="kolab.xml"
 Content-Transfer-Encoding: quoted-printable
 Content-Disposition: attachment; filename="kolab.xml"
 
 <?xml version=3D"1.0" encoding=3D"UTF-8" standalone=3D"no" ?>
 <icalendar xmlns=3D"urn:ietf:params:xml:ns:icalendar-2.0">
 
   <vcalendar>
     <properties>
       <prodid>
         <text>Libkolab-0.1.0 Libkolabxml-0.3.0</text>
       </prodid>
       <version>
         <text>2.0</text>
       </version>
       <x-kolab-version>
         <text>3.0dev1</text>
       </x-kolab-version>
     </properties>
     <components>
       <vevent>
         <properties>
           <uid>
             <text>KOrganizer-1687167952.818</text>
           </uid>
           <created>
             <date-time>2009-09-01T12:52:58Z</date-time>
           </created>
           <dtstamp>
             <date-time>2012-05-05T05:05:05Z</date-time>
           </dtstamp>
           <sequence>
             <integer>0</integer>
           </sequence>
           <class>
             <text>PRIVATE</text>
           </class>
           <categories>
             <text>Appointment</text>
             <text>Business</text>
           </categories>
           <dtstart>
             <parameters>
               <tzid>
                 <text>/kolab.org/Europe/Berlin</text>
               </tzid>
             </parameters>
             <date-time>2009-09-02T10:00:00</date-time>
           </dtstart>
           <dtend>
             <parameters>
               <tzid>
                 <text>/kolab.org/Europe/Berlin</text>
               </tzid>
             </parameters>
             <date-time>2009-09-02T11:00:00</date-time>
           </dtend>
           <transp>
             <text>TRANSPARENT</text>
           </transp>
           <rrule>
             <recur>
               <freq>WEEKLY</freq>
               <count>10</count>
               <byday>WE</byday>
               <byday>FR</byday>
             </recur>
           </rrule>
           <exdate>
             <date>2009-09-04</date>
           </exdate>
           <summary>
             <text>Complex Event</text>
           </summary>
           <description>
             <text>Some notes on this event.</text>
           </description>
           <location>
             <text>Here</text>
           </location>
           <attendee>
             <parameters>
               <cn>
                 <text>Attendee1</text>
               </cn>
               <partstat>
                 <text>NEEDS-ACTION</text>
               </partstat>
               <role>
                 <text>REQ-PARTICIPANT</text>
               </role>
               <rsvp>
                 <boolean>true</boolean>
               </rsvp>
             </parameters>
             <cal-address>mailto:%3Ca1%40example%2Ecom%3E</cal-address>
           </attendee>
           <attendee>
             <parameters>
               <cn>
                 <text>Attendee2</text>
               </cn>
               <partstat>
                 <text>ACCEPTED</text>
               </partstat>
               <role>
                 <text>NON-PARTICIPANT</text>
               </role>
               <rsvp>
                 <boolean>true</boolean>
               </rsvp>
             </parameters>
             <cal-address>mailto:%3Ca2%40example%2Ecom%3E</cal-address>
           </attendee>
           <attendee>
             <parameters>
               <cn>
                 <text>Attendee3</text>
               </cn>
               <partstat>
                 <text>DECLINED</text>
               </partstat>
               <role>
                 <text>REQ-PARTICIPANT</text>
               </role>
             </parameters>
             <cal-address>mailto:%3Ca3%40example%2Ecom%3E</cal-address>
           </attendee>
           <attach>
             <parameters>
               <fmttype>
                 <text>image/png</text>
               </fmttype>
               <x-label>
                 <text>akonadi.png</text>
               </x-label>
             </parameters>
             <uri>cid:7313173.zaagFSsPPv@kolab.resource.akonadi</uri>
           </attach>
         </properties>
         <components>
           <valarm>
             <properties>
               <action>
                 <text>DISPLAY</text>
               </action>
               <description>
                 <text/>
               </description>
               <trigger>
                 <parameters>
                   <related>
                     <text>START</text>
                   </related>
                 </parameters>
                 <duration>-PT900S</duration>
               </trigger>
               <duration>
                 <duration>PT5S</duration>
               </duration>
               <repeat>
                 <integer>0</integer>
               </repeat>
             </properties>
           </valarm>
         </components>
       </vevent>
     </components>
   </vcalendar>
 
 </icalendar>
 
 --nextPart1929983.SbWkbbbi0G
 Content-ID: <7313173.zaagFSsPPv@kolab.resource.akonadi>
 Content-Type: image/png; name="akonadi.png"
 Content-Transfer-Encoding: base64
 Content-Disposition: attachment; filename="akonadi.png"
 
 iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAAAXNSR0IArs4c6QAAAAZiS0dEAP8A
 /wD/oL2nkwAAAAlwSFlzAAAbrwAAG68BXhqRHAAAAAd0SU1FB9gFEQkdFPibCIYAAAMrSURBVDjL
 pZNLaBx1AMZ/M/P/78zOzO7ObDbvNUnTxsUUm6YmaulNxN70UKEIRTyIBy8KHrypRUTw4Ek8CqKn
 XuyhHgQRFTRNAvVFbOnDNO/EfWZmdmczszPrwcdBvPnBd/ng98F3+OB/Svl3cPbSx5mZ+ZkXjKx+
 setHs4cHbaW+7d2s79SvhK29j+6vvXH0nwVPfLh1WsTptdnxuDDxoGv3NZWuf4S3G1C722BrrcrO
 nY1f/ebO05s337n3N6cBLLy1bI7njr4uD4nJhx4by5SGLQYcAzWnI6SGVFUkQKwMRqH/jGHNfXJY
 XwoBBMDc/MhLThpOJ6ZEMzM4moquQpIVxI5Br5QlHrEImw5xuzRJ0n4beBlABRgwxcXhUp5Ww6OX
 QvCXo/TPjUJq6FmJldPJOTZ2znj++HOfnwQQ0y9+e67X6iykpmTCNTnY9xBjeTQVelFC0u3RjxNI
 +6iaQiajYtmGlXa6NxZe/f4pobv24/X9thBSYebUEHc26zT6kLMkaZTQ84+IvYioE9PrRvTTCBSF
 2PczUTxyVuiOsXHju58Y0XTaQcTJM0OsXt9kaqqIrinEQUTYCPGrAR2vRbcbELS7ZGWEj72hzL+5
 MqS0uzvBys/CzbsUxx0KJROvnTBZLtCPU/y6j1erE/g1wrBNdqREWKwktdB4QP3h8qO/K6Z2RR8b
 JmjtUx7KMDtd4MSozuatXX5Zus3tH29Rre7S18GtjFNanAMpr37w2ti+ABAyeT0eLJ7vtWoDX1z9
 BiXV0DRJmqakSRfd1iiMOmSPjeKeqVC/H3oPT8r3ZNyzVID3L1V2nDzPqsPlQ2PiGKoJCQFJ4qPI
 FH0gh3P6ONlHKlQ3ve6EEb57fl7GSh9DAVjaqmrNRsv97Kv1xeurjcthkCyqURs7JzBdE6NokQgJ
 SbQ+O65+euqEszo4UPytWMhvCwBUkSqq1jk356xNDUevLK/sPbl+L7jQ8ZLyUaepGE1xMDZhfDlb
 KSwPFt1tXcp9qWk1VWjhP2e6dneLvhdqLc/L7B3UzfX1qttsdfKKguo4ZlguO36xWAgLth3mbSsy
 TatnyFz6B+BnWV0A/UiAAAAAAElFTkSuQmCC
 
 --nextPart1929983.SbWkbbbi0G--
Personal tools