Skip navigation
Syndie - distributed forums

# Specs

[Syndie messages] [Syndie key files] [Syndie URIs] [Syndie message headers]

## Syndie messages (up)

A `.syndie` file contains signed and potentially encrypted data for passing Syndie channel metadata and posts around. It is made up of two parts- a UTF-8 encoded header and a body. The header begins with a type line, followed by name=value pairs, delimited by the newline character ('\n' or 0x0A). After the pairs are complete, a blank newline is included, followed by the line "Size=$numBytes\n", where $numBytes is the size of the body (base10). After that comes that many bytes making up the body of the enclosed message, followed by two newline delimited signature lines - AuthorizationSig=$signature and AuthenticationSig=$signature. There can be any arbitrary amount of data after the signature lines, but it is not currently interpreted.

The $numBytes body is an encrypted zip archive, though the encryption method depends upon the type line. For posts and metadata messages, the data is AES/256/CBC encrypted (with a 16 byte IV at the beginning). For private messages, the first 512 bytes are ElGamal/2048 encrypted to the channel's encryption key, which has the AES/256 session key and IV within it, and the remaining bytes are AES/256/CBC encrypted.

The AES/256 encrypted area begins with a random number of nonzero padding bytes, followed by 0x0, then the internal payload size (as a 4 byte unsigned integer), followed by the total payload size (the same as the Size header), followed by the actual Zip encoded data, a random number of pad bytes, up to a 16 byte boundary, aka:

        rand(nonzero) padding + 0 + internalSize + totalSize + data + rand

After the AES/256 encrypted area there is an HMAC-SHA256 of the body section, using the SHA256 of the body decryption key concatenated with the IV as the HMAC key.

The authorization signature is verified against the set of public keys associated with the channel. Not all messages must have valid authorization signatures, but unauthorized messages may not be passed along.

The authentication signature may be verified against the Author header (either in the public or encrypted header sets), but not all messages are authenticated.

The unencrypted zip archive may contain the following entries:

- `headers.dat` [used in: posts, private messages, metadata posts]

  Optionally contains headers that are not visible to those who cannot decrypt the message

- `page$n.dat` [used in: posts, private messages]

  Page $n's contents

- `page$n.cfg` [used in: posts, private messages]

  Headers for page $n: Content-type, title, references, etc

- `attach$n.dat` [used in: posts, private messages]

  Attachment $n's contents

- `attach$n.cfg` [used in: posts, private messages]

  Headers for attachment $n: Content-type, language, etc

- `avatar32.png` [used in: posts, private messages, metadata posts]

  Contains a 32x32 pixel avatar for the message or channel

- `references.cfg` [used in: posts, private messages, metadata posts]

  Contains a tree of syndie references, formatted as "[\t]*$name\t$uri\t$refType\t$description\n", where the tab indentation at the beginning of the line determines the tree structure. The refType field can, for instance, be used to differentiate mentions of a positive reference and those recommending blacklisting, etc.

## Syndie key files (up)

When passing around keys for Syndie channels, they can either be transferred in Syndie URIs or in key files. The key files themselves are UTF encoded as follows:

    keytype: [manage|manage-pub|reply|reply-pub|post|post-pub|read]\n
    scope: $base64(channelHash)\n
    raw: $base64(bytes)\n

## Syndie URIs (up)

This defines the URIs safely passable within syndie, capable of referencing specific resources. They contain one of four reference types, plus a bencoded set of attributes:

    Type: url
    Attributes:
    * net:  what network the URL is on, such as "i2p", "tor", "ip", or "freenet" (string)
    * url:  network-specific URL (string)
    * name: [optional] short name of the resource referenced (string)
    * desc: [optional] longer description of the resource (string)
    * tag:  [optional] list of tags attached to the reference (string[])

    Type: channel
    Attributes:
    * channel:     [1] base64 of the SHA256 of the channel's identity public key (string)
    * author:      [1] base64 of the SHA256 of the author's identity public key, if different from the channel (string)
    * msgId:       [1] unique identifier within the channel's scope (or author's scope, if specified) (integer)
    * page:        [optional] page within the message's scope (integer starting at 1)
    * attachment:  [optional] attachment within the message's scope (integer starting at 1)
    * readKeyType: [optional] describes the readKey, e.g. "AES256" (string)
    * readKeyData: [optional] base64 of the key required to read posts in the channel [string)
    * postKeyType: [optional] describes the postKey, e.g. "DSA1024" (string)
    * postKeyData: [optional] base64 of the private key required to post to the channel (string)
    * name:        [optional] short name of the resource referenced (string)
    * desc:        [optional] longer description of the resource (string)
    * tag:         [optional] list of tags attached to the reference (string[])

    [1] If the field is not specified, it must be implicitly derived from the context.
        For instance, a syndie post may omit the channel and msgId when referring to another
        page or attachment on the current message.

    Type: search
    Attributes:
    * scope:        base64 of the SHA256 of the channel's identity public key, or "all" (string[])
    * author:       "authorized", "manager", "owner", "any" (string)
    * postbyscope:  base64 of the SHA256 of the posting author's identity public key (string[])
    * age:          number of days in the past to look back for the post's creation date (integer)
    * agelocal:     like age, but measures when the local archive received the message (integer)
    * unreadonly:   if true, only include unread messages (boolean)
    * taginclude:   matches must include at least one of these tags (string[])
    * tagrequire:   matches must include all of these tags (string[])
    * tagexclude:   matches must not include any of these tags (string[])
    * tagmessages:  the tag filter is applied against individual messages, not threads as a whole (boolean)
    * pagemin:      minimum number of pages in the post (integer)
    * pagemax:      maximum number of pages in the post (integer)
    * attachmin:    minimum number of attachments in the post (integer)
    * attachmax:    maximum number of attachments in the post (integer)
    * refmin:       minimum number of references in the post (integer)
    * refmax:       maximum number of references in the post (integer)
    * keymin:       minimum number of keys in the post (integer)
    * keymax:       maximum number of keys in the post (integer)
    * encrypted:    the message is still encrypted and not readable (boolean)
    * pbe:          the message was encrypted with a passphrase (boolean)
    * private:      the message was encrypted with a channel reply key (boolean)
    * public:       the message was readable by anyone (boolean)
    * authorized:   the message was readable by authorized readers (boolean)
    * threaded:     matches should take threading into consideration (boolean)
    * keyword:      matches messages with the keyword/phrase in the body or subject (string)

    All of the search terms are optional

    Type: archive
    Attributes:
    * net:          what network the URL is on, such as "i2p", "tor", "ip", or "freenet" (string)
    * url:          network-specific URL (string)
    * readKeyType:  [optional] describes the readKey, e.g. "AES256" (string)
    * readKeyData:  [optional] base64 of the key required to pull data from the archive (string)
    * postKeyType:  [optional] describes the postKey, e.g. "AES256" (string)
    * postKeyData:  [optional] base64 of the key required to pull data from the archive (string)
    * identKeyType: [optional] describes the identKey, e.g. "DSA1024" (string)
    * identKeyData: [optional] base64 of the key the archive will identify themselves as (string)
    * name:         [optional] short name of the resource referenced (string)
    * desc:         [optional] longer description of the resource (string)
    * tag:          [optional] list of tags attached to the reference (string[])

    Type: text
    Attributes:
    * name: [optional] short name of the freeform text reference (string)
    * body: [optional] freeform text reference (string)
    * tag:  [optional] list of tags attached to the reference (string[])

    The canonical encoding is: "urn:syndie:$refType:$bencodedAttributes",
    with $refType being one of the five types above, and $bencodedAttributes
    being the bencoded attributes.  Strings are UTF-8, and the bencoded attributes
    are ordered according to the UK locale (in the canonical form).  Keys have
    leading 0x0 bytes stripped prior to Base64 encoding, which must be expanded
    upon decoding.

    Examples:
     urn:syndie:url:d3:url19:http://www.i2p.net/e
     urn:syndie:channel:d7:channel40:12345678901234567890123456789012345678909:messageIdi42e4pagei0ee
     urn:syndie:channel:d10:attachmenti3ee
     urn:syndie:channel:d4:pagei2ee
     urn:syndie:search:d3:tag3i2pe
     urn:syndie:search:d6:status7:watchede

    Within syndie-enabled apps, the urn:syndie: prefix can be dropped:
     url:d3:url19:http://www.i2p.net/e
     channel:d7:channel40:12345678901234567890123456789012345678909:messageIdi42e4pagei0ee
     channel:d10:attachmenti3ee
     channel:d4:pagei2ee
     search:d3:tag3i2pe
     search:d6:status7:watchede

## Syndie message headers (up)

Syndie messages have a defined set of headers, and unknown headers are uninterpreted.

Author AuthenticationMask TargetChannel PostURI References Tags OverwriteURI ForceNewThread RefuseReplies Cancel Subject BodyKey BodyKeyPromptSalt BodyKeyPrompt Identity EncryptKey Name Description Edition PublicPosting PublicReplies AuthorizedKeys ManagerKeys Archives ChannelReadKeys Expiration

In the following list, Required means the header must be included for messages of the allowed types. Allow as hidden means the header may be included in the encrypted `headers.dat` zip headers, rather than in the unencrypted publicly visible headers. Allow on posts means the header can be used on normal posts. Allow on private messages means the header can be used on posts encrypted to a channel's private key. Allow on metadata messages means the header can be used on metadata messages configuring a channel.

- Author
  - Required? no
  - Allow as hidden? yes
  - Allow on posts? yes
  - Allow on private messages? yes
  - Allow on metadata messages? no
  - Content type: base64(channel identity key)
  - Description: Nym that posted the message (must be authenticated with an authenticationSig)
- AuthenticationMask
  - Required? no
  - Allow as hidden? yes
  - Allow on posts? yes
  - Allow on private messages? yes
  - Allow on metadata messages? no
  - Content type: base64(random 40 byte value)
  - Description: If included, the authenticationSignature is XORed against this mask before checking for validity (preventing unauthorized people from seeing if the post was authenticated by $nym by simply checking the signature against $nym's pubkey)
- TargetChannel
  - Required? no
  - Allow as hidden? no
  - Allow on posts? yes
  - Allow on private messages? no
  - Allow on metadata messages? no
  - Content type: base64(channel ident hash)
  - Description: The channel in the PostURI may refer to the author (eg when unauthorized), so this supplements that with the channel that the post is destined for
- PostURI
  - Required? yes
  - Allow as hidden? no
  - Allow on posts? yes
  - Allow on private messages? yes
  - Allow on metadata messages? no
  - Content type: Syndie URI
  - Description: Unique message identifier, including the channel the post is authorized for and the messageId. If the post is not authorized for the target channel, the scope referenced is the author's
- References
  - Required? no
  - Allow as hidden? yes
  - Allow on posts? yes
  - Allow on private messages? yes
  - Allow on metadata messages? no
  - Content type: comma delimited list of Syndie URIs
  - Description: Earliest in the list is most recent ancestor
- Tags
  - Required? no
  - Allow as hidden? yes
  - Allow on posts? yes
  - Allow on private messages? yes
  - Allow on metadata messages? yes
  - Content type: tab or comma delimited list of tags
  - Description: Key phrases describing the message or channel
- OverwriteURI
  - Required? no
  - Allow as hidden? yes
  - Allow on posts? yes
  - Allow on private messages? no
  - Allow on metadata messages? no
  - Content type: Syndie URI
  - Description: Entry that the current post is replacing (if authorized)
- ForceNewThread
  - Required? no
  - Allow as hidden? yes
  - Allow on posts? yes
  - Allow on private messages? yes
  - Allow on metadata messages? no
  - Content type: "true" or "false"
  - Description: If true, this message starts a new thread, even if there are parents
- RefuseReplies
  - Required? no
  - Allow as hidden? yes
  - Allow on posts? yes
  - Allow on private messages? yes
  - Allow on metadata messages? no
  - Content type: "true" or "false"
  - Description: If true, only the author may post a threaded reply to this post
- Cancel
  - Required? no
  - Allow as hidden? yes
  - Allow on posts? yes
  - Allow on private messages? no
  - Allow on metadata messages? yes
  - Content type: tab delimited list of Syndie URIs
  - Description: If the author is allowed, cancel the given message(s)
- Subject
  - Required? no
  - Allow as hidden? yes
  - Allow on posts? yes
  - Allow on private messages? yes
  - Allow on metadata messages? no
  - Content type: text
  - Description: Short description of what the post is about
- BodyKey
  - Required? no
  - Allow as hidden? no
  - Allow on posts? yes
  - Allow on private messages? no
  - Allow on metadata messages? yes
  - Content type: base64(AES256 session key)
  - Description: If specified, the zip entries are encrypted with the given key (and hence, readable by anyone) rather than the channel's normal (unpublicized) session key.
- BodyKeyPromptSalt
  - Required? no
  - Allow as hidden? no
  - Allow on posts? yes
  - Allow on private messages? yes
  - Allow on metadata messages? yes
  - Content type: base64(channel identity key)
  - Description: Fed into the PBE algorithm with the answer to the BodyKeyPrompt
- BodyKeyPrompt
  - Required? no
  - Allow as hidden? no
  - Allow on posts? yes
  - Allow on private messages? yes
  - Allow on metadata messages? yes
  - Content type: text
  - Description: If specified, the body is encrypted with a PBE key generated by answering this query correctly. e.g. "BodyKeyPrompt=1+1=?" would PBE generate the body key from the correct answer ("2")
- Identity
  - Required? yes
  - Allow as hidden? no
  - Allow on posts? no
  - Allow on private messages? no
  - Allow on metadata messages? yes
  - Content type: base64(DSA1024 public key)
  - Description: Master signing public key
- EncryptKey
  - Required? yes
  - Allow as hidden? no
  - Allow on posts? no
  - Allow on private messages? no
  - Allow on metadata messages? yes
  - Content type: base64(Elgamal2048 public key)
  - Description: For private replies
- Name
  - Required? no
  - Allow as hidden? no
  - Allow on posts? no
  - Allow on private messages? no
  - Allow on metadata messages? yes
  - Content type: text
  - Description: Suggested nickname
- Description
  - Required? no
  - Allow as hidden? no
  - Allow on posts? no
  - Allow on private messages? no
  - Allow on metadata messages? yes
  - Content type: text
  - Description: Suggested description
- Edition
  - Required? yes
  - Allow as hidden? no
  - Allow on posts? no
  - Allow on private messages? no
  - Allow on metadata messages? yes
  - Content type: base10 nonnegative integer
  - Description: Higher numbers replace lower numbers
- PublicPosting
  - Required? no
  - Allow as hidden? no
  - Allow on posts? no
  - Allow on private messages? no
  - Allow on metadata messages? yes
  - Content type: "true" or "false"
  - Description: If true, authenticated posts do not need to be authorized (though authorized posts can be unauthenticated)
- PublicReplies
  - Required? no
  - Allow as hidden? no
  - Allow on posts? no
  - Allow on private messages? no
  - Allow on metadata messages? yes
  - Content type: "true" or "false"
  - Description: Like AllowPublicPosting, except replies only (unauthorized users can't create new threads)
- AuthorizedKeys
  - Required? no
  - Allow as hidden? no
  - Allow on posts? no
  - Allow on private messages? no
  - Allow on metadata messages? yes
  - Content type: tab delimited base64(DSA1024 public key)
  - Description: Posts with authorizationSignatures verified by one of these keys are allowed (the pubkeys may be a nym's identity key or may be a separate signing key shared by multiple authors)
- ManagerKeys
  - Required? no
  - Allow as hidden? no
  - Allow on posts? no
  - Allow on private messages? no
  - Allow on metadata messages? yes
  - Content type: tab delimited base64(DSA1024 public key)
  - Description: New metadata messages for this channel may be signed by one of these keys (the channel's identity key is implicitly in this set)
- Archives
  - Required? no
  - Allow as hidden? yes
  - Allow on posts? no
  - Allow on private messages? no
  - Allow on metadata messages? yes
  - Content type: tab delimited Syndie URIs
  - Description: Lists some archives that may be consulted for new posts in the current channel (and/or which may accept replies for the channel)
- ChannelReadKeys
  - Required? no
  - Allow as hidden? yes
  - Allow on posts? no
  - Allow on private messages? no
  - Allow on metadata messages? yes
  - Content type: tab delimited base64(AES256 key)
  - Description: Adds some channel read keys that may be used to encrypt/decrypt posts in the channel (publicizing these makes the channel publicly readable)
- Expiration
  - Required? no
  - Allow as hidden? yes
  - Allow on posts? yes
  - Allow on private messages? yes
  - Allow on metadata messages? yes
  - Content type: YYYYMMDD
  - Description: Suggest a date after which the channel / message can be discarded safely

When referring to `base64`, the content is base64 encoded with an alternate alphabet. The alphabet is the standard one except with "~" replacing "/", and "+" with "-" (for safer URL and file name encoding).