index.html

---
layout: default
title: "TJSON: Tagged JSON with rich type annotations"
---

<div class="container">
  <div class="row">
    <div class="col-lg-12">
      <div class="lead text-xs-center">
        <p>
          <strong>DRAFT:</strong> This format is still in a draft state and subject to change!
        </p>
      </div>
      <div id="introduction">
        <p>
          <strong>TJSON</strong> (Tagged JSON) is a tagging scheme/microformat for enriching
          the types that can be stored in <a href="http://www.json.org">JSON</a> documents.
          It augments the existing types present in JSON, codifiying ad hoc practices already
          commonly used for processing JSON into a schema-free, self-describing format.
        </p>
        <p>
          TJSON documents are amenable to "content-aware hashing" where different encodings of the
          same data (including both TJSON and binary formats like Protocol Buffers, MessagePack,
          BSON, etc) can share the same content hash and therefore the same cryptographic signature.
          This is possible with content hash algorithms that are aware of the underlying structure
          of data, such as
          <a href="https://github.com/benlaurie/objecthash">Ben Laurie's objecthash</a>.
        </p>
        <p>
          TJSON supports the following data types:
        </p>
        <ul>
          <li>
            <strong>Objects:</strong>
            Name/value dictionaries. The names of objects in TJSON carry a postfix "tag" which acts
            as a type annotation for the associated value. See the descriptions of "Strings" below
            for more information.
          </li>
          <li>
            <strong>Arrays:</strong>
            Lists of values: identical to JSON, but typed by their containing objects. Unlike
            JSON, arrays cannot be used as a top-level expression: only objects are allowed.
          </li>
          <li>
            <strong>Sets:</strong>
            Lists of unique values: similar to an array, but repeated elements are disallowed.
          </li>
          <li>
            <strong>Strings:</strong>
            TJSON strings are Unicode and always serialized as UTF-8. When used as the name of a
            member of an object, they carry a mandatory "tag" which functions as a self-describing
            type annotation which provides a type signature for the associated value.
          </li>
          <li>
            <strong>Binary Data:</strong>
            First-class support for 8-bit clean binary data, encoded in a variety of formats
            including hexadecimal (a.k.a. base16), base32, and base64url.
          </li>
          <li>
            <strong>Numbers:</strong>
          </li>
          <ul>
            <li>
              <strong>Integers:</strong>
              TJSON supports the full ranges of both signed and unsigned 64-bit integers
              by serializing them as strings.
            </li>
            <li>
              <strong>Floating points:</strong>
              Floating point numbers in TJSON are identical to JSON, but can always be disambiguated
              from integers.
            </li>
          </ul>
          <li>
            <strong>Timestamps:</strong>
            TJSON has a first-class type for representing date/time timestamp values,
            serialized as a subset of RFC 3339 (an ISO 8601-alike).
          </li>

          <li>
            <strong>Boolean Values:</strong>
            TJSON supports the <em>true</em> and <em>false</em> values from JSON
            (<em>null</em> is expressly disallowed).
          </li>
        </ul>
      </div>

      <div class="subsection">
        <h2 class="page-header">Objects</h2>
        <p>
          Objects are the <sem>only</em> type allowed at the top-level of a TJSON document.
          Many ordinary JSON parsers accept arrays or other types as top-level expressions. This
          is <em>NOT</em> the case in TJSON: objects-only at the top-level.
        </p>
        <p>
          Objects in TJSON use the same syntax as JSON, but each member name contains a "tag"
          which annotates the type of the associated value of the member.
        </p>

        <p>
          Below is an example of an object whose value is a Unicode String:
        </p>

        <div class="syntax-example">
          {&quot;hello-world<span class="tag-prefix">:s</span>&quot;: "Hello, world!"}
        </div>

        <p>
          This example consists of an object whose only member is named <em>"hello-world"</em>
          and whose corresponding value is the <em>string (:s)</em> encoded in UTF-8 whose
          contents are <em>"Hello, world!"</em>
        </p>

        <p>
          Member names in TJSON must be distinct. The use of the same member name more
          than once in the same object is an error, regardless of if the same name is used
          for the same value, same types, or multiple different types. TJSON names are
          single-use only.
        </p>

        <p>
          TJSON uses the case of the first letter of the name of a type to distinguish
          between scalar (single value) and non-scalar (collection) types. The syntax for
          identifying a nested TJSON object is a capital "O" letter: (NOT zero)
        </p>

        <div class="syntax-example">
          {&quot;hello-object<span class="tag-prefix">:O</span>&quot;: {&quot;hello-string<span class="tag-prefix">:s</span>&quot;: "Hello, world!"}}
        </div>
      </div>

      <div class="subsection">
        <h2 class="page-header">Arrays</h2>
        <p>
          Arrays are not allowed as a toplevel expression in TJSON. The following is <em>NOT</em>
          a valid TJSON document, because toplevel arrays are NOT allowed in TJSON:
        </p>

        <div class="syntax-example syntax-invalid">
          [&quot;No toplevel arrays in TJSON!&quot;]
        </div>

        <p>
          Arrays <i>MUST</i> first be wrapped in an object, from which they inherit their type
          information. Arrays are described by an "A" tag (non-scalar types in TJSON are
          capitalized) however this tag alone is not sufficient:
        </p>

        <div class="syntax-example syntax-invalid">
          {&quot;not-quite-valid<span class="tag-prefix">:A</span>&quot;: ["Hello, world!"]}
        </div>

        <p>
          To properly tag TJSON array, you <i>MUST</i> also include the type of its contents in
          the tag. The following is valid array syntax:
        </p>

        <div class="syntax-example">
          {&quot;valid-array<span class="tag-prefix">:A&lt;s&gt;</span>&quot;: ["Hello, world!"]}
        </div>

        <p>
          The above syntax describes an <em>array</em> of <em>strings</em>. It might remind you
          of <em>generic</em> syntax from statically typed programming languages. TJSON contains
          a tiny type system it uses to verify type annotations.
        </p>

        <p>
          The syntax may be nested to support multidimensional arrays:
        </p>

        <div class="syntax-example">
          {&quot;nested-array<span class="tag-prefix">:A&lt;A&lt;s&gt;&gt;</span>&quot;: [["Nested"], ["Array!"]]}
        </div>

        <p>
          Or objects nested within arrays:
        </p>

        <div class="syntax-example">
          {&quot;nested-object<span class="tag-prefix">:A&lt;O&gt;</span>&quot;: [{&quot;nested<span class="tag-prefix">:s</span>&quot;: "object"}]}
        </div>

        <p>
          The inner type parameter may be omitted for empty arrays:
        </p>

        <div class="syntax-example">
          {&quot;empty-array<span class="tag-prefix">:A&lt;&gt;</span>&quot;: []}
        </div>
      </div>

      <div class="subsection">
        <h2 class="page-header">Sets</h2>
        <p>
          Sets use a syntax that's nearly identical to arrays, but require
          elements within the set are unique:
        </p>

        <div class="syntax-example">
          {&quot;valid-set<span class="tag-prefix">:S&lt;s&gt;</span>&quot;: ["One", "Two", "Three"]}
        </div>

        <p>
          Sets containing repeated items are invalid and are rejected by
          compliant parsers:
        </p>

        <div class="syntax-example syntax-invalid">
          {&quot;invalid-set<span class="tag-prefix">:S&lt;s&gt;</span>&quot;: ["One", "One", "One"]}
        </div>
      </div>

      <div class="subsection">
        <h2 class="page-header">Strings</h2>
        <p>
          As an element of an array, or a member of an object, strings have the same syntax as
          they do in JSON. But when used as the name of an object member, strings carry a special
          postfix tag which acts as a type annotation/signature for the value:
        </p>
        <div class="syntax-example">
          {&quot;hello-string<span class="tag-prefix">:s</span>&quot;: "I'm a string!"}
        </div>
        <p>
          Note that a posfix tag is <em>mandatory</em> for all object member names in TJSON and
          prevents any ambiguities between tagged and untagged strings. Parsers which encounter
          untagged names for object members should raise an exception.
        </p>
        <p>
          Unlike JSON, TJSON strings <em>MUST</em> be encoded as
          <a href="https://en.wikipedia.org/wiki/UTF-8">UTF-8</a>.
          Other Unicode encodings (e.g. UCS-2 as seen in JavaScript) are expressly disallowed.
          All TJSON documents should be valid UTF-8, and parsers should reject documents that
          fail to decode as UTF-8.
        </p>
      </div>

      <div class="subsection">
        <h2 class="page-header">Binary Data</h2>
        <p>
          TJSON supports multiple different formats for encoding 8-bit clean binary data.
          Conforming encoders/decoders are required to support them all. The default is
          <strong>base64url</strong>,
          however encoders may be configured with alternative, potentially more visually
          appealing or well-recognized encodings for specific fields.
        </p>

        <h3>Hexadecimal Data (a.k.a. Base16)</h3>
        <p>
          Data tagged as "d16" is encoded in lower-case hexadecimal format:
        </p>
        <div class="syntax-example">
          {&quot;hello-base-sixteen<span class="tag-prefix">:d16</span>&quot;: "48656c6c6f2c20776f726c6421"}
        </div>
        <p>
          TJSON parsers should expressly reject the use of any upper case hexadecimal characters
          and fail with an exception.
        </p>

        <h3>Base32</h3>
        <p>
          Data tagged as "d32" is encoded in "base32" format as specified in
          <a href="https://tools.ietf.org/html/rfc4648">RFC 4648</a>:
        </p>
        <div class="syntax-example">
          {&quot;hello-base-thirty-two<span class="tag-prefix">:d32</span>&quot;: "jbswy3dpfqqho33snrscc"}
        </div>
        <p>
          The encoded data should <em>NOT</em> be padded with "<b>=</b>" characters as it's stored
          within a quote-delimited string so its length is known in advance.
        </p>
        <p>
          TJSON parsers should expressly reject the use of any upper case Base32 characters
          and fail with an exception.
        </p>

        <h3>Base64url</h3>
        <p>
          Data tagged "d64" is encoded in in "base64url" format as specified in
          <a href="https://tools.ietf.org/html/rfc4648">RFC 4648</a>:
        </p>
        <div class="syntax-example">
          {&quot;hello-base-sixty-four-url<span class="tag-prefix">:d64</span>&quot;: "SGVsbG8sIHdvcmxkIQ"}
        </div>
        <p>
          The encoded data should <em>NOT</em> be padded with "<b>=</b>" characters as it's stored
          within a quote-delimited string so its length is known in advance.
        </p>
        <p>
          The non-URL safe variant of Base64 is not supported by TJSON and should be rejected by
          parsers (i.e. if it contains the "<b>+</b>" or "<b>/</b>" characters it should be
          rejected)
        </p>
        <p>
          Because "base64url" is the default encoding for TJSON, the shorthand "d" variant
          <em>SHOULD</em> be used by default unless another format is specified:
        </p>
        <div class="syntax-example">
          {&quot;base-sixty-four-is-default<span class="tag-prefix">:d</span>&quot;: "SGVsbG8sIHdvcmxkIQ"}
        </div>
      </div>

      <div class="subsection">
        <h2 class="page-header">Numbers</h2>
        <p>
          TJSON supports both integers and floating point numbers in separate formats that can
          always be disambiguated.
        </p>
        <h3>Integers</h3>
        <p>
          In TJSON, integers are stored as strings, sidestepping integer precision issues with
          JSON parsers that do floating point conversions.
        </p>
        <p>
          The following is an example of a <strong>signed integer</strong>, which may be any
          value in the range <em>-(2**63)</em> to <em>(2**63)-1</em>.
        </p>
        <div class="syntax-example">
          {&quot;hello-signed-int<span class="tag-prefix">:i</span>&quot;: "42"}
        </div>
        <p>
          The following is an example of an <strong>unsigned integer</strong>, which may be
          any value in the range <em>0</em> to <em>(2**64)-1</em>:
        </p>
        <div class="syntax-example">
          {&quot;hello-unsigned-int<span class="tag-prefix">:u</span>&quot;: "18446744073709551615"}
        </div>
        <p>
          Integers otherwise utilize the <em>int</em> syntax as described in the JSON specification.
        </p>
        <h3>Floating Points</h3>
        <p>
          Floating points use the native number literal syntax provided by JSON. Unlike integers,
          TJSON floats must not be quoted:
        </p>
        <div class="syntax-example">
          {&quot;hello-float<span class="tag-prefix">:f</span>&quot;: 0.42}
        </div>
        <p>
          The full <a href="https://en.wikipedia.org/wiki/IEEE_floating_point">IEEE 754</a>
          64-bit floating point range is supported.
        </p>
      </div>

      <div class="subsection">
        <h2 class="page-header">Boolean Values</h2>
        <p>
          TJSON supports the <span class="inline-syntax">true</span> and
          <span class="inline-syntax">false</span> values from JSON:
        </p>
        <div class="syntax-example">
          {&quot;hello-true<span class="tag-prefix">:b</span>&quot;: true,
          &quot;hello-false<span class="tag-prefix">:b</span>&quot;: false}
        </div>
        <p>
          The <span class="inline-syntax">null</span> value is expressly
          disallowed anywhere inside of a TJSON document.
        </p>
      </div>

      <div class="subsection">
        <h2 class="page-header">Timestamp</h2>
        <p>
          TJSON adds a literal syntax for timestamp values. The format is based on
          <a href="https://www.ietf.org/rfc/rfc3339.txt">RFC 3339</a>, however the
          use of the UTC time zone identifier "<b>Z</b>" is mandatory (i.e. all
          timestamps are Z-normalized):
        </p>
        <div class="syntax-example">
          {&quot;hello-timestamp<span class="tag-prefix">:t</span>&quot;: "2016-10-02T07:31:51Z"}
        </div>
        <p>
          TJSON parsers should expressly reject the use of other time zone identifiers
          and fail with an exception.
        </p>
      </div>
    </div>
  </div>
</div>