Data Formats

Most programs spend a surprising amount of their time turning bytes into structure and structure back into bytes: parsing a config file, reading a JSON request body, writing a CSV report, packing a binary wire frame, encoding a token. This chapter covers the standard-library modules that do that work:

• json - encode, decode, and schema-validate JSON.
• csv - split and join comma-separated rows.
• tomllib - read TOML configuration (read-only).
• configparser - read and write INI files.
• base64 - RFC 4648 Base64, standard and URL-safe.
• binascii - hex encoding and CRC32.
• struct - pack and unpack primitive values to and from bytes.
• uuid - generate and validate UUIDs.

Every one of these is written in Dragon (with a few thin extern "C" bridges for number conversion and float bit-reinterpretation). Because they are ordinary typed Dragon source, the surfaces are concrete and they do not always match Python name-for-name. Each section documents what the code actually exposes and ends with a "Differs from Python" note where the shape is not the one you would reach for out of habit. Read it before you port code over from CPython.

A module is used by importing it and qualifying the call (import json then json.dumps(...)), or by pulling the names in directly (from csv import parse_row).

json

JSON is the module with the widest surface, because Dragon's type system gives it something CPython cannot: your classes are schemas. That yields two tiers, and you should pick deliberately between them. When the shape is known, the schema-directed pair decode[T] / encode[T] moves data between JSON bytes and your own types with no boxing at all. When the shape is genuinely unknown, the boxed Any tree is the honest escape hatch.

Decoding a whole document. loads(s: str) -> Any parses a JSON string into a boxed Any tree: objects become dict[str, Any], arrays become list[Any], and scalars become tagged Any values. Its bytes sibling is loadb(data: bytes) -> Any (what Request.json() uses on the verbatim body). This is the Python-parity convenience tier - every node is one box, so it is fine off hot paths.

The catch is that you cannot index straight into the boxed Any and read a value out of it; the box does not auto-unbox under a subscript. The working pattern for a document you know is an object is to decode it AT dict[str, Any] with decode[dict[str, Any]], and let each value unbox as you assign it to a typed local:

from json import decode

const obj: dict[str, Any] = decode[dict[str, Any]]('{"name": "ada", "age": 36}'.encode("utf-8"))
const name: str = obj["name"]    # the Any value unboxes into a str here
const age: int = obj["age"]      # ...and into an int here
print(name)                      # ada
print(age)                       # 36

decode[dict[str, Any]] raises ValueError if the top-level value is not an object (decode[list[Any]] is the array twin). When a value's type is not known statically, narrow it with isinstance before binding:

from json import decode

const obj: dict[str, Any] = decode[dict[str, Any]]('{"port": 8080}'.encode("utf-8"))
const v: Any = obj["port"]
if isinstance(v, int) {
    const n: int = v
    print(n)               # 8080
}
print("port" in obj)       # True

Decoding a value of known type. When you already know the shape, spell it: decode[int](b"42"), decode[list[int]](b"[1, 2, 3]"), decode[dict[str, str]](...) all skip the boxing entirely and hand you the native value - the same decode[T] the next section introduces, at scalar and container targets.

Decoding into your own types: decode[T]. When the shape you expect is a class, the class itself is the schema. decode[T](body: bytes) -> T reads the raw UTF-8 bytes straight into a T, field by field - no Any tree, no intermediate dict, nothing boxed on the way through:

from json import decode

class User {
    def(id: int, tag: str, active: bool) {
        self.id = id
        self.tag = tag
        self.active = active
    }
}

const u: User = decode[User]('{"id": 7, "tag": "ok", "active": true}'.encode("utf-8"))
print(u.id)      # 7
print(u.tag)     # ok

The compiler synthesizes a decoder for each concrete T at compile time, so every field lands in its native slot: an int field is parsed directly into an i64, a str field into one owned heap string. This is the shape Rust reaches with a serde derive macro and Go reaches with reflection, except here it is stamped from the type you already wrote - no annotation, no registration, no runtime dispatch. The contract:

• Keys may arrive in any order, and keys the class does not declare are skipped without being materialized - a wide payload with three fields you care about costs three fields.
• Constructor defaults apply. A field with a default (port: int = 8080) is optional in the document; a field without one is required, and its absence raises ValueError.
• Optional[T] fields accept null.
• Nested classes recurse (a home: Address field decodes the nested object into an Address), and list[...] fields decode each element at the element type.
• Failure is loud and total. A type mismatch, a missing required field, or malformed JSON raises ValueError with the byte offset - never a partially-filled object, so a bad body cannot impersonate a valid one.

The target does not have to be a class. Any concrete type works at the top level, which keeps wire payloads uniform end to end:

# doc: no-check
from json import decode

const n: int = decode[int](b"42")
const xs: list[int] = decode[list[int]](b"[1, 2, 3]")
const users: list[User] = decode[list[User]](body)    # array of objects
const d: dict[str, str] = decode[dict[str, str]](b'{"x": "one"}')

decode[T] takes bytes, not str, because the payloads it exists for - request bodies, process-lane frames, files - already are bytes; for a string literal, .encode("utf-8") at the callsite is the whole ceremony.

The boxed stamps: a spelled-out Any. Generic code holds only a T - it cannot decide at its call site to use loads instead. So the convenience tier has a generic door: an Any you spell in the type argument stamps a boxed decoder. decode[Any] accepts any document (it delegates to loadb); decode[dict[str, Any]] requires a top-level object; decode[list[Any]] requires an array.

# doc: no-check
from json import decode

# The reply's shape is genuinely unknown at compile time.
const d: dict[str, Any] = decode[dict[str, Any]](body)
const v: Any = d["status"]
if isinstance(v, str) {
    const status: str = v
    print(status)
}

Read the price off the type: every concrete T decodes box-free; an Any you spell buys one box per value, exactly what loads costs. It is an opt-in, never a fallback - decode[dict[str, User]] and `decode[dict[int, Any]]` stay compile errors rather than silently taking the boxed path. This is what makes an FFI call with a loosely-shaped reply possible: ffi.sidecar_call[dict[str, Any]](argv, input, blobs) routes its reply through this stamp.

Encoding. dumps(obj: Any) -> str serializes any scalar or arbitrarily-nested homogeneous list/dict by dispatching on the value's runtime tag:

import json

const scores: dict[str, int] = {"ada": 90, "bob": 75}
print(json.dumps(scores))    # {"ada": 90, "bob": 75}
print(json.dumps([1, 2, 3])) # [1, 2, 3]
print(json.dumps(true))      # true

The tag dispatch is one branch per value, and a monomorphized list[int] / dict[str, int] passes straight through - the runtime walker reads native elements into a single output buffer. For a known shape where the consumer wants bytes, encode[T] below is the box-free pair.

Values dumps has no JSON form for - bytes and class instances - raise a catchable TypeError (Object of type Thing is not JSON serializable), exactly as CPython's json.dumps does. A class instance is not a dead end, though; it is the whole point of the next call.

Encoding from your own types: encode[T]. The write-side mirror of decode[T]. encode[T](value: T) -> bytes serializes a class instance (or any concrete top-level type) by emitting one typed write per field - no boxing, no runtime tag dispatch, compact output:

# doc: no-check
from json import encode, decode

const u: User = User(7, "hi", true)
print(encode[User](u).decode("utf-8"))    # {"id":7,"tag":"hi","active":true}

const back: User = decode[User](encode[User](u))   # lossless round-trip

An Optional field holding None emits null; nested classes nest; list[T] and dict[str, ...] work as fields and at the top level, in both directions (encode[list[User]](users) emits the array of objects). This pair is exactly what the process-lane FFI uses to move your arguments and returns across the boundary - see Calling Python, Go, and Rust; the FFI has no private serializer, just these two public calls.

Beneath both sit two public classes for the rare document you must handle piecewise without a type: json.Cursor, a pull reader over bytes, and json.JsonWriter, a push writer (begin_object() / key(k) / write_int(n) / ... / finish() -> bytes). They are the pieces the compiler stamps decode[T] / encode[T] from. Reach for them when the shape is genuinely irregular but the hot path still cannot afford an Any tree; reach for decode[T] / encode[T] everywhere else.

For building JSON text with interpolated values, the module also hosts the JSON content type: template[JSON] { {"name": "!{name}"} } escapes each interpolation so it can never break the surrounding document - see HTML, CSS, and XML for the content-type family.

Validating with JSON Schema

Decoding tells you the payload was JSON; it says nothing about whether it is the JSON you agreed to accept. The module ships a JSON Schema validator (a practical Draft 7 subset) so a service can check a decoded payload against its contract before acting on it - no third-party jsonschema dependency. There is exactly one entry point: the Schema class. An instance names the dialect it speaks and owns a registry of named schemas - register compiles a schema into the instance immediately, validate runs by name:

import json
from json import Schema, ValidationResult, ValidationError

s: Schema = Schema("draft-07")
s.register("order", {
    type: "object",
    required: ["order"],
    properties: {
        order: {type: "integer", minimum: 1}
    }
})

const good: ValidationResult = s.validate("order", {order: 7})
print(good.ok)                        # True

const bad: ValidationResult = s.validate("order", json.loads('{"order": 0}'))
print(bad.ok)                         # False
const errs: list[ValidationError] = bad.errors
for e in errs {
    print(f"{e.path}: {e.message}")   # $.order: value below minimum
}

Schemas are ordinary Dragon dicts, so the keys are bare identifiers - no quoting - and the payload can be anything: a value straight from json.loads, a dict literal like the {order: 7} above, or a native Dragon list. register(name, schema) also takes a parsed document (s.register("evt", decode[dict[str, Any]](text.encode("utf-8"))) for schema text arriving from a request body or a database column). validate(name, payload) never raises on invalid data - it returns ok plus a list of ValidationError values, each carrying the JSON path of the offending value and a human-readable message, so one call reports every violation instead of stopping at the first. The constructor rejects a dialect it does not know ("draft-07", "2019-09", "2020-12", and "latest" are accepted) with a ValueError.

Because the instance owns a registry, registered schemas can reference each other with {"$ref": "<registered-name>"} - shared shapes are registered once and composed, and a $ref resolves through the registry at validation time (so a schema may reference one registered after it). $ref is not an identifier, so it keeps its quotes - exactly as it would in JavaScript:

import json
from json import Schema, ValidationResult, ValidationError

s: Schema = Schema("draft-07")
s.register("address", {
    type: "object",
    required: ["city"],
    properties: {city: {type: "string", minLength: 1}}
})
s.register("person", {
    type: "object",
    required: ["name", "home"],
    properties: {
        name: {type: "string"},
        home: {"$ref": "address"}
    }
})

const r: ValidationResult = s.validate("person", {name: "ada", home: {city: ""}})
const errs: list[ValidationError] = r.errors
for e in errs {
    print(f"{e.path}: {e.message}")   # $.home.city: string shorter than minLength 1
}

Self-referencing schemas work too - a tree schema whose items is {"$ref": "node"} validates its own children; a pure ref cycle (a schema that is nothing but a $ref back to itself) is reported as an error rather than recursing forever, and a $ref to a name that was never registered reports unknown $ref '<name>' at the offending path.

The supported keywords are the practical core of Draft 7: type (a single name or a list of names), enum (deep equality), $ref (a registered schema name), required and properties for objects, items / minItems / maxItems for arrays, minLength / maxLength for strings, and minimum / maximum for numbers. Unknown keywords are ignored, as in every JSON Schema validator. The semantics follow the spec where it matters: "number" accepts integers, "integer" rejects fractions, and booleans are never integers.

Registration mutates the instance - register at startup, not concurrently from server worker threads; validation is read-only and safe to share.

Differs from Python. json.loads returns an Any tree, not a

dict/list, and a boxed Any does not unbox under a direct subscript -

reach for decode[dict[str, Any]] (a typed dict) or a concrete

decode[T], which is where the typed, zero-box path lives. decode[T] /

encode[T] have no CPython stdlib counterpart at all: they replace the

pydantic / dataclass-loader / json.loads-then-construct idiom with a

decoder synthesized at compile time from your class, reflection-free.

There is no object_hook, indent=, or sort_keys= keyword; dumps and

encode[T] emit compact output. Schema validation likewise replaces the

third-party jsonschema/fastjsonschema dependency.

csv

The csv module works on whole lines and whole rows, not the file-object reader/writer iterators Python uses. Every function takes the delimiter as a required argument - there is no dialect object holding it for you.

parse_row(line, delimiter) -> list[str] splits one line into fields, honoring quoted fields (which may contain the delimiter) and doubled-quote escapes:

import csv

const row: list[str] = csv.parse_row('ada,36,"Lovelace, A."', ",")
print(row)     # ['ada', '36', 'Lovelace, A.']

parse_rows(text, delimiter) -> list[list[str]] does the same for a whole multi-line document, and format_row / format_rows go the other way, quoting a field only when it contains the delimiter, a quote, or a newline:

import csv

const rows: list[list[str]] = csv.parse_rows("a,b\n1,2\n3,4", ",")
print(len(rows))      # 3
print(rows[1])        # ['1', '2']

const fields: list[str] = ["plain", "has,comma", 'with"quote']
print(csv.format_row(fields, ","))   # plain,"has,comma","with""quote"

The delimiter is just a string, so tab-separated data is `parse_row(line, "\t"). For data you trust to have no quoting, parse_row_simple` is a bare split. The convenience helpers field_count(line, delimiter) and has_header(text, delimiter) round out the module.

Differs from Python. No csv.reader / csv.writer / DictReader,

no dialect or Sniffer objects. You call parse_row / format_row

directly and pass the delimiter every time.

tomllib

tomllib is a read-only TOML parser, like Python 3.11's. The one call you need is loads(text: str) -> TomlDoc. It returns a TomlDoc, not a dict - you read values out of it with typed, dotted-key accessors, where a key like server.port addresses port inside a [server] table:

import tomllib

const src: str = 'title = "Dragon"\n[server]\nport = 8080\ndebug = true\nratio = 0.75'
const doc: tomllib.TomlDoc = tomllib.loads(src)

print(doc.get("title"))              # Dragon
print(doc.get_int("server.port"))    # 8080
print(doc.get_bool("server.debug"))  # True
print(doc.get_float("server.ratio")) # 0.75
print(doc.has("server.missing"))     # False
print(doc.all_keys())                # ['title', 'server.port', 'server.debug', 'server.ratio']

The accessor methods are get (raw string), get_int, get_float, get_bool, plus has(key) -> bool and all_keys() -> list[str]. A missing key raises ValueError. The parser handles [section] headers, dotted keys, strings, integers (with _ separators), floats, booleans, and # comments; it does not handle arrays of tables, inline tables, multi-line strings, or datetime values.

Differs from Python. loads returns a TomlDoc with typed

get_int / get_float / get_bool accessors, not a plain dict of

already-typed values, and there is no load(fp) file overload - read the

text yourself and pass it in.

configparser

configparser reads and writes the INI format through a ConfigParser object. Construct one, feed it text with read_string, then query by section and key:

from configparser import ConfigParser

const cfg: ConfigParser = ConfigParser()
cfg.read_string("[db]\nhost = localhost\nport = 5432\npool = on")

print(cfg.sections())                # ['db']
print(cfg.get("db", "host"))         # localhost
print(cfg.get_int("db", "port"))     # 5432
print(cfg.get_bool("db", "pool"))    # True
print(cfg.has_option("db", "host"))  # True

The getters are get, get_int, get_float, and get_bool (which treats true, yes, on, and 1 as true). Both = and : are accepted as key/value separators, and # or ; start a comment line. The class is also mutable: set(section, key, value) and add_section(name) build a config up, and to_string() serializes it back to INI text:

from configparser import ConfigParser

const out: ConfigParser = ConfigParser()
out.set("app", "name", "dragon")
print(out.to_string())
# [app]
# name = dragon

items(section) -> list[list[str]] returns the [key, value] pairs of a section for iteration.

Differs from Python. Values are read with explicit typed getters

(get_int, get_bool, …) rather than getint / getboolean methods,

and there is no [section] subscript view, interpolation, or DEFAULT

section. Input arrives via read_string, not read(filename).

base64

base64 is RFC 4648, working in bytes at both ends to match Python. b64encode(data: bytes) -> bytes encodes; b64decode(data: bytes) -> bytes decodes. Since most text you want to encode starts life as a str, encode it to UTF-8 first and decode the result back when you need to read it:

import base64

const raw: bytes = "hello dragon".encode("utf-8")
const enc: bytes = base64.b64encode(raw)
print(enc.decode("ascii"))     # aGVsbG8gZHJhZ29u

const dec: bytes = base64.b64decode(enc)
print(dec.decode("utf-8"))     # hello dragon

The URL-safe variant swaps +// for -/_, which is what JWT and other URL-embedded tokens use:

import base64

const u: bytes = base64.urlsafe_b64encode(bytes([251, 255, 191]))
print(u.decode("ascii"))       # -_-_

standard_b64encode / standard_b64decode are aliases for the default pair, and urlsafe_b64decode reverses the URL-safe form. Decoding tolerates missing padding (handy for JWS, which elides it).

Differs from Python. The surface matches base64, but encode/decode

are strictly bytes-in, bytes-out - there is no implicit str

acceptance, so call .encode() / .decode() at the boundaries.

binascii

binascii converts between binary and ASCII hex, and computes CRC32. hexlify(data: bytes) -> bytes produces lowercase hex; unhexlify reverses it and conveniently accepts either bytes or str:

import binascii

const h: bytes = binascii.hexlify("Dragon".encode("utf-8"))
print(h.decode("ascii"))         # 447261676f6e

const back: bytes = binascii.unhexlify("447261676f6e")   # str input is fine
print(back.decode("utf-8"))      # Dragon

b2a_hex / a2b_hex are aliases for hexlify / unhexlify. `crc32(data: bytes, value: int = 0) -> int` is the standard IEEE 802.3 CRC32 used by gzip and PNG:

import binascii

print(binascii.crc32("hello".encode("utf-8")))   # 907060870

Differs from Python. The functions match (hexlify, unhexlify,

crc32, b2a_hex, a2b_hex). unhexlify accepting a str directly is a

small ergonomic addition over CPython, which wants bytes.

struct

struct packs and unpacks primitive values to and from bytes using CPython's format mini-language. The public surface is `pack(fmt, *args) -> bytes, unpack(fmt, data) -> list, and calcsize(fmt) -> int`.

The format string opens with an optional byte-order char - < little-endian, > or ! big-endian (network), =/@/none native - followed by format codes: b/B for int8, h/H for int16, i/I/l/L for int32, q/Q for int64, f for float32, d for float64, ? for bool, s for a fixed-width string field, and x for a pad byte. Lowercase is signed, uppercase unsigned.

import struct

# Big-endian uint16, int32, float64.
const packed: bytes = struct.pack(">Hid", 8080, -1, 3.5)
print(len(packed))                # 14
print(struct.calcsize(">Hid"))    # 14

const vals: list[Any] = struct.unpack(">Hid", packed)
const port: int = vals[0]
const code: int = vals[1]
const ratio: float = vals[2]
print(port)                       # 8080
print(code)                       # -1
print(ratio)                      # 3.5

The float formats are the reason this module needs compiler support: a value's raw IEEE-754 bytes are recovered with a register-level bitcast, no numeric conversion. That makes the whole binary-protocol class - msgpack, BSON, CBOR, the MySQL wire format, .npy - expressible in pure Dragon. A malformed format or an argument that does not fit raises struct.error; note that float64 is lowercase d (an uppercase D is not a valid code).

Differs from Python. unpack returns a list[Any], not a tuple -

Dragon has no variadic-arity runtime tuple, so a list is the expressible

analog; index access and iteration are identical. Bind each element to a

typed local (const port: int = vals[0]) to use it.

uuid

uuid generates version-4 (random) UUIDs and validates UUID strings. uuid4() -> str returns the canonical 36-character form:

import uuid

const u: str = uuid.uuid4()
print(len(u))                  # 36
print(uuid.uuid_is_valid(u))   # True
print(uuid.uuid_version(u))    # 4
print(len(uuid.uuid_hex(u)))   # 32  (dashes stripped)

The helpers are uuid_hex (strip dashes to 32 hex chars), uuid_version (the version digit, 1-5, or -1), uuid_is_valid (full 8-4-4-4-12 format check), and uuid_compare(a, b) -> int for canonical lexicographic ordering.

uuid4() draws its 122 random bits from the OS CSPRNG (the same source as secrets), not from the random module - so seeding random has no effect on the UUIDs you get, and two processes started in the same instant produce different values.

Differs from Python. uuid4() returns a plain str, not a UUID

object - there are no .hex / .int / .version attributes; the

uuid_hex / uuid_version module functions take a string instead. Only

uuid4 is implemented (no uuid1/uuid3/uuid5). As in Python, a v4 UUID

is a fine unique identifier but is not a substitute for a purpose-built

token - use secrets.token_urlsafe for anything bearer-like.

At a glance

These modules sit at the seam between raw bytes and structured data, and they all stay typed and compiled the whole way through - a JSON decode or a struct.pack costs what the equivalent hand-written loop costs, with no dynamic interpreter underneath. For the clocks and numbers your structured data so often carries - timestamps, durations, sums, and averages - turn to Dates, Times, and Math.