← Farbfeld · Build · Demo

Farbfeld v2 specification

Farbfeld v2 is a compressed, lossless-by-default sibling of farbfeld that keeps the .ff extension and, like farbfeld, has no container: no magic number, no chunks, no checksums, no metadata. A file is one flag byte, the image size, and one arithmetic-coded stream. The reference implementation is ffimg.js (mkCore, mkPalFrame, decodeFZ3); where this text and the code disagree, the code and the test vectors win. Status: experimental, not a standard, and not part of suckless farbfeld. All integers below are unsigned unless noted.

1. Telling v1 from v2

A v1 file starts with the ASCII letters farbfeld, so its first byte is 0x66. A v2 file's first byte is either exactly 0x06 (the optional metadata prefix, section 9) or has a low nibble of 3, 4, 5 or 7; 0x66 is neither. A decoder therefore needs only the first byte, never the extension. Compression wrappers (.ff.gz) sit outside the format.

byte 0   bits 0-3  kind:  3 = still (8-bit)   4 = animation (8-bit)   5 = still (16-bit)   7 = tiled still (8-bit)
         bit 4     alpha channel present (4 channels, else 3)
         bit 5     palette mode            (kinds 3, 4)
         bit 6     near-lossless           (kinds 3, 4, 7; not with palette)
         bit 7     blended predictor       (kinds 3, 4, 7; not with palette)
                   in palette mode: luma-sorted palette numbering instead (section 4)
then     LEB128 width, LEB128 height                 (both > 0)
kind 4:  LEB128 frame count (1..65535), LEB128 loop count (0 = forever)
kind 7:  one byte e, 4..12: tiles are 2^e x 2^e pixels
bit 6:   one byte N, 1..40: the error bound
kind 7:  one LEB128 byte length per tile (raster order), then the tiles' streams back to back (section 8)
others:  the range-coded stream (section 2)

LEB128 here: 7 bits per byte, least significant group first, high bit = more follows, at most 5 bytes. Limits: width × height × frames ≤ 225 (226 pixels for kind 7, at most 220 tiles). Kind 5 allows no palette, near or blend bit; kind 7 no palette bit; palette mode excludes the near bit. A decoder must reject anything else.

2. Range coder and probability models

A binary range coder in the style of LZMA. The decoder keeps range (32 bit, starts 0xFFFFFFFF) and code (32 bit, the first 4 stream bytes, big-endian). Bytes past the end of the file read as 0: the encoder drops the always-zero first byte LZMA would emit and trims trailing zero bytes. Cut-off files are caught by the end marker (section 8).

decode(p):                 // p = probability that the bit is 0, 12 bit, 1..4095
  bound = (range >>> 12) * p
  if code < bound: range = bound; bit = 0
  else: code -= bound; range -= bound; bit = 1
  while range < 2^24: range = (range << 8) mod 2^32; code = ((code << 8) | next_byte) mod 2^32
  return bit

Every model slot holds two 16-bit counters a and b, both initially 32768. For a single model, p = clamp((a + b) >> 5, 1, 4095). Bits marked mixed in section 3 use the logistic mixer below instead. After the bit is known each used slot updates: bit 0 → a += (65536 − a) >> 4; b += (65536 − b) >> 8; bit 1 → a −= a >> 4; b −= b >> 8.

Logistic mixer

Integer arithmetic only. squash(d) maps −2047..2047 to 0..4095: clamp (d > 2047 → 4095, d < −2047 → 0), then with w = d & 127, i = (d >> 7) + 16 return (T[i]·(128−w) + T[i+1]·w + 64) >> 7 for the table T = 1 2 3 6 10 16 27 45 73 120 194 310 488 747 1101 1546 2047 2549 2994 3348 3607 3785 3901 3975 4022 4050 4068 4079 4085 4089 4092 4093 4094. stretch(p) is its inverse by table: for x = −2047..2047 in order, every p from the previous squash(x)+1 up to squash(x) gets x; leftovers get 2047.

A mixed bit uses three slots (s1, s2, s3) and 7 inputs: for each slot stretch(clamp(a >> 4, 1, 4095)) and the same for b, then the constant 256. Weights are integers (65536 = 1.0), one set of 7 per weight-set index; the first 6 start at 11000, the last at 0. dot = Σ input·weight, p = clamp(squash(dot >> 16), 1, 4095) is the probability of a 0 bit. After the bit: err = ((bit ? 0 : 4095) − p) · 6, weight += (input·err + 32768) >> 16 clamped to ±98304, and all three slots update as above.

A byte tree codes an 8-bit value MSB first with node = 1, using slot base + node (single model) for each bit, then node = node*2 + bit.

3. Pixels (kinds 3 and 4, truecolour)

Channels are coded per pixel in raster order, each pixel's channels in order 0..n−1, from planes G, R−G, B−G (mod 256) and, with alpha, A. With the near-lossless bit set the planes are G, R, B, A unchanged (modular subtraction would wrap an error of 1 across 0/255 into an error of 255). The decoder adds G back: R = (R′ + G) mod 256.

Prediction

For one plane: a = left, b = up, c = up-left. Missing neighbours: with no left, a = up (or 0 in the first row); with no up, b = a; with either missing, c = b. Default predictor is MED: c ≥ max(a,b) → min(a,b); c ≤ min(a,b) → max(a,b); else a+b−c.

Blend (bit 7). Five candidates: MED, a, b, (a+b+1)>>1, clamp(a+b−c, 0, 255). For each, the stored error εk of its earlier prediction at the left, up, up-left and up-right pixels (absent = 0) is summed to sk; ε is min(255, |value − candidate|) recorded after each pixel. If sMED = 0 the prediction is MED. Otherwise Wk = floor(240 / (1+sk)4) and the prediction is floor((2·Σ Wk·vk + ΣWk) / (2·ΣWk)).

Static pixels (animation, frame 2 onward). If the left and up pixels of this plane equal their value in the previous frame (a missing neighbour counts as equal), the prediction is the previous frame's value here, and context set 1 is used instead of 0.

Residual

Lossless: s = (value − pred) wrapped to signed 8 bits. Near-lossless with bound N, step = 2N+1: s = value − pred (no wrap), q = s ≥ 0 ? floor((s+N)/step) : −floor((N−s)/step), and the decoder reconstructs clamp(pred + q·step, 0, 255), which is also what later predictions use. Let the coded value be s (lossless) or q (near). Zigzag: z = s ≥ 0 ? 2s : −2s−1 (0..255) and k = bit length of z (0 for z = 0).

Coding z

With S = 9, channel c, set σ ∈ {0,1} (static) and sc = σ·channels + c (pr below is the final prediction, including the static override):

e(p)  = min(255, |coded value at p| )           stored per plane, p = left / up / up-left / up-right
t     = 2e(L) + 2e(U) + e(UL) + e(UR)           absent = 0
q1    = 0 if t=0; else 1 if t≤2; 2 if ≤5; 3 if ≤10; 4 if ≤20; 5 if ≤40; 6 if ≤80; else 7
q2    = 0 / 1 / 2 for the previous channel's |coded value| at this pixel being 0 / <4 / ≥4   (channel 0: 0)
g     = |a−c| + |b−c|;   qg = 0 if g=0; 1 if ≤2; 2 if ≤6; 3 if ≤14; 4 if ≤30; 5 if ≤62; 6 if ≤126; else 7
qd    = the same 8-level quantiser applied to |a−pr| + |b−pr|

unary: for i = 0..7:  bit (k > i), stop at the first 0 (no terminator after i = 7); mixed, slots
        s1 = U [((sc*8+q1)*3+q2)*S + i]
        s2 = U2[((sc*8+qg)*8+q1)*S + i]
        s3 = U3[((sc*8+qd)*3+q2)*S + i]          weight set = sc*S + i
then the k−1 bits of z below its top bit, high to low:
        first (position k−2):  M1[(sc*8+q1)*S + k]
        others (position j):   M2[k*8 + j]                       (single models)
z = (1 << (k−1)) | those bits (k ≥ 1)

All slots and weights start fresh per image (once per animation, not per frame). The decoded s is z odd ? −((z+1)>>1) : z>>1.

4. Palette mode (bit 5)

Used when the whole image, or all frames, have at most 256 distinct colours; always lossless. Two numberings, chosen by the encoder per file (bit 7): first appearance (bit 7 clear), where colours are numbered in order of first appearance in scan order, frame by frame, and luma-sorted (bit 7 set), where they are sorted by 3R + 6G + B (+ 8A), ties by first appearance. The stream begins with count−1 as a byte tree, then each colour's channels (R, G, B[, A]) as byte trees with a separate tree per channel position (luma-sorted: each value is coded as its difference from the previous colour's, mod 256). Then each pixel:

neighbours a,b,c,d = left, up, up-left, up-right indices (same edge rules as section 3; d = b when missing)
ctx = (a==b) | (b==c)<<1 | (b==d)<<2 | (a==c)<<3 | (no colour numbered yet)<<4
frame ≥ 2:  pv = same position in previous frame; flag "idx == pv" with model EV[ pc ],
            pc = (left unchanged) | (up unchanged)<<1 | (pv == a)<<2            (missing neighbour = unchanged)
then for each of a, b, c, d in turn, skipping values equal to pv or an earlier candidate:
            flag "idx == candidate" with model EC[candidate number][ctx]
first appearance:  flag "idx == highest index so far + 1" with model EN[ctx], else the index as a byte tree
luma-sorted:       d = idx − a (never 0). z = 2d−2 if d > 0, else −2d−1; m = z+1; k = bit length of m (1..9).
                   unary: for i = 1..9 the bit (k > i), stopping at the first 0, with models UD[cx*10 + i], where
                   cx = 0, 1, 2, 3, 4, 5 for |a−b| = 0, 1, <4, <8, <16, else;
                   then the k−1 bits of m below its top bit, high to low, with models MD[k*10 + position]

An index at or above the colour count is an error.

5. Animation (kind 4)

Frames are complete RGBA images, already composited: there are no disposal modes or sub-rectangles. Before each frame comes its delay in centiseconds: a flag "same as the previous delay" (single model, initial previous delay 10), otherwise 16 bits as two byte trees (high byte, then low byte, 512 shared slots). The loop count in the header is the number of plays, 0 meaning forever. An encoder zeroes the colour of fully transparent pixels, since it is invisible.

Frame 2 onward then carries, before its pixels, a description of what changed since the previous frame:

flag MV (single model)  a global motion vector follows
    dx, dy: each zigzag (z = v ≥ 0 ? 2v : −2v−1) as a byte tree; trees at slot bases 0 and 256 of a 512-slot array
reference frame R = the previous frame shifted by (dx, dy): R(x,y) = prev(clamp(x−dx), clamp(y−dy))   (all zero when there is no vector)
flag CHANGED (single model)  0: this frame equals R exactly; nothing more is coded
flag FULL (single model)     1: the box is the whole frame
    otherwise x0, y0, width−1, height−1, each as 16 bits (high byte tree, low byte tree), slot bases 0, 512, 1024, 1536 of a 2048-slot array
then the pixels of the box only; everything outside the box is copied from R.

Inside the box the coder runs as in sections 3 and 4 with R as the previous-frame reference (the static-pixel rule and the palette "same as previous frame" flag), over the box's rows and columns only. Neighbours outside the box are the copied pixels; their error history is zero. The error history is also cleared at the start of every frame. The vector is global (one per frame), not per block, and an encoder only uses it when it clearly beats no motion on a sample of the frame.

6. 16-bit stills (kind 5)

Lossless only. Same as section 3 with 16-bit values: planes G, R−G, B−G mod 65536 (A as is), MED prediction only, residual wrapped to signed 16 bits, S = 17, the unary part up to 16 steps, M2[k*16 + j], and every magnitude fed to a context (e, the previous-channel value, g) shifted right by 8 first. The alpha bit selects 4 channels, with the alpha value 65535 meaning opaque. Reading such a file as 8-bit keeps the high byte.

7. End marker

After the last pixel (and, for palette mode, after the last frame) the encoder codes 16 more bits, high to low, of the constant 0xA55A, each with its own fresh single slot, so every one has probability exactly 2048/4096. A decoder that reads anything else must reject the file. A cut-off or damaged file decodes to noise before it gets here, so it passes with probability about 2−16; a file that lost only trailing zero bytes (which decoders read back as zeros anyway) still decodes correctly.

8. Tiled stills (kind 7)

The image is cut into tiles of 2e × 2e pixels in raster order (the last row and column of tiles may be smaller). Each tile is coded exactly like a still of its own size (section 3, truecolour only, the file's alpha, near and blend bits), with fresh models, no neighbours across tile borders, and its own end marker (section 7), as a separate range-coded stream. The header's table gives each stream's byte length, so any tile can be located and decoded alone, or tiles can be decoded in parallel. Cost: the models start over in every tile, so files get larger (about 2% at 256, 5% at 128, 12% at 64 on one 800×526 test image) and the table adds a byte or two per tile. A stream is read from its own byte range only; bytes past it read as zero.

9. Optional metadata prefix

The only thing in v2 that resembles a container, and it is optional: a file may begin with 0x06, a LEB128 length L, and L bytes of records, followed by an ordinary v2 file (which may not itself start with 0x06). Files without metadata have no prefix and no overhead. Each record is a type byte, a LEB128 length and a payload:

1  colour space   1 byte: 0 sRGB, 1 linear sRGB, 2 Display P3, 3 Rec.2020
2  resolution     LEB128 x, LEB128 y: pixels per inch
3  text           UTF-8 key, a 0 byte, UTF-8 value
4  ICC profile    the profile's bytes
other types: skipped. A record running past L, or L past the end of the file, is an error (L ≤ 2^24).

Without the prefix, colour is sRGB as in farbfeld. v1 and kind-5 files take the same prefix.

10. Test vectors and a second decoder

vectors/index.json lists small files with their bytes in hex and the pixels they must decode to: a 1×1 still, a still with alpha, a two-colour checkerboard, a 12×10 textured still, a 3-frame animation, a luma-sorted palette image, a scrolling animation (motion vector), a tiled still, a still behind a metadata prefix, and a 16-bit still. test.cjs runs them, plus round-trip, truncation and corrupt-file checks, against the reference decoder. ff2dec.py is a separate pure-Python decoder written from this page; crosscheck.cjs decodes generated files in every mode with both and compares the pixels (they agree), though I wrote both, so it checks that this page is sufficient, not that I read it independently. corpus.cjs checks real images against size budgets.

Not specified: premultiplied alpha (it is not), per-block motion, progressive display. This revision (mixer, end marker) is not compatible with the earlier experimental files.