# JSONTokenParser
<span class="source-link">[[Source]](src/json/json_token_parser.md#L-0-1)</span>

Streaming, incremental JSON token parser.

Feed bytes as they arrive with `feed()`; the parser walks the JSON
structure to any depth and pushes tokens to a `JSONTokenNotify`
callback as they complete. It builds no tree — its working memory
is the container-depth stack, the single string or number it is
part-way through, and the fed bytes it has not yet consumed. A
value split across a chunk boundary is held and stitched onto the
next `feed()`, so you never manage leftover bytes.

```pony
let parser = JSONTokenParser(
  object is JSONTokenNotify
    fun ref apply(p: JSONTokenParser, token: JSONToken) =>
      match token
      | let k: JSONTokenKey    => // k.value
      | let s: JSONTokenString => // s.value
      | let n: JSONTokenNumber => // n.value
      | JSONTokenObjectStart   => // ...
      end
  end)
parser.feed(chunk)?
```

It parses a *stream* of top-level values: after one value's
tokens end it continues to the next, so newline-delimited JSON or
a socket delivering messages back to back works with no per-value
setup. Feeding a whole document at once is just the case where
every byte is already in hand.

A number is the one value with no self-delimiter — the parser
cannot know a number is finished until a following non-number byte
arrives, so a number at the very end of the fed bytes is not
emitted until more bytes come or you call `finish()`. `finish()`
says "no more bytes are coming" and completes it.

Malformed input raises from `feed()`/`finish()`; read
`describe_error()`, `token_end()`, and `line()` for the location.
After a raise the parser latches: every later `feed()`/`finish()`
raises again. `abort()` (called from the notifier) stops the
parse the same way. For untrusted input, `JSONParseLimits`
caps nesting depth and the length of a single string or number.

To turn a token stream back into a `JSONValue`, use `JSONReassembler`. For a
whole in-memory document, `JSONParser.parse()` is simpler.


```pony
class ref JSONTokenParser
```

## Constructors

### create
<span class="source-link">[[Source]](src/json/json_token_parser.md#L-0-62)</span>


```pony
new ref create(
  notify: JSONTokenNotify ref,
  limits: JSONParseLimits val = JSONParseLimits)
: JSONTokenParser ref^
```
#### Parameters

*   notify: [JSONTokenNotify](json-JSONTokenNotify.md) ref
*   limits: [JSONParseLimits](json-JSONParseLimits.md) val = JSONParseLimits

#### Returns

* [JSONTokenParser](json-JSONTokenParser.md) ref^

---

## Public Functions

### feed
<span class="source-link">[[Source]](src/json/json_token_parser.md#L-0-71)</span>


Append a chunk of bytes and emit every token now derivable from the bytes in
hand. Holds any partial value to be finished by a later `feed()`. Raises on
malformed input or `abort()`; after that the parser is done and every later
`feed()`/`finish()` raises. Do not call `feed()` or `finish()`
from within the notifier — the parse is running and a re-entrant
call raises.


```pony
fun ref feed(
  data: ByteSeq)
: None val ?
```
#### Parameters

*   data: [ByteSeq](builtin-ByteSeq.md)

#### Returns

* [None](builtin-None.md) val ?

---

### finish
<span class="source-link">[[Source]](src/json/json_token_parser.md#L-0-88)</span>


Signal that no more bytes will be fed. Completes a pending number (the one
value that otherwise waits for a following byte) and emits its
token. Raises if that number's text is not valid, or if
aborted/already done. A value left structurally incomplete (an
open container, an unterminated string) is simply
not completed — the consumer sees the truncation via `incomplete()`. After
`finish()` the parser is done; feeding more raises.


```pony
fun ref finish()
: None val ?
```

#### Returns

* [None](builtin-None.md) val ?

---

### abort
<span class="source-link">[[Source]](src/json/json_token_parser.md#L-0-118)</span>


Signal the parser to stop. The current `feed()`/`finish()` then
raises, and the parser is done. Call this from the notifier when
you have seen enough.


```pony
fun ref abort()
: None val
```

#### Returns

* [None](builtin-None.md) val

---

### token_start
<span class="source-link">[[Source]](src/json/json_token_parser.md#L-0-126)</span>


Byte offset where the current token starts, absolute across all feeds. Valid
only during the notify callback.


```pony
fun box token_start()
: USize val
```

#### Returns

* [USize](builtin-USize.md) val

---

### token_end
<span class="source-link">[[Source]](src/json/json_token_parser.md#L-0-133)</span>


Byte offset just past the current token, absolute across all
feeds. Valid only during the notify callback.


```pony
fun box token_end()
: USize val
```

#### Returns

* [USize](builtin-USize.md) val

---

### line
<span class="source-link">[[Source]](src/json/json_token_parser.md#L-0-140)</span>


Current line number (1-based).


```pony
fun box line()
: USize val
```

#### Returns

* [USize](builtin-USize.md) val

---

### incomplete
<span class="source-link">[[Source]](src/json/json_token_parser.md#L-0-146)</span>


True while a value is part-way through — either mid-scan on a
leaf or with a container still open. After feeding a complete
document and calling `finish()` this is false; if it is still
true, the input ended in the middle of a value. This is the
byte-level truncation check; `JSONReassembler.mid_value()`
reports only whether the reassembler is holding a partial
value's tokens.


```pony
fun box incomplete()
: Bool val
```

#### Returns

* [Bool](builtin-Bool.md) val

---

### describe_error
<span class="source-link">[[Source]](src/json/json_token_parser.md#L-0-158)</span>


Human-readable description of the most recent error.


```pony
fun box describe_error()
: String val
```

#### Returns

* [String](builtin-String.md) val

---

### parse_error
<span class="source-link">[[Source]](src/json/json_token_parser.md#L-0-174)</span>


The most recent error as a `JSONParseError` (message, byte offset, line).
Meaningful after a raise from `feed()`/`finish()`.


```pony
fun box parse_error()
: JSONParseError val
```

#### Returns

* [JSONParseError](json-JSONParseError.md) val

---

