delphin.tokens

class delphin.tokens.YyToken

A tuple of token data in the YY format.

Parameters:

• id – token identifier
• start – start vertex
• end – end vertex
• lnk – <from:to> charspan (optional)
• paths – path membership
• form – surface token
• surface – original token (optional; only if form was modified)
• ipos – length of lrules? always 0?
• lrules – something about lexical rules; always “null”?
• pos – pairs of (POS, prob)

classmethod from_dict(d)

Decode from a dictionary as from to_dict().

to_dict()

Encode the token as a dictionary suitable for JSON serialization.

class delphin.tokens.YyTokenLattice(tokens)

A lattice of YY Tokens.

Parameters:tokens – a list of YyToken objects

classmethod from_list(toks)

Decode from a list as from to_list().

classmethod from_string(s)

Decode from the YY token lattice format.

to_list()

Encode the token lattice as a list suitable for JSON serialization.