delphin.tokens
YY tokens and token lattices.
- class delphin.tokens.YYToken(id, start, end, lnk=None, paths=(1,), form=None, surface=None, ipos=0, lrules=('null',), pos=())[source]
A tuple of token data in the YY format.
- Parameters:
id – token identifier
start – start vertex
end – end vertex
lnk – <from:to> charspan (optional)
paths – path membership
form – surface token
surface – original token (optional; only if
form
was modified)ipos – length of
lrules
? always 0?lrules – something about lexical rules; always “null”?
pos – pairs of (POS, prob)