pip_services3_expressions.tokenizers.generic.SymbolNode module

class pip_services3_expressions.tokenizers.generic.SymbolNode.SymbolNode(parent: Optional[SymbolNode, None], character: int)

Bases: object

A SymbolNode object is a member of a tree that contains all possible prefixes of allowable symbols. Multi-character symbols appear in a SymbolNode tree with one node for each character.

For example, the symbol =:~ will appear in a tree as three nodes. The first node contains an equals sign, and has a child; that child contains a colon and has a child; this third child contains a tilde, and has no children of its own. If the colon node had another child for a dollar sign character, then the tree would contain the symbol =:$.

A tree of SymbolNode objects collaborate to read a (potentially multi-character) symbol from an input stream. A root node with no character of its own finds an initial node that represents the first character in the input. This node looks to see if the next character in the stream matches one of its children. If so, the node delegates its reading task to its child. This approach walks down the tree, pulling symbols from the input that match the path down the tree.

When a node does not have a child that matches the next character, we will have read the longest possible symbol prefix. This prefix may or may not be a valid symbol. Consider a tree that has had =:~ added and has not had =: added. In this tree, of the three nodes that contain =:~, only the first and third contain complete symbols. If, say, the input contains =:a, the colon node will not have a child that matches the ‘a’ and so it will stop reading. The colon node has to “unread”: it must push back its character, and ask its parent to unread. Unreading continues until it reaches an ancestor that represents a valid symbol.

add_descendant_line(value: str, token_type: pip_services3_expressions.tokenizers.TokenType.TokenType)

Add a line of descendants that represent the characters in the given string.

Parameters

• value –

• token_type –

ancestry() → str

Show the symbol this node represents.

Returns

The symbol this node represents.

deepest_read(scanner: pip_services3_expressions.io.IScanner.IScanner) → pip_services3_expressions.tokenizers.generic.SymbolNode.SymbolNode

Find the descendant that takes as many characters as possible from the input.

Parameters

scanner –

ensure_child_with_char(value: int)

Find or create a child for the given character.

find_child_with_char(value) → pip_services3_expressions.tokenizers.generic.SymbolNode.SymbolNode

Find a child with the given character.

Parameters

value –

property token_type

unread_to_valid(scanner: pip_services3_expressions.io.IScanner.IScanner) → pip_services3_expressions.tokenizers.generic.SymbolNode.SymbolNode

Unwind to a valid node; this node is “valid” if its ancestry represents a complete symbol. If this node is not valid, put back the character and ask the parent to unwind.

Parameters

scanner –

property valid