pip_services3_expressions.tokenizers.ISymbolState module

class pip_services3_expressions.tokenizers.ISymbolState.ISymbolState

Bases: pip_services3_expressions.tokenizers.ITokenizerState.ITokenizerState, abc.ABC

Defines an interface for tokenizer state that processes delimiters.

add(value: str, token_type: pip_services3_expressions.tokenizers.TokenType.TokenType)

Add a multi-character symbol.

Parameters

• value – The symbol to add, such as “=:=”

• token_type – The token type