pip_services3_expressions.calculator.tokenizers package

Submodules

Module contents

class pip_services3_expressions.calculator.tokenizers.ExpressionNumberState

Bases: pip_services3_expressions.tokenizers.generic.GenericNumberState.GenericNumberState

Implements an Expression-specific number state object.

next_token(scanner: pip_services3_expressions.io.IScanner.IScanner, tokenizer: pip_services3_expressions.tokenizers.ITokenizer.ITokenizer) → pip_services3_expressions.tokenizers.Token.Token

Gets the next token from the stream started from the character linked to this state.

Parameters

  • scanner – A textual string to be tokenized.

  • tokenizer – A tokenizer class that controls the process.

Returns

The next token from the top of the stream.

class pip_services3_expressions.calculator.tokenizers.ExpressionQuoteState

Bases: pip_services3_expressions.tokenizers.IQuoteState.IQuoteState

Implements an Expression-specific quote string state object.

decode_string(value: str, quote_symbol: int) → Optional[str, None]

Decodes a string value.

Parameters

  • value – A string value to be decoded.

  • quote_symbol – A string quote character.

Returns

An decoded string.

encode_string(value: str, quote_symbol: int) → Optional[str, None]

Encodes a string value.

Parameters

  • value – A string value to be encoded.

  • quote_symbol – A string quote character.

Returns

An encoded string.

next_token(scanner: pip_services3_expressions.io.IScanner.IScanner, tokenizer: pip_services3_expressions.tokenizers.ITokenizer.ITokenizer) → pip_services3_expressions.tokenizers.Token.Token

Gets the next token from the stream started from the character linked to this state.

Parameters

  • scanner – A textual string to be tokenized.

  • tokenizer – A tokenizer class that controls the process.

Returns

The next token from the top of the stream.

class pip_services3_expressions.calculator.tokenizers.ExpressionSymbolState

Bases: pip_services3_expressions.tokenizers.generic.GenericSymbolState.GenericSymbolState

Constructs an instance of this class.

class pip_services3_expressions.calculator.tokenizers.ExpressionTokenizer

Bases: pip_services3_expressions.tokenizers.AbstractTokenizer.AbstractTokenizer

Implement tokenizer to perform lexical analysis for expressions.

class pip_services3_expressions.calculator.tokenizers.ExpressionWordState

Bases: pip_services3_expressions.tokenizers.generic.GenericWordState.GenericWordState

Implements a word state object.

next_token(scanner: pip_services3_expressions.io.IScanner.IScanner, tokenizer: pip_services3_expressions.tokenizers.ITokenizer.ITokenizer) → pip_services3_expressions.tokenizers.Token.Token

Gets the next token from the stream started from the character linked to this state.

Parameters

  • scanner – A textual string to be tokenized.

  • tokenizer – A tokenizer class that controls the process.

Returns

The next token from the top of the stream.