pip_services3_expressions.tokenizers.IWhitespaceState module

class pip_services3_expressions.tokenizers.IWhitespaceState.IWhitespaceState

Bases: pip_services3_expressions.tokenizers.ITokenizerState.ITokenizerState, abc.ABC

Defines an interface for tokenizer state that processes whitespaces (’ ‘, ‘ ‘)

clear_whitespace_chars()

Clears definitions of whitespace characters.

set_whitespace_chars(from_symbol: int, to_symbol: int, enable: bool)

Establish the given characters as whitespace to ignore.

Parameters

  • from_symbol – First character index of the interval.

  • to_symbol – Last character index of the interval.

  • enable – true if this state should ignore characters in the given range.