pip_services3_expressions.tokenizers.Token module

class pip_services3_expressions.tokenizers.Token.Token(type: pip_services3_expressions.tokenizers.TokenType.TokenType, value: str, line: int, column: int)

Bases: object

A token represents a logical chunk of a string. For example, a typical tokenizer would break the string “1.23 <= 12.3” into three tokens: the number 1.23, a less-than-or-equal symbol, and the number 12.3. A token is a receptacle, and relies on a tokenizer to decide precisely how to divide a string into tokens.

property column

The column number where the token is.

equals(obj: Any) → bool

property line

The line number where the token is.

property type

The token type.

property value

The token value.