pip_services3_expressions.tokenizers.generic.GenericQuoteState module
-
class
pip_services3_expressions.tokenizers.generic.GenericQuoteState.GenericQuoteState Bases:
pip_services3_expressions.tokenizers.IQuoteState.IQuoteStateA quoteState returns a quoted string token from a scanner. This state will collect characters until it sees a match to the character that the tokenizer used to switch to this state. For example, if a tokenizer uses a double-quote character to enter this state, then
next_tokenwill search for another double-quote until it finds one or finds the end of the scanner.-
decode_string(value: str, quote_symbol: int) → Optional[str, None] Decodes a string value.
- Parameters
value – A string value to be decoded.
quote_symbol – A string quote character.
- Returns
An decoded string.
-
encode_string(value: str, quote_symbol: int) → Optional[str, None] Encodes a string value.
- Parameters
value – A string value to be encoded.
quote_symbol – A string quote character.
- Returns
An encoded string.
-
next_token(scanner: pip_services3_expressions.io.IScanner.IScanner, tokenizer: pip_services3_expressions.tokenizers.ITokenizer.ITokenizer) → pip_services3_expressions.tokenizers.Token.Token Return a quoted string token from a scanner. This method will collect characters until it sees a match to the character that the tokenizer used to switch to this state.
- Parameters
scanner – A textual string to be tokenized.
tokenizer – A tokenizer class that controls the process.
- Returns
The next token from the top of the stream.
-