pip_services3_expressions.tokenizers.generic.CppCommentState module

class pip_services3_expressions.tokenizers.generic.CppCommentState.CppCommentState

Bases: pip_services3_expressions.tokenizers.generic.GenericCommentState.GenericCommentState

This state will either delegate to a comment-handling state, or return a token with just a slash in it.

get_multi_line_comment(scanner: pip_services3_expressions.io.IScanner.IScanner) → str

Ignore everything up to a closing star and slash, and then return the tokenizer’s next token.

Parameters

scanner –

get_single_line_comment(scanner: pip_services3_expressions.io.IScanner.IScanner) → str

Ignore everything up to an end-of-line and return the tokenizer’s next token.

next_token(scanner: pip_services3_expressions.io.IScanner.IScanner, tokenizer: pip_services3_expressions.tokenizers.ITokenizer.ITokenizer) → pip_services3_expressions.tokenizers.Token.Token

Either delegate to a comment-handling state, or return a token with just a slash in it.

Parameters

• scanner – A textual string to be tokenized.

• tokenizer – A tokenizer class that controls the process.

Returns

The next token from the top of the stream.