clang 18.0.0git
Public Member Functions | List of all members
clang::ast_matchers::dynamic::Parser::CodeTokenizer Class Reference

Simple tokenizer for the parser. More...

Public Member Functions

 CodeTokenizer (StringRef &MatcherCode, Diagnostics *Error)
 CodeTokenizer (StringRef &MatcherCode, Diagnostics *Error, unsigned CodeCompletionOffset)
const TokenInfopeekNextToken () const
 Returns but doesn't consume the next token.
TokenInfo consumeNextToken ()
 Consumes and returns the next token.
TokenInfo SkipNewlines ()
TokenInfo consumeNextTokenIgnoreNewlines ()
TokenInfo::TokenKind nextTokenKind () const

Detailed Description

Simple tokenizer for the parser.

Definition at line 69 of file Parser.cpp.

Constructor & Destructor Documentation

◆ CodeTokenizer() [1/2]

clang::ast_matchers::dynamic::Parser::CodeTokenizer::CodeTokenizer ( StringRef &  MatcherCode,
Diagnostics Error 

Definition at line 71 of file Parser.cpp.

◆ CodeTokenizer() [2/2]

clang::ast_matchers::dynamic::Parser::CodeTokenizer::CodeTokenizer ( StringRef &  MatcherCode,
Diagnostics Error,
unsigned  CodeCompletionOffset 

Definition at line 76 of file Parser.cpp.

Member Function Documentation

◆ consumeNextToken()

TokenInfo clang::ast_matchers::dynamic::Parser::CodeTokenizer::consumeNextToken ( )

Consumes and returns the next token.

Definition at line 87 of file Parser.cpp.

Referenced by consumeNextTokenIgnoreNewlines().

◆ consumeNextTokenIgnoreNewlines()

TokenInfo clang::ast_matchers::dynamic::Parser::CodeTokenizer::consumeNextTokenIgnoreNewlines ( )

◆ nextTokenKind()

TokenInfo::TokenKind clang::ast_matchers::dynamic::Parser::CodeTokenizer::nextTokenKind ( ) const

Definition at line 106 of file Parser.cpp.

References clang::ast_matchers::dynamic::Parser::TokenInfo::Kind.

◆ peekNextToken()

const TokenInfo & clang::ast_matchers::dynamic::Parser::CodeTokenizer::peekNextToken ( ) const

Returns but doesn't consume the next token.

Definition at line 84 of file Parser.cpp.

◆ SkipNewlines()

TokenInfo clang::ast_matchers::dynamic::Parser::CodeTokenizer::SkipNewlines ( )

The documentation for this class was generated from the following file: