clang::ast_matchers::dynamic::Parser::CodeTokenizer Class Reference

Simple tokenizer for the parser. More...

Public Member Functions
	CodeTokenizer (StringRef &MatcherCode, Diagnostics *Error)
	CodeTokenizer (StringRef &MatcherCode, Diagnostics *Error, unsigned CodeCompletionOffset)
const TokenInfo &	peekNextToken () const
	Returns but doesn't consume the next token.
TokenInfo	consumeNextToken ()
	Consumes and returns the next token.
TokenInfo	SkipNewlines ()
TokenInfo	consumeNextTokenIgnoreNewlines ()
TokenInfo::TokenKind	nextTokenKind () const

Detailed Description

Simple tokenizer for the parser.

Definition at line 68 of file Parser.cpp.

Constructor & Destructor Documentation

CodeTokenizer() [1/2]

clang::ast_matchers::dynamic::Parser::CodeTokenizer::CodeTokenizer ( StringRef & MatcherCode, Diagnostics * Error )

inlineexplicit

Definition at line 70 of file Parser.cpp.

CodeTokenizer() [2/2]

clang::ast_matchers::dynamic::Parser::CodeTokenizer::CodeTokenizer ( StringRef & MatcherCode, Diagnostics * Error, unsigned CodeCompletionOffset )

inline

Definition at line 75 of file Parser.cpp.

Member Function Documentation

consumeNextToken()

TokenInfo clang::ast_matchers::dynamic::Parser::CodeTokenizer::consumeNextToken ( )

inline

Consumes and returns the next token.

Definition at line 86 of file Parser.cpp.

Referenced by consumeNextTokenIgnoreNewlines().

consumeNextTokenIgnoreNewlines()

TokenInfo clang::ast_matchers::dynamic::Parser::CodeTokenizer::consumeNextTokenIgnoreNewlines ( )

inline

Definition at line 98 of file Parser.cpp.

References consumeNextToken(), SkipNewlines(), and clang::ast_matchers::dynamic::Parser::TokenInfo::TK_Eof.

nextTokenKind()

TokenInfo::TokenKind clang::ast_matchers::dynamic::Parser::CodeTokenizer::nextTokenKind ( ) const

inline

Definition at line 105 of file Parser.cpp.

peekNextToken()

const TokenInfo & clang::ast_matchers::dynamic::Parser::CodeTokenizer::peekNextToken ( ) const

inline

Returns but doesn't consume the next token.

Definition at line 83 of file Parser.cpp.

SkipNewlines()

TokenInfo clang::ast_matchers::dynamic::Parser::CodeTokenizer::SkipNewlines ( )

inline

Definition at line 92 of file Parser.cpp.

References clang::ast_matchers::dynamic::Parser::TokenInfo::TK_NewLine.

Referenced by consumeNextTokenIgnoreNewlines().

---

The documentation for this class was generated from the following file:

• lib/ASTMatchers/Dynamic/Parser.cpp