clang::syntax::TokenCollector Class Reference

Collects tokens for the main file while running the frontend action. More...

#include "clang/Tooling/Syntax/Tokens.h"

Public Member Functions
	TokenCollector (Preprocessor &P)
	Adds the hooks to collect the tokens.
TokenBuffer	consume () &&
	Finalizes token collection.

Detailed Description

Collects tokens for the main file while running the frontend action.

An instance of this object should be created on FrontendAction::BeginSourceFile() and the results should be consumed after FrontendAction::Execute() finishes.

Definition at line 418 of file Tokens.h.

Constructor & Destructor Documentation

TokenCollector()

TokenCollector::TokenCollector

(

Preprocessor &

PP

)

Adds the hooks to collect the tokens.

Fills in the TokenBuffer by tracing the run of a preprocessor.

Should be called before the preprocessing starts, i.e. as a part of BeginSourceFile() or CreateASTConsumer().

The implementation tracks the tokens, macro expansions and directives coming from the preprocessor and:

• for each token, figures out if it is a part of an expanded token stream, spelled token stream or both. Stores the tokens appropriately.
• records mappings from the spelled to expanded token ranges, e.g. for macro expansions. FIXME: also properly record:
  • #include directives,
  • #pragma, #line and other PP directives,
  • skipped pp regions,
  • ...

Definition at line 681 of file Tokens.cpp.

References clang::Preprocessor::getSourceManager(), and clang::T.

Member Function Documentation

consume()

TokenBuffer TokenCollector::consume ( ) &&

nodiscard

Finalizes token collection.

Should be called after preprocessing is finished, i.e. after running Execute().

Definition at line 893 of file Tokens.cpp.

---

The documentation for this class was generated from the following files:

• include/clang/Tooling/Syntax/Tokens.h
• lib/Tooling/Syntax/Tokens.cpp