clang::TokenConcatenation Class Reference

TokenConcatenation class, which answers the question of "Is it safe to emit two tokens without a whitespace between them, or would that cause implicit concatenation of the tokens?". More...

#include "clang/Lex/TokenConcatenation.h"

Public Member Functions
	TokenConcatenation (const Preprocessor &PP)
bool	AvoidConcat (const Token &PrevPrevTok, const Token &PrevTok, const Token &Tok) const
	AvoidConcat - If printing PrevTok immediately followed by Tok would cause the two individual tokens to be lexed as a single token, return true (which causes a space to be printed between them).

Detailed Description

TokenConcatenation class, which answers the question of "Is it safe to emit two tokens without a whitespace between them, or would that cause implicit concatenation of the tokens?".

For example, it emitting two identifiers "foo" and "bar" next to each other would cause the lexer to produce one "foobar" token. Emitting "1" and ")" next to each other is safe.

Definition at line 30 of file TokenConcatenation.h.

Constructor & Destructor Documentation

TokenConcatenation()

TokenConcatenation::TokenConcatenation

(

const Preprocessor &

PP

)

Definition at line 69 of file TokenConcatenation.cpp.

References clang::Preprocessor::getLangOpts(), and memset().

Member Function Documentation

AvoidConcat()

bool TokenConcatenation::AvoidConcat	(	const Token &	PrevPrevTok,
		const Token &	PrevTok,
		const Token &	Tok
	)		const

AvoidConcat - If printing PrevTok immediately followed by Tok would cause the two individual tokens to be lexed as a single token, return true (which causes a space to be printed between them).

This allows the output of -E mode to be lexed to the same token stream as lexing the input directly would.

This code must conservatively return true if it doesn't want to be 100% accurate. This will cause the output to include extra space characters, but the resulting output won't have incorrect concatenations going on. Examples include "..", which we print with a space between, because we don't want to track enough to tell "x.." from "...".

Definition at line 160 of file TokenConcatenation.cpp.

References GetFirstChar(), clang::Token::getIdentifierInfo(), clang::Token::getKind(), clang::Preprocessor::getLangOpts(), clang::Token::getLength(), clang::Token::getLocation(), clang::SourceLocation::getLocWithOffset(), clang::Preprocessor::getSourceManager(), clang::Token::hasUDSuffix(), clang::Token::is(), clang::Token::isAnnotation(), clang::isDigit(), clang::Token::isNot(), clang::Token::isOneOf(), clang::isPreprocessingNumberBody(), and SM.

---

The documentation for this class was generated from the following files:

• include/clang/Lex/TokenConcatenation.h
• lib/Lex/TokenConcatenation.cpp