clang::syntax::Token Class Reference

A token coming directly from a file or from a macro invocation. More...

#include "clang/Tooling/Syntax/Tokens.h"

Public Member Functions
	Token (SourceLocation Location, unsigned Length, tok::TokenKind Kind)
	Token (const clang::Token &T)
	EXPECTS: clang::Token is not an annotation token.
tok::TokenKind	kind () const
SourceLocation	location () const
	Location of the first character of a token.
SourceLocation	endLocation () const
	Location right after the last character of a token.
unsigned	length () const
llvm::StringRef	text (const SourceManager &SM) const
	Get the substring covered by the token.
FileRange	range (const SourceManager &SM) const
	Gets a range of this token.
std::string	dumpForTests (const SourceManager &SM) const
std::string	str () const
	For debugging purposes.

Static Public Member Functions
static FileRange	range (const SourceManager &SM, const syntax::Token &First, const syntax::Token &Last)
	Given two tokens inside the same file, returns a file range that starts at First and ends at Last.

Detailed Description

A token coming directly from a file or from a macro invocation.

Has just enough information to locate the token in the source code. Can represent both expanded and spelled tokens.

Definition at line 103 of file Tokens.h.

Constructor & Destructor Documentation

Token() [1/2]

syntax::Token::Token	(	SourceLocation	Location,
		unsigned	Length,
		tok::TokenKind	Kind
	)

Definition at line 143 of file Tokens.cpp.

References clang::SourceLocation::isValid().

Token() [2/2]

syntax::Token::Token ( const clang::Token &  T )

explicit

EXPECTS: clang::Token is not an annotation token.

Definition at line 149 of file Tokens.cpp.

References clang::T.

Member Function Documentation

dumpForTests()

std::string syntax::Token::dumpForTests

(

const SourceManager &

SM

)

const

Definition at line 908 of file Tokens.cpp.

References clang::tok::getTokenName(), length(), and SM.

endLocation()

SourceLocation clang::syntax::Token::endLocation ( ) const

inline

Location right after the last character of a token.

Definition at line 113 of file Tokens.h.

References clang::SourceLocation::getLocWithOffset().

kind()

tok::TokenKind clang::syntax::Token::kind ( ) const

inline

Definition at line 109 of file Tokens.h.

Referenced by clang::syntax::deepCopyExpandingMacros(), clang::syntax::TokenBufferTokenManager::getText(), and clang::syntax::TokenBuffer::macroExpansions().

length()

unsigned clang::syntax::Token::length ( ) const

inline

Definition at line 116 of file Tokens.h.

location()

SourceLocation clang::syntax::Token::location ( ) const

inline

Location of the first character of a token.

Definition at line 111 of file Tokens.h.

Referenced by clang::syntax::spelledTokensTouching().

range() [1/2]

FileRange syntax::Token::range

(

const SourceManager &

SM

)

const

Gets a range of this token.

EXPECTS: token comes from a file, not from a macro expansion.

Definition at line 161 of file Tokens.cpp.

References clang::File, length(), and SM.

Referenced by clang::syntax::computeReplacements().

range() [2/2]

FileRange syntax::Token::range ( const SourceManager &  SM, const syntax::Token &  First, const syntax::Token &  Last  )

static

Given two tokens inside the same file, returns a file range that starts at First and ends at Last.

EXPECTS: First and Last are file tokens from the same file, Last starts after First.

Definition at line 169 of file Tokens.cpp.

References clang::First, clang::Last, and SM.

str()

std::string syntax::Token::str

(

)

const

For debugging purposes.

Definition at line 903 of file Tokens.cpp.

References clang::tok::getTokenName(), and length().

text()

llvm::StringRef syntax::Token::text

(

const SourceManager &

SM

)

const

Get the substring covered by the token.

Note that will include all digraphs, newline continuations, etc. E.g. tokens for 'int' and in\ t both have the same kind tok::kw_int, but results of text() are different.

Definition at line 154 of file Tokens.cpp.

References clang::Invalid, length(), and SM.

Referenced by clang::syntax::TokenBufferTokenManager::getText().

---

The documentation for this class was generated from the following files:

• include/clang/Tooling/Syntax/Tokens.h
• lib/Tooling/Syntax/Tokens.cpp