clang::syntax::Token Class Reference

A token coming directly from a file or from a macro invocation. More...

#include "clang/Tooling/Syntax/Tokens.h"

Public Member Functions
	Token (SourceLocation Location, unsigned Length, tok::TokenKind Kind)
	Token (const clang::Token &T)
	EXPECTS: clang::Token is not an annotation token.
tok::TokenKind	kind () const
SourceLocation	location () const
	Location of the first character of a token.
SourceLocation	endLocation () const
	Location right after the last character of a token.
unsigned	length () const
llvm::StringRef	text (const SourceManager &SM) const
	Get the substring covered by the token.
FileRange	range (const SourceManager &SM) const
	Gets a range of this token.
std::string	dumpForTests (const SourceManager &SM) const
std::string	str () const
	For debugging purposes.

Static Public Member Functions
static FileRange	range (const SourceManager &SM, const syntax::Token &First, const syntax::Token &Last)
	Given two tokens inside the same file, returns a file range that starts at First and ends at Last.

Detailed Description

A token coming directly from a file or from a macro invocation.

Has just enough information to locate the token in the source code. Can represent both expanded and spelled tokens.

Definition at line 103 of file Tokens.h.

Constructor & Destructor Documentation

Token() [1/2]

syntax::Token::Token	(	SourceLocation	Location,
		unsigned	Length,
		tok::TokenKind	Kind )

Definition at line 141 of file Tokens.cpp.

Referenced by Token().

Token() [2/2]

syntax::Token::Token ( const clang::Token & T )

explicit

EXPECTS: clang::Token is not an annotation token.

Definition at line 147 of file Tokens.cpp.

References getKind(), clang::T, and Token().

Member Function Documentation

dumpForTests()

std::string syntax::Token::dumpForTests

(

const SourceManager &

SM

)

const

Definition at line 906 of file Tokens.cpp.

References clang::tok::getTokenName(), kind(), length(), SM, and text().

endLocation()

SourceLocation clang::syntax::Token::endLocation ( ) const

inline

Location right after the last character of a token.

Definition at line 113 of file Tokens.h.

References clang::SourceLocation::getLocWithOffset().

kind()

tok::TokenKind clang::syntax::Token::kind ( ) const

inline

Definition at line 109 of file Tokens.h.

Referenced by clang::syntax::deepCopyExpandingMacros(), dumpForTests(), clang::syntax::TokenBufferTokenManager::getText(), clang::syntax::TokenBuffer::macroExpansions(), and str().

length()

unsigned clang::syntax::Token::length ( ) const

inline

Definition at line 116 of file Tokens.h.

location()

SourceLocation clang::syntax::Token::location ( ) const

inline

Location of the first character of a token.

Definition at line 111 of file Tokens.h.

Referenced by range(), and text().

range() [1/2]

FileRange syntax::Token::range

(

const SourceManager &

SM

)

const

Gets a range of this token.

EXPECTS: token comes from a file, not from a macro expansion.

Definition at line 159 of file Tokens.cpp.

References clang::File, length(), location(), and SM.

Referenced by clang::syntax::computeReplacements().

range() [2/2]

FileRange syntax::Token::range ( const SourceManager & SM, const syntax::Token & First, const syntax::Token & Last )

static

Given two tokens inside the same file, returns a file range that starts at First and ends at Last.

EXPECTS: First and Last are file tokens from the same file, Last starts after First.

Definition at line 167 of file Tokens.cpp.

References clang::First, clang::Last, and SM.

str()

std::string syntax::Token::str

(

)

const

For debugging purposes.

Definition at line 901 of file Tokens.cpp.

References clang::tok::getTokenName(), kind(), and length().

text()

llvm::StringRef syntax::Token::text

(

const SourceManager &

SM

)

const

Get the substring covered by the token.

Note that will include all digraphs, newline continuations, etc. E.g. tokens for 'int' and in\ t both have the same kind tok::kw_int, but results of text() are different.

Definition at line 152 of file Tokens.cpp.

References clang::Invalid, length(), location(), and SM.

Referenced by dumpForTests(), and clang::syntax::TokenBufferTokenManager::getText().

---

The documentation for this class was generated from the following files:

• include/clang/Tooling/Syntax/Tokens.h
• lib/Tooling/Syntax/Tokens.cpp