clang  10.0.0svn
Public Member Functions | Static Public Member Functions | List of all members
clang::syntax::Token Class Reference

A token coming directly from a file or from a macro invocation. More...

#include "clang/Tooling/Syntax/Tokens.h"

Public Member Functions

 Token (SourceLocation Location, unsigned Length, tok::TokenKind Kind)
 
 Token (const clang::Token &T)
 EXPECTS: clang::Token is not an annotation token. More...
 
tok::TokenKind kind () const
 
SourceLocation location () const
 Location of the first character of a token. More...
 
SourceLocation endLocation () const
 Location right after the last character of a token. More...
 
unsigned length () const
 
llvm::StringRef text (const SourceManager &SM) const
 Get the substring covered by the token. More...
 
FileRange range (const SourceManager &SM) const
 Gets a range of this token. More...
 
std::string dumpForTests (const SourceManager &SM) const
 
std::string str () const
 For debugging purposes. More...
 

Static Public Member Functions

static FileRange range (const SourceManager &SM, const syntax::Token &First, const syntax::Token &Last)
 Given two tokens inside the same file, returns a file range that starts at First and ends at Last. More...
 

Detailed Description

A token coming directly from a file or from a macro invocation.

Has just enough information to locate the token in the source code. Can represent both expanded and spelled tokens.

Definition at line 100 of file Tokens.h.

Constructor & Destructor Documentation

◆ Token() [1/2]

syntax::Token::Token ( SourceLocation  Location,
unsigned  Length,
tok::TokenKind  Kind 
)

Definition at line 38 of file Tokens.cpp.

References clang::SourceLocation::isValid().

◆ Token() [2/2]

syntax::Token::Token ( const clang::Token T)
explicit

EXPECTS: clang::Token is not an annotation token.

Definition at line 44 of file Tokens.cpp.

References clang::Token::isAnnotation().

Member Function Documentation

◆ dumpForTests()

std::string syntax::Token::dumpForTests ( const SourceManager SM) const

Definition at line 565 of file Tokens.cpp.

References clang::tok::getTokenName(), and clang::comments::tok::text.

◆ endLocation()

SourceLocation clang::syntax::Token::endLocation ( ) const
inline

Location right after the last character of a token.

Definition at line 110 of file Tokens.h.

References clang::SourceLocation::getLocWithOffset().

◆ kind()

tok::TokenKind clang::syntax::Token::kind ( ) const
inline

Definition at line 106 of file Tokens.h.

Referenced by clang::syntax::TokenBuffer::macroExpansions().

◆ length()

unsigned clang::syntax::Token::length ( ) const
inline

◆ location()

SourceLocation clang::syntax::Token::location ( ) const
inline

Location of the first character of a token.

Definition at line 108 of file Tokens.h.

Referenced by clang::syntax::TokenBuffer::expansionStartingAt(), range(), clang::syntax::TokenBuffer::spelledForExpanded(), clang::syntax::FileRange::text(), and text().

◆ range() [1/2]

FileRange syntax::Token::range ( const SourceManager SM) const

Gets a range of this token.

EXPECTS: token comes from a file, not from a macro expansion.

Definition at line 56 of file Tokens.cpp.

References clang::SourceManager::getDecomposedLoc(), length(), and location().

Referenced by range().

◆ range() [2/2]

FileRange syntax::Token::range ( const SourceManager SM,
const syntax::Token First,
const syntax::Token Last 
)
static

Given two tokens inside the same file, returns a file range that starts at First and ends at Last.

EXPECTS: First and Last are file tokens from the same file, Last starts after First.

Definition at line 64 of file Tokens.cpp.

References range().

◆ str()

std::string syntax::Token::str ( ) const

For debugging purposes.

Definition at line 560 of file Tokens.cpp.

References clang::tok::getTokenName(), and length().

Referenced by clang::syntax::operator<<().

◆ text()

llvm::StringRef syntax::Token::text ( const SourceManager SM) const

Get the substring covered by the token.

Note that will include all digraphs, newline continuations, etc. E.g. tokens for 'int' and in\ t both have the same kind tok::kw_int, but results of text() are different.

Definition at line 49 of file Tokens.cpp.

References clang::SourceManager::getCharacterData(), length(), and location().


The documentation for this class was generated from the following files: