clang 20.0.0git
Public Member Functions | Static Public Member Functions | List of all members
clang::syntax::Token Class Reference

A token coming directly from a file or from a macro invocation. More...

#include "clang/Tooling/Syntax/Tokens.h"

Public Member Functions

 Token (SourceLocation Location, unsigned Length, tok::TokenKind Kind)
 
 Token (const clang::Token &T)
 EXPECTS: clang::Token is not an annotation token.
 
tok::TokenKind kind () const
 
SourceLocation location () const
 Location of the first character of a token.
 
SourceLocation endLocation () const
 Location right after the last character of a token.
 
unsigned length () const
 
llvm::StringRef text (const SourceManager &SM) const
 Get the substring covered by the token.
 
FileRange range (const SourceManager &SM) const
 Gets a range of this token.
 
std::string dumpForTests (const SourceManager &SM) const
 
std::string str () const
 For debugging purposes.
 

Static Public Member Functions

static FileRange range (const SourceManager &SM, const syntax::Token &First, const syntax::Token &Last)
 Given two tokens inside the same file, returns a file range that starts at First and ends at Last.
 

Detailed Description

A token coming directly from a file or from a macro invocation.

Has just enough information to locate the token in the source code. Can represent both expanded and spelled tokens.

Definition at line 103 of file Tokens.h.

Constructor & Destructor Documentation

◆ Token() [1/2]

syntax::Token::Token ( SourceLocation  Location,
unsigned  Length,
tok::TokenKind  Kind 
)

Definition at line 143 of file Tokens.cpp.

References clang::SourceLocation::isValid().

◆ Token() [2/2]

syntax::Token::Token ( const clang::Token T)
explicit

EXPECTS: clang::Token is not an annotation token.

Definition at line 149 of file Tokens.cpp.

References clang::T.

Member Function Documentation

◆ dumpForTests()

std::string syntax::Token::dumpForTests ( const SourceManager SM) const

Definition at line 908 of file Tokens.cpp.

References clang::tok::getTokenName(), length(), and SM.

◆ endLocation()

SourceLocation clang::syntax::Token::endLocation ( ) const
inline

Location right after the last character of a token.

Definition at line 113 of file Tokens.h.

References clang::SourceLocation::getLocWithOffset().

◆ kind()

tok::TokenKind clang::syntax::Token::kind ( ) const
inline

◆ length()

unsigned clang::syntax::Token::length ( ) const
inline

Definition at line 116 of file Tokens.h.

◆ location()

SourceLocation clang::syntax::Token::location ( ) const
inline

Location of the first character of a token.

Definition at line 111 of file Tokens.h.

Referenced by clang::syntax::spelledTokensTouching().

◆ range() [1/2]

FileRange syntax::Token::range ( const SourceManager SM) const

Gets a range of this token.

EXPECTS: token comes from a file, not from a macro expansion.

Definition at line 161 of file Tokens.cpp.

References clang::File, length(), and SM.

Referenced by clang::syntax::computeReplacements().

◆ range() [2/2]

FileRange syntax::Token::range ( const SourceManager SM,
const syntax::Token First,
const syntax::Token Last 
)
static

Given two tokens inside the same file, returns a file range that starts at First and ends at Last.

EXPECTS: First and Last are file tokens from the same file, Last starts after First.

Definition at line 169 of file Tokens.cpp.

References clang::First, clang::Last, and SM.

◆ str()

std::string syntax::Token::str ( ) const

For debugging purposes.

Definition at line 903 of file Tokens.cpp.

References clang::tok::getTokenName(), and length().

◆ text()

llvm::StringRef syntax::Token::text ( const SourceManager SM) const

Get the substring covered by the token.

Note that will include all digraphs, newline continuations, etc. E.g. tokens for 'int' and in\ t both have the same kind tok::kw_int, but results of text() are different.

Definition at line 154 of file Tokens.cpp.

References clang::Invalid, length(), and SM.

Referenced by clang::syntax::TokenBufferTokenManager::getText().


The documentation for this class was generated from the following files: