clang-tools 20.0.0git
Classes | Public Types | Public Member Functions | Public Attributes | Static Public Attributes | List of all members
clang::clangd::Token Struct Reference

A single C++ or preprocessor token. More...

#include <Token.h>

Classes

struct  Range
 A half-open range of tokens within a stream. More...
 

Public Types

using Index = uint32_t
 An Index identifies a token within a stream.
 

Public Member Functions

StringRef text () const
 The token text.
 
template<class T >
bool flag (T Mask) const
 
template<class T >
void setFlag (T Mask)
 
const Tokennext () const
 Returns the next token in the stream. this may not be a sentinel.
 
const TokennextNC () const
 Returns the next token in the stream, skipping over comments.
 
const Tokenprev () const
 Returns the previous token in the stream. this may not be a sentinel.
 
const Tokenpair () const
 Returns the bracket paired with this one, if any.
 

Public Attributes

const char * Data = nullptr
 
uint32_t Length = 0
 
uint32_t Line = 0
 Zero-based line number for the start of the token.
 
uint8_t Indent = 0
 Width of whitespace before the first token on this line.
 
uint8_t Flags = 0
 Flags have some meaning defined by the function that produced this stream.
 
Index OriginalIndex = Invalid
 Index into the original token stream (as raw-lexed from the source code).
 
clang::tok::TokenKind Kind = clang::tok::unknown
 The type of token as determined by clang's lexer.
 
int32_t Pair = 0
 If this token is a paired bracket, the offset of the pair in the stream.
 

Static Public Attributes

static constexpr Index Invalid = std::numeric_limits<Index>::max()
 A sentinel Index indicating no token.
 

Detailed Description

A single C++ or preprocessor token.

Unlike clang::Token and syntax::Token, these tokens are not connected to a SourceManager - we are not dealing with multiple files.

Definition at line 50 of file support/Token.h.

Member Typedef Documentation

◆ Index

using clang::clangd::Token::Index = uint32_t

An Index identifies a token within a stream.

Definition at line 52 of file support/Token.h.

Member Function Documentation

◆ flag()

template<class T >
bool clang::clangd::Token::flag ( Mask) const
inline

Definition at line 74 of file support/Token.h.

References Flags.

Referenced by clang::clangd::cook().

◆ next()

const Token & clang::clangd::Token::next ( ) const
inline

Returns the next token in the stream. this may not be a sentinel.

Definition at line 82 of file support/Token.h.

References Kind.

◆ nextNC()

const Token & clang::clangd::Token::nextNC ( ) const
inline

Returns the next token in the stream, skipping over comments.

Definition at line 87 of file support/Token.h.

◆ pair()

const Token * clang::clangd::Token::pair ( ) const
inline

Returns the bracket paired with this one, if any.

Definition at line 100 of file support/Token.h.

References Pair.

Referenced by clang::clangd::getFoldingRanges().

◆ prev()

const Token & clang::clangd::Token::prev ( ) const
inline

Returns the previous token in the stream. this may not be a sentinel.

Definition at line 95 of file support/Token.h.

References Kind.

◆ setFlag()

template<class T >
void clang::clangd::Token::setFlag ( Mask)
inline

Definition at line 77 of file support/Token.h.

References Flags.

Referenced by clang::clangd::lex().

◆ text()

StringRef clang::clangd::Token::text ( ) const
inline

The token text.

Typically from the original source file, but may have been synthesized.

Definition at line 60 of file support/Token.h.

References Data, and Length.

Referenced by clang::clangd::collectIdentifierRanges(), clang::clangd::collectIdentifiers(), clang::clangd::collectMacroReferences(), clang::clangd::cook(), clang::clangd::findNearbyIdentifier(), and clang::clangd::getHover().

Member Data Documentation

◆ Data

const char* clang::clangd::Token::Data = nullptr

Definition at line 61 of file support/Token.h.

Referenced by clang::clangd::cook(), clang::clangd::lex(), and text().

◆ Flags

uint8_t clang::clangd::Token::Flags = 0

Flags have some meaning defined by the function that produced this stream.

Definition at line 70 of file support/Token.h.

Referenced by clang::clangd::cook(), flag(), and setFlag().

◆ Indent

uint8_t clang::clangd::Token::Indent = 0

Width of whitespace before the first token on this line.

Definition at line 68 of file support/Token.h.

Referenced by clang::clangd::lex().

◆ Invalid

constexpr Index clang::clangd::Token::Invalid = std::numeric_limits<Index>::max()
staticconstexpr

A sentinel Index indicating no token.

Definition at line 54 of file support/Token.h.

◆ Kind

clang::tok::TokenKind clang::clangd::Token::Kind = clang::tok::unknown

The type of token as determined by clang's lexer.

Definition at line 103 of file support/Token.h.

Referenced by clang::clangd::cook(), clang::clangd::lex(), next(), and prev().

◆ Length

uint32_t clang::clangd::Token::Length = 0

Definition at line 62 of file support/Token.h.

Referenced by clang::clangd::cook(), clang::clangd::lex(), and text().

◆ Line

uint32_t clang::clangd::Token::Line = 0

Zero-based line number for the start of the token.

This refers to the original source file as written.

Definition at line 66 of file support/Token.h.

Referenced by clang::clangd::getFoldingRanges(), and clang::clangd::lex().

◆ OriginalIndex

Index clang::clangd::Token::OriginalIndex = Invalid

Index into the original token stream (as raw-lexed from the source code).

Definition at line 72 of file support/Token.h.

Referenced by clang::clangd::lex().

◆ Pair

int32_t clang::clangd::Token::Pair = 0

If this token is a paired bracket, the offset of the pair in the stream.

Definition at line 105 of file support/Token.h.

Referenced by pair().


The documentation for this struct was generated from the following file: