clang 19.0.0git
Public Types | Public Member Functions | List of all members
clang::Token Class Reference

Token - This structure provides full information about a lexed token. More...

#include "clang/Lex/Token.h"

Public Types

enum  TokenFlags {
  StartOfLine = 0x01 , LeadingSpace = 0x02 , DisableExpand = 0x04 , NeedsCleaning = 0x08 ,
  LeadingEmptyMacro = 0x10 , HasUDSuffix = 0x20 , HasUCN = 0x40 , IgnoredComma = 0x80 ,
  StringifiedInMacro = 0x100 , CommaAfterElided = 0x200 , IsEditorPlaceholder = 0x400 , IsReinjected = 0x800
}
 

Public Member Functions

tok::TokenKind getKind () const
 
void setKind (tok::TokenKind K)
 
bool is (tok::TokenKind K) const
 is/isNot - Predicates to check if this token is a specific kind, as in "if (Tok.is(tok::l_brace)) {...}".
 
bool isNot (tok::TokenKind K) const
 
bool isOneOf (tok::TokenKind K1, tok::TokenKind K2) const
 
template<typename... Ts>
bool isOneOf (tok::TokenKind K1, Ts... Ks) const
 
bool isAnyIdentifier () const
 Return true if this is a raw identifier (when lexing in raw mode) or a non-keyword identifier (when lexing in non-raw mode).
 
bool isLiteral () const
 Return true if this is a "literal", like a numeric constant, string, etc.
 
bool isAnnotation () const
 Return true if this is any of tok::annot_* kind tokens.
 
bool isRegularKeywordAttribute () const
 Return true if the token is a keyword that is parsed in the same position as a standard attribute, but that has semantic meaning and so cannot be a true attribute.
 
SourceLocation getLocation () const
 Return a source location identifier for the specified offset in the current file.
 
unsigned getLength () const
 
void setLocation (SourceLocation L)
 
void setLength (unsigned Len)
 
SourceLocation getAnnotationEndLoc () const
 
void setAnnotationEndLoc (SourceLocation L)
 
SourceLocation getLastLoc () const
 
SourceLocation getEndLoc () const
 
SourceRange getAnnotationRange () const
 SourceRange of the group of tokens that this annotation token represents.
 
void setAnnotationRange (SourceRange R)
 
const char * getName () const
 
void startToken ()
 Reset all flags to cleared.
 
bool hasPtrData () const
 
IdentifierInfogetIdentifierInfo () const
 
void setIdentifierInfo (IdentifierInfo *II)
 
const void * getEofData () const
 
void setEofData (const void *D)
 
StringRef getRawIdentifier () const
 getRawIdentifier - For a raw identifier token (i.e., an identifier lexed in raw mode), returns a reference to the text substring in the buffer if known.
 
void setRawIdentifierData (const char *Ptr)
 
const char * getLiteralData () const
 getLiteralData - For a literal token (numeric constant, string, etc), this returns a pointer to the start of it in the text buffer if known, null otherwise.
 
void setLiteralData (const char *Ptr)
 
void * getAnnotationValue () const
 
void setAnnotationValue (void *val)
 
void setFlag (TokenFlags Flag)
 Set the specified flag.
 
bool getFlag (TokenFlags Flag) const
 Get the specified flag.
 
void clearFlag (TokenFlags Flag)
 Unset the specified flag.
 
unsigned getFlags () const
 Return the internal represtation of the flags.
 
void setFlagValue (TokenFlags Flag, bool Val)
 Set a flag to either true or false.
 
bool isAtStartOfLine () const
 isAtStartOfLine - Return true if this token is at the start of a line.
 
bool hasLeadingSpace () const
 Return true if this token has whitespace before it.
 
bool isExpandDisabled () const
 Return true if this identifier token should never be expanded in the future, due to C99 6.10.3.4p2.
 
bool isObjCAtKeyword (tok::ObjCKeywordKind objcKey) const
 Return true if we have an ObjC keyword identifier.
 
tok::ObjCKeywordKind getObjCKeywordID () const
 Return the ObjC keyword kind.
 
bool isSimpleTypeSpecifier (const LangOptions &LangOpts) const
 Determine whether the token kind starts a simple-type-specifier.
 
bool needsCleaning () const
 Return true if this token has trigraphs or escaped newlines in it.
 
bool hasLeadingEmptyMacro () const
 Return true if this token has an empty macro before it.
 
bool hasUDSuffix () const
 Return true if this token is a string or character literal which has a ud-suffix.
 
bool hasUCN () const
 Returns true if this token contains a universal character name.
 
bool stringifiedInMacro () const
 Returns true if this token is formed by macro by stringizing or charizing operator.
 
bool commaAfterElided () const
 Returns true if the comma after this token was elided.
 
bool isEditorPlaceholder () const
 Returns true if this token is an editor placeholder.
 

Detailed Description

Token - This structure provides full information about a lexed token.

It is not intended to be space efficient, it is intended to return as much information as possible about each returned token. This is expected to be compressed into a smaller form if memory footprint is important.

The parser can create a special "annotation token" representing a stream of tokens that were parsed and semantically resolved, e.g.: "foo::MyClass<int>" can be represented by a single typename annotation token that carries information about the SourceRange of the tokens and the type object.

Definition at line 36 of file Token.h.

Member Enumeration Documentation

◆ TokenFlags

Enumerator
StartOfLine 
LeadingSpace 
DisableExpand 
NeedsCleaning 
LeadingEmptyMacro 
HasUDSuffix 
HasUCN 
IgnoredComma 
StringifiedInMacro 
CommaAfterElided 
IsEditorPlaceholder 
IsReinjected 

Definition at line 74 of file Token.h.

Member Function Documentation

◆ clearFlag()

void clang::Token::clearFlag ( TokenFlags  Flag)
inline

Unset the specified flag.

Definition at line 254 of file Token.h.

Referenced by setFlagValue().

◆ commaAfterElided()

bool clang::Token::commaAfterElided ( ) const
inline

Returns true if the comma after this token was elided.

Definition at line 313 of file Token.h.

References CommaAfterElided, and getFlag().

◆ getAnnotationEndLoc()

SourceLocation clang::Token::getAnnotationEndLoc ( ) const
inline

◆ getAnnotationRange()

SourceRange clang::Token::getAnnotationRange ( ) const
inline

SourceRange of the group of tokens that this annotation token represents.

Definition at line 166 of file Token.h.

References getAnnotationEndLoc(), and getLocation().

Referenced by clang::Parser::TryAnnotateTypeOrScopeToken().

◆ getAnnotationValue()

void * clang::Token::getAnnotationValue ( ) const
inline

◆ getEndLoc()

SourceLocation clang::Token::getEndLoc ( ) const
inline

◆ getEofData()

const void * clang::Token::getEofData ( ) const
inline

Definition at line 200 of file Token.h.

References is().

◆ getFlag()

bool clang::Token::getFlag ( TokenFlags  Flag) const
inline

◆ getFlags()

unsigned clang::Token::getFlags ( ) const
inline

Return the internal represtation of the flags.

This is only intended for low-level operations such as writing tokens to disk.

Definition at line 262 of file Token.h.

Referenced by clang::ASTWriter::AddToken().

◆ getIdentifierInfo()

IdentifierInfo * clang::Token::getIdentifierInfo ( ) const
inline

Definition at line 187 of file Token.h.

References is(), isAnnotation(), isLiteral(), and isNot().

Referenced by clang::Sema::ActOnPragmaUnused(), AddDefinedMacroToHash(), clang::ASTWriter::AddToken(), clang::MacroArgs::ArgNeedsPreexpansion(), clang::TokenConcatenation::AvoidConcat(), clang::Preprocessor::CheckMacroName(), diagnoseAutoModuleImport(), dumpTokenInto(), EvaluateDefined(), EvaluateFeatureLikeBuiltinMacro(), EvaluateValue(), ExpectFeatureIdentifierInfo(), GetFirstChar(), getIdentifier(), getObjCKeywordID(), clang::Lexer::getSpelling(), clang::Preprocessor::getSpelling(), clang::Preprocessor::HandleDirective(), clang::PragmaNamespace::HandlePragma(), isConfigurationPattern(), clang::format::isFunctionDeclarationName(), clang::MacroInfo::isIdenticalTo(), clang::format::AdditionalKeywords::IsJavaScriptIdentifier(), isObjCAtKeyword(), isSameToken(), isSimpleTypeSpecifier(), isTrivialSingleTokenExpansion(), clang::VAOptDefinitionContext::isVAOptToken(), isWideStringLiteralFromMacro(), clang::format::AdditionalKeywords::isWordLike(), clang::TokenLexer::Lex(), LexModuleNameComponent(), clang::Preprocessor::LexOnOffSwitch(), clang::MacroPPCallbacks::MacroDefined(), clang::detail::MacroExpansionRangeRecorder::MacroExpands(), clang::MacroPPCallbacks::MacroUndefined(), clang::format::mustBeJSIdent(), clang::TokenValue::operator==(), ParseAlignPragma(), parseDeclareSimdClauses(), clang::Parser::parseMapperModifier(), clang::Parser::ParseMSAsmIdentifier(), parseOpenMPReductionId(), clang::Parser::ParseOpenMPReservedLocator(), clang::Parser::ParseOpenMPVarList(), clang::Parser::ParseTopLevelDecl(), clang::Parser::ParseUnqualifiedId(), PragmaLoopHintString(), PrintPreprocessedTokens(), clang::Parser::TryAnnotateTypeOrScopeToken(), clang::Parser::TryAnnotateTypeOrScopeTokenAfterScopeSpec(), and clang::Preprocessor::TypoCorrectToken().

◆ getKind()

tok::TokenKind clang::Token::getKind ( ) const
inline

Definition at line 94 of file Token.h.

Referenced by clang::Sema::ActOnCharacterConstant(), clang::ASTWriter::AddToken(), atOrBeforeSeparation(), clang::TokenConcatenation::AvoidConcat(), clang::Sema::CheckConstraintExpression(), clang::Preprocessor::CheckMacroName(), clang::Lexer::ComputePreamble(), contains(), clang::Preprocessor::DumpToken(), EvaluateDirectiveSubExpr(), EvaluateFeatureLikeBuiltinMacro(), EvaluateHasIncludeCommon(), EvaluateValue(), getBeginningOfFileToken(), getEntityEndLoc(), getIdentifier(), getSpellingSlow(), clang::Preprocessor::HandlePragmaModuleBuild(), highlightLines(), isAnnotation(), isAnyIdentifier(), isConfigurationPattern(), clang::format::isConfiguredQualifier(), clang::MacroInfo::isIdenticalTo(), clang::format::AdditionalKeywords::IsJavaScriptIdentifier(), isLiteral(), clang::format::FormatToken::isObjCAtKeyword(), clang::Preprocessor::IsPreviousCachedToken(), clang::format::isQualifier(), isRegularKeywordAttribute(), isSameToken(), clang::format::FormatToken::isSimpleTypeSpecifier(), isSimpleTypeSpecifier(), clang::TokenValue::operator==(), clang::Parser::ParseConstraintLogicalAndExpression(), parseOpenMPReductionId(), clang::Parser::ParseTopLevelDecl(), clang::Parser::ParseUnqualifiedId(), PrintPreprocessedTokens(), ReadOriginalFileName(), clang::ASTReader::ReadToken(), clang::CodeGen::CoverageMappingModuleGen::setUpCoverageCallbacks(), clang::format::ShouldBreakBeforeBrace(), clang::Parser::SkipMalformedDecl(), clang::Parser::SkipUntil(), clang::MacroArgs::StringifyArgument(), and clang::tokenIsLikeStringLiteral().

◆ getLastLoc()

SourceLocation clang::Token::getLastLoc ( ) const
inline

Definition at line 155 of file Token.h.

References getAnnotationEndLoc(), getLocation(), and isAnnotation().

Referenced by clang::Parser::TryAnnotateTypeOrScopeToken().

◆ getLength()

unsigned clang::Token::getLength ( ) const
inline

◆ getLiteralData()

const char * clang::Token::getLiteralData ( ) const
inline

getLiteralData - For a literal token (numeric constant, string, etc), this returns a pointer to the start of it in the text buffer if known, null otherwise.

Definition at line 225 of file Token.h.

References isLiteral().

Referenced by dumpTokenInto(), fixSeparateAttrArgAndNumber(), GetFirstChar(), clang::Lexer::getSpelling(), clang::Preprocessor::getSpellingOfSingleCharacterNumericConstant(), PrintPreprocessedTokens(), clang::ento::tryExpandAsInteger(), and UseNSOptionsMacro().

◆ getLocation()

SourceLocation clang::Token::getLocation ( ) const
inline

Return a source location identifier for the specified offset in the current file.

Definition at line 132 of file Token.h.

References clang::SourceLocation::getFromRawEncoding().

Referenced by clang::Sema::ActOnCharacterConstant(), clang::Sema::ActOnNumericConstant(), clang::Sema::ActOnPragmaUnused(), clang::arcmt::trans::MigrationContext::addPropertyAttribute(), clang::ASTWriter::AddToken(), clang::TokenConcatenation::AvoidConcat(), clang::Preprocessor::CheckEndOfDirective(), clang::Preprocessor::CheckMacroName(), ComputeDATE_TIME(), clang::Lexer::ComputePreamble(), clang::Parser::ConsumeToken(), clang::Parser::Diag(), clang::Preprocessor::Diag(), diagnoseUnexpectedOperator(), clang::Preprocessor::DiscardUntilEndOfDirective(), clang::DoPrintPreprocessedInput(), clang::Preprocessor::DumpToken(), EvaluateDefined(), EvaluateDirectiveSubExpr(), EvaluateFeatureLikeBuiltinMacro(), EvaluateHasIncludeCommon(), EvaluateValue(), clang::Parser::ExitScope(), clang::Sema::ExpandFunctionLocalPredefinedMacros(), clang::BalancedDelimiterTracker::expectAndConsume(), ExpectFeatureIdentifierInfo(), findDirectives(), findPreviousTokenKind(), clang::arcmt::trans::findSemiAfterLocation(), FixDigraph(), clang::format::UnwrappedLineFormatter::format(), getAnnotationRange(), getBeginningOfFileToken(), getEndLoc(), GetFirstChar(), getLastLoc(), GetLineValue(), clang::StringLiteralParser::getOffsetOfStringByte(), clang::Lexer::getSpelling(), clang::Preprocessor::getSpellingOfSingleCharacterNumericConstant(), clang::Preprocessor::HandleDirective(), clang::Preprocessor::HandlePragmaDependency(), clang::Preprocessor::HandlePragmaHdrstop(), clang::Preprocessor::HandlePragmaIncludeAlias(), clang::Preprocessor::HandlePragmaMark(), clang::Preprocessor::HandlePragmaModuleBuild(), clang::Preprocessor::HandlePragmaPopMacro(), clang::Preprocessor::HandlePragmaSystemHeader(), highlightLines(), clang::TokenLexer::Init(), clang::Preprocessor::IsPreviousCachedToken(), clang::TokenLexer::Lex(), clang::Preprocessor::LexHeaderName(), LexModuleNameComponent(), LocPropertyAttribute(), clang::MacroPPCallbacks::MacroDefined(), clang::detail::MacroExpansionRangeRecorder::MacroExpands(), clang::MacroPPCallbacks::MacroUndefined(), clang::tooling::maybeExtendRange(), NormalizeLine(), ParseAlignPragma(), clang::Parser::ParseAssignmentExpression(), clang::Parser::ParseConditionalExpression(), parseDeclareSimdClauses(), ParseLoopHintValue(), clang::Parser::parseMapperModifier(), clang::Parser::parseMapTypeModifiers(), clang::Parser::ParseMSAsmIdentifier(), clang::Parser::ParseOpenMPParensExpr(), parseOpenMPReductionId(), parseOpenMPSimpleClause(), clang::Parser::ParseOpenMPVarList(), clang::Preprocessor::ParsePragmaPushOrPopMacro(), clang::Preprocessor::parseSimpleIntegerLiteral(), clang::Parser::ParseTopLevelDecl(), clang::Parser::ParseUnqualifiedId(), clang::format::IntegerLiteralSeparatorFixer::process(), ReadLineMarkerFlags(), ReadOriginalFileName(), clang::RewriteMacrosInInput(), clang::arcmt::trans::MigrationContext::rewritePropertyAttribute(), clang::CodeGen::CoverageMappingModuleGen::setUpCoverageCallbacks(), startsWithNewline(), clang::syntax::tokenize(), clang::Parser::TryAnnotateTypeOrScopeToken(), clang::Parser::TryAnnotateTypeOrScopeTokenAfterScopeSpec(), clang::Parser::TryConsumeToken(), and updateConsecutiveMacroArgTokens().

◆ getName()

const char * clang::Token::getName ( ) const
inline

Definition at line 174 of file Token.h.

References clang::tok::getTokenName().

◆ getObjCKeywordID()

tok::ObjCKeywordKind Token::getObjCKeywordID ( ) const

Return the ObjC keyword kind.

getObjCKeywordID - Return the ObjC keyword kind.

Definition at line 70 of file Lexer.cpp.

References getIdentifierInfo(), clang::IdentifierInfo::getObjCKeywordID(), and isAnnotation().

◆ getRawIdentifier()

StringRef clang::Token::getRawIdentifier ( ) const
inline

getRawIdentifier - For a raw identifier token (i.e., an identifier lexed in raw mode), returns a reference to the text substring in the buffer if known.

Definition at line 213 of file Token.h.

References getLength(), and is().

Referenced by clang::arcmt::trans::MigrationContext::addPropertyAttribute(), clang::Lexer::ComputePreamble(), clang::Lexer::getSpelling(), clang::Preprocessor::HandlePragmaModuleBuild(), LocPropertyAttribute(), and clang::arcmt::trans::MigrationContext::rewritePropertyAttribute().

◆ hasLeadingEmptyMacro()

bool clang::Token::hasLeadingEmptyMacro ( ) const
inline

Return true if this token has an empty macro before it.

Definition at line 299 of file Token.h.

References getFlag(), and LeadingEmptyMacro.

Referenced by clang::Preprocessor::LexHeaderName().

◆ hasLeadingSpace()

bool clang::Token::hasLeadingSpace ( ) const
inline

◆ hasPtrData()

bool clang::Token::hasPtrData ( ) const
inline

Definition at line 185 of file Token.h.

◆ hasUCN()

bool clang::Token::hasUCN ( ) const
inline

Returns true if this token contains a universal character name.

Definition at line 306 of file Token.h.

References getFlag(), and HasUCN.

Referenced by clang::Lexer::getSpelling(), and clang::Preprocessor::getSpelling().

◆ hasUDSuffix()

bool clang::Token::hasUDSuffix ( ) const
inline

Return true if this token is a string or character literal which has a ud-suffix.

Definition at line 303 of file Token.h.

References getFlag(), and HasUDSuffix.

Referenced by clang::TokenConcatenation::AvoidConcat(), EvaluateValue(), LexModuleNameComponent(), and clang::Preprocessor::ParsePragmaPushOrPopMacro().

◆ is()

bool clang::Token::is ( tok::TokenKind  K) const
inline

is/isNot - Predicates to check if this token is a specific kind, as in "if (Tok.is(tok::l_brace)) {...}".

Definition at line 99 of file Token.h.

Referenced by clang::Sema::ActOnNameClassifiedAsNonType(), clang::arcmt::trans::MigrationContext::addPropertyAttribute(), clang::TokenConcatenation::AvoidConcat(), buildMSAsmString(), clang::Sema::CheckConstraintExpression(), clang::Preprocessor::CheckEndOfDirective(), clang::Preprocessor::CheckMacroName(), clang::Sema::ClassifyName(), clang::Parser::ConsumeAnyToken(), clang::Preprocessor::CreateString(), diagnoseUnexpectedOperator(), clang::DoPrintPreprocessedInput(), EvaluateDefined(), EvaluateHasIncludeCommon(), EvaluateValue(), findDirectives(), findPreviousTokenKind(), getEofData(), getIdentifier(), getIdentifierInfo(), getRawIdentifier(), clang::Lexer::getSpelling(), clang::Preprocessor::getSpellingOfSingleCharacterNumericConstant(), clang::Preprocessor::HandleMicrosoftCommentPaste(), clang::Preprocessor::HandlePragmaHdrstop(), clang::Preprocessor::HandlePragmaIncludeAlias(), clang::Preprocessor::HandlePragmaModuleBuild(), clang::Preprocessor::HandlePragmaPoison(), highlightLines(), clang::format::FormatToken::is(), IsCommonTypo(), isMapModifier(), clang::TokenLexer::isNextTokenLParen(), isOneOf(), clang::TokenLexer::isParsingPreprocessorDirective(), isResultTypeOrTemplate(), isWideStringLiteralFromMacro(), clang::TokenLexer::Lex(), clang::Preprocessor::LexHeaderName(), LexModuleNameComponent(), LexRawTokensFromMainFile(), LocPropertyAttribute(), clang::Parser::ConditionDeclarationOrInitStatementState::markNotExpression(), clang::tooling::maybeExtendRange(), clang::Parser::MightBeCXXScopeToken(), clang::IncrementalParser::Parse(), clang::Parser::ParseAssignmentExpression(), clang::Parser::ParseConditionalExpression(), clang::Parser::ParseConstraintLogicalAndExpression(), clang::Parser::ParseConstraintLogicalOrExpression(), parseDeclareSimdClauses(), ParseLoopHintValue(), parseMapType(), clang::Parser::parseMapTypeModifiers(), clang::Parser::ParseMSAsmIdentifier(), clang::Parser::ParseOpenACCDirectiveDecl(), clang::Parser::ParseOpenACCDirectiveStmt(), parseOpenMPReductionId(), clang::Parser::ParseOpenMPReservedLocator(), clang::Parser::ParseOpenMPVarList(), clang::Preprocessor::parseSimpleIntegerLiteral(), clang::Parser::ParseTopLevelDecl(), clang::Parser::ParseUnqualifiedId(), clang::format::CommaSeparatedList::precomputeFormattingInfos(), PrintPreprocessedTokens(), clang::format::IntegerLiteralSeparatorFixer::process(), ReadLineMarkerFlags(), clang::Lexer::ReadToEndOfLine(), clang::RewriteIncludesInInput(), clang::RewriteMacrosInInput(), clang::arcmt::trans::MigrationContext::rewritePropertyAttribute(), setEofData(), setRawIdentifierData(), clang::Preprocessor::SkipTokensWhileUsingPCH(), clang::Parser::SkipUntil(), clang::MacroArgs::StringifyArgument(), clang::format::switchesFormatting(), clang::Parser::TryAnnotateTypeOrScopeToken(), and clang::Parser::TryAnnotateTypeOrScopeTokenAfterScopeSpec().

◆ isAnnotation()

bool clang::Token::isAnnotation ( ) const
inline

◆ isAnyIdentifier()

bool clang::Token::isAnyIdentifier ( ) const
inline

Return true if this is a raw identifier (when lexing in raw mode) or a non-keyword identifier (when lexing in non-raw mode).

Definition at line 110 of file Token.h.

References getKind(), and clang::tok::isAnyIdentifier().

◆ isAtStartOfLine()

bool clang::Token::isAtStartOfLine ( ) const
inline

◆ isEditorPlaceholder()

bool clang::Token::isEditorPlaceholder ( ) const
inline

Returns true if this token is an editor placeholder.

Editor placeholders are produced by the code-completion engine and are represented as characters between '<#' and '#>' in the source code. The lexer uses identifier tokens to represent placeholders.

Definition at line 320 of file Token.h.

References getFlag(), and IsEditorPlaceholder.

Referenced by clang::Parser::TryAnnotateTypeOrScopeToken().

◆ isExpandDisabled()

bool clang::Token::isExpandDisabled ( ) const
inline

Return true if this identifier token should never be expanded in the future, due to C99 6.10.3.4p2.

Definition at line 284 of file Token.h.

References DisableExpand, and getFlag().

Referenced by clang::Preprocessor::DumpToken().

◆ isLiteral()

bool clang::Token::isLiteral ( ) const
inline

◆ isNot()

bool clang::Token::isNot ( tok::TokenKind  K) const
inline

Definition at line 100 of file Token.h.

Referenced by clang::arcmt::trans::MigrationContext::addPropertyAttribute(), clang::MacroArgs::ArgNeedsPreexpansion(), clang::TokenConcatenation::AvoidConcat(), clang::Sema::ClassifyName(), clang::Preprocessor::DiscardUntilEndOfDirective(), DoPrintMacros(), dumpTokenInto(), EvaluateDefined(), EvaluateDirectiveSubExpr(), EvaluateFeatureLikeBuiltinMacro(), EvaluateHasIncludeCommon(), EvaluateValue(), clang::PrintDependencyDirectivesSourceMinimizerAction::ExecuteAction(), clang::DumpRawTokensAction::ExecuteAction(), clang::DumpTokensAction::ExecuteAction(), clang::PreprocessOnlyAction::ExecuteAction(), findDirectives(), clang::arcmt::trans::findSemiAfterLocation(), clang::MacroArgs::getArgLength(), getIdentifierInfo(), clang::format::getLastNonComment(), GetLineValue(), clang::Preprocessor::getSpelling(), clang::Preprocessor::HandleMicrosoftCommentPaste(), clang::Preprocessor::HandlePragmaDependency(), clang::Preprocessor::HandlePragmaHdrstop(), clang::Preprocessor::HandlePragmaIncludeAlias(), clang::Preprocessor::HandlePragmaModuleBuild(), clang::Preprocessor::HandlePragmaPoison(), clang::Preprocessor::LexHeaderName(), LexModuleName(), clang::Preprocessor::LexOnOffSwitch(), LexRawTokensFromMainFile(), LocPropertyAttribute(), clang::Parser::ConditionDeclarationOrInitStatementState::markNotExpression(), clang::IncrementalParser::Parse(), ParseAlignPragma(), parseDeclareSimdClauses(), ParseLoopHintValue(), clang::Parser::parseMapperModifier(), clang::Parser::parseMapTypeModifiers(), clang::Parser::ParseMSAsmIdentifier(), parseOpenMPSimpleClause(), clang::Parser::ParseOpenMPVarList(), clang::Preprocessor::ParsePragmaPushOrPopMacro(), parseStepSize(), clang::Parser::ParseUnqualifiedId(), clang::format::IntegerLiteralSeparatorFixer::process(), ReadOriginalFileName(), clang::RewriteIncludesInInput(), clang::RewriteMacrosInInput(), clang::arcmt::trans::MigrationContext::rewritePropertyAttribute(), clang::Parser::SkipUntil(), clang::MacroArgs::StringifyArgument(), clang::TokenRewriter::TokenRewriter(), and clang::Parser::TryConsumeToken().

◆ isObjCAtKeyword()

bool Token::isObjCAtKeyword ( tok::ObjCKeywordKind  objcKey) const

Return true if we have an ObjC keyword identifier.

isObjCAtKeyword - Return true if we have an ObjC keyword identifier.

Definition at line 61 of file Lexer.cpp.

References getIdentifierInfo(), and isAnnotation().

◆ isOneOf() [1/2]

bool clang::Token::isOneOf ( tok::TokenKind  K1,
tok::TokenKind  K2 
) const
inline

◆ isOneOf() [2/2]

template<typename... Ts>
bool clang::Token::isOneOf ( tok::TokenKind  K1,
Ts...  Ks 
) const
inline

Definition at line 104 of file Token.h.

References is(), and isOneOf().

◆ isRegularKeywordAttribute()

bool clang::Token::isRegularKeywordAttribute ( ) const
inline

Return true if the token is a keyword that is parsed in the same position as a standard attribute, but that has semantic meaning and so cannot be a true attribute.

Definition at line 126 of file Token.h.

References getKind(), and clang::tok::isRegularKeywordAttribute().

◆ isSimpleTypeSpecifier()

bool Token::isSimpleTypeSpecifier ( const LangOptions LangOpts) const

Determine whether the token kind starts a simple-type-specifier.

Definition at line 78 of file Lexer.cpp.

References getIdentifierInfo(), getKind(), and clang::IdentifierInfo::isKeyword().

◆ needsCleaning()

bool clang::Token::needsCleaning ( ) const
inline

◆ setAnnotationEndLoc()

void clang::Token::setAnnotationEndLoc ( SourceLocation  L)
inline

◆ setAnnotationRange()

void clang::Token::setAnnotationRange ( SourceRange  R)
inline

◆ setAnnotationValue()

void clang::Token::setAnnotationValue ( void *  val)
inline

Definition at line 238 of file Token.h.

References isAnnotation().

Referenced by clang::Preprocessor::EnterAnnotationToken(), and clang::ASTReader::ReadToken().

◆ setEofData()

void clang::Token::setEofData ( const void *  D)
inline

Definition at line 204 of file Token.h.

References is().

◆ setFlag()

void clang::Token::setFlag ( TokenFlags  Flag)
inline

Set the specified flag.

Definition at line 244 of file Token.h.

Referenced by clang::TokenLexer::Lex(), clang::ASTReader::ReadToken(), and setFlagValue().

◆ setFlagValue()

void clang::Token::setFlagValue ( TokenFlags  Flag,
bool  Val 
)
inline

Set a flag to either true or false.

Definition at line 267 of file Token.h.

References clearFlag(), and setFlag().

Referenced by clang::TokenLexer::Lex(), and clang::Preprocessor::LexHeaderName().

◆ setIdentifierInfo()

void clang::Token::setIdentifierInfo ( IdentifierInfo II)
inline

◆ setKind()

void clang::Token::setKind ( tok::TokenKind  K)
inline

◆ setLength()

void clang::Token::setLength ( unsigned  Len)
inline

◆ setLiteralData()

void clang::Token::setLiteralData ( const char *  Ptr)
inline

Definition at line 229 of file Token.h.

References isLiteral().

Referenced by clang::Preprocessor::CreateString().

◆ setLocation()

void clang::Token::setLocation ( SourceLocation  L)
inline

◆ setRawIdentifierData()

void clang::Token::setRawIdentifierData ( const char *  Ptr)
inline

Definition at line 217 of file Token.h.

References is().

Referenced by clang::Preprocessor::CreateString().

◆ startToken()

void clang::Token::startToken ( )
inline

◆ stringifiedInMacro()

bool clang::Token::stringifiedInMacro ( ) const
inline

Returns true if this token is formed by macro by stringizing or charizing operator.

Definition at line 310 of file Token.h.

References getFlag(), and StringifiedInMacro.

Referenced by isWideStringLiteralFromMacro().


The documentation for this class was generated from the following files: