clang-tools  10.0.0svn
PreprocessorTracker.cpp
Go to the documentation of this file.
1 //===--- PreprocessorTracker.cpp - Preprocessor tracking -*- C++ -*------===//
2 //
3 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
4 // See https://llvm.org/LICENSE.txt for license information.
5 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
6 //
7 //===--------------------------------------------------------------------===//
8 //
9 // The Basic Idea (Macro and Conditional Checking)
10 //
11 // Basically we install a PPCallbacks-derived object to track preprocessor
12 // activity, namely when a header file is entered/exited, when a macro
13 // is expanded, when "defined" is used, and when #if, #elif, #ifdef,
14 // and #ifndef are used. We save the state of macro and "defined"
15 // expressions in a map, keyed on a name/file/line/column quadruple.
16 // The map entries store the different states (values) that a macro expansion,
17 // "defined" expression, or condition expression has in the course of
18 // processing for the one location in the one header containing it,
19 // plus a list of the nested include stacks for the states. When a macro
20 // or "defined" expression evaluates to the same value, which is the
21 // desired case, only one state is stored. Similarly, for conditional
22 // directives, we save the condition expression states in a separate map.
23 //
24 // This information is collected as modularize compiles all the headers
25 // given to it to process. After all the compilations are performed,
26 // a check is performed for any entries in the maps that contain more
27 // than one different state, and for these an output message is generated.
28 //
29 // For example:
30 //
31 // (...)/SubHeader.h:11:5:
32 // #if SYMBOL == 1
33 // ^
34 // error: Macro instance 'SYMBOL' has different values in this header,
35 // depending on how it was included.
36 // 'SYMBOL' expanded to: '1' with respect to these inclusion paths:
37 // (...)/Header1.h
38 // (...)/SubHeader.h
39 // (...)/SubHeader.h:3:9:
40 // #define SYMBOL 1
41 // ^
42 // Macro defined here.
43 // 'SYMBOL' expanded to: '2' with respect to these inclusion paths:
44 // (...)/Header2.h
45 // (...)/SubHeader.h
46 // (...)/SubHeader.h:7:9:
47 // #define SYMBOL 2
48 // ^
49 // Macro defined here.
50 //
51 // The Basic Idea ('Extern "C/C++" {}' Or 'namespace {}') With Nested
52 // '#include' Checking)
53 //
54 // To check for '#include' directives nested inside 'Extern "C/C++" {}'
55 // or 'namespace {}' blocks, we keep track of the '#include' directives
56 // while running the preprocessor, and later during a walk of the AST
57 // we call a function to check for any '#include' directies inside
58 // an 'Extern "C/C++" {}' or 'namespace {}' block, given its source
59 // range.
60 //
61 // Design and Implementation Details (Macro and Conditional Checking)
62 //
63 // A PreprocessorTrackerImpl class implements the PreprocessorTracker
64 // interface. It uses a PreprocessorCallbacks class derived from PPCallbacks
65 // to track preprocessor activity, namely entering/exiting a header, macro
66 // expansions, use of "defined" expressions, and #if, #elif, #ifdef, and
67 // #ifndef conditional directives. PreprocessorTrackerImpl stores a map
68 // of MacroExpansionTracker objects keyed on a name/file/line/column
69 // value represented by a light-weight PPItemKey value object. This
70 // is the key top-level data structure tracking the values of macro
71 // expansion instances. Similarly, it stores a map of ConditionalTracker
72 // objects with the same kind of key, for tracking preprocessor conditional
73 // directives.
74 //
75 // The MacroExpansionTracker object represents one macro reference or use
76 // of a "defined" expression in a header file. It stores a handle to a
77 // string representing the unexpanded macro instance, a handle to a string
78 // representing the unpreprocessed source line containing the unexpanded
79 // macro instance, and a vector of one or more MacroExpansionInstance
80 // objects.
81 //
82 // The MacroExpansionInstance object represents one or more expansions
83 // of a macro reference, for the case where the macro expands to the same
84 // value. MacroExpansionInstance stores a handle to a string representing
85 // the expanded macro value, a PPItemKey representing the file/line/column
86 // where the macro was defined, a handle to a string representing the source
87 // line containing the macro definition, and a vector of InclusionPathHandle
88 // values that represents the hierarchies of include files for each case
89 // where the particular header containing the macro reference was referenced
90 // or included.
91 
92 // In the normal case where a macro instance always expands to the same
93 // value, the MacroExpansionTracker object will only contain one
94 // MacroExpansionInstance representing all the macro expansion instances.
95 // If a case was encountered where a macro instance expands to a value
96 // that is different from that seen before, or the macro was defined in
97 // a different place, a new MacroExpansionInstance object representing
98 // that case will be added to the vector in MacroExpansionTracker. If a
99 // macro instance expands to a value already seen before, the
100 // InclusionPathHandle representing that case's include file hierarchy
101 // will be added to the existing MacroExpansionInstance object.
102 
103 // For checking conditional directives, the ConditionalTracker class
104 // functions similarly to MacroExpansionTracker, but tracks an #if,
105 // #elif, #ifdef, or #ifndef directive in a header file. It stores
106 // a vector of one or two ConditionalExpansionInstance objects,
107 // representing the cases where the conditional expression evaluates
108 // to true or false. This latter object stores the evaluated value
109 // of the condition expression (a bool) and a vector of
110 // InclusionPathHandles.
111 //
112 // To reduce the instances of string and object copying, the
113 // PreprocessorTrackerImpl class uses a StringPool to save all stored
114 // strings, and defines a StringHandle type to abstract the references
115 // to the strings.
116 //
117 // PreprocessorTrackerImpl also maintains a list representing the unique
118 // headers, which is just a vector of StringHandle's for the header file
119 // paths. A HeaderHandle abstracts a reference to a header, and is simply
120 // the index of the stored header file path.
121 //
122 // A HeaderInclusionPath class abstracts a unique hierarchy of header file
123 // inclusions. It simply stores a vector of HeaderHandles ordered from the
124 // top-most header (the one from the header list passed to modularize) down
125 // to the header containing the macro reference. PreprocessorTrackerImpl
126 // stores a vector of these objects. An InclusionPathHandle typedef
127 // abstracts a reference to one of the HeaderInclusionPath objects, and is
128 // simply the index of the stored HeaderInclusionPath object. The
129 // MacroExpansionInstance object stores a vector of these handles so that
130 // the reporting function can display the include hierarchies for the macro
131 // expansion instances represented by that object, to help the user
132 // understand how the header was included. (A future enhancement might
133 // be to associate a line number for the #include directives, but I
134 // think not doing so is good enough for the present.)
135 //
136 // A key reason for using these opaque handles was to try to keep all the
137 // internal objects light-weight value objects, in order to reduce string
138 // and object copying overhead, and to abstract this implementation detail.
139 //
140 // The key data structures are built up while modularize runs the headers
141 // through the compilation. A PreprocessorTracker instance is created and
142 // passed down to the AST action and consumer objects in modularize. For
143 // each new compilation instance, the consumer calls the
144 // PreprocessorTracker's handleNewPreprocessorEntry function, which sets
145 // up a PreprocessorCallbacks object for the preprocessor. At the end of
146 // the compilation instance, the PreprocessorTracker's
147 // handleNewPreprocessorExit function handles cleaning up with respect
148 // to the preprocessing instance.
149 //
150 // The PreprocessorCallbacks object uses an overidden FileChanged callback
151 // to determine when a header is entered and exited (including exiting the
152 // header during #include directives). It calls PreprocessorTracker's
153 // handleHeaderEntry and handleHeaderExit functions upon entering and
154 // exiting a header. These functions manage a stack of header handles
155 // representing by a vector, pushing and popping header handles as headers
156 // are entered and exited. When a HeaderInclusionPath object is created,
157 // it simply copies this stack.
158 //
159 // The PreprocessorCallbacks object uses an overridden MacroExpands callback
160 // to track when a macro expansion is performed. It calls a couple of helper
161 // functions to get the unexpanded and expanded macro values as strings, but
162 // then calls PreprocessorTrackerImpl's addMacroExpansionInstance function to
163 // do the rest of the work. The getMacroExpandedString function uses the
164 // preprocessor's getSpelling to convert tokens to strings using the
165 // information passed to the MacroExpands callback, and simply concatenates
166 // them. It makes recursive calls to itself to handle nested macro
167 // definitions, and also handles function-style macros.
168 //
169 // PreprocessorTrackerImpl's addMacroExpansionInstance function looks for
170 // an existing MacroExpansionTracker entry in its map of MacroExampleTracker
171 // objects. If none exists, it adds one with one MacroExpansionInstance and
172 // returns. If a MacroExpansionTracker object already exists, it looks for
173 // an existing MacroExpansionInstance object stored in the
174 // MacroExpansionTracker object, one that matches the macro expanded value
175 // and the macro definition location. If a matching MacroExpansionInstance
176 // object is found, it just adds the current HeaderInclusionPath object to
177 // it. If not found, it creates and stores a new MacroExpantionInstance
178 // object. The addMacroExpansionInstance function calls a couple of helper
179 // functions to get the pre-formatted location and source line strings for
180 // the macro reference and the macro definition stored as string handles.
181 // These helper functions use the current source manager from the
182 // preprocessor. This is done in advance at this point in time because the
183 // source manager doesn't exist at the time of the reporting.
184 //
185 // For conditional check, the PreprocessorCallbacks class overrides the
186 // PPCallbacks handlers for #if, #elif, #ifdef, and #ifndef. These handlers
187 // call the addConditionalExpansionInstance method of
188 // PreprocessorTrackerImpl. The process is similar to that of macros, but
189 // with some different data and error messages. A lookup is performed for
190 // the conditional, and if a ConditionalTracker object doesn't yet exist for
191 // the conditional, a new one is added, including adding a
192 // ConditionalExpansionInstance object to it to represent the condition
193 // expression state. If a ConditionalTracker for the conditional does
194 // exist, a lookup is made for a ConditionalExpansionInstance object
195 // matching the condition expression state. If one exists, a
196 // HeaderInclusionPath is added to it. Otherwise a new
197 // ConditionalExpansionInstance entry is made. If a ConditionalTracker
198 // has two ConditionalExpansionInstance objects, it means there was a
199 // conflict, meaning the conditional expression evaluated differently in
200 // one or more cases.
201 //
202 // After modularize has performed all the compilations, it enters a phase
203 // of error reporting. This new feature adds to this reporting phase calls
204 // to the PreprocessorTracker's reportInconsistentMacros and
205 // reportInconsistentConditionals functions. These functions walk the maps
206 // of MacroExpansionTracker's and ConditionalTracker's respectively. If
207 // any of these objects have more than one MacroExpansionInstance or
208 // ConditionalExpansionInstance objects, it formats and outputs an error
209 // message like the example shown previously, using the stored data.
210 //
211 // A potential issue is that there is some overlap between the #if/#elif
212 // conditional and macro reporting. I could disable the #if and #elif,
213 // leaving just the #ifdef and #ifndef, since these don't overlap. Or,
214 // to make clearer the separate reporting phases, I could add an output
215 // message marking the phases.
216 //
217 // Design and Implementation Details ('Extern "C/C++" {}' Or
218 // 'namespace {}') With Nested '#include' Checking)
219 //
220 // We override the InclusionDirective in PPCallbacks to record information
221 // about each '#include' directive encountered during preprocessing.
222 // We co-opt the PPItemKey class to store the information about each
223 // '#include' directive, including the source file name containing the
224 // directive, the name of the file being included, and the source line
225 // and column of the directive. We store these object in a vector,
226 // after first check to see if an entry already exists.
227 //
228 // Later, while the AST is being walked for other checks, we provide
229 // visit handlers for 'extern "C/C++" {}' and 'namespace (name) {}'
230 // blocks, checking to see if any '#include' directives occurred
231 // within the blocks, reporting errors if any found.
232 //
233 // Future Directions
234 //
235 // We probably should add options to disable any of the checks, in case
236 // there is some problem with them, or the messages get too verbose.
237 //
238 // With the map of all the macro and conditional expansion instances,
239 // it might be possible to add to the existing modularize error messages
240 // (the second part referring to definitions being different), attempting
241 // to tie them to the last macro conflict encountered with respect to the
242 // order of the code encountered.
243 //
244 //===--------------------------------------------------------------------===//
245 
246 #include "clang/Lex/LexDiagnostic.h"
247 #include "PreprocessorTracker.h"
248 #include "clang/Lex/MacroArgs.h"
249 #include "clang/Lex/PPCallbacks.h"
250 #include "llvm/ADT/SmallSet.h"
251 #include "llvm/Support/StringPool.h"
252 #include "llvm/Support/raw_ostream.h"
253 #include "ModularizeUtilities.h"
254 
255 namespace Modularize {
256 
257 // Some handle types
258 typedef llvm::PooledStringPtr StringHandle;
259 
260 typedef int HeaderHandle;
261 const HeaderHandle HeaderHandleInvalid = -1;
262 
264 const InclusionPathHandle InclusionPathHandleInvalid = -1;
265 
266 // Some utility functions.
267 
268 // Get a "file:line:column" source location string.
269 static std::string getSourceLocationString(clang::Preprocessor &PP,
270  clang::SourceLocation Loc) {
271  if (Loc.isInvalid())
272  return std::string("(none)");
273  else
274  return Loc.printToString(PP.getSourceManager());
275 }
276 
277 // Get just the file name from a source location.
278 static std::string getSourceLocationFile(clang::Preprocessor &PP,
279  clang::SourceLocation Loc) {
280  std::string Source(getSourceLocationString(PP, Loc));
281  size_t Offset = Source.find(':', 2);
282  if (Offset == std::string::npos)
283  return Source;
284  return Source.substr(0, Offset);
285 }
286 
287 // Get just the line and column from a source location.
288 static void getSourceLocationLineAndColumn(clang::Preprocessor &PP,
289  clang::SourceLocation Loc, int &Line,
290  int &Column) {
291  clang::PresumedLoc PLoc = PP.getSourceManager().getPresumedLoc(Loc);
292  if (PLoc.isInvalid()) {
293  Line = 0;
294  Column = 0;
295  return;
296  }
297  Line = PLoc.getLine();
298  Column = PLoc.getColumn();
299 }
300 
301 // Retrieve source snippet from file image.
302 static std::string getSourceString(clang::Preprocessor &PP,
303  clang::SourceRange Range) {
304  clang::SourceLocation BeginLoc = Range.getBegin();
305  clang::SourceLocation EndLoc = Range.getEnd();
306  const char *BeginPtr = PP.getSourceManager().getCharacterData(BeginLoc);
307  const char *EndPtr = PP.getSourceManager().getCharacterData(EndLoc);
308  size_t Length = EndPtr - BeginPtr;
309  return llvm::StringRef(BeginPtr, Length).trim().str();
310 }
311 
312 // Retrieve source line from file image given a location.
313 static std::string getSourceLine(clang::Preprocessor &PP,
314  clang::SourceLocation Loc) {
315  const llvm::MemoryBuffer *MemBuffer =
316  PP.getSourceManager().getBuffer(PP.getSourceManager().getFileID(Loc));
317  const char *Buffer = MemBuffer->getBufferStart();
318  const char *BufferEnd = MemBuffer->getBufferEnd();
319  const char *BeginPtr = PP.getSourceManager().getCharacterData(Loc);
320  const char *EndPtr = BeginPtr;
321  while (BeginPtr > Buffer) {
322  if (*BeginPtr == '\n') {
323  BeginPtr++;
324  break;
325  }
326  BeginPtr--;
327  }
328  while (EndPtr < BufferEnd) {
329  if (*EndPtr == '\n') {
330  break;
331  }
332  EndPtr++;
333  }
334  size_t Length = EndPtr - BeginPtr;
335  return llvm::StringRef(BeginPtr, Length).str();
336 }
337 
338 // Retrieve source line from file image given a file ID and line number.
339 static std::string getSourceLine(clang::Preprocessor &PP, clang::FileID FileID,
340  int Line) {
341  const llvm::MemoryBuffer *MemBuffer = PP.getSourceManager().getBuffer(FileID);
342  const char *Buffer = MemBuffer->getBufferStart();
343  const char *BufferEnd = MemBuffer->getBufferEnd();
344  const char *BeginPtr = Buffer;
345  const char *EndPtr = BufferEnd;
346  int LineCounter = 1;
347  if (Line == 1)
348  BeginPtr = Buffer;
349  else {
350  while (Buffer < BufferEnd) {
351  if (*Buffer == '\n') {
352  if (++LineCounter == Line) {
353  BeginPtr = Buffer++ + 1;
354  break;
355  }
356  }
357  Buffer++;
358  }
359  }
360  while (Buffer < BufferEnd) {
361  if (*Buffer == '\n') {
362  EndPtr = Buffer;
363  break;
364  }
365  Buffer++;
366  }
367  size_t Length = EndPtr - BeginPtr;
368  return llvm::StringRef(BeginPtr, Length).str();
369 }
370 
371 // Get the string for the Unexpanded macro instance.
372 // The soureRange is expected to end at the last token
373 // for the macro instance, which in the case of a function-style
374 // macro will be a ')', but for an object-style macro, it
375 // will be the macro name itself.
376 static std::string getMacroUnexpandedString(clang::SourceRange Range,
377  clang::Preprocessor &PP,
378  llvm::StringRef MacroName,
379  const clang::MacroInfo *MI) {
380  clang::SourceLocation BeginLoc(Range.getBegin());
381  const char *BeginPtr = PP.getSourceManager().getCharacterData(BeginLoc);
382  size_t Length;
383  std::string Unexpanded;
384  if (MI->isFunctionLike()) {
385  clang::SourceLocation EndLoc(Range.getEnd());
386  const char *EndPtr = PP.getSourceManager().getCharacterData(EndLoc) + 1;
387  Length = (EndPtr - BeginPtr) + 1; // +1 is ')' width.
388  } else
389  Length = MacroName.size();
390  return llvm::StringRef(BeginPtr, Length).trim().str();
391 }
392 
393 // Get the expansion for a macro instance, given the information
394 // provided by PPCallbacks.
395 // FIXME: This doesn't support function-style macro instances
396 // passed as arguments to another function-style macro. However,
397 // since it still expands the inner arguments, it still
398 // allows modularize to effectively work with respect to macro
399 // consistency checking, although it displays the incorrect
400 // expansion in error messages.
401 static std::string getMacroExpandedString(clang::Preprocessor &PP,
402  llvm::StringRef MacroName,
403  const clang::MacroInfo *MI,
404  const clang::MacroArgs *Args) {
405  std::string Expanded;
406  // Walk over the macro Tokens.
407  for (const auto &T : MI->tokens()) {
408  clang::IdentifierInfo *II = T.getIdentifierInfo();
409  int ArgNo = (II && Args ? MI->getParameterNum(II) : -1);
410  if (ArgNo == -1) {
411  // This isn't an argument, just add it.
412  if (II == nullptr)
413  Expanded += PP.getSpelling(T); // Not an identifier.
414  else {
415  // Token is for an identifier.
416  std::string Name = II->getName().str();
417  // Check for nexted macro references.
418  clang::MacroInfo *MacroInfo = PP.getMacroInfo(II);
419  if (MacroInfo && (Name != MacroName))
420  Expanded += getMacroExpandedString(PP, Name, MacroInfo, nullptr);
421  else
422  Expanded += Name;
423  }
424  continue;
425  }
426  // We get here if it's a function-style macro with arguments.
427  const clang::Token *ResultArgToks;
428  const clang::Token *ArgTok = Args->getUnexpArgument(ArgNo);
429  if (Args->ArgNeedsPreexpansion(ArgTok, PP))
430  ResultArgToks = &(const_cast<clang::MacroArgs *>(Args))
431  ->getPreExpArgument(ArgNo, PP)[0];
432  else
433  ResultArgToks = ArgTok; // Use non-preexpanded Tokens.
434  // If the arg token didn't expand into anything, ignore it.
435  if (ResultArgToks->is(clang::tok::eof))
436  continue;
437  unsigned NumToks = clang::MacroArgs::getArgLength(ResultArgToks);
438  // Append the resulting argument expansions.
439  for (unsigned ArgumentIndex = 0; ArgumentIndex < NumToks; ++ArgumentIndex) {
440  const clang::Token &AT = ResultArgToks[ArgumentIndex];
441  clang::IdentifierInfo *II = AT.getIdentifierInfo();
442  if (II == nullptr)
443  Expanded += PP.getSpelling(AT); // Not an identifier.
444  else {
445  // It's an identifier. Check for further expansion.
446  std::string Name = II->getName().str();
447  clang::MacroInfo *MacroInfo = PP.getMacroInfo(II);
448  if (MacroInfo)
449  Expanded += getMacroExpandedString(PP, Name, MacroInfo, nullptr);
450  else
451  Expanded += Name;
452  }
453  }
454  }
455  return Expanded;
456 }
457 
458 namespace {
459 
460 // ConditionValueKind strings.
461 const char *
463  "(not evaluated)", "false", "true"
464 };
465 
466 bool operator<(const StringHandle &H1, const StringHandle &H2) {
467  const char *S1 = (H1 ? *H1 : "");
468  const char *S2 = (H2 ? *H2 : "");
469  int Diff = strcmp(S1, S2);
470  return Diff < 0;
471 }
472 bool operator>(const StringHandle &H1, const StringHandle &H2) {
473  const char *S1 = (H1 ? *H1 : "");
474  const char *S2 = (H2 ? *H2 : "");
475  int Diff = strcmp(S1, S2);
476  return Diff > 0;
477 }
478 
479 // Preprocessor item key.
480 //
481 // This class represents a location in a source file, for use
482 // as a key representing a unique name/file/line/column quadruplet,
483 // which in this case is used to identify a macro expansion instance,
484 // but could be used for other things as well.
485 // The file is a header file handle, the line is a line number,
486 // and the column is a column number.
487 class PPItemKey {
488 public:
489  PPItemKey(clang::Preprocessor &PP, StringHandle Name, HeaderHandle File,
490  clang::SourceLocation Loc)
491  : Name(Name), File(File) {
493  }
494  PPItemKey(StringHandle Name, HeaderHandle File, int Line, int Column)
495  : Name(Name), File(File), Line(Line), Column(Column) {}
496  PPItemKey(const PPItemKey &Other)
497  : Name(Other.Name), File(Other.File), Line(Other.Line),
498  Column(Other.Column) {}
499  PPItemKey() : File(HeaderHandleInvalid), Line(0), Column(0) {}
500  bool operator==(const PPItemKey &Other) const {
501  if (Name != Other.Name)
502  return false;
503  if (File != Other.File)
504  return false;
505  if (Line != Other.Line)
506  return false;
507  return Column == Other.Column;
508  }
509  bool operator<(const PPItemKey &Other) const {
510  if (Name < Other.Name)
511  return true;
512  else if (Name > Other.Name)
513  return false;
514  if (File < Other.File)
515  return true;
516  else if (File > Other.File)
517  return false;
518  if (Line < Other.Line)
519  return true;
520  else if (Line > Other.Line)
521  return false;
522  return Column < Other.Column;
523  }
524  StringHandle Name;
525  HeaderHandle File;
526  int Line;
527  int Column;
528 };
529 
530 // Header inclusion path.
531 class HeaderInclusionPath {
532 public:
533  HeaderInclusionPath(std::vector<HeaderHandle> HeaderInclusionPath)
534  : Path(HeaderInclusionPath) {}
535  HeaderInclusionPath(const HeaderInclusionPath &Other) : Path(Other.Path) {}
536  HeaderInclusionPath() {}
537  std::vector<HeaderHandle> Path;
538 };
539 
540 // Macro expansion instance.
541 //
542 // This class represents an instance of a macro expansion with a
543 // unique value. It also stores the unique header inclusion paths
544 // for use in telling the user the nested include path to the header.
545 class MacroExpansionInstance {
546 public:
547  MacroExpansionInstance(StringHandle MacroExpanded,
548  PPItemKey &DefinitionLocation,
549  StringHandle DefinitionSourceLine,
550  InclusionPathHandle H)
551  : MacroExpanded(MacroExpanded), DefinitionLocation(DefinitionLocation),
552  DefinitionSourceLine(DefinitionSourceLine) {
553  InclusionPathHandles.push_back(H);
554  }
555  MacroExpansionInstance() {}
556 
557  // Check for the presence of a header inclusion path handle entry.
558  // Return false if not found.
559  bool haveInclusionPathHandle(InclusionPathHandle H) {
560  for (auto I = InclusionPathHandles.begin(), E = InclusionPathHandles.end();
561  I != E; ++I) {
562  if (*I == H)
563  return true;
564  }
566  }
567  // Add a new header inclusion path entry, if not already present.
568  void addInclusionPathHandle(InclusionPathHandle H) {
569  if (!haveInclusionPathHandle(H))
570  InclusionPathHandles.push_back(H);
571  }
572 
573  // A string representing the macro instance after preprocessing.
574  StringHandle MacroExpanded;
575  // A file/line/column triplet representing the macro definition location.
577  // A place to save the macro definition line string.
578  StringHandle DefinitionSourceLine;
579  // The header inclusion path handles for all the instances.
580  std::vector<InclusionPathHandle> InclusionPathHandles;
581 };
582 
583 // Macro expansion instance tracker.
584 //
585 // This class represents one macro expansion, keyed by a PPItemKey.
586 // It stores a string representing the macro reference in the source,
587 // and a list of ConditionalExpansionInstances objects representing
588 // the unique values the condition expands to in instances of the header.
589 class MacroExpansionTracker {
590 public:
591  MacroExpansionTracker(StringHandle MacroUnexpanded,
592  StringHandle MacroExpanded,
593  StringHandle InstanceSourceLine,
594  PPItemKey &DefinitionLocation,
595  StringHandle DefinitionSourceLine,
596  InclusionPathHandle InclusionPathHandle)
597  : MacroUnexpanded(MacroUnexpanded),
598  InstanceSourceLine(InstanceSourceLine) {
599  addMacroExpansionInstance(MacroExpanded, DefinitionLocation,
600  DefinitionSourceLine, InclusionPathHandle);
601  }
602  MacroExpansionTracker() {}
603 
604  // Find a matching macro expansion instance.
605  MacroExpansionInstance *
606  findMacroExpansionInstance(StringHandle MacroExpanded,
607  PPItemKey &DefinitionLocation) {
608  for (auto I = MacroExpansionInstances.begin(),
609  E = MacroExpansionInstances.end();
610  I != E; ++I) {
611  if ((I->MacroExpanded == MacroExpanded) &&
612  (I->DefinitionLocation == DefinitionLocation)) {
613  return &*I; // Found.
614  }
615  }
616  return nullptr; // Not found.
617  }
618 
619  // Add a macro expansion instance.
620  void addMacroExpansionInstance(StringHandle MacroExpanded,
621  PPItemKey &DefinitionLocation,
622  StringHandle DefinitionSourceLine,
623  InclusionPathHandle InclusionPathHandle) {
624  MacroExpansionInstances.push_back(
625  MacroExpansionInstance(MacroExpanded, DefinitionLocation,
626  DefinitionSourceLine, InclusionPathHandle));
627  }
628 
629  // Return true if there is a mismatch.
630  bool hasMismatch() { return MacroExpansionInstances.size() > 1; }
631 
632  // A string representing the macro instance without expansion.
633  StringHandle MacroUnexpanded;
634  // A place to save the macro instance source line string.
635  StringHandle InstanceSourceLine;
636  // The macro expansion instances.
637  // If all instances of the macro expansion expand to the same value,
638  // This vector will only have one instance.
639  std::vector<MacroExpansionInstance> MacroExpansionInstances;
640 };
641 
642 // Conditional expansion instance.
643 //
644 // This class represents an instance of a condition exoression result
645 // with a unique value. It also stores the unique header inclusion paths
646 // for use in telling the user the nested include path to the header.
647 class ConditionalExpansionInstance {
648 public:
649  ConditionalExpansionInstance(clang::PPCallbacks::ConditionValueKind ConditionValue, InclusionPathHandle H)
650  : ConditionValue(ConditionValue) {
651  InclusionPathHandles.push_back(H);
652  }
653  ConditionalExpansionInstance() {}
654 
655  // Check for the presence of a header inclusion path handle entry.
656  // Return false if not found.
657  bool haveInclusionPathHandle(InclusionPathHandle H) {
658  for (auto I = InclusionPathHandles.begin(), E = InclusionPathHandles.end();
659  I != E; ++I) {
660  if (*I == H)
661  return true;
662  }
664  }
665  // Add a new header inclusion path entry, if not already present.
666  void addInclusionPathHandle(InclusionPathHandle H) {
667  if (!haveInclusionPathHandle(H))
668  InclusionPathHandles.push_back(H);
669  }
670 
671  // A flag representing the evaluated condition value.
672  clang::PPCallbacks::ConditionValueKind ConditionValue;
673  // The header inclusion path handles for all the instances.
674  std::vector<InclusionPathHandle> InclusionPathHandles;
675 };
676 
677 // Conditional directive instance tracker.
678 //
679 // This class represents one conditional directive, keyed by a PPItemKey.
680 // It stores a string representing the macro reference in the source,
681 // and a list of ConditionExpansionInstance objects representing
682 // the unique value the condition expression expands to in instances of
683 // the header.
684 class ConditionalTracker {
685 public:
686  ConditionalTracker(clang::tok::PPKeywordKind DirectiveKind,
687  clang::PPCallbacks::ConditionValueKind ConditionValue,
688  StringHandle ConditionUnexpanded,
689  InclusionPathHandle InclusionPathHandle)
690  : DirectiveKind(DirectiveKind), ConditionUnexpanded(ConditionUnexpanded) {
691  addConditionalExpansionInstance(ConditionValue, InclusionPathHandle);
692  }
693  ConditionalTracker() {}
694 
695  // Find a matching condition expansion instance.
696  ConditionalExpansionInstance *
697  findConditionalExpansionInstance(clang::PPCallbacks::ConditionValueKind ConditionValue) {
698  for (auto I = ConditionalExpansionInstances.begin(),
700  I != E; ++I) {
701  if (I->ConditionValue == ConditionValue) {
702  return &*I; // Found.
703  }
704  }
705  return nullptr; // Not found.
706  }
707 
708  // Add a conditional expansion instance.
709  void
710  addConditionalExpansionInstance(clang::PPCallbacks::ConditionValueKind ConditionValue,
711  InclusionPathHandle InclusionPathHandle) {
713  ConditionalExpansionInstance(ConditionValue, InclusionPathHandle));
714  }
715 
716  // Return true if there is a mismatch.
717  bool hasMismatch() { return ConditionalExpansionInstances.size() > 1; }
718 
719  // The kind of directive.
720  clang::tok::PPKeywordKind DirectiveKind;
721  // A string representing the macro instance without expansion.
722  StringHandle ConditionUnexpanded;
723  // The condition expansion instances.
724  // If all instances of the conditional expression expand to the same value,
725  // This vector will only have one instance.
726  std::vector<ConditionalExpansionInstance> ConditionalExpansionInstances;
727 };
728 
729 class PreprocessorTrackerImpl;
730 
731 // Preprocessor callbacks for modularize.
732 //
733 // This class derives from the Clang PPCallbacks class to track preprocessor
734 // actions, such as changing files and handling preprocessor directives and
735 // macro expansions. It has to figure out when a new header file is entered
736 // and left, as the provided handler is not particularly clear about it.
737 class PreprocessorCallbacks : public clang::PPCallbacks {
738 public:
739  PreprocessorCallbacks(PreprocessorTrackerImpl &ppTracker,
740  clang::Preprocessor &PP, llvm::StringRef rootHeaderFile)
741  : PPTracker(ppTracker), PP(PP), RootHeaderFile(rootHeaderFile) {}
742  ~PreprocessorCallbacks() override {}
743 
744  // Overridden handlers.
745  void InclusionDirective(clang::SourceLocation HashLoc,
746  const clang::Token &IncludeTok,
747  llvm::StringRef FileName, bool IsAngled,
748  clang::CharSourceRange FilenameRange,
749  const clang::FileEntry *File,
750  llvm::StringRef SearchPath,
751  llvm::StringRef RelativePath,
752  const clang::Module *Imported,
753  clang::SrcMgr::CharacteristicKind FileType) override;
754  void FileChanged(clang::SourceLocation Loc,
755  clang::PPCallbacks::FileChangeReason Reason,
756  clang::SrcMgr::CharacteristicKind FileType,
757  clang::FileID PrevFID = clang::FileID()) override;
758  void MacroExpands(const clang::Token &MacroNameTok,
759  const clang::MacroDefinition &MD, clang::SourceRange Range,
760  const clang::MacroArgs *Args) override;
761  void Defined(const clang::Token &MacroNameTok,
762  const clang::MacroDefinition &MD,
763  clang::SourceRange Range) override;
764  void If(clang::SourceLocation Loc, clang::SourceRange ConditionRange,
765  clang::PPCallbacks::ConditionValueKind ConditionResult) override;
766  void Elif(clang::SourceLocation Loc, clang::SourceRange ConditionRange,
767  clang::PPCallbacks::ConditionValueKind ConditionResult,
768  clang::SourceLocation IfLoc) override;
769  void Ifdef(clang::SourceLocation Loc, const clang::Token &MacroNameTok,
770  const clang::MacroDefinition &MD) override;
771  void Ifndef(clang::SourceLocation Loc, const clang::Token &MacroNameTok,
772  const clang::MacroDefinition &MD) override;
773 
774 private:
775  PreprocessorTrackerImpl &PPTracker;
776  clang::Preprocessor &PP;
777  std::string RootHeaderFile;
778 };
779 
780 // Preprocessor macro expansion item map types.
781 typedef std::map<PPItemKey, MacroExpansionTracker> MacroExpansionMap;
782 typedef std::map<PPItemKey, MacroExpansionTracker>::iterator
783 MacroExpansionMapIter;
784 
785 // Preprocessor conditional expansion item map types.
786 typedef std::map<PPItemKey, ConditionalTracker> ConditionalExpansionMap;
787 typedef std::map<PPItemKey, ConditionalTracker>::iterator
788 ConditionalExpansionMapIter;
789 
790 // Preprocessor tracker for modularize.
791 //
792 // This class stores information about all the headers processed in the
793 // course of running modularize.
794 class PreprocessorTrackerImpl : public PreprocessorTracker {
795 public:
796  PreprocessorTrackerImpl(llvm::SmallVector<std::string, 32> &Headers,
797  bool DoBlockCheckHeaderListOnly)
798  : BlockCheckHeaderListOnly(DoBlockCheckHeaderListOnly),
799  CurrentInclusionPathHandle(InclusionPathHandleInvalid),
800  InNestedHeader(false) {
801  // Use canonical header path representation.
802  for (llvm::ArrayRef<std::string>::iterator I = Headers.begin(),
803  E = Headers.end();
804  I != E; ++I) {
805  HeaderList.push_back(getCanonicalPath(*I));
806  }
807  }
808 
809  ~PreprocessorTrackerImpl() override {}
810 
811  // Handle entering a preprocessing session.
812  void handlePreprocessorEntry(clang::Preprocessor &PP,
813  llvm::StringRef rootHeaderFile) override {
814  HeadersInThisCompile.clear();
815  assert((HeaderStack.size() == 0) && "Header stack should be empty.");
816  pushHeaderHandle(addHeader(rootHeaderFile));
817  PP.addPPCallbacks(std::make_unique<PreprocessorCallbacks>(*this, PP,
818  rootHeaderFile));
819  }
820  // Handle exiting a preprocessing session.
821  void handlePreprocessorExit() override { HeaderStack.clear(); }
822 
823  // Handle include directive.
824  // This function is called every time an include directive is seen by the
825  // preprocessor, for the purpose of later checking for 'extern "" {}' or
826  // "namespace {}" blocks containing #include directives.
827  void handleIncludeDirective(llvm::StringRef DirectivePath, int DirectiveLine,
828  int DirectiveColumn,
829  llvm::StringRef TargetPath) override {
830  // If it's not a header in the header list, ignore it with respect to
831  // the check.
832  if (BlockCheckHeaderListOnly && !isHeaderListHeader(TargetPath))
833  return;
834  HeaderHandle CurrentHeaderHandle = findHeaderHandle(DirectivePath);
835  StringHandle IncludeHeaderHandle = addString(TargetPath);
836  for (std::vector<PPItemKey>::const_iterator I = IncludeDirectives.begin(),
837  E = IncludeDirectives.end();
838  I != E; ++I) {
839  // If we already have an entry for this directive, return now.
840  if ((I->File == CurrentHeaderHandle) && (I->Line == DirectiveLine))
841  return;
842  }
843  PPItemKey IncludeDirectiveItem(IncludeHeaderHandle, CurrentHeaderHandle,
844  DirectiveLine, DirectiveColumn);
845  IncludeDirectives.push_back(IncludeDirectiveItem);
846  }
847 
848  // Check for include directives within the given source line range.
849  // Report errors if any found. Returns true if no include directives
850  // found in block.
851  bool checkForIncludesInBlock(clang::Preprocessor &PP,
852  clang::SourceRange BlockSourceRange,
853  const char *BlockIdentifierMessage,
854  llvm::raw_ostream &OS) override {
855  clang::SourceLocation BlockStartLoc = BlockSourceRange.getBegin();
856  clang::SourceLocation BlockEndLoc = BlockSourceRange.getEnd();
857  // Use block location to get FileID of both the include directive
858  // and block statement.
859  clang::FileID FileID = PP.getSourceManager().getFileID(BlockStartLoc);
860  std::string SourcePath = getSourceLocationFile(PP, BlockStartLoc);
861  SourcePath = ModularizeUtilities::getCanonicalPath(SourcePath);
862  HeaderHandle SourceHandle = findHeaderHandle(SourcePath);
863  if (SourceHandle == -1)
864  return true;
865  int BlockStartLine, BlockStartColumn, BlockEndLine, BlockEndColumn;
866  bool returnValue = true;
867  getSourceLocationLineAndColumn(PP, BlockStartLoc, BlockStartLine,
868  BlockStartColumn);
869  getSourceLocationLineAndColumn(PP, BlockEndLoc, BlockEndLine,
870  BlockEndColumn);
871  for (std::vector<PPItemKey>::const_iterator I = IncludeDirectives.begin(),
872  E = IncludeDirectives.end();
873  I != E; ++I) {
874  // If we find an entry within the block, report an error.
875  if ((I->File == SourceHandle) && (I->Line >= BlockStartLine) &&
876  (I->Line < BlockEndLine)) {
877  returnValue = false;
878  OS << SourcePath << ":" << I->Line << ":" << I->Column << ":\n";
879  OS << getSourceLine(PP, FileID, I->Line) << "\n";
880  if (I->Column > 0)
881  OS << std::string(I->Column - 1, ' ') << "^\n";
882  OS << "error: Include directive within " << BlockIdentifierMessage
883  << ".\n";
884  OS << SourcePath << ":" << BlockStartLine << ":" << BlockStartColumn
885  << ":\n";
886  OS << getSourceLine(PP, BlockStartLoc) << "\n";
887  if (BlockStartColumn > 0)
888  OS << std::string(BlockStartColumn - 1, ' ') << "^\n";
889  OS << "The \"" << BlockIdentifierMessage << "\" block is here.\n";
890  }
891  }
892  return returnValue;
893  }
894 
895  // Handle entering a header source file.
896  void handleHeaderEntry(clang::Preprocessor &PP, llvm::StringRef HeaderPath) {
897  // Ignore <built-in> and <command-line> to reduce message clutter.
898  if (HeaderPath.startswith("<"))
899  return;
900  HeaderHandle H = addHeader(HeaderPath);
901  if (H != getCurrentHeaderHandle())
902  pushHeaderHandle(H);
903  // Check for nested header.
904  if (!InNestedHeader)
905  InNestedHeader = !HeadersInThisCompile.insert(H).second;
906  }
907 
908  // Handle exiting a header source file.
909  void handleHeaderExit(llvm::StringRef HeaderPath) {
910  // Ignore <built-in> and <command-line> to reduce message clutter.
911  if (HeaderPath.startswith("<"))
912  return;
913  HeaderHandle H = findHeaderHandle(HeaderPath);
914  HeaderHandle TH;
915  if (isHeaderHandleInStack(H)) {
916  do {
917  TH = getCurrentHeaderHandle();
918  popHeaderHandle();
919  } while ((TH != H) && (HeaderStack.size() != 0));
920  }
921  InNestedHeader = false;
922  }
923 
924  // Lookup/add string.
925  StringHandle addString(llvm::StringRef Str) { return Strings.intern(Str); }
926 
927  // Convert to a canonical path.
928  std::string getCanonicalPath(llvm::StringRef path) const {
929  std::string CanonicalPath(path);
930  std::replace(CanonicalPath.begin(), CanonicalPath.end(), '\\', '/');
931  return CanonicalPath;
932  }
933 
934  // Return true if the given header is in the header list.
935  bool isHeaderListHeader(llvm::StringRef HeaderPath) const {
936  std::string CanonicalPath = getCanonicalPath(HeaderPath);
937  for (llvm::ArrayRef<std::string>::iterator I = HeaderList.begin(),
938  E = HeaderList.end();
939  I != E; ++I) {
940  if (*I == CanonicalPath)
941  return true;
942  }
943  return false;
944  }
945 
946  // Get the handle of a header file entry.
947  // Return HeaderHandleInvalid if not found.
948  HeaderHandle findHeaderHandle(llvm::StringRef HeaderPath) const {
949  std::string CanonicalPath = getCanonicalPath(HeaderPath);
950  HeaderHandle H = 0;
951  for (auto I = HeaderPaths.begin(), E = HeaderPaths.end(); I != E;
952  ++I, ++H) {
953  if (**I == CanonicalPath)
954  return H;
955  }
956  return HeaderHandleInvalid;
957  }
958 
959  // Add a new header file entry, or return existing handle.
960  // Return the header handle.
961  HeaderHandle addHeader(llvm::StringRef HeaderPath) {
962  std::string CanonicalPath = getCanonicalPath(HeaderPath);
963  HeaderHandle H = findHeaderHandle(CanonicalPath);
964  if (H == HeaderHandleInvalid) {
965  H = HeaderPaths.size();
966  HeaderPaths.push_back(addString(CanonicalPath));
967  }
968  return H;
969  }
970 
971  // Return a header file path string given its handle.
972  StringHandle getHeaderFilePath(HeaderHandle H) const {
973  if ((H >= 0) && (H < (HeaderHandle)HeaderPaths.size()))
974  return HeaderPaths[H];
975  return StringHandle();
976  }
977 
978  // Returns a handle to the inclusion path.
979  InclusionPathHandle pushHeaderHandle(HeaderHandle H) {
980  HeaderStack.push_back(H);
981  return CurrentInclusionPathHandle = addInclusionPathHandle(HeaderStack);
982  }
983  // Pops the last header handle from the stack;
984  void popHeaderHandle() {
985  // assert((HeaderStack.size() != 0) && "Header stack already empty.");
986  if (HeaderStack.size() != 0) {
987  HeaderStack.pop_back();
988  CurrentInclusionPathHandle = addInclusionPathHandle(HeaderStack);
989  }
990  }
991  // Get the top handle on the header stack.
992  HeaderHandle getCurrentHeaderHandle() const {
993  if (HeaderStack.size() != 0)
994  return HeaderStack.back();
995  return HeaderHandleInvalid;
996  }
997 
998  // Check for presence of header handle in the header stack.
999  bool isHeaderHandleInStack(HeaderHandle H) const {
1000  for (auto I = HeaderStack.begin(), E = HeaderStack.end(); I != E; ++I) {
1001  if (*I == H)
1002  return true;
1003  }
1004  return false;
1005  }
1006 
1007  // Get the handle of a header inclusion path entry.
1008  // Return InclusionPathHandleInvalid if not found.
1009  InclusionPathHandle
1010  findInclusionPathHandle(const std::vector<HeaderHandle> &Path) const {
1011  InclusionPathHandle H = 0;
1012  for (auto I = InclusionPaths.begin(), E = InclusionPaths.end(); I != E;
1013  ++I, ++H) {
1014  if (I->Path == Path)
1015  return H;
1016  }
1017  return HeaderHandleInvalid;
1018  }
1019  // Add a new header inclusion path entry, or return existing handle.
1020  // Return the header inclusion path entry handle.
1021  InclusionPathHandle
1022  addInclusionPathHandle(const std::vector<HeaderHandle> &Path) {
1023  InclusionPathHandle H = findInclusionPathHandle(Path);
1024  if (H == HeaderHandleInvalid) {
1025  H = InclusionPaths.size();
1026  InclusionPaths.push_back(HeaderInclusionPath(Path));
1027  }
1028  return H;
1029  }
1030  // Return the current inclusion path handle.
1031  InclusionPathHandle getCurrentInclusionPathHandle() const {
1032  return CurrentInclusionPathHandle;
1033  }
1034 
1035  // Return an inclusion path given its handle.
1036  const std::vector<HeaderHandle> &
1037  getInclusionPath(InclusionPathHandle H) const {
1038  if ((H >= 0) && (H <= (InclusionPathHandle)InclusionPaths.size()))
1039  return InclusionPaths[H].Path;
1040  static std::vector<HeaderHandle> Empty;
1041  return Empty;
1042  }
1043 
1044  // Add a macro expansion instance.
1045  void addMacroExpansionInstance(clang::Preprocessor &PP, HeaderHandle H,
1046  clang::SourceLocation InstanceLoc,
1047  clang::SourceLocation DefinitionLoc,
1048  clang::IdentifierInfo *II,
1049  llvm::StringRef MacroUnexpanded,
1050  llvm::StringRef MacroExpanded,
1051  InclusionPathHandle InclusionPathHandle) {
1052  if (InNestedHeader)
1053  return;
1054  StringHandle MacroName = addString(II->getName());
1055  PPItemKey InstanceKey(PP, MacroName, H, InstanceLoc);
1056  PPItemKey DefinitionKey(PP, MacroName, H, DefinitionLoc);
1057  auto I = MacroExpansions.find(InstanceKey);
1058  // If existing instance of expansion not found, add one.
1059  if (I == MacroExpansions.end()) {
1060  std::string InstanceSourceLine =
1061  getSourceLocationString(PP, InstanceLoc) + ":\n" +
1062  getSourceLine(PP, InstanceLoc) + "\n";
1063  std::string DefinitionSourceLine =
1064  getSourceLocationString(PP, DefinitionLoc) + ":\n" +
1065  getSourceLine(PP, DefinitionLoc) + "\n";
1066  MacroExpansions[InstanceKey] = MacroExpansionTracker(
1067  addString(MacroUnexpanded), addString(MacroExpanded),
1068  addString(InstanceSourceLine), DefinitionKey,
1069  addString(DefinitionSourceLine), InclusionPathHandle);
1070  } else {
1071  // We've seen the macro before. Get its tracker.
1072  MacroExpansionTracker &CondTracker = I->second;
1073  // Look up an existing instance value for the macro.
1074  MacroExpansionInstance *MacroInfo =
1075  CondTracker.findMacroExpansionInstance(addString(MacroExpanded),
1076  DefinitionKey);
1077  // If found, just add the inclusion path to the instance.
1078  if (MacroInfo)
1079  MacroInfo->addInclusionPathHandle(InclusionPathHandle);
1080  else {
1081  // Otherwise add a new instance with the unique value.
1082  std::string DefinitionSourceLine =
1083  getSourceLocationString(PP, DefinitionLoc) + ":\n" +
1084  getSourceLine(PP, DefinitionLoc) + "\n";
1085  CondTracker.addMacroExpansionInstance(
1086  addString(MacroExpanded), DefinitionKey,
1087  addString(DefinitionSourceLine), InclusionPathHandle);
1088  }
1089  }
1090  }
1091 
1092  // Add a conditional expansion instance.
1093  void
1094  addConditionalExpansionInstance(clang::Preprocessor &PP, HeaderHandle H,
1095  clang::SourceLocation InstanceLoc,
1096  clang::tok::PPKeywordKind DirectiveKind,
1097  clang::PPCallbacks::ConditionValueKind ConditionValue,
1098  llvm::StringRef ConditionUnexpanded,
1099  InclusionPathHandle InclusionPathHandle) {
1100  // Ignore header guards, assuming the header guard is the only conditional.
1101  if (InNestedHeader)
1102  return;
1103  StringHandle ConditionUnexpandedHandle(addString(ConditionUnexpanded));
1104  PPItemKey InstanceKey(PP, ConditionUnexpandedHandle, H, InstanceLoc);
1105  auto I = ConditionalExpansions.find(InstanceKey);
1106  // If existing instance of condition not found, add one.
1107  if (I == ConditionalExpansions.end()) {
1108  std::string InstanceSourceLine =
1109  getSourceLocationString(PP, InstanceLoc) + ":\n" +
1110  getSourceLine(PP, InstanceLoc) + "\n";
1111  ConditionalExpansions[InstanceKey] =
1112  ConditionalTracker(DirectiveKind, ConditionValue,
1113  ConditionUnexpandedHandle, InclusionPathHandle);
1114  } else {
1115  // We've seen the conditional before. Get its tracker.
1116  ConditionalTracker &CondTracker = I->second;
1117  // Look up an existing instance value for the condition.
1118  ConditionalExpansionInstance *MacroInfo =
1119  CondTracker.findConditionalExpansionInstance(ConditionValue);
1120  // If found, just add the inclusion path to the instance.
1121  if (MacroInfo)
1122  MacroInfo->addInclusionPathHandle(InclusionPathHandle);
1123  else {
1124  // Otherwise add a new instance with the unique value.
1125  CondTracker.addConditionalExpansionInstance(ConditionValue,
1126  InclusionPathHandle);
1127  }
1128  }
1129  }
1130 
1131  // Report on inconsistent macro instances.
1132  // Returns true if any mismatches.
1133  bool reportInconsistentMacros(llvm::raw_ostream &OS) override {
1134  bool ReturnValue = false;
1135  // Walk all the macro expansion trackers in the map.
1136  for (auto I = MacroExpansions.begin(), E = MacroExpansions.end(); I != E;
1137  ++I) {
1138  const PPItemKey &ItemKey = I->first;
1139  MacroExpansionTracker &MacroExpTracker = I->second;
1140  // If no mismatch (only one instance value) continue.
1141  if (!MacroExpTracker.hasMismatch())
1142  continue;
1143  // Tell caller we found one or more errors.
1144  ReturnValue = true;
1145  // Start the error message.
1146  OS << *MacroExpTracker.InstanceSourceLine;
1147  if (ItemKey.Column > 0)
1148  OS << std::string(ItemKey.Column - 1, ' ') << "^\n";
1149  OS << "error: Macro instance '" << *MacroExpTracker.MacroUnexpanded
1150  << "' has different values in this header, depending on how it was "
1151  "included.\n";
1152  // Walk all the instances.
1153  for (auto IMT = MacroExpTracker.MacroExpansionInstances.begin(),
1154  EMT = MacroExpTracker.MacroExpansionInstances.end();
1155  IMT != EMT; ++IMT) {
1156  MacroExpansionInstance &MacroInfo = *IMT;
1157  OS << " '" << *MacroExpTracker.MacroUnexpanded << "' expanded to: '"
1158  << *MacroInfo.MacroExpanded
1159  << "' with respect to these inclusion paths:\n";
1160  // Walk all the inclusion path hierarchies.
1161  for (auto IIP = MacroInfo.InclusionPathHandles.begin(),
1162  EIP = MacroInfo.InclusionPathHandles.end();
1163  IIP != EIP; ++IIP) {
1164  const std::vector<HeaderHandle> &ip = getInclusionPath(*IIP);
1165  auto Count = (int)ip.size();
1166  for (int Index = 0; Index < Count; ++Index) {
1167  HeaderHandle H = ip[Index];
1168  OS << std::string((Index * 2) + 4, ' ') << *getHeaderFilePath(H)
1169  << "\n";
1170  }
1171  }
1172  // For a macro that wasn't defined, we flag it by using the
1173  // instance location.
1174  // If there is a definition...
1175  if (MacroInfo.DefinitionLocation.Line != ItemKey.Line) {
1176  OS << *MacroInfo.DefinitionSourceLine;
1177  if (MacroInfo.DefinitionLocation.Column > 0)
1178  OS << std::string(MacroInfo.DefinitionLocation.Column - 1, ' ')
1179  << "^\n";
1180  OS << "Macro defined here.\n";
1181  } else
1182  OS << "(no macro definition)"
1183  << "\n";
1184  }
1185  }
1186  return ReturnValue;
1187  }
1188 
1189  // Report on inconsistent conditional instances.
1190  // Returns true if any mismatches.
1191  bool reportInconsistentConditionals(llvm::raw_ostream &OS) override {
1192  bool ReturnValue = false;
1193  // Walk all the conditional trackers in the map.
1194  for (auto I = ConditionalExpansions.begin(),
1195  E = ConditionalExpansions.end();
1196  I != E; ++I) {
1197  const PPItemKey &ItemKey = I->first;
1198  ConditionalTracker &CondTracker = I->second;
1199  if (!CondTracker.hasMismatch())
1200  continue;
1201  // Tell caller we found one or more errors.
1202  ReturnValue = true;
1203  // Start the error message.
1204  OS << *HeaderPaths[ItemKey.File] << ":" << ItemKey.Line << ":"
1205  << ItemKey.Column << "\n";
1206  OS << "#" << getDirectiveSpelling(CondTracker.DirectiveKind) << " "
1207  << *CondTracker.ConditionUnexpanded << "\n";
1208  OS << "^\n";
1209  OS << "error: Conditional expression instance '"
1210  << *CondTracker.ConditionUnexpanded
1211  << "' has different values in this header, depending on how it was "
1212  "included.\n";
1213  // Walk all the instances.
1214  for (auto IMT = CondTracker.ConditionalExpansionInstances.begin(),
1215  EMT = CondTracker.ConditionalExpansionInstances.end();
1216  IMT != EMT; ++IMT) {
1217  ConditionalExpansionInstance &MacroInfo = *IMT;
1218  OS << " '" << *CondTracker.ConditionUnexpanded << "' expanded to: '"
1219  << ConditionValueKindStrings[MacroInfo.ConditionValue]
1220  << "' with respect to these inclusion paths:\n";
1221  // Walk all the inclusion path hierarchies.
1222  for (auto IIP = MacroInfo.InclusionPathHandles.begin(),
1223  EIP = MacroInfo.InclusionPathHandles.end();
1224  IIP != EIP; ++IIP) {
1225  const std::vector<HeaderHandle> &ip = getInclusionPath(*IIP);
1226  auto Count = (int)ip.size();
1227  for (int Index = 0; Index < Count; ++Index) {
1228  HeaderHandle H = ip[Index];
1229  OS << std::string((Index * 2) + 4, ' ') << *getHeaderFilePath(H)
1230  << "\n";
1231  }
1232  }
1233  }
1234  }
1235  return ReturnValue;
1236  }
1237 
1238  // Get directive spelling.
1239  static const char *getDirectiveSpelling(clang::tok::PPKeywordKind kind) {
1240  switch (kind) {
1241  case clang::tok::pp_if:
1242  return "if";
1243  case clang::tok::pp_elif:
1244  return "elif";
1245  case clang::tok::pp_ifdef:
1246  return "ifdef";
1247  case clang::tok::pp_ifndef:
1248  return "ifndef";
1249  default:
1250  return "(unknown)";
1251  }
1252  }
1253 
1254 private:
1255  llvm::SmallVector<std::string, 32> HeaderList;
1256  // Only do extern, namespace check for headers in HeaderList.
1258  llvm::StringPool Strings;
1259  std::vector<StringHandle> HeaderPaths;
1260  std::vector<HeaderHandle> HeaderStack;
1261  std::vector<HeaderInclusionPath> InclusionPaths;
1262  InclusionPathHandle CurrentInclusionPathHandle;
1263  llvm::SmallSet<HeaderHandle, 32> HeadersInThisCompile;
1264  std::vector<PPItemKey> IncludeDirectives;
1265  MacroExpansionMap MacroExpansions;
1266  ConditionalExpansionMap ConditionalExpansions;
1267  bool InNestedHeader;
1268 };
1269 
1270 } // namespace
1271 
1272 // PreprocessorTracker functions.
1273 
1274 // PreprocessorTracker desctructor.
1276 
1277 // Create instance of PreprocessorTracker.
1279  llvm::SmallVector<std::string, 32> &Headers,
1280  bool DoBlockCheckHeaderListOnly) {
1281  return new PreprocessorTrackerImpl(Headers, DoBlockCheckHeaderListOnly);
1282 }
1283 
1284 // Preprocessor callbacks for modularize.
1285 
1286 // Handle include directive.
1287 void PreprocessorCallbacks::InclusionDirective(
1288  clang::SourceLocation HashLoc, const clang::Token &IncludeTok,
1289  llvm::StringRef FileName, bool IsAngled,
1290  clang::CharSourceRange FilenameRange, const clang::FileEntry *File,
1291  llvm::StringRef SearchPath, llvm::StringRef RelativePath,
1292  const clang::Module *Imported, clang::SrcMgr::CharacteristicKind FileType) {
1293  int DirectiveLine, DirectiveColumn;
1294  std::string HeaderPath = getSourceLocationFile(PP, HashLoc);
1295  getSourceLocationLineAndColumn(PP, HashLoc, DirectiveLine, DirectiveColumn);
1296  PPTracker.handleIncludeDirective(HeaderPath, DirectiveLine, DirectiveColumn,
1297  FileName);
1298 }
1299 
1300 // Handle file entry/exit.
1301 void PreprocessorCallbacks::FileChanged(
1302  clang::SourceLocation Loc, clang::PPCallbacks::FileChangeReason Reason,
1303  clang::SrcMgr::CharacteristicKind FileType, clang::FileID PrevFID) {
1304  switch (Reason) {
1305  case EnterFile:
1306  PPTracker.handleHeaderEntry(PP, getSourceLocationFile(PP, Loc));
1307  break;
1308  case ExitFile: {
1309  const clang::FileEntry *F =
1310  PP.getSourceManager().getFileEntryForID(PrevFID);
1311  if (F)
1312  PPTracker.handleHeaderExit(F->getName());
1313  } break;
1314  case SystemHeaderPragma:
1315  case RenameFile:
1316  break;
1317  }
1318 }
1319 
1320 // Handle macro expansion.
1321 void PreprocessorCallbacks::MacroExpands(const clang::Token &MacroNameTok,
1322  const clang::MacroDefinition &MD,
1323  clang::SourceRange Range,
1324  const clang::MacroArgs *Args) {
1325  clang::SourceLocation Loc = Range.getBegin();
1326  // Ignore macro argument expansions.
1327  if (!Loc.isFileID())
1328  return;
1329  clang::IdentifierInfo *II = MacroNameTok.getIdentifierInfo();
1330  const clang::MacroInfo *MI = MD.getMacroInfo();
1331  std::string MacroName = II->getName().str();
1332  std::string Unexpanded(getMacroUnexpandedString(Range, PP, MacroName, MI));
1333  std::string Expanded(getMacroExpandedString(PP, MacroName, MI, Args));
1334  PPTracker.addMacroExpansionInstance(
1335  PP, PPTracker.getCurrentHeaderHandle(), Loc, MI->getDefinitionLoc(), II,
1336  Unexpanded, Expanded, PPTracker.getCurrentInclusionPathHandle());
1337 }
1338 
1339 void PreprocessorCallbacks::Defined(const clang::Token &MacroNameTok,
1340  const clang::MacroDefinition &MD,
1341  clang::SourceRange Range) {
1342  clang::SourceLocation Loc(Range.getBegin());
1343  clang::IdentifierInfo *II = MacroNameTok.getIdentifierInfo();
1344  const clang::MacroInfo *MI = MD.getMacroInfo();
1345  std::string MacroName = II->getName().str();
1346  std::string Unexpanded(getSourceString(PP, Range));
1347  PPTracker.addMacroExpansionInstance(
1348  PP, PPTracker.getCurrentHeaderHandle(), Loc,
1349  (MI ? MI->getDefinitionLoc() : Loc), II, Unexpanded,
1350  (MI ? "true" : "false"), PPTracker.getCurrentInclusionPathHandle());
1351 }
1352 
1353 void PreprocessorCallbacks::If(clang::SourceLocation Loc,
1354  clang::SourceRange ConditionRange,
1355  clang::PPCallbacks::ConditionValueKind ConditionResult) {
1356  std::string Unexpanded(getSourceString(PP, ConditionRange));
1357  PPTracker.addConditionalExpansionInstance(
1358  PP, PPTracker.getCurrentHeaderHandle(), Loc, clang::tok::pp_if,
1359  ConditionResult, Unexpanded, PPTracker.getCurrentInclusionPathHandle());
1360 }
1361 
1362 void PreprocessorCallbacks::Elif(clang::SourceLocation Loc,
1363  clang::SourceRange ConditionRange,
1364  clang::PPCallbacks::ConditionValueKind ConditionResult,
1365  clang::SourceLocation IfLoc) {
1366  std::string Unexpanded(getSourceString(PP, ConditionRange));
1367  PPTracker.addConditionalExpansionInstance(
1368  PP, PPTracker.getCurrentHeaderHandle(), Loc, clang::tok::pp_elif,
1369  ConditionResult, Unexpanded, PPTracker.getCurrentInclusionPathHandle());
1370 }
1371 
1372 void PreprocessorCallbacks::Ifdef(clang::SourceLocation Loc,
1373  const clang::Token &MacroNameTok,
1374  const clang::MacroDefinition &MD) {
1375  clang::PPCallbacks::ConditionValueKind IsDefined =
1376  (MD ? clang::PPCallbacks::CVK_True : clang::PPCallbacks::CVK_False );
1377  PPTracker.addConditionalExpansionInstance(
1378  PP, PPTracker.getCurrentHeaderHandle(), Loc, clang::tok::pp_ifdef,
1379  IsDefined, PP.getSpelling(MacroNameTok),
1380  PPTracker.getCurrentInclusionPathHandle());
1381 }
1382 
1383 void PreprocessorCallbacks::Ifndef(clang::SourceLocation Loc,
1384  const clang::Token &MacroNameTok,
1385  const clang::MacroDefinition &MD) {
1386  clang::PPCallbacks::ConditionValueKind IsNotDefined =
1387  (!MD ? clang::PPCallbacks::CVK_True : clang::PPCallbacks::CVK_False );
1388  PPTracker.addConditionalExpansionInstance(
1389  PP, PPTracker.getCurrentHeaderHandle(), Loc, clang::tok::pp_ifndef,
1390  IsNotDefined, PP.getSpelling(MacroNameTok),
1391  PPTracker.getCurrentInclusionPathHandle());
1392 }
1393 } // end namespace Modularize
StringHandle DefinitionSourceLine
SourceLocation Loc
&#39;#&#39; location in the include directive
StringHandle InstanceSourceLine
static std::string getCanonicalPath(llvm::StringRef FilePath)
Convert header path to canonical form.
static std::string getSourceString(clang::Preprocessor &PP, clang::SourceRange Range)
StringHandle Name
int Column
static void getSourceLocationLineAndColumn(clang::Preprocessor &PP, clang::SourceLocation Loc, int &Line, int &Column)
const HeaderHandle HeaderHandleInvalid
std::vector< HeaderHandle > Path
static std::string getSourceLocationFile(clang::Preprocessor &PP, clang::SourceLocation Loc)
static std::string replace(llvm::StringRef Haystack, llvm::StringRef Needle, llvm::StringRef Repl)
Definition: TestIndex.cpp:30
ModularizeUtilities class definition.
static std::string getMacroUnexpandedString(clang::SourceRange Range, clang::Preprocessor &PP, llvm::StringRef MacroName, const clang::MacroInfo *MI)
clang::tok::PPKeywordKind DirectiveKind
StringHandle MacroUnexpanded
static cl::opt< bool > BlockCheckHeaderListOnly("block-check-header-list-only", cl::init(false), cl::desc("Only warn if #include directives are inside extern or namespace" " blocks if the included header is in the header list."))
const InclusionPathHandle InclusionPathHandleInvalid
StringHandle MacroExpanded
std::vector< MacroExpansionInstance > MacroExpansionInstances
bool operator<(const Ref &L, const Ref &R)
Definition: Ref.h:58
bool operator==(const Ref &L, const Ref &R)
Definition: Ref.h:61
bool IsAngled
true if this was an include with angle brackets
static std::string getSourceLine(clang::Preprocessor &PP, clang::SourceLocation Loc)
StringHandle ConditionUnexpanded
PathRef FileName
std::vector< InclusionPathHandle > InclusionPathHandles
static const char *const ConditionValueKindStrings[]
clang::PPCallbacks::ConditionValueKind ConditionValue
std::vector< llvm::StringRef > Strings
size_t Offset
static PreprocessorTracker * create(llvm::SmallVector< std::string, 32 > &Headers, bool DoBlockCheckHeaderListOnly)
PPItemKey DefinitionLocation
static GeneratorRegistry::Add< MDGenerator > MD(MDGenerator::Format, "Generator for MD output.")
CharSourceRange Range
SourceRange for the file name.
llvm::Optional< std::string > getCanonicalPath(const FileEntry *F, const SourceManager &SourceMgr)
Get the canonical path of F.
Definition: SourceCode.cpp:625
Macro expansions and preprocessor conditional consistency checker.
unsigned Length
static std::string getSourceLocationString(clang::Preprocessor &PP, clang::SourceLocation Loc)
static std::string getMacroExpandedString(clang::Preprocessor &PP, llvm::StringRef MacroName, const clang::MacroInfo *MI, const clang::MacroArgs *Args)
Preprocessor tracker for modularize.
std::vector< ConditionalExpansionInstance > ConditionalExpansionInstances
const SymbolIndex * Index
Definition: Dexp.cpp:84
llvm::PooledStringPtr StringHandle