We now emit:
t.m:6:15: warning: field width should have type 'int', but argument has type 'unsigned int'
printf(STR, (unsigned) 1, 1);
^ ~~~~~~~~~~~~
t.m:3:18: note: instantiated from:
#define STR "abc%*ddef"
^
which has the correct location in the string literal in the note line.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@64936 91177308-0d34-0410-b5e6-96231b3b80d8
*end* of a macro instantiation, not the start of it. This is
really all about bug-for-bug compatibility with GCC, but not
doing this breaks the FreeBSD kernel.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@64604 91177308-0d34-0410-b5e6-96231b3b80d8
Now instead of just tracking the expansion history, also track the full
range of the macro that got replaced. For object-like macros, this doesn't
change anything. For _Pragma and function-like macros, this means we track
the locations of the ')'.
This is required for PR3579 because apparently GCC uses the line of the ')'
of a function-like macro as the location to expand __LINE__ to.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@64601 91177308-0d34-0410-b5e6-96231b3b80d8
a target.
Make Preprocessor.cpp define a new __INTPTR_TYPE__ macro based on this.
On linux/32, set intptr_t to int, instead of long. This fixes PR3563.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@64495 91177308-0d34-0410-b5e6-96231b3b80d8
wine sources. This was happening because HighlightMacros was
calling EnterMainFile multiple times on the same preprocessor
object and getting an assert due to the new #line stuff (the
file in question was bison output with #line directives).
The fix for this is to not reenter the file. Instead,
relex the tokens in raw mode, swizzle them a bit and repreprocess
the token stream. An added bonus of this is that rewrite macros
will now hilight the macro definition as well as its uses. Woo.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@64480 91177308-0d34-0410-b5e6-96231b3b80d8
to use this stat information in the PTH file using a 'StatSysCallCache' object.
Performance impact (Cocoa.h, PTH):
- number of stat calls reduces from 1230 to 425
- fsyntax-only: time improves by 4.2%
We can reduce the number of stat calls to almost zero by caching negative stat
calls and directory stat calls in the PTH file as well.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@64353 91177308-0d34-0410-b5e6-96231b3b80d8
actually *slightly* slower than the binary search. Since this is algorithmically
better, further performance tuning should be able to make this faster.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@64326 91177308-0d34-0410-b5e6-96231b3b80d8
line markers, including maintenance of the virtual include stack.
For something like this:
# 42 "bar.c" 1
# 142 "bar2.c" 1
#warning zappa
# 92 "bar.c" 2
#warning gonzo
# 102 "foo.c" 2
#warning bonkta
we now produce these three warnings:
#1:
In file included from foo.c:3:
In file included from bar.c:42:
bar2.c:143:2: warning: #warning zappa
#warning zappa
^
#2:
In file included from foo.c:3:
bar.c:92:2: warning: #warning gonzo
#warning gonzo
^
#3:
foo.c:102:2: warning: #warning bonkta
#warning bonkta
^
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@63722 91177308-0d34-0410-b5e6-96231b3b80d8
.def file for each library. This means that adding a diagnostic
to sema doesn't require all the other libraries to be rebuilt.
Patch by Anders Johnsen!
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@63111 91177308-0d34-0410-b5e6-96231b3b80d8
as reported to the user and as manipulated by #line. This is what __FILE__,
__INCLUDE_LEVEL__, diagnostics and other things should follow (but not
dependency generation!).
This patch also includes several cleanups along the way:
- SourceLocation now has a dump method, and several other places
that did similar things now use it.
- I cleaned up some code in AnalysisConsumer, but it should probably be
simplified further now that NamedDecl is better.
- TextDiagnosticPrinter is now simplified and cleaned up a bit.
This patch is a prerequisite for #line, but does not actually provide
any #line functionality.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@63098 91177308-0d34-0410-b5e6-96231b3b80d8
Performance impact for -fsyntax-only on Cocoa.h (with Cocoa.h in the PTH file):
- PTH generation time improves by 5%
- PTH reading improves by 0.3%.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@63072 91177308-0d34-0410-b5e6-96231b3b80d8
instantiation history in an effort to speed up c99-intconst-1.c.
Now that multiple nested instantiations are allowed, we just
make them and don't pay the cost of lookups. With the other
changes that went in before this, reverting this is actually
a speedup for c99-intconst-1.c, speeding it up from 1.96s to 1.80s,
and preserves much better loc info.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@63036 91177308-0d34-0410-b5e6-96231b3b80d8
Token now has a class of kinds for "literals", which include
numeric constants, strings, etc. These tokens can optionally have
a pointer to the start of the token in the lexer buffer. This
makes it faster to get spelling and do other gymnastics, because we
don't have to go through source locations.
This change is performance neutral, but will make other changes
more feasible down the road.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@63028 91177308-0d34-0410-b5e6-96231b3b80d8
This reduces fsyntax-only time on c99-intconst-1.c from 2.43s down to
2.01s (20%), reducing the number of fileid lookups from 2529040 linear
and 64771121 binary to 5625902 linear and 4151182 binary.
This knocks getFileID down to only 4.6% of compile time on this testcase.
At this point, malloc/free is over 35% of compile time, primarily allocating
MacroArgs objects and their argument preexpansion vectors.
I don't feel like malloc avoiding right now, so I'm just going to call
this good.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62994 91177308-0d34-0410-b5e6-96231b3b80d8
of a macro. Since these tokens may themselves be from macro
expansions, we need to resolve down to the spelling loc when the
macro ends up being instantiated. Instead of resolving this for
each token expanded from the macro definition, just do it once when
the macro is defined. This speeds up clang on c99-intconst-1.c from
2.66s to 2.43s (9.5%), reducing the FileID lookups from 407244 linear and
114175649 binary to 2529040 linear and 64771121 binary.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62993 91177308-0d34-0410-b5e6-96231b3b80d8
per token lexed from it. This speeds up clang on c99-intconst-1.c from
the GCC testsuite from 3.64s to 2.66s (36%). This reduces the number of
binary search FileID lookups from 251570522 to 114175649 on this testcase.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62992 91177308-0d34-0410-b5e6-96231b3b80d8
ground work for implementing #line, and fixes the "out of macro ID's"
problem.
There is nothing particularly tricky about the code, other than the
very performance sensitive SourceManager::getFileID() method.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62978 91177308-0d34-0410-b5e6-96231b3b80d8
Refactor how the preprocessor changes a token from being an tok::identifier to a
keyword (e.g. tok::kw_for). Instead of doing this in HandleIdentifier, hoist this
common case out into the caller, so that every keyword doesn't have to go through
HandleIdentifier. This drops time in HandleIdentifier from 1.25ms to .62ms, and
speeds up clang -Eonly with PTH by about 1%.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62855 91177308-0d34-0410-b5e6-96231b3b80d8
tells us whether Preprocessor::HandleIdentifier needs to be called.
Because this method is only rarely needed, this saves a call and a
bunch of random checks. This drops the time in HandleIdentifier
from 3.52ms to .98ms on cocoa.h on my machine.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62675 91177308-0d34-0410-b5e6-96231b3b80d8
Changes to IdentifierTable:
- High-level summary: StringMap never owns IdentifierInfos. It just
references them.
- The string map now has StringMapEntry<IdentifierInfo*> instead of
StringMapEntry<IdentifierInfo>. The IdentifierInfo object is
allocated using the same bump pointer allocator as used by the
StringMap.
Changes to IdentifierInfo:
- Added an extra pointer to point to the
StringMapEntry<IdentifierInfo*> in the string map. This pointer
will be null if the IdentifierInfo* is *only* used by the PTHLexer
(that is it isn't in the StringMap).
Algorithmic changes:
- Non-PTH case:
IdentifierInfo::get() will always consult the StringMap first to
see if we have an IdentifierInfo object. If that StringMapEntry
references a null pointer, we allocate a new one from the BumpPtrAllocator
and update the reference in the StringMapEntry.
- PTH case:
We do the same lookup as with the non-PTH case, but if we don't get
a hit in the StringMap we do a secondary lookup in the PTHManager for
the IdentifierInfo. If we don't find an IdentifierInfo we create a
new one as in the non-PTH case. If we do find and IdentifierInfo
in the PTHManager, we update the StringMapEntry to refer to it so
that the IdentifierInfo will be found on the next StringMap lookup.
This way we only do a binary search in the PTH file at most once
for a given IdentifierInfo. This greatly speeds things up for source
files containing a non-trivial amount of code.
Performance impact:
While these changes do add some extra indirection in
IdentifierTable to access an IdentifierInfo*, I saw speedups even
in the non-PTH case as well.
Non-PTH: For -fsyntax-only on Cocoa.h, we see a 6% speedup.
PTH (with Cocoa.h in token cache): 11% speedup.
I also did an experiment where we did -fsyntax-only on a source file
including a large header and Cocoa.h, but the token cache did not
contain the larger header. For this file, we were seeing a performance
*regression* when using PTH of 3% over non-PTH. Now we are seeing
a performance improvement of 9%!
Tests:
The serialization tests are now failing. I looked at this extensively,
and I my belief is that this change is unmasking a bug rather than
introducing a new one. I have disabled the serialization tests for now.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62636 91177308-0d34-0410-b5e6-96231b3b80d8
instead of assembling from bytes. This speeds up -Eonly PTH reading
of cocoa.h by about 2ms, which is 4.2%.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62447 91177308-0d34-0410-b5e6-96231b3b80d8
internally. This is just a cleanup that reduces the need to cast to
unsigned char before assembling a larger integer.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62442 91177308-0d34-0410-b5e6-96231b3b80d8
the chunk ID not the file ID. This exposes problems in
TextDiagnosticPrinter where it should have been using the canonical
file ID but wasn't. Fix these along the way.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62427 91177308-0d34-0410-b5e6-96231b3b80d8
method. This lets us clean up the interface and make it more obvious that
this method is *really really* _Pragma specific.
Note that _Pragma handling uglifies the Lexer in the critical path. It would
be very interesting to consider making _Pragma remapping be a new special
lexer class of its own.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62425 91177308-0d34-0410-b5e6-96231b3b80d8
"FileID" a concept that is now enforced by the compiler's type checker
instead of yet-another-random-unsigned floating around.
This is an important distinction from the "FileID" currently tracked by
SourceLocation. *That* FileID may refer to the start of a file or to a
chunk within it. The new FileID *only* refers to the file (and its
#include stack and eventually #line data), it cannot refer to a chunk.
FileID is a completely opaque datatype to all clients, only SourceManager
is allowed to poke and prod it.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62407 91177308-0d34-0410-b5e6-96231b3b80d8
content cache directly. Content cache has a 1-1 mapping with fileentries,
whereas multiple FileIDs can be the same FileEntry.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62401 91177308-0d34-0410-b5e6-96231b3b80d8
if warnings in system headers are disabled. isIdenticalTo can end up
calling the expensive getSpelling method, and other bad stuff and is
completely unneeded if the warning will be discarded anyway. rdar://6502956
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62347 91177308-0d34-0410-b5e6-96231b3b80d8
into its ctor. Also, make it handle validity checking of pascal
strings instead of making clients do it.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62332 91177308-0d34-0410-b5e6-96231b3b80d8
the "physical" location of tokens, refer to the "spelling" location.
This is more concrete and useful, tokens aren't really physical objects!
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62309 91177308-0d34-0410-b5e6-96231b3b80d8
- IdentifierInfo can now (optionally) have its string data not be
co-located with itself. This is for use with PTH. This aspect is a
little gross, as getName() and getLength() now make assumptions
about a possible alternate representation of IdentifierInfo.
Perhaps we should make IdentifierInfo have virtual methods?
IdentifierTable:
- Added class "IdentifierInfoLookup" that can be used by
IdentifierTable to perform "string -> IdentifierInfo" lookups using
an auxilliary data structure. This is used by PTH.
- Perform tests show that IdentifierTable::get() does not slow down
because of the extra check for the IdentiferInfoLookup object (the
regular StringMap lookup does enough work to mitigate the impact of
an extra null pointer check).
- The upshot is that now that some IdentifierInfo objects might be
owned by the IdentiferInfoLookup object. This should be reviewed.
PTH:
- Modified PTHManager::GetIdentifierInfo to *not* insert entries in
IdentifierTable's string map, and instead create IdentifierInfo
objects on the fly when mapping from persistent IDs to
IdentifierInfos. This saves a ton of work with string copies,
hashing, and StringMap lookup and resizing. This change was
motivated because when processing source files in the PTH cache we
don't need to do any string -> IdentifierInfo lookups.
- PTHManager now subclasses IdentifierInfoLookup, allowing clients of
IdentifierTable to transparently use IdentifierInfo objects managed
by the PTH file. PTHManager resolves "string -> IdentifierInfo"
queries by doing a binary search over a sorted table of identifier
strings in the PTH file (the exact algorithm we use can be changed
as needed).
These changes lead to the following performance changes when using PTH on Cocoa.h:
- fsyntax-only: 10% performance improvement
- Eonly: 30% performance improvement
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62273 91177308-0d34-0410-b5e6-96231b3b80d8
lexical order of the corresponding identifier strings. This will be used for a
forthcoming optimization. This slows down PTH generation time by 7%. We can
revert this change if the optimization proves to not be valuable.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62248 91177308-0d34-0410-b5e6-96231b3b80d8
- Use canonical FileID when using getSpelling() caching. This
addresses some cache misses we were seeing with -fsyntax-only on
Cocoa.h
- Added Preprocessor::getPhysicalCharacterAt() utility method for
clients to grab the first character at a specified sourcelocation.
This uses the PTH spelling cache.
- Modified Sema::ActOnNumericConstant() to use
Preprocessor::getPhysicalCharacterAt() instead of
SourceManager::getCharacterData() (to get PTH hits).
These changes cause -fsyntax-only to not page in any sources from
Cocoa.h. We see a speedup of 27%.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62193 91177308-0d34-0410-b5e6-96231b3b80d8
- Refactor caching logic into a helper class PTHSpellingSearch
- Allow "random accesses" in the spelling cache, thus catching the remaining
cases where 'getSpelling' wasn't hitting the PTH cache
For -Eonly, PTH, Cocoa.h:
- This reduces wall time by 3% (user time unchanged, sys time reduced)
- This reduces the amount of paged source by 1112K.
The remaining 1112K still being paged in is from somewhere else
(investigating).
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62009 91177308-0d34-0410-b5e6-96231b3b80d8
performance gain. Here's what we see for -Eonly on Cocoa.h (using PTH):
- wall time decreases by 21% (26% speedup overall)
- system time decreases by 35%
- user time decreases by 6%
These reductions are due to not paging source files just to get spellings for
literals. The solution in place doesn't appear to be 100% yet, as we still see
some of the pages for source files getting mapped in. Using -print-stats, we see
that SourceManager maps in 7179K less bytes of source text (reduction of 75%).
Will investigate why the remaining 25% are getting paged in.
With these changes, here's how PTH compares to non-PTH on Cocoa.h:
-Eonly: PTH takes 64% of the time as non-PTH (54% speedup)
-fsyntax-only: PTH takes 89% of the time as non-PTH (11% speedup)
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@61913 91177308-0d34-0410-b5e6-96231b3b80d8
- Added stub PTHLexer::getSpelling() that will be used for fetching cached
spellings from the PTH file. This doesn't do anything yet.
- Added a hook in Preprocessor::getSpelling() to call PTHLexer::getSpelling()
when using a PTHLexer.
- Updated PTHLexer to read the offsets of spelling tables in the PTH file.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@61911 91177308-0d34-0410-b5e6-96231b3b80d8
- This reduces the PTH size for Cocoa.h by 7%.
- The increases PTH -Eonly speed for Cocoa.h by 0.8%.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@61377 91177308-0d34-0410-b5e6-96231b3b80d8
- Encode the token length with 2 bytes instead of 4.
- This reduces the size of the .pth file for Cocoa.h by 12%.
- This speeds up PTH time (-Eonly) on Cocoa.h by 1.6%.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@61364 91177308-0d34-0410-b5e6-96231b3b80d8
- In PTHLexer::Lex read all of the token data from PTH file before
constructing the token. The idea is to enhance locality.
- Do not use Read8/Read32 in PTHLexer::Lex. Inline these operations manually.
- Change PTHManager::ReadIdentifierInfo() to PTHManager::GetIdentifierInfo().
They are functionally the same except that PTHLexer::Lex() reads the
persistent id.
These changes result in a 3.3% speedup for PTH on Cocoa.h (-Eonly).
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@61363 91177308-0d34-0410-b5e6-96231b3b80d8
- Embed 'eom' tokens in PTH file.
- Use embedded 'eom' tokens to not lazily generate them in the PTHLexer.
This means that PTHLexer can always advance to the next token after
reading a token (instead of buffering tokens using a copy).
- Moved logic of 'ReadToken' into Lex. GetToken & ReadToken no longer exist.
- These changes result in a 3.3% speedup (-Eonly) on Cocoa.h.
- The code is a little gross. Many cleanups are possible and should be done.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@61360 91177308-0d34-0410-b5e6-96231b3b80d8
- Added a side-table per each token-cached file with the preprocessor conditional stack. This tracks what #if's are matched with what #endifs and where their respective tokens are in the PTH file. This will allow for quick skipping of excluded conditional branches in the Preprocessor.
- Performance testing shows the addition of this information (without actually utilizing it) leads to no performance regressions.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@60911 91177308-0d34-0410-b5e6-96231b3b80d8
- Added virtual method 'getSourceLocation()' (no arguments) that gets the location of the next "observable" location (e.g., next character, next token).
PPLexerChange.cpp:
- Implemented FIXME by using PreprocessorLexer::getSourceLocation() to get the location in the file we are returning to after lexing a #included file. This appears to be slightly faster than having the branch (i.e., 'if(CurLexer)'). It's also not a really hot part of the Preprocessor.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@60860 91177308-0d34-0410-b5e6-96231b3b80d8
Use an array instead of a DenseMap to cache persistent IDs -> IdentifierInfo*. This leads to a 4% speedup at -fsyntax-only using PTH.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@60452 91177308-0d34-0410-b5e6-96231b3b80d8
Chris Lattner
Fariborz Jahanian
Ted Kremenek
Mike Stump
Eli Friedman
Daniel Dunbar
Steve Naroff