Bug 8227 dreftool - rickg's phase 1 preventative crash maintenance
r=bernd
This is dreftool, resurrected with the help of kherron, bernd, and myself.
Licensing has been switched per http://bugzilla.mozilla.org/show_bug.cgi?id=8227#c45
to MPL/GPL/LGPL.
Old copies of xpcom classes have been removed, dreftool now uses the normal ones.
New:
* Builds on Linux and uses NSPR, so it should be possible to use it on most
systems.
* Member variables are screened.
* Tries to recognize NS_ENSURE early outs.
* Can parse a single file.
* The summary can indicate what it was told to parse.
* Links to LXR can be generated. Using this feature is not recommended as pieces of
mozilla tend to be on branches and the links will be bad. If for non branched content,
the links will drift as the files are changed.
* Links to Bonsai against the file revision will be generated if CVS records are
detected. This is good, unless you have local modifications in your tree.
Known issues:
* Aliasing isn't detected, so
|a = new something; *foo = a; if (!*foo) return NS_ERROR_FAILURE;| is flagged.
e.x.
http://bonsai.mozilla.org/cvsblame.cgi?file=mozilla/accessible/src/html/nsHTMLSelectAccessible.cpp&rev=1.33&mark=920-921,922,923,925#920
* Assigning to dereferenced objects
|*foo = new something| is not quite correctly handled (perhaps it's misinterpreted as
|foo = new something|. e.x.
http://bonsai.mozilla.org/cvsblame.cgi?file=/mozilla/extensions/transformiix/source/xslt/txStandaloneXSLTProcessor.cpp&rev=1.11&mark=88,92,96,99,84#91
* Various other allocators aren't caught: strdup and co., nsCRT::, nsMemory::, ...
* There's no way to provide a user defined list of allocators.
People should of course not file bugs simply based on dreftool, they should file bugs
after verifying that the complaints from dreftool are valid.
2003-11-13 01:38:24 +03:00
|
|
|
/* ***** BEGIN LICENSE BLOCK *****
|
|
|
|
* Version: MPL 1.1/GPL 2.0/LGPL 2.1
|
|
|
|
*
|
|
|
|
* The contents of this file are subject to the Mozilla Public License Version
|
|
|
|
* 1.1 (the "License"); you may not use this file except in compliance with
|
|
|
|
* the License. You may obtain a copy of the License at
|
|
|
|
* http://www.mozilla.org/MPL/
|
|
|
|
*
|
|
|
|
* Software distributed under the License is distributed on an "AS IS" basis,
|
|
|
|
* WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
|
|
|
|
* for the specific language governing rights and limitations under the
|
|
|
|
* License.
|
|
|
|
*
|
|
|
|
* The Original Code is Dreftool.
|
|
|
|
*
|
|
|
|
* The Initial Developer of the Original Code is
|
|
|
|
* Rick Gessner <rick@gessner.com>.
|
|
|
|
* Portions created by the Initial Developer are Copyright (C) 1998
|
|
|
|
* the Initial Developer. All Rights Reserved.
|
|
|
|
*
|
|
|
|
* Contributor(s):
|
|
|
|
* Kenneth Herron <kjh-5727@comcast.net>
|
|
|
|
* Bernd Mielke <bmlk@gmx.de>
|
|
|
|
*
|
|
|
|
* Alternatively, the contents of this file may be used under the terms of
|
|
|
|
* either the GNU General Public License Version 2 or later (the "GPL"), or
|
|
|
|
* the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
|
|
|
|
* in which case the provisions of the GPL or the LGPL are applicable instead
|
|
|
|
* of those above. If you wish to allow use of your version of this file only
|
|
|
|
* under the terms of either the GPL or the LGPL, and not to allow others to
|
|
|
|
* use your version of this file under the terms of the MPL, indicate your
|
|
|
|
* decision by deleting the provisions above and replace them with the notice
|
|
|
|
* and other provisions required by the GPL or the LGPL. If you do not delete
|
|
|
|
* the provisions above, a recipient may use your version of this file under
|
|
|
|
* the terms of any one of the MPL, the GPL or the LGPL.
|
|
|
|
*
|
|
|
|
* ***** END LICENSE BLOCK ***** */
|
|
|
|
|
1999-08-04 23:44:15 +04:00
|
|
|
|
|
|
|
#include "CCPPTokenizer.h"
|
|
|
|
#include "CScanner.h"
|
|
|
|
#include "CToken.h"
|
|
|
|
#include "tokens.h"
|
|
|
|
#include <ctype.h>
|
|
|
|
|
|
|
|
|
|
|
|
CCPPTokenizer::CCPPTokenizer() : mTokens(0) {
|
|
|
|
}
|
|
|
|
|
|
|
|
/********************************************************
|
|
|
|
* Process: Destroy the tokenizer
|
|
|
|
* Usage:
|
|
|
|
* Calls:
|
|
|
|
* Notes:
|
|
|
|
*********************************************************/
|
|
|
|
CCPPTokenizer::~CCPPTokenizer() {
|
|
|
|
CToken* theToken=0;
|
Bug 8227 dreftool - rickg's phase 1 preventative crash maintenance
r=bernd
This is dreftool, resurrected with the help of kherron, bernd, and myself.
Licensing has been switched per http://bugzilla.mozilla.org/show_bug.cgi?id=8227#c45
to MPL/GPL/LGPL.
Old copies of xpcom classes have been removed, dreftool now uses the normal ones.
New:
* Builds on Linux and uses NSPR, so it should be possible to use it on most
systems.
* Member variables are screened.
* Tries to recognize NS_ENSURE early outs.
* Can parse a single file.
* The summary can indicate what it was told to parse.
* Links to LXR can be generated. Using this feature is not recommended as pieces of
mozilla tend to be on branches and the links will be bad. If for non branched content,
the links will drift as the files are changed.
* Links to Bonsai against the file revision will be generated if CVS records are
detected. This is good, unless you have local modifications in your tree.
Known issues:
* Aliasing isn't detected, so
|a = new something; *foo = a; if (!*foo) return NS_ERROR_FAILURE;| is flagged.
e.x.
http://bonsai.mozilla.org/cvsblame.cgi?file=mozilla/accessible/src/html/nsHTMLSelectAccessible.cpp&rev=1.33&mark=920-921,922,923,925#920
* Assigning to dereferenced objects
|*foo = new something| is not quite correctly handled (perhaps it's misinterpreted as
|foo = new something|. e.x.
http://bonsai.mozilla.org/cvsblame.cgi?file=/mozilla/extensions/transformiix/source/xslt/txStandaloneXSLTProcessor.cpp&rev=1.11&mark=88,92,96,99,84#91
* Various other allocators aren't caught: strdup and co., nsCRT::, nsMemory::, ...
* There's no way to provide a user defined list of allocators.
People should of course not file bugs simply based on dreftool, they should file bugs
after verifying that the complaints from dreftool are valid.
2003-11-13 01:38:24 +03:00
|
|
|
while((theToken=(CToken*)mTokens.Pop())){
|
1999-08-04 23:44:15 +04:00
|
|
|
delete theToken;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
int consumeToken(CScanner& aScanner,CToken** aToken){
|
|
|
|
static nsCAutoString gOperatorChars("/?.<>[]{}~^+=-!%&*(),|:");
|
|
|
|
|
Bug 8227 dreftool - rickg's phase 1 preventative crash maintenance
r=bernd
This is dreftool, resurrected with the help of kherron, bernd, and myself.
Licensing has been switched per http://bugzilla.mozilla.org/show_bug.cgi?id=8227#c45
to MPL/GPL/LGPL.
Old copies of xpcom classes have been removed, dreftool now uses the normal ones.
New:
* Builds on Linux and uses NSPR, so it should be possible to use it on most
systems.
* Member variables are screened.
* Tries to recognize NS_ENSURE early outs.
* Can parse a single file.
* The summary can indicate what it was told to parse.
* Links to LXR can be generated. Using this feature is not recommended as pieces of
mozilla tend to be on branches and the links will be bad. If for non branched content,
the links will drift as the files are changed.
* Links to Bonsai against the file revision will be generated if CVS records are
detected. This is good, unless you have local modifications in your tree.
Known issues:
* Aliasing isn't detected, so
|a = new something; *foo = a; if (!*foo) return NS_ERROR_FAILURE;| is flagged.
e.x.
http://bonsai.mozilla.org/cvsblame.cgi?file=mozilla/accessible/src/html/nsHTMLSelectAccessible.cpp&rev=1.33&mark=920-921,922,923,925#920
* Assigning to dereferenced objects
|*foo = new something| is not quite correctly handled (perhaps it's misinterpreted as
|foo = new something|. e.x.
http://bonsai.mozilla.org/cvsblame.cgi?file=/mozilla/extensions/transformiix/source/xslt/txStandaloneXSLTProcessor.cpp&rev=1.11&mark=88,92,96,99,84#91
* Various other allocators aren't caught: strdup and co., nsCRT::, nsMemory::, ...
* There's no way to provide a user defined list of allocators.
People should of course not file bugs simply based on dreftool, they should file bugs
after verifying that the complaints from dreftool are valid.
2003-11-13 01:38:24 +03:00
|
|
|
char theChar=0;
|
1999-08-04 23:44:15 +04:00
|
|
|
int result=aScanner.getChar(theChar);
|
|
|
|
if(kNoError==result){
|
|
|
|
if((isspace(theChar)) || (theChar>=kSpace)) {
|
|
|
|
if('#'==theChar)
|
|
|
|
*aToken=(CToken*)new CCompilerDirectiveToken();
|
|
|
|
else if((theChar=='_') || (isalpha(theChar)))
|
|
|
|
*aToken=(CToken*)new CIdentifierToken();
|
|
|
|
else if(isdigit(theChar))
|
|
|
|
*aToken=(CToken*)new CNumberToken();
|
|
|
|
else if((kLF==theChar) || (kCR==theChar))
|
|
|
|
*aToken=(CToken*)new CNewlineToken();
|
|
|
|
else if(isspace(theChar))
|
|
|
|
*aToken=(CToken*)new CWhitespaceToken();
|
Bug 8227 dreftool - rickg's phase 1 preventative crash maintenance
r=bernd
This is dreftool, resurrected with the help of kherron, bernd, and myself.
Licensing has been switched per http://bugzilla.mozilla.org/show_bug.cgi?id=8227#c45
to MPL/GPL/LGPL.
Old copies of xpcom classes have been removed, dreftool now uses the normal ones.
New:
* Builds on Linux and uses NSPR, so it should be possible to use it on most
systems.
* Member variables are screened.
* Tries to recognize NS_ENSURE early outs.
* Can parse a single file.
* The summary can indicate what it was told to parse.
* Links to LXR can be generated. Using this feature is not recommended as pieces of
mozilla tend to be on branches and the links will be bad. If for non branched content,
the links will drift as the files are changed.
* Links to Bonsai against the file revision will be generated if CVS records are
detected. This is good, unless you have local modifications in your tree.
Known issues:
* Aliasing isn't detected, so
|a = new something; *foo = a; if (!*foo) return NS_ERROR_FAILURE;| is flagged.
e.x.
http://bonsai.mozilla.org/cvsblame.cgi?file=mozilla/accessible/src/html/nsHTMLSelectAccessible.cpp&rev=1.33&mark=920-921,922,923,925#920
* Assigning to dereferenced objects
|*foo = new something| is not quite correctly handled (perhaps it's misinterpreted as
|foo = new something|. e.x.
http://bonsai.mozilla.org/cvsblame.cgi?file=/mozilla/extensions/transformiix/source/xslt/txStandaloneXSLTProcessor.cpp&rev=1.11&mark=88,92,96,99,84#91
* Various other allocators aren't caught: strdup and co., nsCRT::, nsMemory::, ...
* There's no way to provide a user defined list of allocators.
People should of course not file bugs simply based on dreftool, they should file bugs
after verifying that the complaints from dreftool are valid.
2003-11-13 01:38:24 +03:00
|
|
|
else if(-1<gOperatorChars.FindChar(theChar)) {
|
1999-08-04 23:44:15 +04:00
|
|
|
PRUnichar nextChar;
|
|
|
|
aScanner.peek(nextChar);
|
|
|
|
if((kForwardSlash==theChar) && ((kForwardSlash==nextChar) || (kAsterisk==nextChar))){
|
|
|
|
*aToken=(CToken*)new CCommentToken(nextChar);
|
|
|
|
}
|
|
|
|
else *aToken=(CToken*)new COperatorToken();
|
|
|
|
}
|
|
|
|
else if((kQuote==theChar) || (kApostrophe==theChar))
|
|
|
|
*aToken=(CToken*)new CQuotedStringToken();
|
|
|
|
else if(kSemicolon==theChar) {
|
|
|
|
*aToken=(CToken*)new CSemicolonToken();
|
|
|
|
}
|
|
|
|
if(*aToken){
|
|
|
|
result=(*aToken)->consume(theChar,aScanner);
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
return result;
|
|
|
|
}
|
|
|
|
|
Bug 8227 dreftool - rickg's phase 1 preventative crash maintenance
r=bernd
This is dreftool, resurrected with the help of kherron, bernd, and myself.
Licensing has been switched per http://bugzilla.mozilla.org/show_bug.cgi?id=8227#c45
to MPL/GPL/LGPL.
Old copies of xpcom classes have been removed, dreftool now uses the normal ones.
New:
* Builds on Linux and uses NSPR, so it should be possible to use it on most
systems.
* Member variables are screened.
* Tries to recognize NS_ENSURE early outs.
* Can parse a single file.
* The summary can indicate what it was told to parse.
* Links to LXR can be generated. Using this feature is not recommended as pieces of
mozilla tend to be on branches and the links will be bad. If for non branched content,
the links will drift as the files are changed.
* Links to Bonsai against the file revision will be generated if CVS records are
detected. This is good, unless you have local modifications in your tree.
Known issues:
* Aliasing isn't detected, so
|a = new something; *foo = a; if (!*foo) return NS_ERROR_FAILURE;| is flagged.
e.x.
http://bonsai.mozilla.org/cvsblame.cgi?file=mozilla/accessible/src/html/nsHTMLSelectAccessible.cpp&rev=1.33&mark=920-921,922,923,925#920
* Assigning to dereferenced objects
|*foo = new something| is not quite correctly handled (perhaps it's misinterpreted as
|foo = new something|. e.x.
http://bonsai.mozilla.org/cvsblame.cgi?file=/mozilla/extensions/transformiix/source/xslt/txStandaloneXSLTProcessor.cpp&rev=1.11&mark=88,92,96,99,84#91
* Various other allocators aren't caught: strdup and co., nsCRT::, nsMemory::, ...
* There's no way to provide a user defined list of allocators.
People should of course not file bugs simply based on dreftool, they should file bugs
after verifying that the complaints from dreftool are valid.
2003-11-13 01:38:24 +03:00
|
|
|
int32 CCPPTokenizer::tokenize(CScanner& aScanner){
|
1999-08-04 23:44:15 +04:00
|
|
|
int result=0;
|
|
|
|
while(kNoError==result){
|
|
|
|
CToken* theToken=0;
|
|
|
|
result=consumeToken(aScanner,&theToken);
|
|
|
|
if(theToken) {
|
|
|
|
mTokens.Push(theToken);
|
|
|
|
}
|
|
|
|
}
|
|
|
|
return result;
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
CToken* CCPPTokenizer::getTokenAt(int anIndex){
|
|
|
|
if((anIndex>=0) && (anIndex<mTokens.GetSize())){
|
|
|
|
return (CToken*)mTokens.ObjectAt(anIndex);
|
|
|
|
}
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
|
|
|
|