A library that helps tokenize text using Text Mate grammars.
Перейти к файлу
Alex Dima d9f6fcb537 House keeping 2016-08-17 15:54:37 +02:00
.vscode Extract plist parser to npm module & adopt it 2016-08-17 11:42:05 +02:00
benchmark Adopt oniguruma with OnigScanner 2015-12-06 22:06:48 +01:00
release Run tests with mocha 2016-08-17 15:49:52 +02:00
src Run tests with mocha 2016-08-17 15:49:52 +02:00
test-cases Run tests with mocha 2016-08-17 15:49:52 +02:00
.gitignore Initial commit 2015-09-16 13:41:40 +02:00
.npmignore House keeping 2016-08-17 15:54:37 +02:00
LICENSE.md Update LICENSE.md 2015-11-06 14:32:12 +01:00
README.md Update README.md 2015-10-30 17:28:52 +01:00
ThirdPartyNotices.txt Add ThirdPartyNotices.txt 2015-11-18 14:38:28 +01:00
gulpfile.js House keeping 2016-08-17 15:54:37 +02:00
package.json Run tests with mocha 2016-08-17 15:49:52 +02:00

README.md

VSCode TextMate

An interpreter for grammar files as defined by TextMate. Supports loading grammar files from JSON or PLIST format. Cross - grammar injections are currently not supported.

Installing

npm install vscode-textmate

Using

var Registry = require('vscode-textmate').Registry;
var registry = new Registry();
var grammar = registry.loadGrammarFromPathSync('./javascript.tmbundle/Syntaxes/JavaScript.plist');

var lineTokens = grammar.tokenizeLine('function add(a,b) { return a+b; }');
for (var i = 0; i < lineTokens.tokens.length; i++) {
	var token = lineTokens.tokens[i];
	console.log('Token from ' + token.startIndex + ' to ' + token.endIndex + ' with scopes ' + token.scopes);
}

Using asynchronously

Sometimes, it is necessary to manage the list of known grammars outside of vscode-textmate. The sample below shows how this works:

var Registry = require('vscode-textmate').Registry;

var registry = new Registry({
	getFilePath: function (scopeName) {
		// Return here the path to the grammar file for `scopeName`
		if (scopeName === 'source.js') {
			return './javascript.tmbundle/Syntaxes/JavaScript.plist';
		}
		return null;
	}
});

// Load the JavaScript grammar and any other grammars included by it async.
registry.loadGrammar('source.js', function(err, grammar) {
	if (err) {
		console.error(err);
		return;
	}

	// at this point `grammar` is available...
});

Tokenizing multiple lines

To tokenize multiple lines, you must pass in the previous returned ruleStack.

var ruleStack = null;
for (var i = 0; i < lines.length; i++) {
	var r = grammar.tokenizeLine(lines[i], ruleStack);
	console.log('Line: #' + i + ', tokens: ' + r.tokens);
	ruleStack = r.ruleStack;
}

API doc

See the .d.ts file

Developing

  • Clone the repository
  • Run npm install
  • Compile in the background with npm run watch
  • Run tests with npm test
  • Run benchmark with npm run benchmark

License

MIT