## edit-dictionary.sh
Instead of editing the .dic file directly, allow user to provide a list of
words. The script expands the existing .dic file, adds the new words and
compress it again using the affix rules.
Numerals at the beginning of the file and "no suggestion" words need to be
special-cased, since compressing the word list creates different results.
## make-new-dict.sh
Extract suggestions exclusions from the existing Mozilla dictionary, then
add them back to the dictionary generated by SCOWL. This removes the need
to maintain an external list of exclusions (mozilla-exclusions.txt).
It also allows to exclude these offensive words from both lists of added
and removed words by Mozilla.
Also:
- Break if the scowl folder is missing.
- Remove backup folders to make sure the install script can't be run twice.
## install-new-dict.sh
Break if the scowl folder is missing.
Differential Revision: https://phabricator.services.mozilla.com/D165883
Added .sh extension to all scripts.
edit-dictionary.sh:
* Convert to utf-8 before editing, and back to iso-8859-1 before saving
* Place a copy of the utf-8 dictionary inside the utf8 folder, and store the iso-8859-1 in place
make-new-dict.sh:
* Use .txt extension for support wordlists, and place them in a subfolder
* Exclude words in mozilla-exclusions.txt from the generated dictionary
* Save 5-mozilla-*.txt files to utf-8
Depends on D165304
Differential Revision: https://phabricator.services.mozilla.com/D165305
Francesco Lodolo (:flod)
Sylvestre Ledru