With JSM global sharing, the object returned by Cu.import() is a
NonSyntacticVariablesObject, rather than a global. Various code tries
to use properties from a JSM global via an import.
Cu.importGlobalProperties can also be used in some places.
MozReview-Commit-ID: HudCXO2GKN0
--HG--
extra : rebase_source : 6b5fa6f5509397504cb461a761f6cc2399f18c40
The issue occurs when nsITimer is fired earlier than the backoff time. In that
case, the update doesn't proceed and we never make another attempt because the
backoff update timer was oneshot.
We fix the issue in two ways:
- Add a tolerance of 1 second in case the timer fires too early.
- Set another oneshot timer whenever we are prevented from updating due to
backoff.
MozReview-Commit-ID: E2ogNRsHJVK
--HG--
extra : rebase_source : c81fa77934f6c39e1c5d07b19785a01546e02542
The issue occurs when nsITimer is fired earlier than the backoff time. In that
case, the update doesn't proceed and we never make another attempt because the
backoff update timer was oneshot.
We fix the issue in two ways:
- Add a tolerance of 1 second in case the timer fires too early.
- Set another oneshot timer whenever we are prevented from updating due to
backoff.
MozReview-Commit-ID: E2ogNRsHJVK
--HG--
extra : rebase_source : 17aa70d8583cc84e28e57410de66eaac63bd18bb
This fixes usages of `Find`, `RFind` and the equality operator that kind of
work right now but will break with the proper type checking of a templatized
version of the string classes.
For `Find` and `RFind` it appears that `nsCString::(R)Find("foo", 0)` calls
were being coerced to the `Find(char*, bool, int, int)` versions. The intent was
probably to just start searching from position zero.
For the equality operator, the type of nullptr is nullptr_t rather than
char(16_t)* so we'd need to add an operator overload that takes nullptr_t. In
this case just using `IsVoid` is probably more appropriate.
--HG--
extra : rebase_source : 50f78519084012ca669da0a211c489520c11d6b6
Version 4 of the Google Safe Browsing server will return a 404 if any of the
application reputation lists are requested on Android. As a result, we should
avoid these threat types from being sent along with ANDROID_PLATFORM.
MozReview-Commit-ID: 6TUBVxe455y
--HG--
extra : rebase_source : dee095c008f4d328f359c66d20f0cc2dfcd109f3
extra : source : 5d6338807c6b21b7672236cac01b13bd75155225
These are all easy cases where an nsXPIDLCString local variable is set via
getter_Copies() and then is only used in ways that nsCStrings can also be used
(i.e. no null checks or implicit conversions to |char*|).
In every case the patch trivially replaces the nsXPIDLCString with an
nsCString. (Also, there are a couple of unused nsXPIDLCString variables that
the patch simply removes.)
The test is based on "test-harmful-simple" so we can add testing entries
in a human-readable way.
MozReview-Commit-ID: IJcYrp04uFs
--HG--
extra : rebase_source : afc4329af4cd5fc31a783a8f0f2e3a4bdcb2a432
extra : source : 55a2e275514d2154a23a656e48f685cc6032e5c3
As our threattype-listname conversion design, "goog-harmful-proto" is allocated
for this new threat type. This threat type is mainly for mobile.
MozReview-Commit-ID: G9GbgmHHHfp
--HG--
extra : rebase_source : 0681fcd9322b94451a86eafe57bf1ccc4b89db30
extra : intermediate-source : 28b0502d9add81beeae58a2c33f9fd5839d4d544
extra : source : 646f02f15131aa98ad37015b0a641304a3271796
This mechanically replaces nsILocalFile with nsIFile in
*.js, *.jsm, *.sjs, *.html, *.xul, *.xml, and *.py.
MozReview-Commit-ID: 4ecl3RZhOwC
--HG--
extra : rebase_source : 412880ea27766118c38498d021331a3df6bccc70
We have a minimum requirement of VS 2015 for Windows builds, which supports
the z length modifier for format specifiers. So we don't need SizePrintfMacros.h
any more, and can just use %zu and friends directly everywhere.
MozReview-Commit-ID: 6s78RvPFMzv
--HG--
extra : rebase_source : 009ea39eb4dac1c927aa03e4f97d8ab673de8a0e
Doing this makes various operations more efficient, as we don't have to
allocate, copy strings, and deallocate needlessly. The remaining
instances of nsCString temporaries are in logging code, which didn't
seem worthwhile to deal with here.
This patch makes the following changes to the macros.
- Removes PROFILER_LABEL_FUNC. It's only suitable for use in functions outside
classes, due to PROFILER_FUNCTION_NAME not getting class names, and it was
mostly misused.
- Removes PROFILER_FUNCTION_NAME. It's no longer used, and __func__ is
universally available now anyway.
- Combines the first two string literal arguments of PROFILER_LABEL and
PROFILER_LABEL_DYNAMIC into a single argument. There was no good reason for
them to be separate, and it forced a '::' in the label, which isn't always
appropriate. Also, the meaning of the "name_space" argument was interpreted
in an interesting variety of ways.
- Adds an "AUTO_" prefix to PROFILER_LABEL and PROFILER_LABEL_DYNAMIC, to make
it clearer they construct RAII objects rather than just being function calls.
(I myself have screwed up the scoping because of this in the past.)
- Fills in the 'js::ProfileEntry::Category::' qualifier within the macro, so
the caller doesn't need to. This makes a *lot* more of the uses fit onto a
single line.
The patch also makes the following changes to the macro uses (beyond those
required by the changes described above).
- Fixes a bunch of labels that had gotten out of sync with the name of the
class and/or function that encloses them.
- Removes a useless PROFILER_LABEL use within a trivial scope in
EventStateManager::DispatchMouseOrPointerEvent(). It clearly wasn't serving
any useful purpose. It also serves as extra evidence that the AUTO_ prefix is
a good idea.
- Tweaks DecodePool::SyncRunIf{Preferred,Possible} so that the labelling is
done within them, instead of at their callsites, because that's a more
standard way of doing things.
--HG--
extra : rebase_source : 318d1bc6fc1425a94aacbf489dd46e4f83211de4
All the instances are converted as follows.
- nsSubstring --> nsAString
- nsCSubstring --> nsACString
--HG--
extra : rebase_source : cfd2238c52e3cb4d13e3bd5ddb80ba6584ab6d91
This patch includes following fix in classifierHelper.js:
1. Avoid the reuse of same chunk numbers
2. Remove unused removeUrlFromDB function
MozReview-Commit-ID: XK1oHBa8gf
--HG--
extra : rebase_source : bf11c44ae98f34de43ac65ad4558ee53894e5568
The Flash infobar is showing up a bit too often. There are a few changes planned to address this. One of them is a list that will contain sites that should never display infobars. This patch allows the list to be downloaded from Shavar and updated via the URL classifier.
MozReview-Commit-ID: BgAaysyRzIE
--HG--
extra : rebase_source : a5f569a3bc60134938e9dc937cb5cfb84222a52b
extra : amend_source : 543e3c38b63d7b958c633c38e066ab9b47a76a01
We write a lot of 4-bytes prefixes to file which call many system calls.
We should use a buffer and only write to file if the buffer is full or
finish writing. nsIBufferedOutputStream is a good candidate to do that
MozReview-Commit-ID: CzGOd7iXVTv
--HG--
extra : rebase_source : 25f1ce804b9a53e0a0a4023a1aa91f1a0ed98547
This was found by Coverity Scan (CID 1409773, 1409774).
MozReview-Commit-ID: LA7oPMobbie
--HG--
extra : rebase_source : 0ca4245e445f0b0da0073cee7b5ee95c621af75f
These methods do not appear to be used.
When JSM global sharing is enabled, these methods contaminate the
global Function.prototype, which breaks Marionette object
serialization.
MozReview-Commit-ID: CAfJ2FCkhlK
--HG--
extra : rebase_source : 38acb4616ee14283d0f67e1b2144972c7d139f84
After adopting the new thread model for safebrowsing, we will create a new
lookup cache for update so we can still check lookup cache at the same time.
Prefix set, completions will be generated when we open the new lookup cache
but it won't include cache, so we will loss cache after that.
This patch will copy cache data from old lookup cache to new lookup
cache while update.
MozReview-Commit-ID: L0WpiHOGIGm
--HG--
extra : rebase_source : ebaf249535561b87a983a3f24dacb8fbab7c096e
This test aims to test the priority of the http channels created from XHR and Fetch inside a tracking script is lowest.
--HG--
extra : rebase_source : 556e503a69eda9bd1ec554f87e668bda66926ce1
Based on the telemetry that landed as part of bug 1336904, Safe Browsing
updates are failing too often: https://mzl.la/2qGkOPS
This should enable browsers on slower networks to reach the update
servers while still putting a reasonable bound on how long the update
thread can be blocked.
MozReview-Commit-ID: 6puVtpMT87K
--HG--
extra : rebase_source : 4b68bb6778b45068bb8a2d0dbdfddfe24a6b022f
resetDatabase() is used to clear out the Safe Browsing database and cache in tests.
Since bug 1333328 however we can no longer rely on updates clearing the cache.
There are two solutions to address this issue:
1. resetDatabase() calls another test-only function: reloadDatabase(). Since the cache
is in memory, reloading the URL classifier will automatically clear the cache.
2. During an update, remove cache entries which are not in the database.
I prefer taking the first one because if we implement the second
approach, an update will take longer since we need to check if each prefix
in the cache can also be found in the database. I think this is not necessary
because prefixes not in the database will eventually be removed when they
are expired.
MozReview-Commit-ID: BjsDKDMr205
--HG--
extra : rebase_source : d4cf8d9876d1427a7917fc74ecb7bc1bf91bf371
In V2 we shuffled the hash entries before sending the request to obscure the real
entry from noises. We should also hide the real entry in V4. Using sort() is
enough for both V2 and V4 because the array contains exactly 5 entries in almost
all cases.
MozReview-Commit-ID: 4uOXIF83KQL
--HG--
extra : rebase_source : 97c6439965e864863dc99f194356eb1efb0235df
Technically speaking, we use the new async API 'nsIURIClassifier.asyncClassifyLocalWithTables'
to replace the old sync API. However, since we cannot guarantee the async call will be done when
we neet its result, we need a sync call as a fallback in this case. This is a sub-optimal
solution and we will be investigating if there's a better solution if the sync call
is used too frequently.
MozReview-Commit-ID: L1uQ2eaYr1e
--HG--
extra : rebase_source : 148e0e85796c932ea85d123092f479e1373ecec9
extra : intermediate-source : 53007a31b576fcd4f16ad6523cccd0a9b90c66f0
extra : source : 1175b9c64b31da2ca2ab88f78488aed6fdc405bc
This patch includes two test cases:
1. Apply an empty update through Classifier interface, which is the normal use case.
2. Apply an empty update through LookupCacheV4::ApplyUpdate, this ensure update algorithm is
correct when applying an empty update. This scenario actually shouldn't happen in
normal use case because it will be skipped by Classifier::CheckValidUpdate.
MozReview-Commit-ID: 9khsuVatX0u
--HG--
extra : rebase_source : 8db87dd2b8b4aec4546257e9f06dcc839b2ea181
With the activation of Ask-to-Activate mode by default, we'll also activate the fallback rule that favors fallback content when the object has not provided a src, so we need to prepare this test for that
MozReview-Commit-ID: JmmxJEiziHW
--HG--
rename : layout/generic/crashtests/simple_blank.swf => toolkit/components/url-classifier/tests/browser/simple_blank.swf
This function is arguably nicer than calling NS_ProcessNextEvent
manually, is slightly more efficient, and will enable better auditing
for NS_ProcessNextEvent when we do Quantum DOM scheduling changes.
Functions and Preference related to mTableFreshness and gFreshnessGuarantee
could be removed since we will no longer require them.
MozReview-Commit-ID: IAa0UHLSQ56
--HG--
extra : rebase_source : daa959340c92cfcb751fe8d0548d30762db3783e
In this patch, we will make Safebrowsing V2 caching use the same algorithm as V4.
So we remove "mMissCache" for negative caching and TableFresness check for
positive caching.
But Safebrowsing V2 doesn't contain negative/positive cache duration information in
gethash response. So we hard-code a fixed value, 15 minutes, as cache duration.
In this way, we can sync the mechanism we handle caching for V2 and V4.
An extra effort for V2 here is that we need to manually record prefixes misses
because we won't get any response for those prefixes(implemented in
nsUrlClassifierLookupCallback::CacheMisses).
--HG--
extra : rebase_source : 0fb7938464ad24a01a51493ecb1b5c0197c16123
This is in preparation for being able to be replaced with SpinEventLoopUntil(),
which is going to be shipped in bug 1359490.
MozReview-Commit-ID: AChVqh4kfVb
--HG--
extra : rebase_source : 03f8d36c29d74fed34ebed6ae3cb31ca829c3e41
These timeouts will ensure that we don't block the Safe Browsing update thread
for too long when we encounter slow or bad network conditions.
MozReview-Commit-ID: AJfR193cTf8
--HG--
extra : rebase_source : 5ae14f204ed88691fac4e9ba8c8202b8aa79b657
mResults is lookupResultArray which is created when we find matched prefix in the database.
mCacheResults stores response of gethash request.
We record threat type match telemetry only when completion is found in both V2 and V4,
that is we got one confirmed result for V2 and one for V4. And then we could record threat
types by iterating through mCacheResultsArray.
But if one of lookupResult is from cache, for example, completion is found in 'goog-malware-proto',
then we won't trigger a gethash request for it because it is in the cache.
In this scenario, mCacheResults will not have results from V4 so when we try to record
threat types we won't find V4 ones.
In this patch we only record telemetry when gethash is sent for both V2 and V4. This may limit
the usefulness of that probe a little bit, but we shouldn't make the code less efficient just
to be able to measure telemetry better.
MozReview-Commit-ID: Ib8SGUaxb4c
--HG--
extra : rebase_source : 03265ddd5253d3ae9ae4a7a20bca6ca19d2e7595
We only register google table if we have a valid google api keys
MozReview-Commit-ID: CJmJErxSOqa
--HG--
extra : rebase_source : e7cb2965ea5206afd47bc37d712677f939c3c26e
We will cache all preferences which will be read during classifing channel
- Store them into static variables nsUrlClassifierDBService
- Use a singleton class to manage/update preferrences in nsChannelClassifier
MozReview-Commit-ID: GvyBI3rVpYh
--HG--
extra : rebase_source : 0cec0724bd47f55c7b1666e700d172698a708efb
The about:url-classifier supports following functions:
1. Provider section
- Show update status for each provider, update status include
last update time, next update time and last update status
- Update button to manually trigger an update for the provider.
2. Debug section
- Set MOZ_LOG Modules
- Set MOZ_LOG_FILE
MozReview-Commit-ID: AHiveKEHSNC
--HG--
extra : rebase_source : b56e918de7f4eb30ad8eebc6d24ffaf2e76f4f4e
In Bug 1311935, We change positive/negative cache duration from milli-second to second.
But the value doesn't covert back to milli-second when store to telemetry(telemetry use
milli-second).
MozReview-Commit-ID: KR6xn9pwhUd
--HG--
extra : rebase_source : 378149dc29d61cbca31b8aa913df946ceff556f3
Recent change of safebrowsing thread model may cause nsUrlClassifierDBService::ReloadDatabase
API fail if there is an ongoing update at the same.
Fix this issue by adding retry in testcase.
MozReview-Commit-ID: CZGMpQvuzum
--HG--
extra : rebase_source : c59937f3e11bafcb2ad501872ffa29bafe84dad8
The Flash block tests sometimes timeout in debug runs. So far it has always happened the same way. All assertions in the test run (and pass), but the test times out during cleanup. Bumping up the timeout for these tests should fix this problem.
MozReview-Commit-ID: F04nSzSyLtr
--HG--
extra : rebase_source : 181546e1fe4b56686a06cb71e059a4b07f6fb957
When starting up, SafeBrowsing.jsm will try to use DBService to add testing entries. Meanwhile,
listmanager will request StreamUpdater to download lists with a random initial delay.
The requests that listmanager issue to StreamUpdater will be queued up
if DBserve is busy and will be retried when StreamUpdater is notified that
the previous update is complete. However, in some edge cases,
the queued requests may not be processed until the next update request from listmanager.
For example, SafeBrowsing.jsm calls DBService.beginUpdate at t0 and the update is
complete at t1. If listmanager sends all requests via StreamUpdate between t0 and t1,
they will all be queued up and no further request can trigger the queued ones.
So in this patch I add a timer to re-trigger FetchNextRequest() in case StreamUpdater is not
notified the previous update is complete.
MozReview-Commit-ID: 3hHsS5N7WRI
--HG--
extra : rebase_source : 1c1ac7212626f1bf83b355dd27b1ddeae6df2e03
mTableRefreshness, a non-thread-safe object, might be accessed on worker thread
and update thread cocurrently. To solve this issue, on update thread we only
insert data to mNewTableRefreshness and merge to mTableRefreshness on
the worker thread later.
MozReview-Commit-ID: 9WgoeYfWVfK
--HG--
extra : rebase_source : b7a8b4cd9d0fb1471cb81ee239f8343ff9a7b38a
When full match is found in both v2 and v4, the threat types returned should also be the same.
If threat types are different, the telemetry record this by setting a bit flags which indicates
what threat types are being returned.
If threat types are the same, this telemetry will record 0.
MozReview-Commit-ID: Laz77yoCg00
--HG--
extra : rebase_source : 1bcace8464c9d6472a745f66fd547b186e5b3297
In Bug 1323953, we always send 4-bytes prefix for completion and the prefix is also
used as the key to store cache result from gethash request.
Since it is always 4-bytes, we could convert it to integer for simplicity.
MozReview-Commit-ID: Lkvrg0wvX5Z
--HG--
extra : rebase_source : 7002b1f55bc0f88ed90340082574cc931f8db87a
LookupCacheV4::Has implements safebrowsing v4 caching logic.
1. Check if fullhash match any prefix in local database:
- If not, the URL is safe.
2. Check if prefix is in the cache(prefix is always the first 4-byte of
the fullhash, Bug 1323953):
- If not, send fullhash request
3. Check if fullhash is in the positive cache:
- If fullhash is found and it is not expired, the URL is not safe.
- If fullhash is found and it is expired, send fullhash request.
4. If fullhash is not found, check negative cache expired time:
- If negative cache time is not expired, the URL is safe.
- If negative cache time is expired, send fullhash request.
MozReview-Commit-ID: GRX7CP8ig49
--HG--
extra : rebase_source : 6d3ae8929b11731584810c44d46a1526f7d83e12
This patch includes following changes:
1. nsUrlClassifierHashCompleter.js
nsUrlClassifierHashCompleter.idl
- Add completionV4 interface for hashCompleter to pass response data to
DB service.
- Process response data includes negative cache duration, matched full
hashes and cache duration for each match. Full matches are passed through
nsIFullHashMatch interface.
- Change _requests.responses from array contains matched fullhashes to
dictionary so that it can store additional information likes negative cache
duration.
2. nsUrlClassifierDBService.cpp
- Implement CompletionV4 interface, store response data to CacheResultV4
object. Expired duration to expired time is handled here.
- Add CacheResultToTableUpdate function to convert V2 & V4 cache result
to TableUpdate object.
3. LookupCache.h
- Extend CacheResult to CacheResultV2 and CacheResultV4 so we can store
response data in CompletionV2 and CompletionV4.
4. HashStore.h
- Add API and member variable in TableUpdateV4 to store response data.
TableUpdate object is used by DB service to pass update data or gethash
response to Classifier, so we need to extend TableUpdateV4 to be able
to store fullHashes.find response.
6. Entry.h
- Define the structure about how we cache fullHashes.find response.
MozReview-Commit-ID: FV4yAl2SAc6
--HG--
extra : rebase_source : 02676ded9bc0580c24b4c906199a4abaf004e296
This patch fixes that Classifier::ActiveTables doesn't return v4 tables.
Classifier::mActiveTablesCache is generated by scanning safebrowsing directory.
We use Classifier::ScanStoreDir to do the work, but it will ignore subdirectory.
Since v4 tables are stored in subdirectory 'google4', mActiveTablesCache doesn't
include v4 tables.
Fix this issue by checking subdirectory recursively in ScanStoreDir.
MozReview-Commit-ID: I6pa6e4bFND
--HG--
extra : rebase_source : 49d19101bec780e21db668b2daff3c46bbdb3fd2
LookupCacheV4::Has implements safebrowsing v4 caching logic.
1. Check if fullhash match any prefix in local database:
- If not, the URL is safe.
2. Check if prefix is in the cache(prefix is always the first 4-byte of
the fullhash, Bug 1323953):
- If not, send fullhash request
3. Check if fullhash is in the positive cache:
- If fullhash is found and it is not expired, the URL is not safe.
- If fullhash is found and it is expired, send fullhash request.
4. If fullhash is not found, check negative cache expired time:
- If negative cache time is not expired, the URL is safe.
- If negative cache time is expired, send fullhash request.
MozReview-Commit-ID: GRX7CP8ig49
--HG--
extra : rebase_source : bcb5fa7aa2b7b116d862e3382447611424603c2d
This patch includes following changes:
1. nsUrlClassifierHashCompleter.js
nsUrlClassifierHashCompleter.idl
- Add completionV4 interface for hashCompleter to pass response data to
DB service.
- Process response data includes negative cache duration, matched full
hashes and cache duration for each match. Full matches are passed through
nsIFullHashMatch interface.
- Change _requests.responses from array contains matched fullhashes to
dictionary so that it can store additional information likes negative cache
duration.
2. nsUrlClassifierDBService.cpp
- Implement CompletionV4 interface, store response data to CacheResultV4
object. Expired duration to expired time is handled here.
- Add CacheResultToTableUpdate function to convert V2 & V4 cache result
to TableUpdate object.
3. LookupCache.h
- Extend CacheResult to CacheResultV2 and CacheResultV4 so we can store
response data in CompletionV2 and CompletionV4.
4. HashStore.h
- Add API and member variable in TableUpdateV4 to store response data.
TableUpdate object is used by DB service to pass update data or gethash
response to Classifier, so we need to extend TableUpdateV4 to be able
to store fullHashes.find response.
6. Entry.h
- Define the structure about how we cache fullHashes.find response.
MozReview-Commit-ID: KgR1NASl7GC
--HG--
extra : rebase_source : 424db14e4af2ffd691c384414d50f64083d5d20b
This patch fixes that Classifier::ActiveTables doesn't return v4 tables.
Classifier::mActiveTablesCache is generated by scanning safebrowsing directory.
We use Classifier::ScanStoreDir to do the work, but it will ignore subdirectory.
Since v4 tables are stored in subdirectory 'google4', mActiveTablesCache doesn't
include v4 tables.
Fix this issue by checking subdirectory recursively in ScanStoreDir.
MozReview-Commit-ID: I6pa6e4bFND
--HG--
extra : rebase_source : 49d19101bec780e21db668b2daff3c46bbdb3fd2
LookupCacheV4::Has implements safebrowsing v4 caching logic.
1. Check if fullhash match any prefix in local database:
- If not, the URL is safe.
2. Check if prefix is in the cache(prefix is always the first 4-byte of
the fullhash, Bug 1323953):
- If not, send fullhash request
3. Check if fullhash is in the positive cache:
- If fullhash is found and it is not expired, the URL is not safe.
- If fullhash is found and it is expired, send fullhash request.
4. If fullhash is not found, check negative cache expired time:
- If negative cache time is not expired, the URL is safe.
- If negative cache time is expired, send fullhash request.
MozReview-Commit-ID: HFpqaOGOtUa
--HG--
extra : rebase_source : 726acb50489c4fa20c99fc8c4d96a3c8351655f5
This patch includes following changes:
1. nsUrlClassifierHashCompleter.js
nsUrlClassifierHashCompleter.idl
- Add completionV4 interface for hashCompleter to pass response data to
DB service.
- Process response data includes negative cache duration, matched full
hashes and cache duration for each match. Full matches are passed through
nsIFullHashMatch interface.
- Change _requests.responses from array contains matched fullhashes to
dictionary so that it can store additional information likes negative cache
duration.
2. nsUrlClassifierDBService.cpp
- Implement CompletionV4 interface, store response data to CacheResultV4
object. Expired duration to expired time is handled here.
- Add CacheResultToTableUpdate function to convert V2 & V4 cache result
to TableUpdate object.
3. LookupCache.h
- Extend CacheResult to CacheResultV2 and CacheResultV4 so we can store
response data in CompletionV2 and CompletionV4.
4. HashStore.h
- Add API and member variable in TableUpdateV4 to store response data.
TableUpdate object is used by DB service to pass update data or gethash
response to Classifier, so we need to extend TableUpdateV4 to be able
to store fullHashes.find response.
6. Entry.h
- Define the structure about how we cache fullHashes.find response.
MozReview-Commit-ID: 8pUJITn8c1n
--HG--
extra : rebase_source : efc1ef8e47cd5d0774a931f235e7e0bbdc56fedf
This patch fixes that Classifier::ActiveTables doesn't return v4 tables.
Classifier::mActiveTablesCache is generated by scanning safebrowsing directory.
We use Classifier::ScanStoreDir to do the work, but it will ignore subdirectory.
Since v4 tables are stored in subdirectory 'google4', mActiveTablesCache doesn't
include v4 tables.
Fix this issue by checking subdirectory recursively in ScanStoreDir.
MozReview-Commit-ID: I6pa6e4bFND
--HG--
extra : rebase_source : 6844e03f1e3c28ebb25ac3885c35a80c42a857d9
We add a new "on-off" protocol PURLClassifierLocal which calls
nsIURIClassifier.asyncClassifyLocalWithTables on construction and
calls back on destruction. Pretty much the same design as PURLClassifier.
In order to avoid code duplication, the actor implementation is templatized
and |MaybeInfo| in PURLClassifier.ipdl is moved around.
Test case is included and the custom event target is not in place for labelling.
The custom event target will be done in Bug 1353701.
MozReview-Commit-ID: IdHYgdnBV7S
--HG--
extra : rebase_source : ab1c896305b9f76cab13a92c9bd88c2d356aacb7
A new function Classifier::AsyncApplyUpdates() is implemented for async update.
Besides, all public Classifier interfaces become "worker thread only" and
we remove DBServiceWorker::ApplyUpdatesBackground/Foreground.
In DBServiceWorker::FinishUpdate, instead of calling Classifier::ApplyUpdates,
we call Classifier::AsyncApplyUpdates and install a callback for notifying
the update observer when update is finished. The callback will occur on the
caller thread (i.e. worker thread.)
As for the shutdown issue, when the main thread is notified to shut down,
we at first *synchronously* dispatch an event to the worker thread to
shut down the update thread. After getting synchronized with all other
threads, we send last two events "CancelUpdate" and "CloseDb" to notify
dangling update (i.e. BeginUpdate is called but FinishUpdate isn't)
and do cleanup work.
MozReview-Commit-ID: DXZvA2eFKlc
--HG--
extra : rebase_source : cd2e27a6b679d2c96e769854d1582ed2dcda12bb
The probes keep track of V2 and V4 server response to update and getHash
request. The probes are keyed by provider
MozReview-Commit-ID: 3Uxjm6H77lE
--HG--
extra : rebase_source : 647523a24bad8334dbb34758954a5f5db0e9b96c
Previously, we operated under the understanding that with Flash blocking activated, non-whitelisted documents would be set to CTA. We are changing that such that now, documents will only be CTA'ed if Flash is set to "Ask to Activate".
Flash blocking will now behave according to the following chart:
User Setting Flash block Whitelisted sites Blacklisted sites Unlisted sites
"Never ..." Off Deny Deny Deny
"Ask ..." Off Ask Ask Ask
"Always ..." Off Allow Allow Allow
"Never ..." On Deny Deny Deny
"Ask ..." On Allow Deny Ask
"Always ..." On Allow Deny Allow
This patch also completely reworks the flash blocking testing. Test data and most code remains consolidated, but will be run in multiple different tests. This avoids having to extend the timeout for Flash block testing to an extremely long length. The new Flash block testing additionally tests each of the six cases (rows) in the table above.
MozReview-Commit-ID: 5aPGUEiUiCv
--HG--
extra : rebase_source : d5f5711855d108a9a33b9d28aae6e312cc9cb432
Enable safebrowsing v4 completion but ignore the result by checking preference in
nsUrlClassifierLookupCallback::Completion may cause telemetry measure incorrect match
result since v4 completions will always be ignored.
So in this patch we move the preference check after telemetry is measured and then we
ignore the result.
MozReview-Commit-ID: J29JitvW3Lc
--HG--
extra : rebase_source : 0cf8797de476980ce25d83099e6526321308327c
URLCLASSIFIER_PREFIX_MATCH is replaced by URLCLASSIFIER_FULL_MATCH(Bug 1311931)
so it should be removed.
MozReview-Commit-ID: 4F1B07bGwRP
--HG--
extra : rebase_source : 2d3e5cda14df4e874bf31145ccb5330acec369f7
mTelemetryProvider might be empty if |CancelUpdate| is called during update.
In that scenario, the call flow will look like:
CancelUpdate -> UpdateError -> DownloadDone -> set mTelemetryProvider to empty ->
OnStartRequest -> record telemetry with empty key.
So in this patch, move mTelemetryProvider.Truncate() to OnStopRequest.
MozReview-Commit-ID: JDgNUqR4SKa
--HG--
extra : rebase_source : 42aa4c898f403a1189e67e355246ebed5aa4af44
nsTArray::AppendElement may cause memory reallocation if out of capacity.
In nsUrlClassifierStreamUpdater::FetchNextRequest(), we take the reference of
the first element of mPendingRequests and pass its member as reference to
DownloadUpdate(), where mPendingRequests.AppendElement will be called.
If the AppendElement in DownloadUpdate() causes realloc, the reference
becomes dangling.
The most efficient fix is to "move" the reference's (i.e. request)
member variables to DownloadUpdate() but I think in this case we can just
take the value from the array and pass it around with no given that the
array element contains simply a couple of strings and pointers.
MozReview-Commit-ID: KEZ5d3l3HoI
--HG--
extra : rebase_source : 4bf61a8a6f6bc57523dfbb7e5b0b40b7ce77a57a
The previous implementation regarding to the Flash Blocking Subdocument list blocked all subdocuments that matched the list. This patch changes that so that subdocuments are only blocked if they are on the Subdocument Block List and also are loaded in a Third-Party context.
The changes to cert8.db and key3.db add the https certificate for subdocument.example.com so that testing can verify that a scheme mismatch between the document and its parent results in a third-party classification.
MozReview-Commit-ID: IXnA4iPzB4y
--HG--
extra : rebase_source : 103c1e184d4219e6db9d00da1ea54674a0e216dd
Patch originally by Rahid Hasan. Updated by Akshendra Pratap Singh, jdm and Standard8.
MozReview-Commit-ID: F0edTCSsXzy
--HG--
extra : rebase_source : 7123a8162a4cdbcfb08f9a765d158e179c46b202
For simplicity, this test is being added to test_classifier.html which
already has all of the infrastructure necessary for setting up a test
domain as a tracking domain.
Chris Peterson
Prathiksha
Thomas Nguyen
Andrew McCreight
Sebastian Hengst
Steve Armand
Francois Marier
Eric Rahm
Henry Chang
Wes Kocher
Nicholas Nethercote
Dan Banner
Masatoshi Kimura
Carsten "Tomcat" Book
Bevis Tseng
Kartikaya Gupta
Marco Castelluccio
Sylvestre Ledru
Andrea Marchesini
Nathan Froyd
Bill McCloskey
DimiL
Florian Quèze
Felipe Gomes
Kirk Steuber
Ryan VanderMeulen
Kershaw Chang
Chung-Sheng Fu
Jeff Walden
Michael Layzell
Iris Hsiao
Joel Maher
"unknown@unknown.com"
Sébastien Blin
Mark Banner
Benjamin Smedberg
David Major
Tom Tromey
Tim Huang
Phil Ringnalda
Markus Stange
Ehsan Akhgari