Граф коммитов

22 Коммитов

Автор SHA1 Сообщение Дата
Prajwal7842 6a8bd88b87
Update environment.yaml (#760) 2020-10-08 11:00:56 +02:00
Stefan Zabka 1df845a3b2
Release v0.12.0 with FF80 (#727)
* Upgrade to FF79

* Fixed test_cache_hits_recorded

* Preparing release v0.12.0 with FF80

* Added regression note
2020-08-26 16:02:03 +02:00
Sarah Bird ad5681f4f9
v0.11.0 / Firefox 78.0.1 release (#706)
* Update to FF 78_0_1
* npm updates
* Repin python
* New version number.
* Make CHANGELOG a markdown file, so it renders nicely
2020-07-08 18:33:27 -05:00
Sarah Bird aefa048612
Arbitrary WebAPI JS instrumentation (#642)
* Add mdn-browser-compat-data

* js_instrument_modules as list

* Add mdn-browser-compat

* Pass a list of instrumentingFunctions

* Script to generate api data

* Working give or take

Getting errors like

OpenWPM: Error name: TypeError post_request_ajax.html:237:17
OpenWPM: Error message: can't redefine non-configurable property
"UNSENT" post_request_ajax.html:238:17

* Small naming cleanup

* Handle non-configurable properties

* Lint

* Add aspirational API

* Begin migration to new JSInstrumentationRequest interface.

* We build and mandate LogSettings.
* We have a new JSInstrumentatinRequest that everything runs through
* Preset, fingerprinting, will be specified in JSON

* Continue making progress

Enum for Operation

* Begin implementing jsModuleRequest validation.

Changing my mind - all validation and construction to be done python
side.
This will reduce JS overhead at runtime.

* Big cleanout- js-instrumentation work moving to python.

* Continue update to python js-instrumentation

* Lint

Can't do all the things I want to with typing due to scope
when content is loaded into page.

* noqa on wip jsinstrumentation file

* Begin updating existing js instrument tests.

* Small cleanups

* Fix naming in calling instrumentJS

* No display mode native for testing

* Restore py test file to orig.

* Support null propertiesToInstrument

* Re-work instrumentObject tests

* Clean-up text in test page.

* Add default to getLogSettings function

* Don't re-assign logSettings.propertiesToInstrument

* Revert "Don't re-assign logSettings.propertiesToInstrument"

This reverts commit 87ccdabf9a.

* Better assign propertiesToInstrument

* Small cleanup

* Make new logSettings object

* Prettify

* Small clean

* -- BREAK -- JS Rework complete

With this commit, the JS side of this PR is complete.
Tests are still failing as fingerprinting implementation has not been
completed on the python side, but all test_js_* tests are passing due
to the core JS API rework being in place.

* Write-out mdn compat data to js_instrumentation .py

* Dry out js test code

* Consolidate JS tests

* Finish missing renames, and add test js via browser_params

* pep8

* New files and failing tests.

* Add a json schema for js_instrument_modules

* Latest py tests

* Flake8

* Ongoing progress.

* More code, more tests.

* flake

* Rename mdn file

* Add latest tests - just implement fingerprinting.json

* flake8

* Add fingerprinting.json (incomplete)

Mimetypes and plugins

* Correct logSettings property name

* Restore create_xpi as function

Needed by manual_test

* Make explicit option for logging to console

* Process browser_params in task manager

* Start being able to pass browser_params to selenium

Also update manual_test to use click

* Revert "Make explicit option for logging to console"

This reverts commit c840fbc5d8.

* Get manual_test working with browser_params

From toplevel directory run:

`python -m test.manual_test --selenium --browser-params --browser-params-file=debug_params.json`

* More robust test for simple fingerprinting output

Can't guarantee order of string output

* Add timing information when testing

* Make recheck really fast.

You'll never hit this recheck as it all happens before page load.

* Handle all inputs properly

* Debug with all window params instrumented

* Load xpi we just built

* Check for ff version support

* Save a bunch of properties

* Relax constraints on what we can instrument.

Let failing happen during instrumentation by using subscript notation.
Don't restrict to MDN list.

* Correct stringifying

* Better name example params, fix some bugs, sample a_f

Some example browser_params - a_f is just working - but crushes
on a page like google.com. g_l and m_z haven't been vetted yet.

* flake8

* Move example browser_params file out of harms way

* Add failing test for regression I introduced.

* Fix for regression.

* Add simple mimeTypes and plugins to fingerprinting.

* Lint JS

* Rm mdn_browser_comat stuff no longer needed

* Remove example_browser_params

They're not used in tests, were just for my testing.

* Load JS_INSTRUMENT_MODULES from JSON string

* Rename JS_INSTRUMENT_MODULES to JS_INSTRUMENT_SETTINGS

* Fixes #28 - Instrument all window.navigator properties.

* Finish removing unused mdn-compat pieces.

* EventID as a shadow variable

* Flake8

* Remove $ prefix and rename

$instrumentionRequests -> jsInstrumentationSettings

* Rename jsInstrumentationRequests->jsInstrumentationSettings

* TS Lint

* Remove use of "request".

Rename python side as per discussion with @englehardt.
Privatize most methods
Numpy docstrings for public methods

* Convert assertions to ValueErrors

* Rename file/folder and fingerprinting -> collection_fingerprinting

file JSInstrumentation.py -> js_instrumentation/__init__.py
collections have their own folder

* Clean-up naming in schema

* Add processing of json schema to documentation

* Rename js_instrumentation again and ref schema location

* Pass JSON not a js string

* Do copying to xpi in npm postbuild step

* Fix import in manual_test

* Revert "Pass JSON not a js string"

This reverts commit 8eb4edb542.

* Add titles to schema pieces

* Add docs for js_instrument_settings

* Bit more README cleanup

* Update README.md

Co-authored-by: Steven Englehardt <englehardt@gmail.com>

* Move updating schema docs section

* Add title
* Fix typo in mac-osx hyperlink

* Make the single-key dictionary clearer

* Remove versions from npm package files

* Clean up instrument_existing_window_property.html and js

We're not using the js in two htmls now, so unify like other test files

* Fix pyside instrumentation test, add more clarificaiton to README

* pyside test must instrument browser apis
* add more to readme to clarify instrumenting

* Use example.com and example.org as localDomains

* context-manage open, and flake8

Co-authored-by: Steven Englehardt <englehardt@gmail.com>
2020-07-08 15:55:22 -07:00
Steven Englehardt 2e522aecc8
v0.10.0 / Firefox 77.0.1 release (#682)
* v0.10.0 release

* Add new section to TOC

* Remove old section from TOC

* Add a note about release cadence

* Ignore python user packages from ~/.local when building env

See: https://github.com/conda/conda/issues/448#issuecomment-195848539

* Fix generated environment

* Pin to localstack 0.11.1 since 0.11.2 seems to be causing issues

* Pinning localstack dependencies as well

* Pinning only  to fix timeouts

* Pinning pyvirtualdisplay due to incompatible interface change

* Add comment to describe new env var

* Pin localstack to 0.11.1.1

* Ignore user site packages in install.sh and for dev dependencies

* Add a troubleshooting note related to 689

* Extend README comment to request feedback
2020-06-22 15:29:44 -07:00
Shubhank Saxena abd613df2e
Add codecov report (#661)
* Added coverage for js tests
Added codecov as python dependency and uploaded the test

* added codecov yaml file

* updated dependency for codecov

* updated dependency for codecov

* end semicolon fix

* fixed xml generation

* position change of args

* minor tweaks

* fixed codecov option

* added pytest cov

* added pytest cov

* added semicolon

* report generation
2020-06-22 14:43:11 -07:00
Sarah Bird b7da58902e Remove xvfb from conda environment. 2020-05-15 15:43:42 -05:00
Sarah Bird 7ccfc14c7d Fix missing dependencies 2020-05-14 01:55:20 -05:00
Sarah Bird 83bc5c8bee Bugfix prune-environment 2020-05-14 00:59:52 -05:00
Sarah Bird 705de01e03 Try a pruning strategy
Alternative cross-platform strategy
2020-05-14 00:14:30 -05:00
Sarah Bird 475057b087 No builds in environment.yaml 2020-05-13 22:52:42 -05:00
Sarah Bird 9b47966ddf Add missing localstack dependencies 2020-05-13 21:26:16 -05:00
Sarah Bird d00370589a Add missing pillow dependency 2020-05-13 19:51:43 -05:00
Sarah Bird b9647980ce Add psutil dependency 2020-05-13 19:41:07 -05:00
Sarah Bird e77edfacfc Add dev dependencies to environment 2020-05-13 19:10:15 -05:00
Sarah Bird 747a4cae41 Add pinning capabilities 2020-05-13 18:12:30 -05:00
Sarah Bird 330049995a Add new install script 2020-05-13 16:55:03 -05:00
Sarah Bird 8c51249e7b Merge branch 'master' into conda-install-dependencies 2020-05-13 16:07:55 -05:00
Sarah Bird f75cfe9300 Finish mac dependencies 2020-05-13 15:10:16 -05:00
Sarah Bird 87702988fd Fixes #638, replaces utilities/domain_utils.py with domain_utils 2020-05-13 14:34:05 -05:00
Sarah Bird 288a1e8f6d WIP 2020-05-13 14:19:34 -05:00
Sarah Bird b31e9f8f3f Start moving dependencies to environment.yaml 2020-04-17 13:07:15 -05:00