Граф коммитов

38 Коммитов

Автор SHA1 Сообщение Дата
Wayne Hong fde30dd4bd
Add support for allow_list, allow_list_match, regex_flags in REST API (#1478) 2024-11-11 22:41:48 +02:00
Omri Mendels 33808c2837
Removed python 3.8 support (EOL) and added 3.12 (#1479) 2024-11-04 11:47:02 +02:00
Omri Mendels f29e112fd3
Update conf files location (#1358) 2024-04-18 10:42:18 +03:00
Paulo Costa 24a76a8f44
Added "keep" anonymizer (#1062) 2023-04-30 10:25:00 +03:00
Omri Mendels 105d9455c5
Install transformers model into the docker image (#912) 2023-01-25 09:06:02 +02:00
Sharon Hart 11dfa64d63
Image Redactor - REST API to support web applications payload (#1009)
* Image redactor - REST API to support web application

* revert port

* lints

* Fix versioning, bump pillow and analyzer

* lower score

* Fix versioning, bump pillow and analyzer

* try to fix test

* add e2e test

Co-authored-by: sharon <sharon.hart@microsoft.com>
Co-authored-by: Omri Mendels <omri374@users.noreply.github.com>
2023-01-18 11:24:55 +02:00
Omri Mendels 77f1a8e125
Remove constraint on empty texts for the anonymizer (#942) 2022-10-25 13:32:30 +03:00
Omri Mendels 0c7117e81a
Compare recognizers to results using a unique key instead of name (#940) 2022-10-24 15:40:11 +03:00
Omri Mendels 8c661fef34
Updates to deny_list recognizer (#858) 2022-05-02 17:05:38 +03:00
Guy Bertental a0c071b798
refactor and modularize context words logic (#828) 2022-02-22 09:12:50 +02:00
Anat Balzam 81c8c49ebb
URL Recognizer (#825)
* initial url recognizer

* fix linting

* CR comments

* Added to api-docs

* Adding a different validator to url recognizer

* change credit

* remove domain_recognizer.py

* More elegant regex

* Add change log

Co-authored-by: Anat Balzam <anatbalzam@microsoft.com>
Co-authored-by: Omri Mendels <omri374@users.noreply.github.com>
2022-02-21 13:26:23 +02:00
Sharon Hart cedcb4cb4c
Remove AnonymizerResult in favor of OperatorResult (#795)
* Remove AnonymizerResult in favor of OperatorResult

* fix test

* documentation changes + align rest api

* operator_name

Co-authored-by: sharon <sharon.hart@microsoft.com>
2021-11-16 11:17:07 +02:00
Omri Mendels 31e8675930
Changes to Anonymizer packages/imports + added repr to classes. (#763)
* modules changes, doc changes

* updates on notebook and e2e

* backward compatibility of OperatorConfig

* revert file structure

* revert customize_presidio_analyzer, doesn't belong in this PR

* update to spark notebook

* updates to notebook

* Update getting_entity_values.ipynb

* Update getting_entity_values.ipynb

* grouped imports

* Update test_context_support.py
2021-09-26 14:49:48 +03:00
Rakan b8d76c6a7e
Issue 754 (#755) 2021-09-05 12:58:17 +03:00
Sharon Hart aea7804219
Add Medical License as a Predefined Recognizer (#719) 2021-05-27 11:42:42 +03:00
zhangabner 1d40193cb0
add pii_verify_engine (#646) 2021-05-06 13:47:45 +03:00
Omri Mendels d5d9551d57
Ad-hoc recognizers (#677) 2021-05-05 10:18:01 +03:00
Nava Vaisman Levy cc9a6012ca
Custom anonymizer with lambda type (#654) 2021-05-04 10:07:28 +03:00
Shiran Rubin 771d38c5bd
Add deanonymize (#639)
* Add decryption mechanism.
On purpose we separate it from the anonymizer/

* Fix tests

* Remove unneeded params
fix docs and tests

* Add decryption engine to match the anonymizer engine.
Refactor code so the text builder and text manipulator will be mutual.

* Change engine result to be unified with different items inheriting from result item.

* Fix decrypt elements and implmentations

* Add hints

* Fix naming

* Change manipulators to operators.

* Change manipulators to operators.

* Move initialization of text manipulator
Change the analyzer results json to receive the list and not the entire content.
Fix tests

* Move getting operators classes into the operator factory.

* Fix entities.
Make sure all engines can use the same entities by using factories, builders and inheritance more.

* revert unneeded change

* Code cleanup

* Add tests and clean up code.

* Move anonymizer configuration to inherit from operator metadata and add decryptor config

* More cleanup

* More cleanup

* revert

* cleanup

* Fix imports and renaming

* Start working on linting.

* Start working on linting.

* add tests

* add tests

* add tests

* add tests

* remove unneeded formating.

* fix linting

* try to understand why it fails in pipe

* try to understand why it fails in pipe

* Fix equal check for recognizer result

* revert change for test investigation

* fix imports and add test

* fix dictionary

* add test for dictionary fix

* Fix example

* Fix E2E entities

* Fix PR comments

* Fix PR comments

* Fix PR comments

* Fix PR comments

* Fix PR comments

* Fix PR comments - use comprehensions

* Fix PR comments - Change names and move classes accordingly.

* fix linting

* fix linting

* use dictionary comprehensions

* Fix documentation and nameing.

* Fix documentation and nameing.

* remove unused code.

* PR comments

* fix linting

* PR review -> be background compatible.

* fix linting

* fix linting due to merge

* Fix PR comments

* linting

* remove my name from code.

* Add engine test for encrypt decrypt in integration tests
Add helper method to change anonymized entity into encrypt result.

* add docs

* fix import

* Remove unneeded field

* Replace decrypt with deanonymize.

* Replace decrypt with deanonymize.

* revert

* Change to anonymizer_results

* Remove entities and merge them together.

* Remove entities and merge them together + renaming

* Remove entities and merge them together + renaming

* fix linting

* fix documentation and naming

* fix documentation and naming

* fix documentation and naming

* fix documentation and naming

* fix documentation and naming

* fix documentation and naming

* fix documentation and naming

* fix documentation and naming

* fix tests

* fix tests

* fix tests

* rename

* fix docs

* Fix PR comments

* Fix linting

* Fix linting

* fux PR comments

* fux PR comments

* fix PR comments

* fix PR comments

* add tests

* add tests

* fix PR comments

* fix PR comments

* fix PR comments

* fix PR comments

* bump version for breaking changes
2021-04-06 15:45:10 +03:00
Sharon Hart 553e089921
Analyzer and Anonymizer Python 3.9 support (#636)
* Python 3.9 support

* Revert image redactor

* test lower versions.

* ignore test

* downgrade spacy

* downgrade spacy

* downgrade spacy

* downgrade spacy

* remove version section

* e2e on 3.9

* add classifiers

Co-authored-by: sharon <sharon.hart@microsoft.com>
Co-authored-by: Omri Mendels <omri374@users.noreply.github.com>
2021-04-04 11:29:42 +03:00
Sharon Hart 9eff2d41a1
Return application/json mime type in all endpoints (#626)
* Return application/json mime type from all endpoints

* Fix response

* Use dumps in analyze

* mimetype for image redactor

Co-authored-by: sharon <sharon.hart@microsoft.com>
2021-03-22 15:25:18 +02:00
Itye Richter bf46a61a6d
added e2e package test (#618) 2021-03-17 12:30:26 +02:00
Itye Richter 13920dbec9
Anonymizer Indexes in response (WIP) (#604)
* added indexes

* fix tests

* linting

* e2e fixes

* fix e2e tests

* pr fix

* linting

* added types

* added types

* more tests

* pr fixes

* more tests

* pr fixes

* Update presidio-anonymizer/presidio_anonymizer/entities/anonymizer_result.py

Co-authored-by: Omri Mendels <omri374@users.noreply.github.com>

* pr fixes

* pr fixes

* pr fixes

* pr fixes

* swagger update

* swagger

* type fix

Co-authored-by: Itye <itye@ityer-2.local>
Co-authored-by: Omri Mendels <omri374@users.noreply.github.com>
2021-03-11 16:15:07 +02:00
Sharon Hart 059a38f9e8
Add Decryption for Single PII (#601)
* Add decryption

* Fix e2e tests

* change todo

* Add e2e test
Fix error text

* Fix error text

Co-authored-by: sharon <sharon.hart@microsoft.com>
2021-03-10 10:06:28 +02:00
Sharon Hart 9e2bbb1dab
Anonymizer - Encrypt anonymizer (#596)
* Add AES Encryption Logics

* Update NOTICE

* Key as method argument

* Anonymizer - Add encrypt anonymizer

* Anonymizer - Add encrypt anonymizer

* space

* Add documentation, tests

* Revert "Add AES Encryption Logics"

This reverts commit 18d991bc

* rename private method

* add key

* V1 to V2

* Add AES Encryption Logics

* Rebase to AES API change

* Add assertion on encryption

* change test names

Co-authored-by: sharon <sharon.hart@microsoft.com>
2021-03-08 11:39:45 +02:00
Omri Mendels a6d720eb86
Align analyzer and anonymizer RecognizerResult objects (#591) 2021-03-03 18:28:47 +02:00
Nava Vaisman Levy cdec0ef260
Anonymizer should return original text if analyzer results are missing (#573)
* anonymizer handle empty analyzer-results
2021-03-01 17:57:00 +02:00
Omri Mendels 61ed5452b8
Feature/rename interpretability (#553) 2021-02-25 13:00:45 +02:00
Shiran Rubin a3eb6e09db
remove FPE from anonymizer since we do not support it. (#552)
Co-authored-by: Nava Vaisman Levy <navalev@users.noreply.github.com>
2021-02-24 19:54:43 +02:00
Shiran Rubin 5084208b6d
V2 image redactor e2e (#543)
Add E2E tests for image redactor.
2021-02-23 13:36:14 +02:00
Nava Vaisman Levy f1d2783a17
fix- e2e tets (#533)
* minor doc update, fix issue with e2e tests
2021-02-18 11:33:50 +02:00
Shiran Rubin 7093281062
V2 transformation to anonymizer (#526)
* Clean the engine a little by moving the text handling to another entity.
First draft.
2021-02-17 13:51:52 +02:00
Sharon Hart 706a31a03a
Remove stub e2e integration tests (#522) 2021-02-16 18:59:02 +02:00
Itye Richter cafdd88607
new e2e flows (#506)
* new e2e flows

* pr fixes

Co-authored-by: Itye <itye@ityer-2.local>
2021-02-15 12:34:14 +02:00
Sharon Hart b96af984c8
Run E2E Integration Tests on Deployment (#502)
* Run e2e integration tests after deployment

* Add stage name

* build

* Take from vars, bump

* rebase

* Change path

* retest

Co-authored-by: sharon <sharon.hart@microsoft.com>
2021-02-11 18:02:52 +02:00
Nava Vaisman Levy 319321e34b
Dynamicly load anonymizers (#485)
* dynamicly load anonymizers list
2021-02-09 14:04:56 +02:00
Nava Vaisman Levy ffa53c103d
Anonymizer - additional integration tests scenarios (#482) 2021-02-07 18:05:06 +02:00
Sharon Hart aae7cf9136
Rename 'functional-tests' to 'e2e-tests' (#480)
* e2e tests pre-commit

* rename functional-test to e2e-tests

Co-authored-by: sharon <sharon.hart@microsoft.com>
2021-02-07 11:57:09 +02:00