29bfb51675
Individual feature importance interpret QA ( #2186 )
...
* add
Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com>
* update
Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com>
* cleanup
Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com>
* lintfix
Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com>
* lintfix
Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com>
* lintfix
Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com>
* fix row change err
Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com>
* address comments
Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com>
---------
Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com>
2023-07-24 17:32:44 +00:00
37c340c8d6
Vision Model Overview E2E Tests - Refactor ( #2185 )
...
* mo test refactor ckpt
* test fixes
* test flags for vision
* test fixes
* auto lint fixes
* removed duplicate constants
* comment fixes
* test & build fixes
* test fixes
* auto lint fixes
* comment updates
2023-07-24 15:25:14 +00:00
90db5e67e4
Vision Data Analysis / Data Balance E2E Tests + Refactor ( #2183 )
...
* refactor ckpt
* removed inapplicable imports
* removed vision spec folder
* spec refactor
* readded spec files for test checks
* isVision fix
* auto lint fixes
* added vision flags to datasets
2023-07-20 11:57:09 -04:00
8898e8226e
Image Classification Notebook for UI testing #2175 ( #2176 )
...
* ic nbk ckpt
* release command support
* python lint fixes
* dataset support
* ci vision nbk gate support
* auto lint fixes
* vision date support + import fixes
* lint fix
* run command syntax fix
* lint fixes
* vision_notebooks mark support & gate update
2023-07-17 14:52:05 +00:00
e433fdf547
Multiclass Image Classification Notebook for UI testing ( #2175 )
...
* multiclass nbk ckpt
* multiclass filename update
* nbk e2e support
* python lint fixes
* multilabel dataset support
* multilabel test support
* vision nbk gate support
* added component id
* disabled error analysis function
* nbk name fix
* nbk name fix
* vision gate support
* import fixes
* import fix
* visitVision import fix
* auto lint fixes
* doc & test fixes
* spec path fix
* dataset reference fix
* run command syntax fix
* auto lint fixes
* vision_notebooks mark support & gate update
2023-07-14 16:10:36 -04:00
5dc415964e
[Object Detection] Model Overview Cache Support Extension for torchmetrics ( #2170 )
...
* cache ext ckpt
* cache ext working ckpt
* python lint fixes
* auto lint fixes
2023-07-13 22:30:10 +00:00
f3d8cbba88
Update text insights and error_analysis_manager to handle pd.DataFrame predictions and prediction with type of string ( #2179 )
...
* Update rai_text_insights.py
* Update error_analysis_manager.py
* Update rai_text_insights.py
2023-07-13 13:10:28 -07:00
b7b5b856b5
ModelAssessmentVision OD + UI Test Template ( #2174 )
...
* initial nbk commit
* vision test template
* added vision support for visit
* lint fixes
* auto lint fixes
* removed spurious import
* fridge OD test case
* release command updates
* python lint fixes
* removed unused imports
* added vision CI nbk test gate
* gate name update
* commented error analysis function
* auto lint fixes
* ci e2e nbk vision gate setup
* .github/workflows/CI-e2e-notebooks-vision.yml
* dataset reference & flight fix
* export fixes
* import lint fix
* auto lint fixes
* removed py 3.6 support from vision
2023-07-13 15:38:49 -04:00
9bfd7801c8
add base values and importances for ending tokens to QA explanations ( #2178 )
2023-07-12 12:25:59 -04:00
4b76dc7fb4
Localized file check-in by OneLocBuild Task: Build definition ID 21368: Build ID 99116202 ( #2177 )
2023-07-11 12:13:51 -07:00
71efc3b66d
Doc update for `responsibleai_text` ( #2169 )
...
* Doc update for rai vision
* RAI text update
2023-07-10 20:22:27 +00:00
5828c2114a
Individual feature importance interpret QA - part1 ( #2167 )
...
* add individual feature importance for QA
temporally use a dropdown to select token
* make QA UI compatible with text classification
temporally using hardcode isQA flag as the prop for identifying qa from an upper level is TBD
* fix left textbox data
fix left textbox importance data and add description for qa interpret
* add isAllDataCohort (#2094 )
Signed-off-by: RubyZ10 <zhenzhu@microsoft.com>
* retain selections (#2096 )
Signed-off-by: RubyZ10 <zhenzhu@microsoft.com>
* Adding Class Name To OD Dropdown (#2098 )
* changing dropdown to include class name
* prettier
* Update erroranalysis-dashboard-regression-superconductor.ipynb (#2102 )
* Port over logic of `load_adult_income_dataset` from `dice-ml` to unblock gates (#2106 )
* Port over logic of `load_adult_income_dataset` from `dice-ml` to unblock gates
Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>
* Fix linting and add TODO
Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>
* Add fixture for dataset fetch
Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>
---------
Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>
* refactor cohortlist (#2103 )
Signed-off-by: RubyZ10 <zhenzhu@microsoft.com>
* Fix dimension mismatch error when dropped_features contains cat_features (#2099 )
* fix error when dropped_features contains cat_features
* fix lint
* add tests
* add tests
* fix UT error
* Add more dataset utilities in `rai_test_utils` (#2101 )
Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>
* release erroranalysis 0.4.4 (#2105 )
* update responsibleai and raiwidgets to erroranalysis 0.4.4 (#2107 )
* enable confusion matrix for binary and multiclass text and vision RAI dashboards (#2108 )
* release raiwidgets and responsibleai 0.28.0 (#2109 )
* Release `rai-test-utils` 0.3.0 (#2110 )
* Update rai_test_utils in erroranalysis and responsibleai to 0.3.0 (#2111 )
* Update rai_test_utils in erroranalysis and responsibleai to 0.3.0
* Update requirements-dev.txt
* update econml to latest 0.14.1 release (#2112 )
* causal fix (#2113 )
Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com>
* [Image Explorer] CanvasTools Image Loading support for Object Detection (#2097 )
* bbox vott template ckpt
* canvastools image load ckpt
* loading image from scratch ckpt
* canvastools image loading support
* coordinate fixes
* regiondata call + path fix
* code cleanup
* image loading ckpt
* callback image loading support
* lint fixes
* disabling internal imports for vott
* file refactor lint fix
* auto lint fixes
* lint fixes
* image dims arg for frontend bbox
* lint fixes
* lint fixes
* image dimension support
* canvas module added
* comment fix
* async image loading support
* lint fixes
* lint fixes
* max-lines lint fixes
* lint fixes
* Add columns names to `create_multiclass_classification_dataset` (#2114 )
Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>
* [Object Detection] [Image Explorer] Bounding Box Support with CanvasTools on Flyout (#2116 )
* bbox vott template ckpt
* canvastools image load ckpt
* loading image from scratch ckpt
* canvastools image loading support
* coordinate fixes
* regiondata call + path fix
* code cleanup
* image loading ckpt
* callback image loading support
* lint fixes
* bounding box working ckpt for fridge
* disabling internal imports for vott
* freeze rough ckpt
* file refactor lint fix
* auto lint fixes
* freeze cleanup
* lint fixes
* image dims arg for frontend bbox
* lint fixes
* lint fixes
* image dimension support
* imagedim support comments
* image dimension support
* image dimension support
* canvas module added
* image dimension working ckpt
* comment fix
* cleanup
* async image loading support
* lint fixes
* lint fixes
* reverted max line override
* code cleanups
* lint fixes
* auto lint fixes
* comment fixes
* add ResponsibleAI Text package as initial commit
* update responsibleai-text package to responsibleai v0.25.0
* add model-agnostic explanation support for wrapped models with simple text tokenizer (#1 )
* add build and release github actions to responsibleai-text package (#2 )
* Enable error analysis for QA (#5 )
* add error analysis for QA model
* validate github actions, fix linting and builds (#4 )
* added question type feature extractor (#6 )
Added regex based question type feature extractor
---------
Co-authored-by: Jarvis Gao <jarvisg495@gmail.com>
Co-authored-by: Jarvis Gao <86736305+JarvisG495@users.noreply.github.com>
* remove train data parameter from RAITextInsights (#7 )
* release responsibleai-text v0.1.0 (#8 )
* cache error analysis predictions for RAITextInsights to optimize performance when retraining tree or creating new matrix (#9 )
* update responsibleai-text package to responsibleai v0.27.0 (#11 )
* release responsibleai-text v0.1.1 (#12 )
* Add QA feature context overlap and improve question type regex (#14 )
* Update .gitignore
* Improve regex matching
modify regex to cover more cases
* add feature context_overlap
* add unit tests for QA feature extractors
* fix CI
* fix test case
* fix task type for multilabel and multiclass classification when passed to UX (#15 )
* release responsibleai-text 0.1.2 (#16 )
* Python backend metrics for question answering scenario (#17 )
* Fixed metrics and added bertscore
* Update setup.py to add extras for qa metrics
* Update CI-python.yml to install qa extras
* fix linting error
* linting fixup
* fix lint
* fix flake8 linting
* fix install command in CI
---------
Co-authored-by: akhil-2709 <aanthireddy@umass.edu>
Co-authored-by: Neutrinowo <86736305+JarvisG495@users.noreply.github.com>
* added-syntactic-complexity (#18 )
* added-syntactic-complexity-feature
Added syntactic complexity features, i.e., average and maximum parse tree depth for both Context and Questions.
* fixed-lint
* add metadata support to RAITextInsights (#19 )
* fix feature metadata serialization logic in RAITextInsights which was causing exceptions in error analysis (#22 )
* add text_column parameter to RAITextInsights to allow user to specify the text columns to differentiate from metadata (#24 )
* release responsibleai-text 0.1.3 (#21 )
* fix responsibleai-text linting and move directory to
responsible-ai-toolbox
* Add loading spinner for causal policy tab (#2121 )
* Add loading spinner for causal policy tab
Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>
* Address code review comments
Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>
---------
Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>
* Refactor- Add datasetCohorts to ModelAssessementContext && create datasetCohort && add compositeFilter in DatasetCohort (#2119 )
* refactor
Signed-off-by: RubyZ10 <zhenzhu@microsoft.com>
* address comments
Signed-off-by: RubyZ10 <zhenzhu@microsoft.com>
---------
Signed-off-by: RubyZ10 <zhenzhu@microsoft.com>
* remove console log (#2125 )
Signed-off-by: RubyZ10 <zhenzhu@microsoft.com>
* Update msdata-rai-widgets-onelocbuild.yml for Azure Pipelines
* fix build failures due to lint errors for onelocbuild yml (#2137 )
* Skipping python 3.7 tests on MACOS (#2129 )
* [WIP] Debugging _bz2 failures
* Update CI-python.yml
* Update CI-python.yml
* Update CI-python.yml
* Update CI-python.yml
* Update CI-python.yml
* Update CI-python.yml
* `rai_test_utils`: Convert all namespaces in `__all__` list as strings (#2128 )
Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>
* User Flows / Architectures
* [Object Detection] DFD Documentation (#2138 )
* dfd
* yarn lint
* [Object Detection] Explanation Documentation (#2136 )
* docs for expl
* fixing name
* docs correcting image
* yarn lintfix
* Update docs/explanation-object-detection-README.md
Co-authored-by: Advitya Gemawat <agemawat@microsoft.com>
* removing dpv2
---------
Co-authored-by: Advitya Gemawat <agemawat@microsoft.com>
* CanvasTools documentation
* isQA as prop
Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com>
* update and refine
Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com>
* remove state isQA
Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com>
* clean
Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com>
---------
Signed-off-by: RubyZ10 <zhenzhu@microsoft.com>
Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>
Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com>
Co-authored-by: Neutrinowo <86736305+JarvisG495@users.noreply.github.com>
Co-authored-by: Ruby Zhu <76190371+RubyZ10@users.noreply.github.com>
Co-authored-by: natalie-isak <123979655+natalie-isak@users.noreply.github.com>
Co-authored-by: tongy-msft <91754176+tongyu-microsoft@users.noreply.github.com>
Co-authored-by: Gaurav Gupta <47334368+gaugup@users.noreply.github.com>
Co-authored-by: Ilya Matiach <ilmat@microsoft.com>
Co-authored-by: Advitya Gemawat <agemawat@microsoft.com>
Co-authored-by: Sreeja Gaddamidi <37956427+sreeja-g@users.noreply.github.com>
Co-authored-by: Jarvis Gao <jarvisg495@gmail.com>
Co-authored-by: mehektulsyan <31842027+mehektulsyan@users.noreply.github.com>
Co-authored-by: akhil-2709 <aanthireddy@umass.edu>
2023-07-10 14:36:57 -04:00
d5643aeca5
update responsibleai-text and responsibleai-vision packages to responsibleai 0.29.0 ( #2172 )
2023-07-09 22:37:10 -04:00
d882424f1c
[Question Answering] Cache support for Model Overview Metrics ( #2166 )
2023-07-07 11:06:47 -04:00
4506b0dcd5
update to version 0.29.0 ( #2168 )
...
* update to version 0.29.0
* indent and add causal section
* more indent fixes
2023-07-06 21:33:36 +00:00
0d5e41a38e
raise error instead of warning when a user has missing data and add c… ( #2143 )
...
* raise error instead of warning when a user has missing data and add check for train data in addition to test
* address comments
2023-07-05 17:19:20 -07:00
9bfca4b5ce
Doc update for rai vision ( #2164 )
2023-07-05 18:19:23 +00:00
a7cb6ecf5e
fix job names in release scripts ( #2165 )
...
Co-authored-by: Roman Lutz <romanlutz13@gmail.com>
2023-07-05 10:55:05 -04:00
bd066380c3
[Object Detection] Cache support for Model Overview Metrics ( #2158 )
...
Cache support for Model Overview Metrics
2023-07-03 22:11:46 +00:00
811b986f4a
release responsibleai-text 0.1.4 ( #2163 )
2023-07-03 16:45:14 -04:00
f4c2618801
release responsibleai-vision 0.2.3 ( #2162 )
2023-07-03 15:25:38 -04:00
fda337ddb7
Update the python version to 3.8 in release workflows ( #2161 )
...
Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>
2023-07-03 18:31:41 +00:00
af228eb221
add device parameter to responsibleai_vision package and pass to model wrappers ( #2157 )
2023-07-03 12:23:30 -04:00
c7bd5ab40f
Localized file check-in by OneLocBuild Task: Build definition ID 21368: Build ID 97902894 ( #2160 )
2023-07-03 10:22:10 -04:00
99fba6aeaf
update responsibleai-text and responsibleai-vision packages to latest responsibleai 0.28.0 release ( #2159 )
2023-06-29 17:21:29 -04:00
24e17017fc
add release scripts for responsibleai-text and responsibleai-vision pypi packages ( #2156 )
2023-06-29 15:28:46 -04:00
058814adcd
add retry logic to common vision utils to make tests more robust ( #2154 )
2023-06-29 13:20:41 -04:00
ea91fda1dd
add responsibleai-vision package to gated CI tests ( #2152 )
2023-06-29 13:18:50 -04:00
8b059d6f5d
fix build failures due to new joblib 1.3.0 release failing on windows with python 3.7 ( #2155 )
2023-06-29 10:53:45 -04:00
234c439411
add validation target column type in the classification scenario ( #2127 )
...
* add validation for the classification scenario when a user input a float-type target column
* use is_float_dtype
* add check if the float can be converted to integer
2023-06-27 13:20:16 -07:00
4aa5fb058c
add responsibleai-text package tests to build gate ( #2139 )
2023-06-27 13:27:55 -04:00
9c5940b3f5
Robot page exception ( #2148 )
2023-06-26 18:38:40 -07:00
af9b099054
Update dice-ml to 0.10.0 ( #2145 )
...
* Update dice-ml to 0.10.0
Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>
* Fix lint
Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>
* Fix failing test
Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>
---------
Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>
2023-06-26 23:18:38 +00:00
5bedefb59f
add responsibleai-vision package to responsible-ai-toolbox ( #2135 )
2023-06-26 17:09:00 -04:00
5671b4cb63
Model Overview Object Detection Documentation
2023-06-26 20:14:42 +00:00
32408d52d6
CanvasTools documentation
2023-06-26 18:13:08 +00:00
80f4b9ef9e
[Object Detection] Explanation Documentation ( #2136 )
...
* docs for expl
* fixing name
* docs correcting image
* yarn lintfix
* Update docs/explanation-object-detection-README.md
Co-authored-by: Advitya Gemawat <agemawat@microsoft.com>
* removing dpv2
---------
Co-authored-by: Advitya Gemawat <agemawat@microsoft.com>
2023-06-26 11:34:12 -04:00
217e2b8fed
[Object Detection] DFD Documentation ( #2138 )
...
* dfd
* yarn lint
2023-06-26 11:33:16 -04:00
4962a6c4a1
User Flows / Architectures
2023-06-26 14:39:46 +00:00
b3040cc447
`rai_test_utils`: Convert all namespaces in `__all__` list as strings ( #2128 )
...
Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>
2023-06-24 01:02:16 +00:00
9e6eef71d5
Skipping python 3.7 tests on MACOS ( #2129 )
...
* [WIP] Debugging _bz2 failures
* Update CI-python.yml
* Update CI-python.yml
* Update CI-python.yml
* Update CI-python.yml
* Update CI-python.yml
* Update CI-python.yml
2023-06-23 23:36:40 +00:00
a1837e656e
fix build failures due to lint errors for onelocbuild yml ( #2137 )
2023-06-23 16:21:31 +00:00
9a3c43e48e
Update msdata-rai-widgets-onelocbuild.yml for Azure Pipelines
2023-06-21 15:27:53 -07:00
fceba5df82
remove console log ( #2125 )
...
Signed-off-by: RubyZ10 <zhenzhu@microsoft.com>
2023-06-21 19:24:07 +00:00
8631a184d7
Refactor- Add datasetCohorts to ModelAssessementContext && create datasetCohort && add compositeFilter in DatasetCohort ( #2119 )
...
* refactor
Signed-off-by: RubyZ10 <zhenzhu@microsoft.com>
* address comments
Signed-off-by: RubyZ10 <zhenzhu@microsoft.com>
---------
Signed-off-by: RubyZ10 <zhenzhu@microsoft.com>
2023-06-20 17:59:53 +00:00
f844c3422a
Add loading spinner for causal policy tab ( #2121 )
...
* Add loading spinner for causal policy tab
Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>
* Address code review comments
Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>
---------
Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>
2023-06-20 01:57:41 +00:00
10bce0343a
fix responsibleai-text linting and move directory to
...
responsible-ai-toolbox
2023-06-16 10:32:06 -04:00
6dea5207b9
release responsibleai-text 0.1.3 ( #21 )
2023-06-16 10:32:06 -04:00
eea0bff4ba
add text_column parameter to RAITextInsights to allow user to specify the text columns to differentiate from metadata ( #24 )
2023-06-16 10:32:06 -04:00
ce76dd0fa8
fix feature metadata serialization logic in RAITextInsights which was causing exceptions in error analysis ( #22 )
2023-06-16 10:32:06 -04:00
Vinutha Karanth
Advitya Gemawat
tongy-msft
Ilya Matiach
Ruby Zhu
hawestra
Gaurav Gupta
natalie-isak