Граф коммитов

329 Коммитов

Автор SHA1 Сообщение Дата
Ilya Matiach 6026326d23
upgrade nx to 12.10.0 (#2561) 2024-05-01 13:09:35 -04:00
Ilya Matiach 84428aa63f
add genai metrics endpoint in UI for model overview metrics (#2517) (#2520) 2024-01-30 14:00:41 -05:00
Ilya Matiach f1cf49c3f4
update UI to support genai task for RAI text dashboard (#2504) (#2508) 2024-01-26 13:15:25 -05:00
Ilya Matiach 49906bb3a4
upgrade plotly to latest version 2.27.1 (#2484) 2024-01-10 14:42:14 -05:00
Ilya Matiach afcc4ba35e
fix socketio cors authentication for object detection and qa metrics requests (#2455) 2023-12-13 10:12:35 -05:00
Ilya Matiach 73dc6211aa
fallback to request if socket connection fails for object detection and question answering metrics for backcompat (#2409) 2023-11-10 13:34:27 -05:00
Advitya Gemawat 4fa84f9c1d
Dashboard Port Updates (#2406)
* port update

* port fix
2023-11-02 16:50:33 -04:00
Ilya Matiach a65a087b64
add socket implementation to RAI dashboard object detection and question answering metrics calls (#2394) 2023-10-26 15:42:35 -04:00
Roman Lutz 426c8421ea
Forecasting: generate time series as cohorts (#2355)
* add pandas timestamps and test

* generate time series in UI if not passed in by Python (Studio only)

* fix bug related to datetime feature detection, add another test scenario

* lintfix

* undo change

* undo change

* add a test case

* prettier

* lintfix

* address comments

* undo changes to time / time series ID feature checks
2023-10-03 12:55:22 -04:00
Advitya Gemawat 896237b995
Sorting support for Object Detection labels (#2376)
* sort support ckpt

* build fix

* python lint fixes

* mock data fix
2023-10-02 11:42:00 -04:00
Advitya Gemawat 40af9df23b
Backend and OD Notebook Dataset fixes (#2360)
* pandas operation fix

* cohort fix

* windows test fix

* backend & dataset fixes

* reverted test file

* python lint fixes

* eslint updates

* auto lint fixes

* auto lint fixes

* test update

* removed file
2023-09-29 14:35:00 -04:00
Advitya Gemawat dbb1373c89
Numerical Exif Metadata support for Image Dashboards (#2358)
* numerical exif backend support

* UI & mock data updates

* auto lint fixes

* python lint fixes
2023-09-28 11:17:35 -04:00
Advitya Gemawat cfe1037f26
NoLabel fix for Object Detection (#2356) 2023-09-26 10:06:22 -04:00
Ilya Matiach 81912e7f27
add model explanation data to test app for question answering example on squad dataset (#2326) 2023-09-25 13:34:48 -04:00
Advitya Gemawat 057cbaca3b
Categorical Exif Metadata support for Vision Dashboard (#2348)
* rai vision insights metadata ckpt

* deepcopy support

* dashboard rendering ckpt

* Flyout Info fix

* mock data update with cat feats

* backend ckpt

* python lint fixes

* isort fix

* MO test updates

* OD MO test fix

* auto lint fixes
2023-09-24 22:19:32 -04:00
Advitya Gemawat a6ca894f4f
Detection info inside Object Detection Flyout (#2339)
* ckpt

* refactor + unit tests

* auto lint fixes

* ckpt

* od label UI ckpt

* od agg label working ckpt

* working backend

* UI update to remove pred label

* added UI tests

* lint fixes

* lint fixes

* MO test tweak for vision

* comment fixes

* comment fixes

* Flyout object update ckpt

* reverted MO changes

* mock data test fixes

* lint fixes

* lint fix

* auto lint fixes

* auto lint fixes
2023-09-21 10:58:51 -04:00
Advitya Gemawat ce3beaa6d3
Error Labeling integration into Image Data Explorer for Agg Labels (#2334)
* ckpt

* refactor + unit tests

* auto lint fixes

* ckpt

* od label UI ckpt

* od agg label working ckpt

* working backend

* UI update to remove pred label

* added UI tests

* lint fixes

* lint fixes

* MO test tweak for vision

* comment fixes

* comment fixes

* auto lint fixes

* reverted MO test changes

* auto lint fixes
2023-09-18 17:48:08 -04:00
Ilya Matiach eae21b7079
add text feature importance e2e unit tests for blbooksgenre jupyter notebook (#2327) 2023-09-12 16:21:43 -04:00
Ilya Matiach 7b7054fbb6
add text feature importance e2e unit tests for covid jupyter notebook (#2323) 2023-09-07 23:06:54 -04:00
Ilya Matiach ff48382c1e
add text feature importance e2e unit tests for dbpedia jupyter notebook (#2313) 2023-09-06 10:48:03 -04:00
Ilya Matiach efbc451628
move individual feature importance tests for text dashboard to common e2e package folder to re-use for unit and notebook tests (#2299) 2023-08-31 14:12:37 -04:00
Ilya Matiach f7bb1e8fd6
add e2e UI notebook tests to covid19 event multilabel text classification notebook (#2282) 2023-08-29 17:12:16 -04:00
Ilya Matiach bc4744ed2d
add e2e UI notebook tests to blbooksgenre text classification notebook (#2273) 2023-08-25 11:38:18 -04:00
Ilya Matiach e5af55817d
add e2e UI notebook tests to DBPedia text classification notebook (#2265) 2023-08-23 15:53:13 -04:00
Roman Lutz ef215504c2
Add forecasting to raiwidgets and add a forecasting notebook (#2252)
* Add postga build trigger (#1755) (#1756)

Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>

Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>

Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>

* Add model wrapper for wrapping predictions and test data (#1762)

* Add model wrapper for wrapping predictions and test data

Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>

* Fix failing tests

Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>

Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>

* Change description of cohort selection panel in Aggregate Feature Importance (#1770)

Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>

Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>

* Support cohort filtering of string target in rai_insights (#1771)

* Port tests

Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>

* Fix rai_insights

Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>

Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>

* Simplify tests in test_cohort_filter.py (#1772)

* Simply tests in test_cohort_filter.py

Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>

* Change test name

Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>

Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>

* Forecasting Dashboard

This commit adds the Forecasting Dashboard capabilities in both UI and SDK.

* remove unrelated file

* basic set of changes to get it working again

* forecasting_grains -> time_series_id_column_names

* fix breaking changes in latest main, add quantile prediction, fix date conversion, remove obsolete managers

* fix UI components (including table and what-if creation, chart still left tbd)

* get dashboard to work with multiple time series and switching back and forth

* remove data explorer and model overview, create and edit cohort functionalities from forecasting

* lintfix

* localizzation

* remove console.log

* remove question

* remove unrelated changes, fix package.json

* string fixes and package.json adjustment

* localization

* localization

* more localization and removing of unrelated files

* remove unrelated files

* remove unrelated file changes

* remove is_forecasting_true_y

* lintfix

* add dropdown label

* feature metadata input validation updates

* cache transformation predictions

* lintfix

* remove isUndefinedOrEmpty

* remove isUndefinedOrEmpty

* rename isAllDataCohort file as requested

* lintfix

* first draft of notebook

* get notebook to run, generate cohorts (not finished)

* ntoebook

* convert filter args and column names properly

* fix datetime handling

* fix forecast request handling

* working version for bug bash

* add wheels

* add UI changes

* update notebook to fix model download

* 15 iterations

* lintfix

* add example of operation method string

* add check for context

* add explanatory comment

* move description creation logic into getDescription

* lintfix

* remove datetime_features from snapshot

* lintfix and other weirdnesses

* undo print removal

* remove wheels, lintfix, small cleanup

* use META_JSON directly instead of declaring a variable

* move constants to common files and reuse

* lowercase f

* isort

* remove datetime_features

* sktime notebook (1st half)

* isort

* debugging the notebook issue

* use parentheses instead of backslashes

* flake8

* isort

* sktime progress

* pre-unified RAIInsights commit

* pre-unified RAIInsights commit 2

* first pass over unification

* unify RAIInsights and RAIForecastingInsights

* adjustments to serialization

* cleanup, test fixes, consolidation, TODO fixes

* fix tests

* isort

* flake8

* feature metadata and categorical_features consolidation

* fix serialization of feature metadata

* fix tests

* flake8

* tests for feature metadata and cat features

* flake8

* remove remnants of RAIForecastingInsights

* fix string

* move other utils back to rai insights

* undo ModelTask import changes

* undo ModelTask references to raiutils

* undo references to ModelTask in raituils

* remove raiutils changes since they're in a separate branch

* datetime_features and time_series_id_features replacement

* pytest.mark.skip

* furthe datetime_features fixes

* TS ID col name fix

* try out asfreq

* notebook try asfreq interpolation

* attempt to condense index into a single col

* add hidden feature flag, fix tests

* remove experimental file, isort

* undo string change

* restructure wrappers and field to file mapping code

* remove unnecessary TODO

* get AML notebook to work

* get AML notebook fully working

* resolve some issues with sktime

* notebook

* remove duplicated test case

* flake8

* isort

* flalke8

* wrap zip object with list

* use ModelTask from raiutils

* remove prints

* flake8

* isort

* use FORECASTING from raiutils.models

* isort

* fix sktime issue

* get past Python errors

* fix predictions array shape

* fix UI-Python mismatch with renamed time and time series ID column names (now called "features")

* lintfix

* undo changes that aren't needed any longer

* remove AML notebook

* update notebook by removing unnecessary content

* notebook tests, basic setup

* ids for tests

* fix notebook based on latest changes in the repo

* add test for adding a what if case

* flake8 and cleanup

* isort

* lintfix

* fix test

* move notebook

* fix nb path

* fix kernelspec

* add pmdarima dependency

* skip forecasting notebook on py 3.6

* remove commented out line

* skip explanation

* add reason kw arg name

* lintfix

* fix case for vision, text

* fix syntax error

* fix python version comparison since it doesn't work with <=

---------

Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>
Co-authored-by: Gaurav Gupta <47334368+gaugup@users.noreply.github.com>
Co-authored-by: Gaurav Gupta <gaugup@microsoft.com>
Co-authored-by: John Wang <t-wangjohn@microsoft.com>
2023-08-22 23:46:08 -04:00
Advitya Gemawat 4e6a9399fb
Vision Data Explorer Table & Class View tests (#2220)
* ckpt

* test ckpt

* constant button test ckpt

* test ckpt

* image explorer test ckpt

* readding constants

* test fixes

* line limit fix

* python lint fixes

* added flyout constants

* reverted flyout test

* auto lint fixes

* cv2 installation tweak

* removed cv2 dependency from vision

* cv2 for image shap exp

* additional tests

* auto lint fixes

* constant & test updates

* arg fix

* removed unused import

* test update

* test cleanup

* auto lint fixes

* test ckpt

* lint & constant fixes

* test fix

* test exp

* dynamic inference test

* test fix

* package update

* test fix

* constant fix

* removed non table/class tests

* auto lint fixes
2023-08-09 10:30:51 -04:00
Advitya Gemawat 3be82a1ab2
Error Analysis testing for IC & multilabel (#2205)
* enabled EA testing for IC & multilabel

* added image test files

* enabled EA for ic & ml nbks

* auto lint fixes
2023-08-07 13:37:16 -07:00
Roman Lutz b19a8fcb86
Fix heatmap by removing erroneous color configuration (#2201)
* remove erroneous color configuration

* add mock data for OD and turn off heatmap for OD

* lintfix

* lint
2023-08-03 02:53:57 +00:00
Advitya Gemawat 37c340c8d6
Vision Model Overview E2E Tests - Refactor (#2185)
* mo test refactor ckpt

* test fixes

* test flags for vision

* test fixes

* auto lint fixes

* removed duplicate constants

* comment fixes

* test & build fixes

* test fixes

* auto lint fixes

* comment updates
2023-07-24 15:25:14 +00:00
Advitya Gemawat 90db5e67e4
Vision Data Analysis / Data Balance E2E Tests + Refactor (#2183)
* refactor ckpt

* removed inapplicable imports

* removed vision spec folder

* spec refactor

* readded spec files for test checks

* isVision fix

* auto lint fixes

* added vision flags to datasets
2023-07-20 11:57:09 -04:00
Advitya Gemawat 8898e8226e
Image Classification Notebook for UI testing #2175 (#2176)
* ic nbk ckpt

* release command support

* python lint fixes

* dataset support

* ci vision nbk gate support

* auto lint fixes

* vision date support + import fixes

* lint fix

* run command syntax fix

* lint fixes

* vision_notebooks mark support & gate update
2023-07-17 14:52:05 +00:00
Advitya Gemawat e433fdf547
Multiclass Image Classification Notebook for UI testing (#2175)
* multiclass nbk ckpt

* multiclass filename update

* nbk e2e support

* python lint fixes

* multilabel dataset support

* multilabel test support

* vision nbk gate support

* added component id

* disabled error analysis function

* nbk name fix

* nbk name fix

* vision gate support

* import fixes

* import fix

* visitVision import fix

* auto lint fixes

* doc & test fixes

* spec path fix

* dataset reference fix

* run command syntax fix

* auto lint fixes

* vision_notebooks mark support & gate update
2023-07-14 16:10:36 -04:00
Advitya Gemawat b7b5b856b5
ModelAssessmentVision OD + UI Test Template (#2174)
* initial nbk commit

* vision test template

* added vision support for visit

* lint fixes

* auto lint fixes

* removed spurious import

* fridge OD test case

* release command updates

* python lint fixes

* removed unused imports

* added vision CI nbk test gate

* gate name update

* commented error analysis function

* auto lint fixes

* ci e2e nbk vision gate setup

* .github/workflows/CI-e2e-notebooks-vision.yml

* dataset reference & flight fix

* export fixes

* import lint fix

* auto lint fixes

* removed py 3.6 support from vision
2023-07-13 15:38:49 -04:00
Advitya Gemawat d882424f1c
[Question Answering] Cache support for Model Overview Metrics (#2166) 2023-07-07 11:06:47 -04:00
Advitya Gemawat bd066380c3
[Object Detection] Cache support for Model Overview Metrics (#2158)
Cache support for Model Overview Metrics
2023-07-03 22:11:46 +00:00
Advitya Gemawat 5aca92cff9
[Object Detection] [Image Explorer] Bounding Box Support with CanvasTools on Flyout (#2116)
* bbox vott template ckpt

* canvastools image load ckpt

* loading image from scratch ckpt

* canvastools image loading support

* coordinate fixes

* regiondata call + path fix

* code cleanup

* image loading ckpt

* callback image loading support

* lint fixes

* bounding box working ckpt for fridge

* disabling internal imports for vott

* freeze rough ckpt

* file refactor lint fix

* auto lint fixes

* freeze cleanup

* lint fixes

* image dims arg for frontend bbox

* lint fixes

* lint fixes

* image dimension support

* imagedim support comments

* image dimension support

* image dimension support

* canvas module added

* image dimension working ckpt

* comment fix

* cleanup

* async image loading support

* lint fixes

* lint fixes

* reverted max line override

* code cleanups

* lint fixes

* auto lint fixes

* comment fixes
2023-06-15 15:12:15 -04:00
Advitya Gemawat f55e4be135
[Image Explorer] CanvasTools Image Loading support for Object Detection (#2097)
* bbox vott template ckpt

* canvastools image load ckpt

* loading image from scratch ckpt

* canvastools image loading support

* coordinate fixes

* regiondata call + path fix

* code cleanup

* image loading ckpt

* callback image loading support

* lint fixes

* disabling internal imports for vott

* file refactor lint fix

* auto lint fixes

* lint fixes

* image dims arg for frontend bbox

* lint fixes

* lint fixes

* image dimension support

* canvas module added

* comment fix

* async image loading support

* lint fixes

* lint fixes

* max-lines lint fixes

* lint fixes
2023-06-13 18:07:13 -04:00
hawestra 3db80ccef7
recommit to sign (#2092) 2023-06-01 13:06:12 -07:00
Gaurav Gupta bde66f749d
Revert "fix outline bug" (#2086)
* Revert "fix outline bug and add contributors to readme (#2081)"

This reverts commit ee7969a692.

* Add the contributors

Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>

* Revert another PR

Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>

---------

Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>
2023-05-30 18:18:21 +00:00
hawestra 06cceab5e3
move background color to outer div (#2084) 2023-05-24 17:04:54 -07:00
Gaurav Gupta d6d88694ab
Add dataset metadata to `IDataset` (#2071)
* Add dataset metadata to `IDataset`

Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>

* Fix tests

Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>

---------

Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>
2023-05-22 21:18:55 +00:00
Ilya Matiach c93f95b5a6
fix multilabel erroranalysis metrics dropdown appearing blank (#2063) 2023-05-17 11:29:57 -04:00
mehektulsyan 23990b73cb
Integrate Question answering metrics from python backend (#2054)
* integrating qa metrics with python backend

* fix

* added bert score

* changed metrics text definitions

* fixup for parameter - selection_indexes

---------

Co-authored-by: Neutrinowo <86736305+JarvisG495@users.noreply.github.com>
Co-authored-by: Ilya Matiach <ilmat@microsoft.com>
2023-05-11 10:41:33 -04:00
mehektulsyan bea3f428c2
QA Scenario metrics template (#2034)
* adding question answering RAI text dashboard metrics with umass team

* fixup

* adding question answering RAI text dashboard metrics with umass team

* fixup

* fixup

* # This is a combination of 3 commits.
# This is the 1st commit message:

adding question answering RAI text dashboard metrics with umass team

# The commit message #2 will be skipped:

# fixup

# The commit message #3 will be skipped:

# fixup

* fixup

* fixup

* fixed review comments: parameters

* new line fixup

* lint fixup

---------

Co-authored-by: Ilya Matiach <ilmat@microsoft.com>
2023-04-27 10:35:58 -04:00
Advitya Gemawat e6973412bc
[Model Overview] Object Detection Metrics support (#2025)
* OD support to surface metrics

* endpoint schema ckpt

* flask endpoint ckpt

* flask endpoint invocation fix

* od fn template rearrangement

* ckpt

* added endpoint return value

* endpoint return update

* ckpt

* onchange functions ckpt

* backend state update

* fn ckpt

* state function update

* state update fix

* endpoint refactor ckpt

* code cleanup ckpt

* endpoint condition update

* filter change & state var fix

* input saving ckpt

* lint fixes

* N/A support & telemetry reference update

* transparent color support for numbers

* lint fixes

* state refactor

* type & style updates

* lint fixes

* removed console logs
2023-04-14 12:07:36 -04:00
natalie-isak 7a56868c44
Drop Down For Object Detection (#2010)
* selecting objects

* adding new mock data

* working!

* adding event listeners - working:

* working button clicks that propagate bw flyout and dashboard

* raiwidgets litning

* fixing ts errors

* Revert "Merge branch 'main' into mult_img_dropdown_v2"

This reverts commit 1da22ffd43, reversing
changes made to b89e643328.

* working changes

* Revert "Revert "Merge branch 'main' into mult_img_dropdown_v2""

This reverts commit 5ff238ebe9.

* cleanup

* cleanup

* libs

* adding tags

* flake

* prettier

* flyout style

* linting again

* reformat to reduce line complex

* linting hopefully last time

* adding localization

* label expl

* styles

* trying to fix proto warning

* reducing number of fake data

* fixing formatting

* yarn lintfixi

* proto poisining

* protocheck

* proto

* proto stuff fixing back to way it was before

* obj sel

* comments from pr
2023-04-06 13:37:01 -07:00
Ilya Matiach dd7d6d514b
add scrollbar to RAI text dashboard text highlighting area (#2023) 2023-04-04 13:29:42 -04:00
Advitya Gemawat 1202fb66b4
Object Detection Model Overview Flask template (#2004)
* flask endpoint infra template

* docstring and lint fixes

* lint fixes

* lint fixes

* lint autofixes

* error statement fix
2023-03-21 13:58:02 -07:00
Ruby Zhu a373cd0ea2
[Refactor]1.Add DatasetCohort 2. Move FeatureFlights to core (#2003)
* move flight to core and add DatasetCohort

Signed-off-by: RubyZ10 <zhenzhu@microsoft.com>

* fix import

Signed-off-by: RubyZ10 <zhenzhu@microsoft.com>

* fix probY

Signed-off-by: RubyZ10 <zhenzhu@microsoft.com>

---------

Signed-off-by: RubyZ10 <zhenzhu@microsoft.com>
2023-03-14 13:25:33 -07:00
natalie-isak c9a02a5098
Adding Object Detection Fridge Data (#1998)
* sample data

* linting

* fixing build error
2023-03-10 16:08:38 -05:00