responsible-ai-widgets/CONTRIBUTING.md

Contributing

Contributor license agreement

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.

When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

If you have previously committed changes that were not signed follow these steps to sign them retroactively after setting up your GPG key as described in the GitHub documentation.

Setting up a GPG key has three stages:

1. Generate the key
2. Tell GitHub about the key
3. Instruct Git to sign using your key

Note that the GitBash shell installed by Git on Windows already has GPG installed, so there is no need to install GPG separately.

Code of conduct

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.

Acceptance criteria

All pull requests need to abide by the following criteria to be accepted:

• passing pipelines on the GitHub pull request
• signed Contributor License Agreement (CLA)
• approval from at least one maintainer
• compatibility with light / dark / high-contrast themes
• fits with overall look-and-feel of the widget
• accessibility (to be clarified)
• support for localization in code (translations need not be provided)
• tests for added / changed functionality

Development process

First ensure you have npm installed (which in turn may require installing node). Using npm you can install yarn as follows:

npm install -g yarn

If yarn --version succeeds you can proceed. If not, you may have to follow the instructions printed by your shell. If you're using Powershell you may have to bypass the execution policy to allow yarn to execute. One way to do this is Set-ExecutionPolicy -ExecutionPolicy Bypass.

For all further steps yarn install is a prerequisite. Run the yarn install command from your repository root directory.

To run the dashboards locally run the following from the root of the repository on your machine:

yarn start

which can take a few seconds before printing out

$ nx serve

> nx run dashboard:serve
**
Web Development Server is listening at http://localhost:4200/
**

at which point you can follow the link to your browser and select the dashboard and version of your choice.

To check for linting issues and auto-apply fixes where possible run

yarn lintfix

To build a specific app run

yarn build <app-name>  // e.g. fairness, interpret

or alternatively yarn buildall to build all of them. Since most apps have dependencies on mlchartlib it makes sense to run yarn buildall at least once.

Testing

Run e2e tests locally with mock data

1. git clone https://github.com/microsoft/responsible-ai-toolbox
2. cd responsible-ai-toolbox
3. yarn install
4. yarn build
5. To execute tests run yarn e2eall. Sometimes it is preferable to watch the execution and select only individual test cases. This is possible using yarn e2e --watch

cypress window will open locally - select test file to run the tests

Run e2e tests locally with notebook data

1. git clone https://github.com/microsoft/responsible-ai-toolbox
2. cd responsible-ai-toolbox (It is recommended to create a new virtual environment and install the dependencies)
3. yarn install
4. yarn buildall or yarn build widget
5. pip install -e responsibleai to install responsibleai locally.
6. pip install -e raiwidgets to install raiwidgets locally.
7. pip install jupyter
8. cd notebooks\responsibleaidashboard
9. To execute tests run yarn e2e-widget. Sometimes it is preferable to watch the execution and select only individual test cases. This is possible using yarn e2e-widget --watch

cypress window will open locally - select test file to run the tests

Test UX and SDK changes

For any new change, which involves changing any of the python SDK components and UI components, the manual testing of the code change can be done using the following steps:

1. git clone https://github.com/microsoft/responsible-ai-toolbox
2. cd responsible-ai-toolbox (It is recommended to create a new virtual environment and install the dependencies)
3. You should commit all your current set of changes for SDK and UX using git commit.
4. Clean all untracked files using git clean -fdx
5. Run yarn install and yarn buildall to build the UX changes.
6. Run pip install -e responsibleai to install responsibleai locally.
7. Run pip install -e raiwidgets to install raiwidgets locally.
8. Run pip install -e erroranalysis to install erroranalysis locally.
9. Run pip install -e rai_core_flask to install rai_core_flask locally.
10. Install jupyter using pip install jupyter
11. Open any notebook using python SDK and any widget from responsible-ai-toolbox and test your changes.

The steps from 3 to 11 need to be repeated if you incrementally change UI or SDK.

Debugging

There are several different ways to debug the dashboards:

1. Use Chrome + React Developer Tools. The debugging experience can be a bit flaky at times, but when it works it allows you to set breakpoints and check all variables at runtime.

2. Adding console.log(...) statements and check the console during execution. Please remember to remove the statements later on.

3. Alternatively, you can set objects as part of window to inspect them through the console at runtime (as opposed to having to specify what to print with console.log at compile time).

Flighting

It is possible to create feature flights to use certain functionality under development before exposing it to all users immediately. To do so, go to responsible-ai-toolbox\libs\model-assessment\src\lib\ModelAssessmentDashboard\FeatureFlights.ts and add your flight. After that you can use it in Typescript code as follows:

isFlightActive(flightName, this.context.featureFlights);

To pass the flight into the ResponsibleAIDashboard, simply add the keyword argument feature_flights and separate all the flights you wish to pass with ampersand (&), e.g., feature_flights="flight1&flight2&flight3".

In the dashboard test environment (using yarn start) you have a dropdown to select which flights should be active.