Summary:
E2E tests in OSS are expensive and flaky.
They already prevented some broken changes to land on main, but as of today:
- they are always green, so they are not bloking
- nobody is looking at the reporting job
- the reporting job takes a lot of time to run and prevent other useful signals to be available soon
- it is expensive
So we decide to disable them for the time being, while we iterate on those with Callstack and MSFT.
## Changelog:
[Internal] - Disable E2E tests
Pull Request resolved: https://github.com/facebook/react-native/pull/41153
Test Plan: CircleCI stays green
Reviewed By: cortinico
Differential Revision: D50552818
Pulled By: cipolleschi
fbshipit-source-id: 7160a8074492c3c9a55485d8a17a6883eb4b35b5
Summary:
Right now, every PR runs the whole test suite. For example, a changelog PR, will run all the tests. As of last month, that meant quite a few $s per single run.
With this PR, we are going to leverage dynamic configuration and file filtering to create a config.yml on the flight, depending on the files changed by the commit/pr.
They way it works is the following:
- It starts a setup workflow in CircleCI.
- This workflow fetch the list of files that have been changed in the current commit.
- It executes a bunch of filtering and computation to understand which tests makes sense to run.
- It creates a config on the flight to run those.
- It continue the pipeline on that config.
Currently, the way it works is the following:
- If a `.md` file has been modified => run nothing
- If only files in the `ReactAndroid` folder are modified => run tests for android only
- If only files in the `React` folder are modified or `ruby` files are modified => run only iOS tests
- If only js files, not in the scripts folder are modified => run only JS tests
- if only files in the e2e folder are modified => run only e2e tests
- else => run everything.
Of course, we can play and modify those filters t make sure that they reflect the work and the tests to the best we can.
bypass-github-exports-checks
## Changelog:
[Internal] - Split circleci config and run test selectively.
Pull Request resolved: https://github.com/facebook/react-native/pull/39042
Test Plan:
- [X] Tested on the local branch for general sanity check.
- [X] Import it in fbsource
- [x] Create a stacked diff which changes only a md file => verify that no tests are run.
- [x] Create a stacked diff which changes only files in ReactAndroid => verify that only android tests run.
- [x] Create a stacked diff which changes only files in React => verify that only iOS tests run.
- [x] Create a stacked diff which changes only ruby files => verify that only iOS tests run.
- [x] Create a stacked diff which changes ruby files and file in React => verify that only iOS tests run.
- [x] Create a stacked diff which changes only files JS not in the script folder => verify that JS tests run.
- [x] Create a stacked diff which changes only JS files in the script folder => verify that the whole suite starts.
- [x] Create a stacked diff which changes only files in the E2E folder => verify that only E2E files runs.
- [x] Trigger a nightly pipeline => verify that parameters are passed to the generated config.
Reviewed By: NickGerleman
Differential Revision: D48394437
Pulled By: cipolleschi
fbshipit-source-id: 771f3e68daa8318d2b73dd91ce85a41488110c04
Summary:
Right now, every PR runs the whole test suite. For example, a changelog PR, will run all the tests. As of last month, that meant quite a few $s per single run.
With this PR, we are going to leverage dynamic configuration and file filtering to create a config.yml on the flight, depending on the files changed by the commit/pr.
They way it works is the following:
- It starts a setup workflow in CircleCI.
- This workflow fetch the list of files that have been changed in the current commit.
- It executes a bunch of filtering and computation to understand which tests makes sense to run.
- It creates a config on the flight to run those.
- It continue the pipeline on that config.
Currently, the way it works is the following:
- If a `.md` file has been modified => run nothing
- If only files in the `ReactAndroid` folder are modified => run tests for android only
- If only files in the `React` folder are modified or `ruby` files are modified => run only iOS tests
- If only js files, not in the scripts folder are modified => run only JS tests
- if only files in the e2e folder are modified => run only e2e tests
- else => run everything.
Of course, we can play and modify those filters t make sure that they reflect the work and the tests to the best we can.
bypass-github-export-checks
## Changelog:
[Internal] - Split circleci config and run test selectively.
Pull Request resolved: https://github.com/facebook/react-native/pull/38793
Test Plan:
- [X] Tested on the local branch for general sanity check.
- [X] Import it in fbsource
- [x] Create a stacked diff which changes only a md file => verify that no tests are run.
- [x] Create a stacked diff which changes only files in ReactAndroid => verify that only android tests run.
- [x] Create a stacked diff which changes only files in React => verify that only iOS tests run.
- [x] Create a stacked diff which changes only ruby files => verify that only iOS tests run.
- [x] Create a stacked diff which changes ruby files and file in React => verify that only iOS tests run.
- [x] Create a stacked diff which changes only files JS not in the script folder => verify that JS tests run.
- [x] Create a stacked diff which changes only JS files in the script folder => verify that the whole suite starts.
- [x] Create a stacked diff which changes only files in the E2E folder => verify that only E2E files runs.
- [x] Trigger a nightly pipeline => verify that parameters are passed to the generated config.
Reviewed By: rshest
Differential Revision: D48118162
Pulled By: cipolleschi
fbshipit-source-id: 73c8071a7e80cd930fe538f77d7bb5de75f22ab7
Riccardo Cipolleschi
Riccardo Cipolleschi