Граф коммитов

45 Коммитов

Автор SHA1 Сообщение Дата
Daniel LaLiberte 0baad0b731
Dlaliberte playwright 06 22 (#3184)
* Initial install of playwright visual tests

* Close to working.

* who knows...

* after updates

* Extend devcontainer docker compose file for playwright (#3192)

* Extend devcontainer docker compose file for playwright

Create another docker compose file for playwright.
That new one extends the existing docker compose

Should be able to run:

`npm run pwtests`

* adjust gh action

* adjust gh action

* remove volumes

* add timeout

* fix volumes

* Fix docker compose environment

* ignore stuff and volume mounting

* Attempting to login reliably

* Auto cleanups.

* Finally starting to work

* Thought it was getting better, but maybe not

* Seems reliable now

* Higher success rate

* Add test of entering feature name. Remove new feature page content test.

* Add playwright-report dir to Dockerfile.

* Comment out mkdir -p /work/playwright-report line.

* Update screenshots.

* Increasing timeout for entering feature name

* try running as user (#3248)

* try running as user

* again

* try

* change volume ownership

* tidy up

* Adjust login process to be more reliable.

* Add 5 second delay before first test runs.

* Wait longer for first login.

* Add auto-logout during login if already logged in

* Screenshot after login click

* Add testName to screenshots. Upload artifacts. Fix form method="POST".

* Add doc to README for Updating the test_html_rendering files.

* Maybe increase chance of login.

* Add a couple more delays.

* Repeat auto-logout during login until no longer logged in, before logging in again.

* Add some more delays.

* Use loginButton.click().  Add delay before after-click screenshot.

* Add console info an errors to login process.

* Capture console messages, relay to terminal.

* Capture console messages properly.

* Add some more console messages during login.

* Ignore error on jsonValue.

* Reduce most delays. Add more logs. Add top-level playwright call.

* More logging to figure out why login fails

* More logging to detect if ever logged in.

* Add more logging to each stage of testing

* Use history.replaceState before changing location.

* Use location.reload() after a timeout.

* After login, just load home page.

* Add playwright-report to artifacts.

* Turn off playwright-report during CI

* Log response after login

* Only log first response after request that should log in.

* Look for 'currentuser/permissions' request/response.

* Delay after mock_login and before reload of home page

* Comment out header and cookie logs.

* Act like logged when in DEV_MODE and login attempted.

* Fix mock mock login.

* Undo fake mock_login.

* rearrange docker (#3265)

* Updates after changes by James.

* Fix previous updates.

* Remove pwtests-shutdown from "stop" command.

* Remove pwtest from client-src/elements. (moved to packages/playwright/tests)

* update paths for run.sh

* Add pwtests-shutdown to top-level stop command.

* Remove "playwright" from ./wait-for-app.sh

* Add logging of user login status.

* add playwright_display for debugging

* add server debugging

* Remove extra "/__screenshots__" in path.

* Update the path for test-results artifacts in the playwright.yml

* add health checks

* fix action

* fix run.sh

* Add beforeAll method, disabled.

* add dsadmin

* Add top level pwtests command, not working yet.

* cleanup and undo changes

* use transactions

* always reuse server

* temp: limit the workers

* Partial cleanup.  Reduce timeouts and delays a bit.

* More cleanup.  Reduce timeout/delay.

* Update the correct README file.  Update html rendering tests.

* Remove unnecessary changes.

* set script-shell to fix source not found and call npm run setup (#3292)

* set script-shell to fix source not found and call npm run setup

There has always been `source: not found` in the previous CI jobs

The tests previously worked because
npm run deps installs the requirements into the default environment and not the virtual environment.
And since we actually did not call `source` correctly, it was okay because
the default environment had the packages installed.

The problem started to arise because we started to call `source`
correctly in some places but not all places. This fixes by setting the
shell consistently whenever we call `source`

More info:
https://morgan.cugerone.com/blog/troubleshooting-source-not-found-when-in-the-context-of-package-json-scripts/

* do each separately

* Revert "do each separately"

This reverts commit 4fc75f7407.

* do ci job

* consistent to the right datastore mode

* verbose

* kill the emulator

* remove verbose

* Reenable Google login

* Finish reenabling Google signin button. Add margin for Firefox.

* Only show Google signin if playwright is not active

* Preserve url on login.  Use default num of workers.  Retry tests in CI

* Some updates after comments

* optimize docker changes

* Rename account-indicator. Shorten some delays.

* revert the workers

* Make playwright.yml push and pull_request triggers be like others.

* Use getByTestId instead of locator with data-testid.

---------

Co-authored-by: James C Scott III <jcscottiii@users.noreply.github.com>
Co-authored-by: James Scott <jamescscott@google.com>
2023-09-06 15:35:06 -04:00
Kyle Ju 7553d5eec2
Use 3.10 in start-server (#2527)
Co-authored-by: Kyle Ju <kyleju@chromium.org>
2022-11-30 12:46:58 -08:00
James C Scott III ad9ee9ae31
Remove all references to django (#2358)
* Remove all references to django

Replace any functionality with flask

* more cleanup

* fix template test after removing empty comment

* remove unused test_app
2022-10-19 13:51:39 -04:00
Kyle Ju 88014a2844
Add Redis to notifier production setup (#2258)
Co-authored-by: Kyle Ju <kyleju@chromium.org>
2022-09-20 09:55:24 -07:00
Kyle Ju e5f9a7b642
Create Redis staging setup (#2211)
* Redis staging setup

* Add an if for staging

* Implement Redis caching APIs (#2212)

* Redis APIs

* Nit

* Partially migrate Ramcache usage to Redis (#2214)

* Migrate all fields except feature-related cache

* Fix

* Address comments

Co-authored-by: Kyle Ju <kyleju@chromium.org>

Co-authored-by: Kyle Ju <kyleju@chromium.org>

Co-authored-by: Kyle Ju <kyleju@chromium.org>
2022-09-12 13:56:50 -07:00
Jason Robbins 697cf9b38d
Refactor remaining models.py code into core_models.py (#2160)
* Refactor remaining models.py to core_models.py.

* Move tests to the right _test file.
2022-08-23 16:00:02 -07:00
Jason Robbins df4aec4cf2
Send and receive emails in py3 using GAE legacy libraries (#1923)
* Progress

* Send and receive emails in py3 using GAE legacy libraries.

* No need to run py2 unit tests when there are none

* Addressed review comments.
2022-06-10 14:56:40 -07:00
Jason Robbins f29a2ff916
Avoid needing to run py2 on workstations. (#1734)
* Avoid needing to run py2 on workstations.

* Use gunicorn instead of dev_appserver

* Add comment to explain that dev-default.yaml is not currently used.

* Use venv pip to install py3 deps.

* Auto-activate and stop referencing lib/.

* Make gunicorn use libs in cs-env.
2022-02-15 13:31:27 -08:00
Jason Robbins 10a6decb01
Further separate py2 code. (#1657)
* Further separate py2 code.

* Try making corresponding changes to py2 workflow

* Add TODOs for a couple of deps that are still needed for now.
2022-01-05 16:22:01 -08:00
Jason Robbins e77396a8cc
Make app deployable with py3. (#1559)
* Make app deployable with py3.

* Handle empty API posts
2021-09-24 15:19:17 -07:00
Jason Robbins 548828aaa0
Some easy pylint fixes. (#1557) 2021-09-23 12:04:33 -07:00
Kyle Ju b6bb10195a
Run 2to3 to detect py2 to py3 changes (#1551)
* Run 2to3 -w -n .

* Update internals/fetchmetrics.py

* Update internals/models.py

* Update internals/notifier.py

* Update pages/blink_handler.py

* Update internals/fetchmetrics.py
2021-09-22 18:23:14 -07:00
Jason Robbins 87cdfd97d4
Delete unneeded code in scripts/. (#1546) 2021-09-21 17:49:41 -07:00
Kyle Ju 06ce5bc8db
Run tests on py3 (#1534)
* 8 exceptions and 3 test failures
2021-09-21 15:49:40 -07:00
Jason Robbins 1286dc308c
Add indexes needed for /upcoming page (#1478)
* Add indexes needed for /upcoming page.

* Added deploy_indexes.sh
2021-08-19 17:43:21 -07:00
Jason Robbins ecd96ba8dd
Split app into py2 and py3 services. (#1460)
* Split app into py2 and py3 services.

* fix typo

* Also deploy app-py3 and upgrade skipfiles to .gcloudignore.

* fix typo

* addressed review comments
2021-08-17 15:35:08 -07:00
Jason Robbins c99738bf0f
Don't set DATASTORE_EMULATOR_HOST on GAE (#1403) 2021-07-09 12:38:30 -07:00
Jason Robbins 580dc06567
Tell dev_appserver to start datastore emulator (#1380) 2021-07-01 13:16:46 -07:00
Shivam Agarwal fd6018bd3d
20210618 gae ndb to cloud ndb (#1365)
* Added Cloud NDB to requirements

* google-cloud-core 1.7.0 requires google-auth<2.0dev,>=1.24.0, but you'll have google-auth 1.20.1 which is incompatible.

* Running DataStore Emulator on port 15606 and setting environment variable for the same

* Replaced GAE NDB imports with Cloud NDB imports

* Solved Error: No Context for Ramcache

* Solved No NDB Context Error for Cloud NDB

* Fixed Error No object QueryOptions

* Removed clear_datstore flag

* Fixed to_dict() method

* Added explicit call to method for stashing values while editing a feature

* Using Cloud NDB in testbed

* Updated README with JRE instructions and setting the DJANGO_SETTTINGS_MODULE environment variable for django.setup

* Added Env Variable for datastore emulator

* Migrated accounts_api_test.py

* Migrated approvals_api_test.py

* Migrated Comments.py and Cues_api_test.py

* Migrated Features.py

* Migrated metricsdata_test.py

* Migrated stars_api_test.py

* Fixed Deprecation warning for query options

* Migrated token_refresh_api_test.py

* Migrated basehandlers_test.py

* Migrated permissions_test.py

* Migrated ramcache_test.py

* Migrated secrets_test.py

* Migrated xsrf_test.py

* Migrated fetchmetrics_test.py

* Migrated models_test.py

* Migrated notifier_test.py

* Migrated processes_test.py

* Migrated featuredetail_test.py

* Migrated featurelist_test.py

* Migrated guide_test.py

* Migrated intentpreview_test.py

* Migrated samples_test.py

* Removed some unused stubs

* Set the consistency of datastore emulator to be 1

* Resetting the database emulator before running the tests

* Added npm targets for starting and stopping emulator

* Updated README and Developer Docs

* Removed unused imports

* Made NDB calls DRY
2021-06-28 19:05:04 -07:00
Shivam Agarwal 299516574f
GAE DB Client to GAE NDB (#1352)
* Migrated models.py

* Migrated bulkloader_helpers.py

* Migrated accounts_api.py

* Migrated models.py

* Migrated ramcache.py

* Migrated secrets.py

* Migrated basehandlers.py

* Migrated metricsdata.py

* Migrated fetchmetrics.py

* Migrated notifiers

* blink_handler

* guide.py

* users.py

* fix_data.py

* appengine.py

* Migrated all order() function calls

* Fixed filterby[]

* Migrated all filter() queries

* Migrated some left out lines

* Fixed Bug: json_dump would result in TypeError

* Fixed bug - key() is not callable

* Bug Fix: Query Filters were not working

* Fixed Bug: JSON output of feature list would not contain feature id

* Removed unused ndb import

* Fixed Bug: old_val would have properties wrappped in _BaseValue

* Migrated Tests

* Fixed Test: capstone has no attribute delete()

* Fixed Bug: A filter won't work

* Fixed Bug: Index out of range if self.Blink_components length is 0

* Fixed Bug: Some order statements won't work

* Assigning default value to Feature_blink_components

* Fixed Bug: A filter won't work

* Fetching singleton using ancestor for strong consistency

* Moved the ancestor query for strong consistency to replace the correct original query

* Replaced _pre_put_hook with _from_pb

* Fix: NDB picked up old properties while looping over self.properties

* Fix: NDB would generate diffs if None is compared with empty strings
2021-06-17 13:47:02 -07:00
Jason Robbins cbebbdc522
Consolidate some ajax calls into cs-client.js. (#1322)
* Consolidate some ajax calls into cs-client.js.

* Remove references to deleted JS files.
2021-05-19 09:24:33 -07:00
Shivam Agarwal 8fbebb7989
20210418 google sign in (#1275)
* Added Google Platform Library

* Added Meta Element for Client ID

* Added Google Sign-In Button

* Authenticating the id_token on our backend

* Saving id_token in flask session, using the id_token to fetch the current user and replaced the usages of AppEngine Users API (not from *_tests.py)

* Correct the flow on pressing SignIn and SignOut

* Code refactor

* Added Comment for is_current_user_admin

* Supporting GAE Users library for post request

* Made some fixes

* Changed Admin User condition

* Reloading only on 200 response code

* Do not require sign in and xsrf token while sending post request for login

* Sign Out using Google Sign In if cookie is not set after login

* Clearing the session if the id_token stored in the session variable becomes invalid or expires

* Replaced GAE Users from tests

* Replaced GAE users with framework users in tests.py
2021-05-06 16:37:30 -07:00
Jason Robbins 55c9cd2925
Refactor models.py into internals/. (#1236)
* Refactor models.py into internals/.

* Trivial change to re-trigger travis.

* Updated an import
2021-03-24 13:13:02 -07:00
Jason Robbins 0c924bb9b1
Remove some unused files. (#1200) 2021-03-12 08:06:00 -08:00
Jason Robbins 28840790b5 Remove unneeded firebase environment variables 2021-02-24 16:31:01 -05:00
Jason Robbins 9cbc5345de Make metrics fetch more robust and debuggable. 2021-01-20 19:21:14 -05:00
Jason Robbins c610e44e99
Upgrade from GAE taskqueue to Google Cloud Tasks (#1141)
* Upgrade from GAE taskqueue to Google Cloud Tasks.

* added unit tests

* Add link to Monorail file that I referenced.

* Add grpcio to travis build

* Try installing grpcio locally first

* Try installing grpcio via sudo

* Try a smaller set of deps for travis.

* Debug travis config

* Avoided missing imports while running unit tests on travis

* Removed travis debugging lines
2021-01-15 15:50:36 -08:00
Jason Robbins 9bb4a50807 Updated sole use of integer division 2020-12-01 14:25:01 -05:00
Jason Robbins cb9d596f43 [py3] Add import __future__ lines to all files and update print statements. 2020-12-01 14:18:14 -05:00
Jason Robbins c43054f228
Set up staging server, and send staging email to archive. (#765)
This is progress on issue #756.
2020-02-24 09:22:24 -08:00
Eric Bidelman a33a626259 Fixes #517. Moves to gcloud 2017-10-17 09:57:53 -07:00
Eric Bidelman 46f72420c7 Push notifications: sent to subscribers when feature is updated 2017-08-06 10:35:40 -07:00
Eric Bidelman e5c98a5dc0 Cleanup new deploy, shared skip_files, fixes for prod 2017-07-02 22:14:35 -07:00
Eric Bidelman 6b9aba52aa Feature owners: db notifications and task queue.
- Intercepts calls to Feature.put() and diffs changes in properties for updates
- Creates async task queue and email notifies feature owners
- Switch to pip and custom django install instead of GAE version
2017-07-02 21:25:30 -07:00
Eric Bidelman 9c106765e9 Update fixdata script 2017-06-30 21:09:05 -07:00
Eric Bidelman 932ab0391a Update deploy script. Rev version 2016-07-21 14:06:04 -07:00
Eric Bidelman f74139bdfb Omit --oauth2 flag. Default now 2015-12-03 08:53:32 -08:00
Eric Bidelman ab2c7c3e46 Feedback. 2015-08-03 09:59:03 -07:00
Eric Bidelman 123f4c8b3c More 1.0
- Use iron-icons instead of font awesome
- Bug fixes and element changes
- Remove uneeded files
2015-07-29 23:14:40 -07:00
Francois Beaufort 03dd210818 Added cron job to fetch chromium histograms 2015-02-10 18:31:50 +01:00
Eric Bidelman 41e2cd0802 Add db data correcting script 2014-04-16 22:03:31 -07:00
Eric Bidelman de2472fe3c Fixes #103
Add build step, update vulcanize output
Modularize imports and CSS
Reduces requests to ~13
2014-03-24 09:23:22 -07:00
Eric Bidelman 6e8b6346d1 Use vulcanized import for main features pages 2014-03-01 21:54:17 -08:00
Eric Bidelman d34adf6627 Update deploy script to use oauth2 2013-09-27 16:50:58 -07:00
Eric Bidelman 83e1f51459 Initial commit of uma stuff 2013-04-12 10:19:21 -07:00