Граф коммитов

6182 Коммитов

Автор SHA1 Сообщение Дата
Ben Hearsum 39404c8d2a bug 1942364: switch to autograph gcp production for signing
This needs to wait for https://github.com/mozilla-releng/scriptworker-scripts/pull/1123 to be deployed before it can.
2025-01-29 14:05:41 +00:00
Mark Banner c28a94f2af Bug 1938922 - Implement sub-variant handling in the search engine selector. 2025-01-28 09:19:22 +00:00
Mark Banner c5c7bbec08 Bug 1938922 - Change JSONEngineUrls.search to be an option to reflect how the main configuration works. 2025-01-28 09:19:22 +00:00
Ryan VanderMeulen 56307e4fd1 Update Gradle to version 8.12.1 2025-01-24 15:53:09 +00:00
Mergify 5c4c34b84a ci(mergify): upgrade configuration to current format 2025-01-24 15:34:49 +00:00
Mark Hammond 7de354de7a Bug 1941655 - relax places DB constraints to what they were before bug 1935797 2025-01-21 21:15:02 +00:00
Johannes Jörg Schmidt cb41b0ac90 new method is_empty for LoginsStore
to check whether the database is empty
2025-01-21 09:17:42 +00:00
Johannes Jörg Schmidt 8edd1bcc03 update changelog describing Logins component changes 2025-01-21 09:17:42 +00:00
Johannes Jörg Schmidt e451e45330 flatten Login struct
The current Login structure is nested:

```
Login {
  RecordFields record;
  LoginFields fields;
  SecureLoginFields sec_fields;
}
```

and thus exposes internal data structuring to the consumer. Since we make the encryption transparent for the consumer (Android, iOS, desktop) here, such a separation no longer makes sense here and above can be simplified to

```
Login {
    // record fields
    string id;
    i64 times_used;
    i64 time_created;
    i64 time_last_used;
    i64 time_password_changed;

    // login fields
    string origin;
    string? http_realm;
    string? form_action_origin;
    string username_field;
    string password_field;

    // secure login fields
    string password;
    string username;
}
```

The advantage of eliminating this separation lies, on the one hand, in the simplification of the API and the resulting easier use of the component, and on the other hand, in the easier changeability of the internal data structure. If, for example, we decide later to encrypt additional fields, such a change is possible without having to adapt the consumers.
2025-01-21 09:17:42 +00:00
Johannes Jörg Schmidt ccf4467f54 introduce EncryptorDecryptorTrait for Logins component
This prepares the Logins component for the desktop and simplifies its
API.

BREAKING CHANGE:
This commit introduces breaking changes to the Logins component:

During initialization, it receives an additional argument, a
EncryptorDecryptorTrait implementation. In addition, several LoginsStore
API methods have been changed to not require an encryption key argument
anymore, and return Logins objects instead of EncryptedLogins.

Additionally, a new API method has been added to the LoginsStore,
`has_logins_by_base_domain(&self, base_domain: &str)`, which can be used
to check for the existence of a login for a given base domain.

**EncryptorDecryptor**

With the introduction of the EncryptorDecryptor trait, encryption
becomes transparent. That means, the LoginStore API receives some
breaking changes as outlined above.  A ManagedEncryptorDecryptor will
provide an EncryptorDecryptor implementation which uses the currently
used crypto methods, given a KeyManager implementation. This eases
adaption for mobile.  Furthermore, we provide a StaticKeyManager
implementation, which can be used in tests and in cases where the key is
- you name it - static.  Constructors Now an implementation of the above
property must be passed to the constructors. To do this, the signatures
are extended as follows:

```
pub fn new(path: impl AsRef<Path>, encdec: Arc<dyn EncryptorDecryptor>) -> ApiResult<Self>
pub fn new_from_db(db: LoginDb, encdec: Arc<dyn EncryptorDecryptor>) -> Self
pub fn new_in_memory(encdec: Arc<dyn EncryptorDecryptor>) -> ApiResult<Self>
```

**LoginStore API Methods**
This allows the LoginStore API to be simplified as follows, making
encryption transparent by eliminating the need to pass the key and
allowing the methods to return decrypted login objects.

```
pub fn list(&self) -> ApiResult<Vec<Login>>
pub fn get(&self, id: &str) -> ApiResult<Option<Login>>
pub fn get_by_base_domain(&self, base_domain: &str) -> ApiResult<Vec<Login>>
pub fn find_login_to_update(&self, entry: LoginEntry) -> ApiResult<Option<Login>>
pub fn update(&self, id: &str, entry: LoginEntry) -> ApiResult<Login>
pub fn add(&self, entry: LoginEntry) -> ApiResult<Login>
pub fn add_or_update(&self, entry: LoginEntry) -> ApiResult<Login>
```

We will stop Uniffi-exposing the crypto primitives encrypt, decrypt,
encrypt_struct and decrypt_struct. Also EncryptedLogin will not be
exposed anymore.  Checking for the Existence of Logins for a given Base
Domain In order to check for the existence of stored logins for a given
base domain, we provide an additional store method,
has_logins_by_base_domain(&self, base_domain: &str), which does not
utilize the EncryptorDecryptor.

Another by-change is in the `check_canary` function: here we do not
throw anymore if a wrong key is used but return false.
2025-01-21 09:17:42 +00:00
Ben Dean-Kawamura 25934715ec Adding AMP FTS metrics
These seem like they could be useful for the experiment.  I'm not sure
we'll be able to hook them up but we might as well try.

Refactored the logic to get the full keywords and added tests for it
since we're now using it in two places.
2025-01-17 14:04:31 +00:00
Ben Dean-Kawamura f63adf8664 AMP FTS experiment logic
This sets it up so we always ingest the FTS data and use the
`SuggestionProviderConstraints` passed to the query to determine how to
perform the query.  This seems like the simplest approach and it doesn't
increase ingestion time that much.  The benchmarks on my machine went
from 339.29 ms to 465.60 ms.
2025-01-16 21:01:11 +00:00
Mark Hammond d7049d23dc Increase bookmark sync log levels to try and narrow down bug 1941655. 2025-01-15 22:00:31 +00:00
Aaditya Dhingra e37ba792a5 Update task assembleNimbusTools to be configuration-cache friendly with fenix. 2025-01-15 16:22:02 +00:00
Alex Cottner d54043ef34 Updating remote-settings root cert hash to the root cert prod just switched to today. 2025-01-15 09:18:55 +00:00
Johannes Jörg Schmidt fc22cdb518 support keydb feature
This adds a feature `keydb` to rc_crypto's nss create, which enables the
`ensure_initialized_with_profile_dir` initialize function. This
configures NSS to use a profile and persist keys into key4.db.

Also adding methods for managing AES256 keys with NSS:
* `authentication_with_primary_password_is_needed`: check wheather primary password is enabled
* `authenticate_with_primary_password`: authenticate with primary password against NSS key database
* `get_or_create_aes256_key`: retrieve a key from key4.db or, if not present, create one
2025-01-14 13:45:26 +00:00
Ben Dean-Kawamura 645d266ac9 One more fix for the taskcluster build.
For some weird reason, the old command line was enabling the features
for all binaries.  Since the remote settings CLI depends on rc_crypto,
this was causing failures when trying to build NSS.  Adding the `-p`
flag, fixes this.
2025-01-13 19:11:04 +00:00
Ben Dean-Kawamura d71591e983 Removing nss dependency from sql-support
We stopped using this in 4bc525e8ee, but
the build-dependency was never removed.  It's now causing taskcluster
build issues with nimbus-cli.
2025-01-13 16:50:05 +00:00
Ben Dean-Kawamura 1fa5f6725d Bug 1937916 - Add build-id to Android builds
Upgrade rust-android-gradle and set the `generateBuildId` flag.

See https://github.com/mozilla/rust-android-gradle/pull/159 for details.
2025-01-13 16:42:50 +00:00
Ben Dean-Kawamura 2403db86a7 Updating rust-android-gradle
The new version works with Python 3.13
2025-01-13 14:33:33 +00:00
Mathieu Leplatre 9093cd5dfb Remote Settings: list component in README 2025-01-13 11:47:13 +00:00
Mathieu Leplatre af9c3172b2 Fix sync() issue: server expects timestamp between quotes 2025-01-10 17:52:16 +00:00
Mathieu Leplatre 662a05a477 Remote settings: add signature verification support 2025-01-10 17:52:16 +00:00
Ryan VanderMeulen 2665ba9ba0 Update AGP to version 8.8.0 2025-01-09 21:41:36 +00:00
Mark Banner 5e3fb6c9df Bug 1919541 - Implement variant handling in the search engine selector. 2025-01-07 21:36:43 +00:00
Ben Dean-Kawamura f9539fbd29 Bug 1931373 - Add FTS matching data
Added extra data to `Suggestion::Fakespot` to capture how the FTS match
was made.  The plan is to use this as a facet for our metrics to help us
consider how to tune the matching logic (i.e. maybe we should not use
stemming, maybe we should reqiure that terms are close together).

Added Suggest CLI flag to print out the FTS match info.
2025-01-07 14:59:10 +00:00
Mathieu Leplatre d305778af1 Remote Settings: store full changesets instead of just records 2025-01-07 12:12:13 +00:00
Ben Dean-Kawamura 7d70184c7c Delete suggest bench tmp dirs
#6479 reduced the overhead for running benchmarks, but a side effect was
that it left directories filled with sqlite DBs in `/tmp`.  This makes
sure we delete those directories.
2025-01-07 01:58:35 +00:00
Ryan VanderMeulen 920491f289 Update Gradle to version 8.12 2025-01-06 20:39:36 +00:00
Johannes Jörg Schmidt 3baf862456 Add NSS low-level bindings for dealing with primary password
This adds two low-level bindings to NSS for dealing with the primary
password:
* `PK11_NeedLogin`: for checking wheather a primary password is set
* `PK11_CheckUserPassword`: for authorization with primary password

Additionally the following two low-level NSS bindings are added to deal
with key persistence:
* `PK11_NeedUserInit`
* `PK11_SetSymKeyNickname`
2025-01-06 19:55:03 +00:00
Heitor Neiva 660d056d5f [ci] Fix mark-as-shipped task name 2025-01-06 19:27:10 +00:00
Ryan VanderMeulen 11cc4f1557 Start release v136.0 2025-01-06 15:27:52 +00:00
Ryan VanderMeulen d5dd8d4a44 Update AndroidX testing dependencies 2025-01-02 19:34:39 +00:00
Ryan VanderMeulen 8e0c0c1795 Update Mockito to version 5.15.2 2025-01-02 19:34:39 +00:00
Ryan VanderMeulen ca48ea617e Update JUnit to version 5.11.4 2025-01-02 19:34:39 +00:00
Johannes Jörg Schmidt 266faf594a extend nss bindings with methods for key management
To manage AES-256 GCM keys with NSS in the way that is currently implemented on the desktop, we need extended functionality of the Rust bindings to NSS. Specifically, these are functions for key generation with persistence, for listing persisted keys, and for wrapping and unwrapping to get at the key material of stored keys.
2025-01-02 18:35:25 +00:00
Ben Dean-Kawamura c451ef7e96 Remote settings: remove the fetched column
This simple change is a good way to introduce the sql_support crate and
start the migration system.

Made a couple other fixes along the way:
 - Don't make the CLI `main` method async, that causes panics when
   running the `get` or `sync` commands since those will create a second
   runtime within the async runtime.
 - Added `remote-settings-data`, which is generated by the CLI, to .gitignore
 - Fixed sql_support testing bug for schemas with no indexes.
2025-01-02 15:35:53 +00:00
Peter Kehl ae30c4092c [[test]] -> [[bench]] in Cargo.toml 2024-12-23 06:08:33 +00:00
lougeniac64 03cf4a3624 Exposed webext-storage component calls for desktop 2024-12-20 19:30:48 +00:00
Ben Dean-Kawamura 176f82f69d Fix taskcluster breakage from #6524
Build a JAR file with libjnidispatch and libmegazord, targetted at
Desktop arches so that devs can run unit tests with it.  Export this as
a configuration and Maven package.  This allows android-components code
to run the unit tests and also simplifies the gradle code for our own
components.

Named this full-megazord-libsForTest, which I think is a bit more clear
than full-megazord-forUnitTests.
2024-12-20 18:38:30 +00:00
Ben Dean-Kawamura 83ad69b756 Taskcluster fixes based on the recent issues
One reason we didn't catch this sooner we relied posting `/taskcluster
ci full` in a GH comment, but that didn't actually run all the tasks.

- Added a set of test parameters for the github issue comment
- Run all tasks for issue comments
2024-12-20 18:38:30 +00:00
Ben Dean-Kawamura 35bb959d60 Bug 1936053: RemoteSettingsClient.sync() should merge records
Merge new records with the old ones, rather than unconditionally storing
the new records.  This fixes buggy behavior when sync does an
incremental update.
2024-12-19 15:13:19 +00:00
Ben Dean-Kawamura 77e53f2bd3 Refactor RemoteSettingsClient::get_records
I'm going to update some of the surrounding code and I figure now is a
good time to simplify this.

One difference is when `sync_if_empty=true` and we have cached records
from the server but there were 0 records in the response.  Before, we
would sync again, now we don't.  I believe this is the correct behavior:
we only want to sync when the cache is empty, not if the record list is
empty.  In any case, I don't think this case will happen in practice.
2024-12-19 15:13:19 +00:00
Ryan VanderMeulen 812610aa58 Update AndroidX Core to version 1.15.0 2024-12-18 20:35:54 +00:00
Ryan VanderMeulen 05096f33d1 Target Android SDK 35 2024-12-18 20:35:54 +00:00
Ryan VanderMeulen 6220723887 Update Android SDK commandlinetools to version 17.0 2024-12-18 20:35:54 +00:00
Ben Dean-Kawamura 58868c9997 Upgrade AGP to 8.7.3 2024-12-18 19:11:06 +00:00
Ben Dean-Kawamura 7bc69fe08a Update our megazord/UniFFI gradle code
* Moved `dependsOnMegazord` from `publish.gradle` into
  `component-common.gradle`.  I don't think it makes sense to mix this
  with the publishing code.
* Use configurations to handle the native megazord dependency.
* Manually copy the native megazord into the resource dir when running
  unit tests.  The strategy we were using before doesn't work after
  Gradle 8.1.
* dependsOnMegazord no longer adds a native-support dependency.  This is
  only used by viaduct, so I moved the dependency into viaduct's
  build.gradle.
* Removed the jnaForTest code.  We no longer need this hack now that
  `implementation` and `testImplementation` can specify different types.
2024-12-18 19:11:06 +00:00
Ben Dean-Kawamura b8c4894e7e Removing kotlin-dsl plugin
This is causing the gradle builds to fail on my Android Studio install,
and AFAICT it's not being used anywhere.
2024-12-18 19:11:06 +00:00
Temisan Iwere bf4dff0a7c [DISCO-3113] relevancy: add method to retrieve bandit data 2024-12-17 16:15:04 +00:00