Граф коммитов

48 Коммитов

Автор SHA1 Сообщение Дата
Lina Butler 07ab69d93e suggest: Ensure that each store test uses a unique database. 2023-08-17 23:10:25 +00:00
Lina Butler d9a3a3f3fe suggest: Replace databases with incompatible schema versions.
Our prototype integrations currently fail to open databases with older
schema versions. Let's throw away and replace these databases
instead of failing.
2023-08-10 19:29:13 +00:00
Lina Butler 65723831c5 suggest: Add docs for adding new suggestion types. 2023-08-09 05:27:50 +00:00
Lina Butler e30c65a467 suggest: Move `SuggestionProvider` into its own module.
The `rs` module contains crate-internal types related to Remote
Settings. Since `SuggestionProvider` is part of the public interface,
and has non-trivial methods, let's give it its own module.
2023-08-09 05:27:50 +00:00
Lina Butler 4b28d41f58 suggest: Turn `Suggestion` into an enum.
The previous representation was specific to AMP-Wikipedia.
2023-08-09 05:27:50 +00:00
Lina Butler 4981e78a9c suggest: Prepare to support other types of suggestions.
This commit:

* Renames some of our crate-internal types and methods to clarify that
  they're specific to AMP-Wikipedia suggestions.
* Generalizes `DownloadedSuggestDataAttachment` to a `SuggestAttachment`
  type that can accommodate any suggestion.
* Removes `icon_id` from the common `suggestions` table, because not
  all suggestions that we'll support in the future have icon data. For
  example, Pocket suggestions don't have icons at all, and AMO
  suggestions have URLs instead of binary data.
2023-08-09 05:27:50 +00:00
Lina Butler a5a8cf90b7 suggest: Use `INSERT OR REPLACE` in `SuggestDao::put_meta()`.
`REPLACE` and `INSERT OR REPLACE` are equivalent, but the latter
matches `put_icon()`.
2023-08-09 01:51:58 +00:00
Lina Butler 170d06ebad suggest: Fix re-ingesting icons.
We used `INSERT` instead of `INSERT OR REPLACE` to store icons, so
trying to re-ingest an updated icon would fail with a unique constraint
error.
2023-08-09 01:51:58 +00:00
Lina Butler 96530396bd suggest: Ignore unknown and malformed Remote Settings records.
We'll want to handle these eventually, as part of a bigger rework for
unknown records and suggestion fields in DISCO-2564. For now, though,
let's at least advance the last sync time when we see them, to be
consistent with how we handle other records.
2023-08-08 23:28:38 +00:00
Lina Butler 76b318cc02 suggest: Adopt the new Remote Settings additions.
This commit refactors the Suggest component to use the Remote Settings
conveniences added in the last three commits.

* `get_records_with_options` replaces `get_records_raw_with_options`.
  As part of this change, Suggest now uses the `RemoteSettingsResponse`
  and `RemoteSettingsRecord` types from the Remote Settings component,
  instead of defining its own types and deserializing them from the
  Viaduct response.
* `SuggestRemoteSettingsClient` becomes a thin wrapper around the Remote
  Settings client, and calls its methods directly. This helps keep the
  mocking surface small.
2023-08-08 22:25:01 +00:00
Tif Tran 71480864ec Suggest db schema change to accomodate 'custom details' for providers 2023-08-02 00:53:29 +00:00
Lina Butler 3aa422e14d suggest: Use `expect-test` for snapshot testing. 2023-07-28 06:11:28 +00:00
Lina Butler dfb3af4a56 suggest: Add test coverage for the store. 2023-07-28 06:11:28 +00:00
Lina Butler 97a0c968d8 suggest: Use a mock Remote Settings client instead of Mockito in tests.
This commit rewrites the `ingest` test to use the
`SuggestRemoteSettingsClient` and `SuggestStoreInner` primitives added
in the last commit.
2023-07-28 06:11:28 +00:00
Lina Butler fcb302e4f3 suggest: Introduce a `SuggestRemoteSettingsClient` trait for testing.
Instead of setting up a mock Remote Settings HTTP server in each test,
we can use a trait to abstract over the Remote Settings client, and use
different implementations in production and test code. This will make it
much easier to write our unit tests.

This commit:

* Implements the `SuggestRemoteSettingsClient` trait for the
  production client (`remote_settings::Client`).
* Factors out the `SuggestStore` implementation into an inner
  `SuggestStoreInner` type that's generic over the Remote Settings
  client.
* Refactors the public `SuggestStore` type to delegate to a concrete
  instance of the inner store for the production client
  (`SuggestStoreInner<remote_settings::Client>`).
2023-07-28 06:11:28 +00:00
Lina Butler 3f3a5736d3 suggest: Move Remote Settings-related types into a separate module.
This commit moves the types and updates their visibility, without any
logic changes.
2023-07-28 06:11:28 +00:00
Lina Butler 1857afb83e suggest: Make all `SuggestionQuery` fields required. 2023-07-20 23:56:12 +00:00
Lina Butler f070aa623d suggest: Clarify type names for downloaded suggestions.
This commit renames:

* `RemoteSuggestion` to `DownloadedSuggestion` and
  `SuggestDataAttachmentContents` to `DownloadedSuggestDataAttachment`,
  for consistency with the "downloading" terminology.
* `RemoteRecordId` to `SuggestRecordId`, for consistency with the other
  Remote Settings types.
2023-07-20 23:56:12 +00:00
Lina Butler c744edc04c suggest: Improve rustdoc comments. 2023-07-20 23:56:12 +00:00
Lina Butler fcae78c735 suggest: Include the collection name in the "last ingest" metadata key.
A bit of future-proofing, in case we support multiple Remote Settings
collections in the future.
2023-07-20 23:56:12 +00:00
Lina Butler 478c796b4b suggest: Don't pass `SuggestionQuery` [ByRef].
This matches how we pass dictionaries across the FFI boundary in our
other components.
2023-07-20 23:56:12 +00:00
Lina Butler 45d152a692 suggest: Clarify "fetching" in docs and type names.
"Fetching" is a bit of an overloaded term that can refer to querying
ingested suggestions, or downloading records from Remote Settings. To
reduce confusion, let's replace those uses with more precise
terminology:

* "Downloading" for when we request suggestions from Remote Settings.
* "Ingesting" for storing the downloaded suggestions in the database.
* "Querying" for when the application requests ingested suggestions.
2023-07-20 23:56:12 +00:00
Lina Butler 70c6f96a3d suggest: Rename `IngestLimits` to `SuggestIngestionConstraints`.
This commit also generalizes `IngestLimits.records` to
`SuggestIngestionConstraints.max_suggestions`, since the partitioning
scheme and number of suggestions per Remote Settings attachment is an
implementation detail.
2023-07-20 23:56:12 +00:00
Lina Butler 6702a30f01 suggest: Open SQLite connections with `SQLITE_OPEN_NO_MUTEX`.
`SuggestDb` already protects the connection with a Rust mutex, so
SQLite's per-connection mutex is unnecessary.
2023-07-20 23:56:12 +00:00
Lina Butler 9de0cfe1b7 suggest: Drop `serde_derive`. 2023-07-20 23:56:12 +00:00
Lina Butler 96baca8f8a suggest: Use inline table syntax for dependencies in `Cargo.toml`. 2023-07-20 23:56:12 +00:00
Lina Butler 1eaf57ec2e suggest: Refactor database transaction management.
This commit makes it possible to combine multiple database accesses
into a single transaction. Previously, each `SuggestDb` method would
lock the database connection, and optionally run its own transaction
before releasing the lock. This meant that a single logical operation,
like ingesting all suggestions from an attachment, would involve
multiple transactions.

This commit moves the database accesses into a new `SuggestDao` type,
which takes a reference to an existing locked connection. The new
`SuggestDb::{read, write}` methods handle locking and transaction
management, and take a closure that receives a `SuggestDao` for reading
and writing.

Co-authored-by: Tif Tran <ttran@mozilla.com>
2023-07-20 23:56:12 +00:00
Lina Butler 81e02ec4be suggest: Switch to Parking Lot. 2023-07-20 23:56:12 +00:00
Lina Butler 69a64370aa suggest: Add MPL license headers. 2023-07-20 23:56:12 +00:00
Lina Butler affaf74d42 suggest: Add README and docs for public interfaces. 2023-07-20 23:56:12 +00:00
Lina Butler 1205700c1e suggest: Clarify internal vs. public interfaces.
This commit changes the visibility of crate-internal types and methods
from `pub` to `pub(crate)`, and clarifies the difference between the two
error types:

* `SuggestApiError` is the public error type, for all errors that should
  be handled by the application.
* `Error` is the internal error type, used in the implementation of the
  component, and is not part of the public interface.
2023-07-20 23:56:12 +00:00
Lina Butler a90ebe464f suggest: Support clearing the suggestions store.
This commit exposes `SuggestDb.clear()` as `SuggestStore.clear()`, and
ensures that it removes all icons in addition to suggestions and
metadata.
2023-07-20 23:56:12 +00:00
Lina Butler f07dd2bf90 suggest: Refactor `full_keyword`.
This commit adds documentation and tests for `full_keyword`, and
reimplements it using the Rust iterator methods.
2023-07-20 23:56:12 +00:00
Lina Butler d6ccae7b19 suggest: Add ingestion test with mock Remote Settings data. 2023-07-20 23:56:12 +00:00
Lina Butler 366052272c suggest: Make `block_id` an integer. 2023-07-20 23:56:12 +00:00
Lina Butler b7c64dbf0c suggest: Replace the `SuggestionQuery` builder with a dictionary.
The Gecko JS UniFFI bindings don't support synchronous function calls
yet (https://bugzilla.mozilla.org/show_bug.cgi?id=1842433), so the
builder API will be a little awkward to use on Desktop.

This commit replaces the builder with a dictionary, with default
fields to make it more ergonomic.
2023-07-20 23:56:12 +00:00
Lina Butler 0be2ca812d suggest: Experiment with a `SuggestionQuery` builder API. 2023-07-20 23:56:12 +00:00
Lina Butler aad81e8ab0 suggest: Rename `SuggestionProvider` to `SuggestStore`.
In other uses, "provider" refers to the source of a suggestion, like
adM, Wikipedia, AMO, etc. Since `SuggestionProvider` aggregates
suggestions from all these sources, calling it a "provider" could be
confusing, especially if we support filtering suggestions from specific
sources in the future.
2023-07-20 23:56:12 +00:00
Lina Butler 8f9299e17d suggest: Rename `SuggestError` to `SuggestApiError`. 2023-07-20 23:56:12 +00:00
Lina Butler 2c08a5ece4 suggest: Flag sponsored suggestions. 2023-07-20 23:56:12 +00:00
Lina Butler 1188dd2816 suggest: Fix Clippy lints. 2023-07-20 23:56:12 +00:00
Lina Butler 5de430fdb0 suggest: Derive built-in traits for `Suggestion`. 2023-07-20 23:56:12 +00:00
Lina Butler 8a98fedccb suggest: Store and return icons. 2023-07-20 23:56:12 +00:00
Lina Butler 8014bba339 suggest: Use the same full keyword heuristics as Desktop. 2023-07-20 23:56:12 +00:00
Lina Butler 2da2064692 suggest: Add UniFFI bindings for `SuggestionProvider`. 2023-07-20 23:56:12 +00:00
Lina Butler aa8c6315c8 suggest: Build config changes for Kotlin UniFFI bindings.
This commit adds an empty UniFFI interface for the new Suggest
component.
2023-07-20 23:56:12 +00:00
Lina Butler dffb6086cc suggest: Open database connections lazily.
Before, creating a provider would block the calling thread until the
database was set up. Now, migrations will run the first time the
database is accessed, which we already expect to happen on a background
thread.
2023-07-20 23:56:12 +00:00
Lina Butler 15f6adec51 Initial scaffolding for a Suggest component.
This commit adds a new component to fetch search suggestions from
Remote Settings and store them in an SQLite database.
2023-07-20 23:56:12 +00:00