Implemented the high-level API and the client functionality. Storage is
a big TODO.
Added a CLI to test it, you can run it using `cargo remote-settings`.
I just saw that Merino expects a country to be passed along with city and region
in weather requests. Rather than assuming "US" and hardcoding that in desktop,
which would be fine for now, let's just go ahead and add the matching geoname's
country code to `Suggestion::Weather`. It's a small change.
This doesn't modify query parsing at all. Country names still aren't supported
in queries. We can cross that bridge if/when we come to it.
The last hack didn't fix the iOS build. I looked through xcframework
zip from the nightly build artifacts
(https://firefoxci.taskcluster-artifacts.net/dHGArzP3R0Cmwcr56RPQJA/0/public/build/nightly.json)
and it seems like it was correctly copying the `RustViaductFFI.h` file,
but not correctly updating module.modulemap.
The main issue was the `awk` command was printing out the updated
contents to STDOUT, but not actually modifying `module.modulemap`. I
fixed that and also moved away from `awk`, since I'm worried about
differences between gawk and nawk.
Tested this by running the following commands and diffing the generated
code from before this change:
```
cargo build --release -p megazord
cargo uniffi-bindgen generate --library target/release/libmegazord.so --crate remote_settings --language kotlin --out-dir [out-dir]
```
There were a few differences:
- The field order for RemoteSettingsConfig changed
- Some docstrings were missing
- Some of the method parameter names changed
To fix this, I updated the Rust code to match the deleted UDL file.
This does a number of things to support weather queries that include cities and
regions. Sorry for how big it is. I could split it up but I think it's more
helpful to see everything in context.
I added `geoname.rs` to support ingesting and querying
[GeoNames](https://www.geonames.org/). This is how we detect cities and regions,
and how we map cities to regions when a query contains only a city. Potentially
we could use it for other suggestion types.
I wanted to mirror the raw geonames tables (the ones in the geonames dataset,
not the ones in our DB) pretty much exactly rather than trying to overfit them
into something specific to weather suggestions. I think that will serve us
better long term.
The code makes some assumptions about the geonames data we store in remote
settings. I tried to document them. And as long as weather and Suggest are US
only, we'll only upload US geonames to RS. We can scale up or down the number of
geonames in RS. For example, we might use some significantly large population as
a threshold.
Weather suggestions are matched on any of the following:
* A weather keyword. `min_keyword_length` in the weather RS data controls
whether prefix matching is allowed. This is pretty much the same as before
except we were relying on desktop to do the prefix threshold check, and I
moved that here.
* A full city name (or abbreviation like "nyc" -- anything that's exactly equal
to a geoname alternate name)
* A full city name + any prefix of its region
After any of the above are typed, you can continue to type any prefix of any of
the others and that will match too. Once you add a word that isn't recognized,
matching stops.
I added a query parsing helper called `filter_map_chunks()`. I renamed
`keyword.rs` to `util.rs` and added it there. Hopefully its doc explains what it
does, and it would be nice if it could be useful for other providers in the
future.
I added a couple of cache structs, `WeatherCache` and `GeonameCache`, as an
optimization to avoid fetching common values from the DB that are used to handle
every weather query.
I added some tables to store keyword and geoname metrics: max keyword length and
max word count. Weather query handling uses them to quickly discard search
strings that are too long so we don't end up with any performance problems
trying to parse huge strings that people sometimes paste into the urlbar. We
could simply hardcode a const (like yelp.rs does), but I think it's nice to do
this.
Providers can now map to more than one record type. This lets us ingest geonames
when weather is ingested.
Bumped UniFFI to 0.28.2
Added a tool to run uniffi-bindgen in library mode. It can input either
a specific library path or the megazord crate name.
Use that simplify several build scripts -- especially the generate docs
ones. The best part of this is that we no longer have to maintain
hand-written modulemaps, which makes adding a new component harder than
it needs to be.
Split out the uniffi-bindgen commands from `build-xcframework.sh`. This
way you can run them standalone and see the results, even if you don't
have XCode setup.
One change is that
automation/swift-components-docs/generate-swift-project.sh now uses
`megazord_ios` rather than `megazord`. I think this should result in
slightly more accurate docs, since historically some components in the
Android megazord aren't in the iOS one (Although, I think they match at
the present).
This happens early during startup and Glean will now delay executing the
actual recording until after libxul is loaded externally.
This avoids forcing that load of Glean/libxul too early.
Use fields inside the error enum, rather than the UniFFI "flat" errors
which ignore the inner data. This is prep for moving to proc-macros.
Switched to using error_support and define both internal and public
error types.
Switch app-services consumers to using the `RemoteSettings` type rather
than `Client`. This way they're using the same public API as everyone
else. This required adding a couple more pub RemoteSettings methods
that I didn't expose via UniFFI. I didn't feel like adding new features
to this client, because I'm hoping to replace it (#6378).
I was going to update the adding a new component docs, but while I was
reading them that the best way to simplify them would be to automate the
process.
* Added the `cargo start-bindings` tool.
* Removed the "Converting an existing Component to UniFFI" HOWTO. I
believe that all our existing components have been converted so
there's no need to keep updating this.
* Added a section about dependencies and the build.gradle file.
* Removed the `pub use` items from the megazord lib.rs files. I don't
believe they were needed.
Bastian Gruber
Charlie Humphreys
Ryan VanderMeulen
Julien Cristau
dependabot[bot]
Ben Dean-Kawamura
Drew Willcoxon
Jan-Erik Rediger
Tarik Eshaq
Donal Meehan
Mark Banner