зеркало из https://github.com/mozilla/gecko-dev.git
279 строки
11 KiB
ReStructuredText
279 строки
11 KiB
ReStructuredText
=============================================
|
|
Directory Links Architecture and Data Formats
|
|
=============================================
|
|
|
|
Directory links are enhancements to the new tab experience that combine content
|
|
Firefox already knows about from user browsing with external content. There are
|
|
3 kinds of links:
|
|
|
|
- directory links fill in additional tiles on the new tab page if there would
|
|
have been empty tiles because the user has a clean profile or cleared history
|
|
- suggested links are shown if certain triggering criteria matches the user's
|
|
browsing behavior, i.e., if the user has a top site that matches one of
|
|
several possible sites. E.g., only show a sports suggestion if the user has a
|
|
sport site as a top site
|
|
- enhanced links replace a matching user's visible history tile from the same
|
|
site but only the visual aspects: title, image, and rollover image
|
|
|
|
To power the above features, DirectoryLinksProvider module downloads, at most
|
|
once per 24 hours, the directory source links as JSON with enough data for
|
|
Firefox to determine what should be shown or not. This module also handles
|
|
reporting back data about the tiles via asynchronous pings that don't return
|
|
data from the server.
|
|
|
|
For the directory source and ping endpoints, the default preference values point
|
|
to Mozilla key-pinned servers with encryption. No cookies are set by the servers
|
|
and Firefox enforces this by making anonymous requests.
|
|
|
|
- default directory source endpoint:
|
|
https://tiles.services.mozilla.com/v3/links/fetch/%LOCALE%/%CHANNEL%
|
|
- default directory ping endpoint: https://tiles.services.mozilla.com/v3/links/
|
|
|
|
|
|
Preferences
|
|
===========
|
|
|
|
There are two main preferences that control downloading links and reporting
|
|
metrics.
|
|
|
|
``browser.newtabpage.directory.source``
|
|
---------------------------------------
|
|
|
|
This endpoint tells Firefox where to download directory source file as a GET
|
|
request. It should return JSON of the appropriate format containing the relevant
|
|
links data. The value can be a data URI, e.g., an empty JSON object effectively
|
|
turns off remote downloading: ``data:text/plain,{}``
|
|
|
|
The preference value will have %LOCALE% and %CHANNEL% replaced by the
|
|
appropriate values for the build of Firefox, e.g.,
|
|
|
|
- directory source endpoint:
|
|
https://tiles.services.mozilla.com/v3/links/fetch/en-US/release
|
|
|
|
``browser.newtabpage.directory.ping``
|
|
-------------------------------------
|
|
|
|
This endpoint tells Firefox where to report Tiles metrics as a POST request. The
|
|
data is sent as a JSON blob. Setting it to empty effectively turns off reporting
|
|
of Tiles data.
|
|
|
|
A path segment will be appended to the endpoint of "view" or "click" depending
|
|
on the type of ping, e.g.,
|
|
|
|
- ``view`` ping endpoint: https://tiles.services.mozilla.com/v3/links/view
|
|
- ``click`` ping endpoint: https://tiles.services.mozilla.com/v3/links/click
|
|
|
|
|
|
Data Flow
|
|
=========
|
|
|
|
When Firefox starts, it checks for a cached directory source file. If one
|
|
exists, it checks for its timestamp to determine if a new file should be
|
|
downloaded.
|
|
|
|
If a directory source file needs to be downloaded, a GET request is made then
|
|
cacheed and unpacked the JSON into the different types of links. Various checks
|
|
filter out invalid links, e.g., those with http-hosted images or those that
|
|
don't fit the allowed suggestions.
|
|
|
|
When a new tab page is built, DirectoryLinksProvider module provides additional
|
|
link data that is combined with history link data to determine which links can
|
|
be displayed or not.
|
|
|
|
When a new tab page is shown, a ``view`` ping is sent with relevant tiles data.
|
|
Similarly, when the user clicks on various parts of tiles (to load the page,
|
|
pin, block, etc.), a ``click`` ping is sent with similar data. Both of these can
|
|
trigger downloading of fresh directory source links if 24 hours have elapsed
|
|
since last download.
|
|
|
|
Users can turn off the ping with in-new-tab-page controls.
|
|
|
|
As the new tab page is rendered, any images for tiles are downloaded if not
|
|
already cached. The default servers hosting the images are Mozilla CDN that
|
|
don't use cookies: https://tiles.cdn.mozilla.net/ and Firefox enforces that the
|
|
images come from mozilla.net or data URIs when using the default directory
|
|
source.
|
|
|
|
|
|
Source JSON Format
|
|
==================
|
|
|
|
Firefox expects links data in a JSON object with top level keys each providing
|
|
an array of tile objects. The keys correspond to the different types of links:
|
|
``directory``, ``suggested``, and ``enhanced``.
|
|
|
|
Example
|
|
-------
|
|
|
|
Below is an example directory source file::
|
|
|
|
{
|
|
"directory": [
|
|
{
|
|
"bgColor": "",
|
|
"directoryId": 498,
|
|
"enhancedImageURI": "https://tiles.cdn.mozilla.net/images/d11ba0b3095bb19d8092cd29be9cbb9e197671ea.28088.png",
|
|
"imageURI": "https://tiles.cdn.mozilla.net/images/1332a68badf11e3f7f69bf7364e79c0a7e2753bc.5316.png",
|
|
"title": "Mozilla Community",
|
|
"type": "affiliate",
|
|
"url": "http://contribute.mozilla.org/"
|
|
}
|
|
],
|
|
"enhanced": [
|
|
{
|
|
"bgColor": "",
|
|
"directoryId": 776,
|
|
"enhancedImageURI": "https://tiles.cdn.mozilla.net/images/44a14fc405cebc299ead86514dff0e3735c8cf65.10814.png",
|
|
"imageURI": "https://tiles.cdn.mozilla.net/images/20e24aa2219ec7542cc8cf0fd79f0c81e16ebeac.11859.png",
|
|
"title": "TurboTax",
|
|
"type": "sponsored",
|
|
"url": "https://turbotax.intuit.com/"
|
|
}
|
|
],
|
|
"suggested": [
|
|
{
|
|
"adgroup_name": "open-source browser",
|
|
"bgColor": "#cae1f4",
|
|
"check_inadjacency": true,
|
|
"directoryId": 702,
|
|
"explanation": "Suggested for %1$S enthusiasts who visit sites like %2$S",
|
|
"frecent_sites": [
|
|
"addons.mozilla.org",
|
|
"air.mozilla.org",
|
|
"blog.mozilla.org",
|
|
"bugzilla.mozilla.org",
|
|
"developer.mozilla.org",
|
|
"etherpad.mozilla.org",
|
|
"hacks.mozilla.org",
|
|
"hg.mozilla.org",
|
|
"mozilla.org",
|
|
"planet.mozilla.org",
|
|
"quality.mozilla.org",
|
|
"support.mozilla.org",
|
|
"treeherder.mozilla.org",
|
|
"wiki.mozilla.org"
|
|
],
|
|
"frequency_caps": {"daily": 3, "total": 10},
|
|
"imageURI": "https://tiles.cdn.mozilla.net/images/9ee2b265678f2775de2e4bf680df600b502e6038.3875.png",
|
|
"time_limits": {"start": "2014-01-01T00:00:00.000Z", "end": "2014-02-01T00:00:00.000Z"},
|
|
"title": "Thanks for testing!",
|
|
"type": "affiliate",
|
|
"url": "https://www.mozilla.com/firefox/tiles"
|
|
}
|
|
]
|
|
}
|
|
|
|
Link Object
|
|
-----------
|
|
|
|
Each link object has various values that Firefox uses to display a tile:
|
|
|
|
- ``url`` - string url for the page to be loaded when the tile is clicked. Only
|
|
https and http URLs are allowed.
|
|
- ``title`` - string that appears below the tile.
|
|
- ``type`` - string relationship of the link to Mozilla. Expected values:
|
|
affiliate, organic, sponsored.
|
|
- ``imageURI`` - string url for the tile image to show. Only https and data URIs
|
|
are allowed.
|
|
- ``enhancedImageURI`` - string url for the image to be shown before the user
|
|
hovers. Only https and data URIs are allowed.
|
|
- ``bgColor`` - string css color for additional fill background color.
|
|
- ``directoryId`` - id of the tile to be used during ping reporting
|
|
|
|
Suggested Link Object Extras
|
|
----------------------------
|
|
|
|
A suggested link has additional values:
|
|
|
|
- ``adgroup_name`` - string to override the hardcoded display name of the
|
|
triggering set of sites in Firefox.
|
|
- ``check_inadjacency`` - boolean if true prevents the suggested link from being
|
|
shown if the new tab page is showing a site from an inadjacency list.
|
|
- ``explanation`` - string to override the default explanation that appears
|
|
below a Suggested Tile. %1$S is replaced by the triggering adgroup name and
|
|
%2$S is replaced by the triggering site.
|
|
- ``frecent_sites`` - array of strings of the sites that can trigger showing a
|
|
Suggested Tile if the user has the site in one of the top 100 most-frecent
|
|
pages.
|
|
- ``frequency_caps`` - an object consisting of daily and total frequency caps
|
|
that limit the number of times a Suggested Tile can be shown in the new tab
|
|
per day and overall.
|
|
- ``time_limits`` - an object consisting of start and end timestamps specifying
|
|
when a Suggested Tile may start and has to stop showing in the newtab.
|
|
The timestamp is expected in ISO_8601 format: '2014-01-10T20:00:00.000Z'
|
|
|
|
The inadjacency list is packaged with Firefox as base64-encoded 1-way-hashed
|
|
sites that tend to have adult, gambling, alcohol, drug, and similar content.
|
|
Its location: chrome://browser/content/newtab/newTab.inadjacent.json
|
|
|
|
The preapproved arrays follow a policy for determining what topic grouping is
|
|
allowed as well as the composition of a grouping. The topics are broad
|
|
uncontroversial categories, e.g., Mobile Phone, News, Technology, Video Game,
|
|
Web Development. There are at least 5 sites within a grouping, and as many
|
|
popular sites relevant to the topic are included to avoid having one site be
|
|
clearly dominant. These requirements provide some deniability of which site
|
|
actually triggered a suggestion during ping reporting, so it's more difficult to
|
|
determine if a user has gone to a specific site.
|
|
|
|
|
|
Ping JSON Format
|
|
================
|
|
|
|
Firefox reports back an action and the state of tiles on the new tab page based
|
|
on the user opening a new tab or clicking a tile. The top level keys of the
|
|
ping:
|
|
|
|
- ``locale`` - string locale of the Firefox build
|
|
- ``tiles`` - array of tiles ping objects
|
|
|
|
An additional key at the top level indicates which action triggered the ping.
|
|
The value associated to the action key is the 0-based index into the tiles array
|
|
of which tile triggered the action. Valid actions: block, click, pin, sponsored,
|
|
sponsored_link, unpin, view. E.g., if the second tile is being clicked, the ping
|
|
will have ``"click": 1``
|
|
|
|
Example
|
|
-------
|
|
|
|
Below is an example ``click`` ping with 3 tiles: a pinned suggested tile
|
|
followed by a history tile and a directory tile. The first tile is being
|
|
blocked::
|
|
|
|
{
|
|
"locale": "en-US",
|
|
"tiles": [
|
|
{
|
|
"id": 702,
|
|
"pin": 1,
|
|
"past_impressions": {"total": 5, "daily": 1},
|
|
},
|
|
{},
|
|
{
|
|
"id": 498,
|
|
}
|
|
],
|
|
"block": 0
|
|
}
|
|
|
|
Tiles Ping Object
|
|
-----------------
|
|
|
|
Each tile of the new tab page is reported back as part of the ping with some or
|
|
none of the following optional values:
|
|
|
|
- ``id`` - id that was provided as part of the downloaded link object (for all
|
|
types of links: directory, suggested, enhanced); not present if the tile was
|
|
created from user behavior, e.g., visiting pages
|
|
- ``past_impressions`` - number of impressions (new tab "views") a suggested
|
|
tile was shown before it was clicked, pinned or blocked. Where the "total"
|
|
counter is the overall number of impressions accumulated prior to a click action,
|
|
and "daily" counter is the number impressions occurred on same calendar day of
|
|
a click. This infomration is submitted once per a suggested tile upon click,
|
|
pin or block
|
|
- ``pinned`` - 1 if the tile is pinned; not present otherwise
|
|
- ``pos`` - integer position if the tile is not in the natural order, e.g., a
|
|
pinned tile after an empty slot; not present otherwise
|
|
- ``score`` - integer truncated score based on the tile's frecency; not present
|
|
if 0
|
|
- ``url`` - empty string if it's an enhanced tile; not present otherwise
|