Clear out some non-fn references to Algolia (#20592)

* Clear out some non-fn references to Algolia

* Update .github/workflows/dry-run-sync-search-indices.yml

Co-authored-by: James M. Greene <JamesMGreene@github.com>

* Update contributing/search.md

Co-authored-by: James M. Greene <JamesMGreene@github.com>

Co-authored-by: James M. Greene <JamesMGreene@github.com>
This commit is contained in:
Kevin Heis 2021-07-29 13:20:55 -07:00 коммит произвёл GitHub
Родитель 42a079716b
Коммит 1ed18e1448
Не найден ключ, соответствующий данной подписи
Идентификатор ключа GPG: 4AEE18F83AFDEB23
13 изменённых файлов: 43 добавлений и 43 удалений

Просмотреть файл

@ -7,7 +7,7 @@ const eventPayload = JSON.parse(await fs.readFile(process.env.GITHUB_EVENT_PATH,
// This workflow-run script does the following: // This workflow-run script does the following:
// 1. Gets an array of labels on a PR. // 1. Gets an array of labels on a PR.
// 2. Finds one with the relevant Algolia text; if none found, exits early. // 2. Finds one with the relevant search text; if none found, exits early.
// 3. Gets the version substring from the label string. // 3. Gets the version substring from the label string.
const labelText = 'sync-english-index-for-' const labelText = 'sync-english-index-for-'
@ -19,18 +19,18 @@ if (!(labelsArray && labelsArray.length)) {
} }
// Find the relevant label // Find the relevant label
const algoliaLabel = labelsArray const searchLabel = labelsArray
.map((label) => label.name) .map((label) => label.name)
.find((label) => label.startsWith(labelText)) .find((label) => label.startsWith(labelText))
// Exit early if no relevant label is found // Exit early if no relevant label is found
if (!algoliaLabel) { if (!searchLabel) {
process.exit(0) process.exit(0)
} }
// Given: sync-english-index-for-enterprise-server@3.0 // Given: sync-english-index-for-enterprise-server@3.0
// Returns: enterprise-server@3.0 // Returns: enterprise-server@3.0
const versionToSync = algoliaLabel.split(labelText)[1] const versionToSync = searchLabel.split(labelText)[1]
// Store the version so we can access it later in the workflow // Store the version so we can access it later in the workflow
setOutput('versionToSync', versionToSync) setOutput('versionToSync', versionToSync)

Просмотреть файл

@ -34,7 +34,7 @@
- PLACEHOLDER - PLACEHOLDER
``` ```
**Note:** All of the content in this file will be updated when the release notes are created in the megabranch including the filename `PLACEHOLDER.yml`. You can update the date or leave it as-is and wait to update it when the release notes are finalized. **Note:** All of the content in this file will be updated when the release notes are created in the megabranch including the filename `PLACEHOLDER.yml`. You can update the date or leave it as-is and wait to update it when the release notes are finalized.
- [ ] Create the Algolia search indices for the new release: - [ ] Create the search indices for the new release:
``` ```
npm run sync-search-ghes-release npm run sync-search-ghes-release
``` ```
@ -51,7 +51,7 @@
``` ```
sync-english-index-for-<PLAN@RELEASE> sync-english-index-for-<PLAN@RELEASE>
``` ```
☝️ This will run a workflow **on every push to the PR** that will sync **only** the English index for the new version to Algolia. This will make the GHES content searchable on staging throughout content creation, and will ensure the search updates go live at the same time the content is published. See [`contributing/search.md`](https://github.com/github/docs-internal/blob/main/contributing/search.md) for details. ☝️ This will run a workflow **on every push to the PR** that will sync **only** the English index for the new version. This will make the GHES content searchable on staging throughout content creation, and will ensure the search updates go live at the same time the content is published. See [`contributing/search.md`](https://github.com/github/docs-internal/blob/main/contributing/search.md) for details.
- [ ] In `github/github`, to create a new GHES release follow these steps: - [ ] In `github/github`, to create a new GHES release follow these steps:
- [ ] Copy the previous release's root document to a new root document for this release `cp app/api/description/ghes-<LATEST RELEASE NUMBER>.yaml app/api/description/ghes-<NEXT RELEASE NUMBER>.yaml`. - [ ] Copy the previous release's root document to a new root document for this release `cp app/api/description/ghes-<LATEST RELEASE NUMBER>.yaml app/api/description/ghes-<NEXT RELEASE NUMBER>.yaml`.
@ -79,7 +79,7 @@ If the `OpenAPI dev mode check / check-schema-versions` check fails with the fol
#### `Node.js tests / test content` failures #### `Node.js tests / test content` failures
If the `Node.js tests / test content` check fails with the following message, the `lib/enterprise-dates.json` file is not up-to-date: If the `Node.js tests / test content` check fails with the following message, the `lib/enterprise-dates.json` file is not up-to-date:
> FAIL tests/content/algolia-search.js ● algolia has remote indexNames in every language for every supported GHE version > FAIL tests/content/search.js ● search has remote indexNames in every language for every supported GHE version
This file should be automatically updated, but you can also run `script/update-enterprise-dates.js` to update it. **Note:** If the test is still failing after running this script, look at the dates for this release. If the date is still inaccurate, it may be an issue with the source at https://github.com/github/enterprise-releases/blob/master/docs/supported-versions.md#release-lifecycle-dates. If that is the case, manually update the dates in the `lib/enterprise-dates.json` file. This file should be automatically updated, but you can also run `script/update-enterprise-dates.js` to update it. **Note:** If the test is still failing after running this script, look at the dates for this release. If the date is still inaccurate, it may be an issue with the source at https://github.com/github/enterprise-releases/blob/master/docs/supported-versions.md#release-lifecycle-dates. If that is the case, manually update the dates in the `lib/enterprise-dates.json` file.
@ -94,4 +94,4 @@ This file should be automatically updated, but you can also run `script/update-e
### 🚢 🛳️ 🚢 Shipping the release branch ### 🚢 🛳️ 🚢 Shipping the release branch
- [ ] The `github/docs-internal` repo is frozen, and the `Repo Freeze Check / Prevent merging during deployment freezes (pull_request_target)` test is expected to fail. Use admin permissions to ship the release branch with this failure. - [ ] The `github/docs-internal` repo is frozen, and the `Repo Freeze Check / Prevent merging during deployment freezes (pull_request_target)` test is expected to fail. Use admin permissions to ship the release branch with this failure.
- [ ] Once smoke tests have passed, you can unfreeze the repos by deleting the `FREEZE` secret in both the `github/docs-internal` and `github/docs` repos. To delete the secrets, click the repo **Settings** tab and then click **Secrets** in the left sidebar. Click **Remove** next to the `FREEZE` secret. - [ ] Once smoke tests have passed, you can unfreeze the repos by deleting the `FREEZE` secret in both the `github/docs-internal` and `github/docs` repos. To delete the secrets, click the repo **Settings** tab and then click **Secrets** in the left sidebar. Click **Remove** next to the `FREEZE` secret.

Просмотреть файл

@ -1,6 +1,6 @@
name: (Dry run) Algolia name: (Dry run) Search indexes
# **What it does**: On request, dry run Algolia to check for issues with search indexing. # **What it does**: On request, dry run to check for issues with search indexing.
# **Why we have it**: It helps us debug issues with search indexing. # **Why we have it**: It helps us debug issues with search indexing.
# **Who does it impact**: Docs engineering. # **Who does it impact**: Docs engineering.

2
.github/workflows/enterprise-dates.yml поставляемый
Просмотреть файл

@ -3,7 +3,7 @@ name: Enterprise date updater
# **What it does**: Runs on a schedule to update lib/enterprise-dates.json. # **What it does**: Runs on a schedule to update lib/enterprise-dates.json.
# **Why we have it**: The lib/enterprise-dates.json file needs to be up-to-date # **Why we have it**: The lib/enterprise-dates.json file needs to be up-to-date
# for the .github/workflows/open-enterprise-issue.yml workflow and the # for the .github/workflows/open-enterprise-issue.yml workflow and the
# tests/content/algolia-search.js test. # tests/content/search.js test.
# **Who does it impact**: Docs engineering, docs content. # **Who does it impact**: Docs engineering, docs content.
on: on:

Просмотреть файл

@ -1,4 +1,4 @@
name: Algolia name: Sync search indexes
# **What it does**: This updates our search indexes after each deployment. # **What it does**: This updates our search indexes after each deployment.
# **Why we have it**: We want our search indexes kept up to date. # **Why we have it**: We want our search indexes kept up to date.
@ -40,4 +40,4 @@ jobs:
channel: ${{ secrets.DOCS_ALERTS_SLACK_CHANNEL_ID }} channel: ${{ secrets.DOCS_ALERTS_SLACK_CHANNEL_ID }}
bot-token: ${{ secrets.SLACK_DOCS_BOT_TOKEN }} bot-token: ${{ secrets.SLACK_DOCS_BOT_TOKEN }}
color: failure color: failure
text: The last Algolia workflow run for ${{github.repository}} failed. Search actions for `workflow:Algolia` text: The last search index workflow run for ${{github.repository}} failed. Search actions for `workflow:search`

Просмотреть файл

@ -1,4 +1,4 @@
name: Algolia Sync Single English Index name: Sync Single English Index
# **What it does**: # **What it does**:
# **Why we have it**: # **Why we have it**:
@ -31,9 +31,9 @@ jobs:
cache: npm cache: npm
- name: Install dependencies - name: Install dependencies
run: npm ci run: npm ci
- name: Get version from Algolia label if present; only continue if the label is found. - name: Get version from search label if present; only continue if the label is found.
id: getVersion id: getVersion
run: $GITHUB_WORKSPACE/.github/actions-scripts/enterprise-algolia-label.js run: $GITHUB_WORKSPACE/.github/actions-scripts/enterprise-search-label.js
- if: ${{ steps.getVersion.outputs.versionToSync }} - if: ${{ steps.getVersion.outputs.versionToSync }}
name: Sync English index for single version name: Sync English index for single version
env: env:

Просмотреть файл

@ -77,7 +77,7 @@ COPY --chown=node:node --from=builder /usr/src/docs/.next /usr/src/docs/.next
# We should always be running in production mode # We should always be running in production mode
ENV NODE_ENV production ENV NODE_ENV production
# Use Lunr instead of Algolia # Hide iframes, add warnings to external links
ENV AIRGAP true ENV AIRGAP true
# Copy only what's needed to run the server # Copy only what's needed to run the server

Просмотреть файл

@ -15,5 +15,5 @@ Here, you'll find additional information that might be helpful as you work on a
- [node versions](./node-versions.md) - our site runs on Node.js - [node versions](./node-versions.md) - our site runs on Node.js
- [permalinks](./permalinks.md) - permalinks for article versioning - [permalinks](./permalinks.md) - permalinks for article versioning
- [redirects](./redirects.md) - configuring redirects in the site - [redirects](./redirects.md) - configuring redirects in the site
- [search](./search.md) - our search functionality is powered by [Algolia](https://www.algolia.com) - [search](./search.md) - our local site search functionality
- [troubleshooting](./troubleshooting.md) - some help for troubleshooting failed and stalled status checks - [troubleshooting](./troubleshooting.md) - some help for troubleshooting failed and stalled status checks

Просмотреть файл

@ -2,7 +2,7 @@
## Overview ## Overview
This site's search functionality is powered by [Algolia](https://www.algolia.com), a third-party service. This site's search functionality.
To see all existing search-related issues and pull requests, visit [github.com/github/docs/labels/search](https://github.com/github/docs/labels/search). To see all existing search-related issues and pull requests, visit [github.com/github/docs/labels/search](https://github.com/github/docs/labels/search).
@ -14,7 +14,7 @@ To see all existing search-related issues and pull requests, visit [github.com/g
## How to search ## How to search
The site search is part of every version of docs.github.com. This endpoint responds in JSON format, and fronts Algolia and Lunr. We recommend using this endpoint over directly integrating with Algolia or Lunr, as the endpoint will be more stable. On any page, you can use the search box to search the documents we've indexed. The site search is part of every version of docs.github.com. This endpoint responds in JSON format, and fronts our search querying functionality. We recommend using this endpoint, as the endpoint will be more stable. On any page, you can use the search box to search the documents we've indexed.
You can also query our search endpoint directly at: You can also query our search endpoint directly at:
`https://docs.github.com/search?version=<VERSION>&language=<LANGUAGE CODE>&filters=topics:<TOPIC>&query=<QUERY>` `https://docs.github.com/search?version=<VERSION>&language=<LANGUAGE CODE>&filters=topics:<TOPIC>&query=<QUERY>`
@ -33,25 +33,25 @@ https://docs.github.com/search?version=dotcom&language=en&filters=topics:'oauth
### Using the topics search filter ### Using the topics search filter
Using the attribute `topics` in your query will only return results that have the matching topic value. The `topics` attribute is configured as a [`filter only` facet in Algolia](https://www.algolia.com/doc/guides/managing-results/refine-results/filtering/). When the topic contains spaces, you must use quotes. For Algolia, [here](https://www.algolia.com/doc/api-reference/api-parameters/filters/#handle-attributes-with-spaces) is an example for filters containing spaces. Algolia also requires [boolean operators](https://www.algolia.com/doc/api-reference/api-parameters/filters) to combine more than one filter. Using the attribute `topics` in your query will only return results that have the matching topic value. When the topic contains spaces, you must use quotes. Filters with spaces or combining filters requires special syntax.
## Production deploys ## Production deploys
A [GitHub Actions workflow](.github/workflows/sync-algolia-search-indices.yml) triggered by pushes to the `main` branch syncs the search data to Algolia. This process generates structured data for all pages on the site, compares that data to what's currently on Algolia, then adds, updates, or removes indices based on the diff of the local and remote data, being careful not to create duplicate records and avoiding any unnecessary (and costly) indexing operations. A [GitHub Actions workflow](.github/workflows/sync-search-indices.yml) triggered by pushes to the `main` branch syncs the search data. This process generates structured data for all pages on the site, compares that data to what's currently on search, then adds, updates, or removes indices based on the diff of the local and remote data, being careful not to create duplicate records and avoiding any unnecessary (and costly) indexing operations.
The Actions workflow progress can be viewed (by GitHub employees) in the [Actions tab](https://github.com/github/docs/actions?query=workflow%3AAlgolia) of the repo. The Actions workflow progress can be viewed (by GitHub employees) in the [Actions tab](https://github.com/github/docs/actions?query=workflow%3Asearch) of the repo.
Because the workflow runs after a branch is merged to `main`, there is a slight delay for search data updates to appear on the site. Because the workflow runs after a branch is merged to `main`, there is a slight delay for search data updates to appear on the site.
## Manual sync from a checkout ## Manual sync from a checkout
It is also possible to manually sync the indices to Algolia from your local checkout of the repo, before your branch is merged to `main`. It is also possible to manually sync the indices from your local checkout of the repo, before your branch is merged to `main`.
**Prerequisite:** Make sure the environment variables `ALGOLIA_APPLICATION_ID` and `ALGOLIA_API_KEY` are set in your `.env` file. You can find these values on [Algolia](https://www.algolia.com/apps/ZI5KPY1HBE/api-keys/all). **Prerequisite:** Make sure the environment variables `ALGOLIA_APPLICATION_ID` and `ALGOLIA_API_KEY` are set in your `.env` file. You can find these values on [Algolia](https://www.algolia.com/apps/ZI5KPY1HBE/api-keys/all). _Remove this paragraph when we switch to Lunr._
### Build without sync (dry run) ### Build without sync (dry run)
To build all the indices without uploading them to Algolia's servers (this takes about an hour): To build all the indices without uploading them (this takes about an hour):
``` ```
npm run sync-search-dry-run npm run sync-search-dry-run
``` ```
@ -65,11 +65,11 @@ Substitute a currently supported version for `<PLAN@RELEASE>` and a currently su
### Build and sync ### Build and sync
To build all the indices and sync them to Algolia (this also takes about an hour): To build all the indices and sync them (this also takes about an hour):
``` ```
npm run sync-search npm run sync-search
``` ```
To build indices for a specific language and/or version and sync them to Algolia: To build indices for a specific language and/or version and sync them:
``` ```
VERSION=<PLAN@RELEASE LANGUAGE=<TWO-LETTER CODE> npm run sync-search VERSION=<PLAN@RELEASE LANGUAGE=<TWO-LETTER CODE> npm run sync-search
``` ```
@ -95,17 +95,17 @@ Why do we need this? For our daily shipping needs, it's tolerable that search up
### Actions workflow files ### Actions workflow files
- [`.github/workflows/sync-algolia-search-indices.yml`](.github/workflows/sync-algolia-search-indices.yml) - Builds and syncs search indices whenever the `main` branch is pushed to (that is, on production deploys). - [`.github/workflows/sync-search-indices.yml`](.github/workflows/sync-search-indices.yml) - Builds and syncs search indices whenever the `main` branch is pushed to (that is, on production deploys).
- [`.github/workflows/dry-run-sync-algolia-search-indices.yml`](.github/workflows/dry-run-sync-algolia-search-indices.yml) - This workflow can be run manually (via `workflow_dispatch`) to do a dry run build of all the indices. Useful for confirming that the indices can build without erroring out. - [`.github/workflows/dry-run-sync-search-indices.yml`](.github/workflows/dry-run-sync-search-indices.yml) - This workflow can be run manually (via `workflow_dispatch`) to do a dry run build of all the indices. Useful for confirming that the indices can build without erroring out.
- [`.github/workflows/sync-single-english-algolia-index.yml`](.github/workflows/sync-single-english-algolia-index.yml) - This workflow is run when a label in the right format is applied to a PR. See "[Label-triggered Actions workflow](#label-triggered-actions-workflow)" for details. - [`.github/workflows/sync-single-english-index.yml`](.github/workflows/sync-single-english-index.yml) - This workflow is run when a label in the right format is applied to a PR. See "[Label-triggered Actions workflow](#label-triggered-actions-workflow)" for details.
### Code files ### Code files
- [components/lib/search.ts](components/lib/search.ts) - The browser-side code that enables search. - [components/lib/search.ts](components/lib/search.ts) - The browser-side code that enables search.
- [lib/search/algolia-client.js](lib/search/algolia-client.js) - A thin wrapper around the [algoliasearch](https://ghub.io/algoliasearch) Node.js module for interacting with the Algolia API. - [lib/search/client.js](lib/search/client.js) - A thin wrapper around the Node.js module for interacting with the search API.
- [lib/search/algolia-search-index.js](lib/search/algolia-search-index.js) - A class for generating structured search data from repository content and syncing it with the remote Algolia service. This class has built-in validation to ensure that all records are valid before they're uploaded. This class also takes care of removing deprecated records, and compares existing remote records with the latest local records to avoid uploading records that haven't changed. - [lib/search/search-index.js](lib/search/search-index.js) - A class for generating structured search data from repository content and syncing it. This class has built-in validation to ensure that all records are valid before they're uploaded. This class also takes care of removing deprecated records, and compares existing remote records with the latest local records to avoid uploading records that haven't changed.
- [script/sync-search-indices.js](script/sync-search-indices.js) - The script used by the Actions workflow to update search indices on our Algolia account. This can also be [run in the development environment](#development). - [script/sync-search-indices.js](script/sync-search-indices.js) - The script used by the Actions workflow to update search indices. This can also be [run in the development environment](#development).
- [tests/algolia-search.js](tests/algolia-search.js) - Tests! - [tests/content/search.js](tests/content/search.js) - Tests!
## Indices ## Indices
@ -144,7 +144,7 @@ Each record represents a section of a page. Sections are derived by splitting up
## Notes ## Notes
- It's not strictly necessary to set an `objectID` as Algolia will create one automatically, but by creating our own we have a guarantee that subsequent invocations of this upload script will overwrite existing records instead of creating numerous duplicate records with differing IDs. - It's not strictly necessary to set an `objectID` as the search index will create one automatically, but by creating our own we have a guarantee that subsequent invocations of this upload script will overwrite existing records instead of creating numerous duplicate records with differing IDs.
- Algolia has typo tolerance. Try spelling something wrong and see what you get! - Our search querying has typo tolerance. Try spelling something wrong and see what you get!
- Algolia has lots of controls for customizing each index, so we can add weights to certain attributes and create rules like "title is more important than body", etc. But it works pretty well as-is without any configuration. - Our search querying has lots of controls for customizing each index, so we can add weights to certain attributes and create rules like "title is more important than body", etc. But it works pretty well as-is without any configuration.
- Algolia has support for "advanced query syntax" for exact matching of quoted expressions and exclusion of words preceded by a `-` sign. This is off by default but we have it enabled in our browser client. This and many other settings can be configured in Algolia.com web interface. The settings in the web interface can be overridden by the search endpoint. See [middleware/search.js]([middleware/search.js). - Our search querying has support for "advanced query syntax" for exact matching of quoted expressions and exclusion of words preceded by a `-` sign. This is off by default but we have it enabled in our browser client. The settings in the web interface can be overridden by the search endpoint. See [middleware/search.js]([middleware/search.js).

Просмотреть файл

@ -3,7 +3,7 @@ export const maxContentLength = 5000
export const namePrefix = 'github-docs' export const namePrefix = 'github-docs'
export default { export default {
// records must be truncated to avoid going over Algolia's 10K limit // records must be truncated to avoid going over 10K limit
maxRecordLength, maxRecordLength,
maxContentLength, maxContentLength,
namePrefix, namePrefix,

Просмотреть файл

@ -47,7 +47,7 @@ export default function csp(req, res, next) {
const { requestedVersion } = isArchivedVersion(req) const { requestedVersion } = isArchivedVersion(req)
// Exception for Algolia instantsearch in deprecated Enterprise docs (Node.js era) // Exception for deprecated Enterprise docs (Node.js era)
if ( if (
versionSatisfiesRange(requestedVersion, '<=2.19') && versionSatisfiesRange(requestedVersion, '<=2.19') &&
versionSatisfiesRange(requestedVersion, '>2.12') versionSatisfiesRange(requestedVersion, '>2.12')

Просмотреть файл

@ -1,5 +1,5 @@
#!/usr/bin/env node #!/usr/bin/env node
// This module accepts an Algolia search record object as input and // This module accepts a search record object as input and
// returns a ranking score which influences how results are sorted. // returns a ranking score which influences how results are sorted.
// higher in this list == higher search ranking // higher in this list == higher search ranking

Просмотреть файл

@ -4,7 +4,7 @@ import { namePrefix } from '../../lib/search/config.js'
import remoteIndexNames from '../../lib/search/cached-index-names.json' import remoteIndexNames from '../../lib/search/cached-index-names.json'
const languageCodes = Object.keys(xLanguages) const languageCodes = Object.keys(xLanguages)
describe('algolia', () => { describe('search', () => {
test('has remote indexNames in every language for every supported GHE version', () => { test('has remote indexNames in every language for every supported GHE version', () => {
expect(supported.length).toBeGreaterThan(1) expect(supported.length).toBeGreaterThan(1)
supported.forEach((version) => { supported.forEach((version) => {