зеркало из https://github.com/github/docs.git
Clear out some non-fn references to Algolia (#20592)
* Clear out some non-fn references to Algolia * Update .github/workflows/dry-run-sync-search-indices.yml Co-authored-by: James M. Greene <JamesMGreene@github.com> * Update contributing/search.md Co-authored-by: James M. Greene <JamesMGreene@github.com> Co-authored-by: James M. Greene <JamesMGreene@github.com>
This commit is contained in:
Kevin Heis
GitHub
Родитель
42a079716b
Коммит
1ed18e1448
|
|
@ -7,7 +7,7 @@ const eventPayload = JSON.parse(await fs.readFile(process.env.GITHUB_EVENT_PATH,
|
|||
|
||||
// This workflow-run script does the following:
|
||||
// 1. Gets an array of labels on a PR.
|
||||
// 2. Finds one with the relevant Algolia text; if none found, exits early.
|
||||
// 2. Finds one with the relevant search text; if none found, exits early.
|
||||
// 3. Gets the version substring from the label string.
|
||||
|
||||
const labelText = 'sync-english-index-for-'
|
||||
|
|
@ -19,18 +19,18 @@ if (!(labelsArray && labelsArray.length)) {
|
|||
}
|
||||
|
||||
// Find the relevant label
|
||||
const algoliaLabel = labelsArray
|
||||
const searchLabel = labelsArray
|
||||
.map((label) => label.name)
|
||||
.find((label) => label.startsWith(labelText))
|
||||
|
||||
// Exit early if no relevant label is found
|
||||
if (!algoliaLabel) {
|
||||
if (!searchLabel) {
|
||||
process.exit(0)
|
||||
}
|
||||
|
||||
// Given: sync-english-index-for-enterprise-server@3.0
|
||||
// Returns: enterprise-server@3.0
|
||||
const versionToSync = algoliaLabel.split(labelText)[1]
|
||||
const versionToSync = searchLabel.split(labelText)[1]
|
||||
|
||||
// Store the version so we can access it later in the workflow
|
||||
setOutput('versionToSync', versionToSync)
|
||||
|
|
@ -34,7 +34,7 @@
|
|||
- PLACEHOLDER
|
||||
```
|
||||
**Note:** All of the content in this file will be updated when the release notes are created in the megabranch including the filename `PLACEHOLDER.yml`. You can update the date or leave it as-is and wait to update it when the release notes are finalized.
|
||||
- [ ] Create the Algolia search indices for the new release:
|
||||
- [ ] Create the search indices for the new release:
|
||||
```
|
||||
npm run sync-search-ghes-release
|
||||
```
|
||||
|
|
@ -51,7 +51,7 @@
|
|||
```
|
||||
sync-english-index-for-<PLAN@RELEASE>
|
||||
```
|
||||
☝️ This will run a workflow **on every push to the PR** that will sync **only** the English index for the new version to Algolia. This will make the GHES content searchable on staging throughout content creation, and will ensure the search updates go live at the same time the content is published. See [`contributing/search.md`](https://github.com/github/docs-internal/blob/main/contributing/search.md) for details.
|
||||
☝️ This will run a workflow **on every push to the PR** that will sync **only** the English index for the new version. This will make the GHES content searchable on staging throughout content creation, and will ensure the search updates go live at the same time the content is published. See [`contributing/search.md`](https://github.com/github/docs-internal/blob/main/contributing/search.md) for details.
|
||||
|
||||
- [ ] In `github/github`, to create a new GHES release follow these steps:
|
||||
- [ ] Copy the previous release's root document to a new root document for this release `cp app/api/description/ghes-<LATEST RELEASE NUMBER>.yaml app/api/description/ghes-<NEXT RELEASE NUMBER>.yaml`.
|
||||
|
|
@ -79,7 +79,7 @@ If the `OpenAPI dev mode check / check-schema-versions` check fails with the fol
|
|||
#### `Node.js tests / test content` failures
|
||||
|
||||
If the `Node.js tests / test content` check fails with the following message, the `lib/enterprise-dates.json` file is not up-to-date:
|
||||
> FAIL tests/content/algolia-search.js ● algolia › has remote indexNames in every language for every supported GHE version
|
||||
> FAIL tests/content/search.js ● search › has remote indexNames in every language for every supported GHE version
|
||||
|
||||
This file should be automatically updated, but you can also run `script/update-enterprise-dates.js` to update it. **Note:** If the test is still failing after running this script, look at the dates for this release. If the date is still inaccurate, it may be an issue with the source at https://github.com/github/enterprise-releases/blob/master/docs/supported-versions.md#release-lifecycle-dates. If that is the case, manually update the dates in the `lib/enterprise-dates.json` file.
|
||||
|
||||
|
|
@ -94,4 +94,4 @@ This file should be automatically updated, but you can also run `script/update-e
|
|||
### 🚢 🛳️ 🚢 Shipping the release branch
|
||||
|
||||
- [ ] The `github/docs-internal` repo is frozen, and the `Repo Freeze Check / Prevent merging during deployment freezes (pull_request_target)` test is expected to fail. Use admin permissions to ship the release branch with this failure.
|
||||
- [ ] Once smoke tests have passed, you can unfreeze the repos by deleting the `FREEZE` secret in both the `github/docs-internal` and `github/docs` repos. To delete the secrets, click the repo **Settings** tab and then click **Secrets** in the left sidebar. Click **Remove** next to the `FREEZE` secret.
|
||||
- [ ] Once smoke tests have passed, you can unfreeze the repos by deleting the `FREEZE` secret in both the `github/docs-internal` and `github/docs` repos. To delete the secrets, click the repo **Settings** tab and then click **Secrets** in the left sidebar. Click **Remove** next to the `FREEZE` secret.
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
name: (Dry run) Algolia
|
||||
name: (Dry run) Search indexes
|
||||
|
||||
# **What it does**: On request, dry run Algolia to check for issues with search indexing.
|
||||
# **What it does**: On request, dry run to check for issues with search indexing.
|
||||
# **Why we have it**: It helps us debug issues with search indexing.
|
||||
# **Who does it impact**: Docs engineering.
|
||||
|
||||
|
|
@ -3,7 +3,7 @@ name: Enterprise date updater
|
|||
# **What it does**: Runs on a schedule to update lib/enterprise-dates.json.
|
||||
# **Why we have it**: The lib/enterprise-dates.json file needs to be up-to-date
|
||||
# for the .github/workflows/open-enterprise-issue.yml workflow and the
|
||||
# tests/content/algolia-search.js test.
|
||||
# tests/content/search.js test.
|
||||
# **Who does it impact**: Docs engineering, docs content.
|
||||
|
||||
on:
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
name: Algolia
|
||||
name: Sync search indexes
|
||||
|
||||
# **What it does**: This updates our search indexes after each deployment.
|
||||
# **Why we have it**: We want our search indexes kept up to date.
|
||||
|
|
@ -40,4 +40,4 @@ jobs:
|
|||
channel: ${{ secrets.DOCS_ALERTS_SLACK_CHANNEL_ID }}
|
||||
bot-token: ${{ secrets.SLACK_DOCS_BOT_TOKEN }}
|
||||
color: failure
|
||||
text: The last Algolia workflow run for ${{github.repository}} failed. Search actions for `workflow:Algolia`
|
||||
text: The last search index workflow run for ${{github.repository}} failed. Search actions for `workflow:search`
|
||||
|
|
@ -1,4 +1,4 @@
|
|||
name: Algolia Sync Single English Index
|
||||
name: Sync Single English Index
|
||||
|
||||
# **What it does**:
|
||||
# **Why we have it**:
|
||||
|
|
@ -31,9 +31,9 @@ jobs:
|
|||
cache: npm
|
||||
- name: Install dependencies
|
||||
run: npm ci
|
||||
- name: Get version from Algolia label if present; only continue if the label is found.
|
||||
- name: Get version from search label if present; only continue if the label is found.
|
||||
id: getVersion
|
||||
run: $GITHUB_WORKSPACE/.github/actions-scripts/enterprise-algolia-label.js
|
||||
run: $GITHUB_WORKSPACE/.github/actions-scripts/enterprise-search-label.js
|
||||
- if: ${{ steps.getVersion.outputs.versionToSync }}
|
||||
name: Sync English index for single version
|
||||
env:
|
||||
|
|
@ -77,7 +77,7 @@ COPY --chown=node:node --from=builder /usr/src/docs/.next /usr/src/docs/.next
|
|||
# We should always be running in production mode
|
||||
ENV NODE_ENV production
|
||||
|
||||
# Use Lunr instead of Algolia
|
||||
# Hide iframes, add warnings to external links
|
||||
ENV AIRGAP true
|
||||
|
||||
# Copy only what's needed to run the server
|
||||
|
|
|
|||
|
|
@ -15,5 +15,5 @@ Here, you'll find additional information that might be helpful as you work on a
|
|||
- [node versions](./node-versions.md) - our site runs on Node.js
|
||||
- [permalinks](./permalinks.md) - permalinks for article versioning
|
||||
- [redirects](./redirects.md) - configuring redirects in the site
|
||||
- [search](./search.md) - our search functionality is powered by [Algolia](https://www.algolia.com)
|
||||
- [search](./search.md) - our local site search functionality
|
||||
- [troubleshooting](./troubleshooting.md) - some help for troubleshooting failed and stalled status checks
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@
|
|||
|
||||
## Overview
|
||||
|
||||
This site's search functionality is powered by [Algolia](https://www.algolia.com), a third-party service.
|
||||
This site's search functionality.
|
||||
|
||||
To see all existing search-related issues and pull requests, visit [github.com/github/docs/labels/search](https://github.com/github/docs/labels/search).
|
||||
|
||||
|
|
@ -14,7 +14,7 @@ To see all existing search-related issues and pull requests, visit [github.com/g
|
|||
|
||||
## How to search
|
||||
|
||||
The site search is part of every version of docs.github.com. This endpoint responds in JSON format, and fronts Algolia and Lunr. We recommend using this endpoint over directly integrating with Algolia or Lunr, as the endpoint will be more stable. On any page, you can use the search box to search the documents we've indexed.
|
||||
The site search is part of every version of docs.github.com. This endpoint responds in JSON format, and fronts our search querying functionality. We recommend using this endpoint, as the endpoint will be more stable. On any page, you can use the search box to search the documents we've indexed.
|
||||
You can also query our search endpoint directly at:
|
||||
`https://docs.github.com/search?version=<VERSION>&language=<LANGUAGE CODE>&filters=topics:<TOPIC>&query=<QUERY>`
|
||||
|
||||
|
|
@ -33,25 +33,25 @@ https://docs.github.com/search?version=dotcom&language=en&filters=topics:'oauth
|
|||
|
||||
### Using the topics search filter
|
||||
|
||||
Using the attribute `topics` in your query will only return results that have the matching topic value. The `topics` attribute is configured as a [`filter only` facet in Algolia](https://www.algolia.com/doc/guides/managing-results/refine-results/filtering/). When the topic contains spaces, you must use quotes. For Algolia, [here](https://www.algolia.com/doc/api-reference/api-parameters/filters/#handle-attributes-with-spaces) is an example for filters containing spaces. Algolia also requires [boolean operators](https://www.algolia.com/doc/api-reference/api-parameters/filters) to combine more than one filter.
|
||||
Using the attribute `topics` in your query will only return results that have the matching topic value. When the topic contains spaces, you must use quotes. Filters with spaces or combining filters requires special syntax.
|
||||
|
||||
## Production deploys
|
||||
|
||||
A [GitHub Actions workflow](.github/workflows/sync-algolia-search-indices.yml) triggered by pushes to the `main` branch syncs the search data to Algolia. This process generates structured data for all pages on the site, compares that data to what's currently on Algolia, then adds, updates, or removes indices based on the diff of the local and remote data, being careful not to create duplicate records and avoiding any unnecessary (and costly) indexing operations.
|
||||
A [GitHub Actions workflow](.github/workflows/sync-search-indices.yml) triggered by pushes to the `main` branch syncs the search data. This process generates structured data for all pages on the site, compares that data to what's currently on search, then adds, updates, or removes indices based on the diff of the local and remote data, being careful not to create duplicate records and avoiding any unnecessary (and costly) indexing operations.
|
||||
|
||||
The Actions workflow progress can be viewed (by GitHub employees) in the [Actions tab](https://github.com/github/docs/actions?query=workflow%3AAlgolia) of the repo.
|
||||
The Actions workflow progress can be viewed (by GitHub employees) in the [Actions tab](https://github.com/github/docs/actions?query=workflow%3Asearch) of the repo.
|
||||
|
||||
Because the workflow runs after a branch is merged to `main`, there is a slight delay for search data updates to appear on the site.
|
||||
|
||||
## Manual sync from a checkout
|
||||
|
||||
It is also possible to manually sync the indices to Algolia from your local checkout of the repo, before your branch is merged to `main`.
|
||||
It is also possible to manually sync the indices from your local checkout of the repo, before your branch is merged to `main`.
|
||||
|
||||
**Prerequisite:** Make sure the environment variables `ALGOLIA_APPLICATION_ID` and `ALGOLIA_API_KEY` are set in your `.env` file. You can find these values on [Algolia](https://www.algolia.com/apps/ZI5KPY1HBE/api-keys/all).
|
||||
**Prerequisite:** Make sure the environment variables `ALGOLIA_APPLICATION_ID` and `ALGOLIA_API_KEY` are set in your `.env` file. You can find these values on [Algolia](https://www.algolia.com/apps/ZI5KPY1HBE/api-keys/all). _Remove this paragraph when we switch to Lunr._
|
||||
|
||||
### Build without sync (dry run)
|
||||
|
||||
To build all the indices without uploading them to Algolia's servers (this takes about an hour):
|
||||
To build all the indices without uploading them (this takes about an hour):
|
||||
```
|
||||
npm run sync-search-dry-run
|
||||
```
|
||||
|
|
@ -65,11 +65,11 @@ Substitute a currently supported version for `<PLAN@RELEASE>` and a currently su
|
|||
|
||||
### Build and sync
|
||||
|
||||
To build all the indices and sync them to Algolia (this also takes about an hour):
|
||||
To build all the indices and sync them (this also takes about an hour):
|
||||
```
|
||||
npm run sync-search
|
||||
```
|
||||
To build indices for a specific language and/or version and sync them to Algolia:
|
||||
To build indices for a specific language and/or version and sync them:
|
||||
```
|
||||
VERSION=<PLAN@RELEASE LANGUAGE=<TWO-LETTER CODE> npm run sync-search
|
||||
```
|
||||
|
|
@ -95,17 +95,17 @@ Why do we need this? For our daily shipping needs, it's tolerable that search up
|
|||
|
||||
### Actions workflow files
|
||||
|
||||
- [`.github/workflows/sync-algolia-search-indices.yml`](.github/workflows/sync-algolia-search-indices.yml) - Builds and syncs search indices whenever the `main` branch is pushed to (that is, on production deploys).
|
||||
- [`.github/workflows/dry-run-sync-algolia-search-indices.yml`](.github/workflows/dry-run-sync-algolia-search-indices.yml) - This workflow can be run manually (via `workflow_dispatch`) to do a dry run build of all the indices. Useful for confirming that the indices can build without erroring out.
|
||||
- [`.github/workflows/sync-single-english-algolia-index.yml`](.github/workflows/sync-single-english-algolia-index.yml) - This workflow is run when a label in the right format is applied to a PR. See "[Label-triggered Actions workflow](#label-triggered-actions-workflow)" for details.
|
||||
- [`.github/workflows/sync-search-indices.yml`](.github/workflows/sync-search-indices.yml) - Builds and syncs search indices whenever the `main` branch is pushed to (that is, on production deploys).
|
||||
- [`.github/workflows/dry-run-sync-search-indices.yml`](.github/workflows/dry-run-sync-search-indices.yml) - This workflow can be run manually (via `workflow_dispatch`) to do a dry run build of all the indices. Useful for confirming that the indices can build without erroring out.
|
||||
- [`.github/workflows/sync-single-english-index.yml`](.github/workflows/sync-single-english-index.yml) - This workflow is run when a label in the right format is applied to a PR. See "[Label-triggered Actions workflow](#label-triggered-actions-workflow)" for details.
|
||||
|
||||
### Code files
|
||||
|
||||
- [components/lib/search.ts](components/lib/search.ts) - The browser-side code that enables search.
|
||||
- [lib/search/algolia-client.js](lib/search/algolia-client.js) - A thin wrapper around the [algoliasearch](https://ghub.io/algoliasearch) Node.js module for interacting with the Algolia API.
|
||||
- [lib/search/algolia-search-index.js](lib/search/algolia-search-index.js) - A class for generating structured search data from repository content and syncing it with the remote Algolia service. This class has built-in validation to ensure that all records are valid before they're uploaded. This class also takes care of removing deprecated records, and compares existing remote records with the latest local records to avoid uploading records that haven't changed.
|
||||
- [script/sync-search-indices.js](script/sync-search-indices.js) - The script used by the Actions workflow to update search indices on our Algolia account. This can also be [run in the development environment](#development).
|
||||
- [tests/algolia-search.js](tests/algolia-search.js) - Tests!
|
||||
- [lib/search/client.js](lib/search/client.js) - A thin wrapper around the Node.js module for interacting with the search API.
|
||||
- [lib/search/search-index.js](lib/search/search-index.js) - A class for generating structured search data from repository content and syncing it. This class has built-in validation to ensure that all records are valid before they're uploaded. This class also takes care of removing deprecated records, and compares existing remote records with the latest local records to avoid uploading records that haven't changed.
|
||||
- [script/sync-search-indices.js](script/sync-search-indices.js) - The script used by the Actions workflow to update search indices. This can also be [run in the development environment](#development).
|
||||
- [tests/content/search.js](tests/content/search.js) - Tests!
|
||||
|
||||
## Indices
|
||||
|
||||
|
|
@ -144,7 +144,7 @@ Each record represents a section of a page. Sections are derived by splitting up
|
|||
|
||||
## Notes
|
||||
|
||||
- It's not strictly necessary to set an `objectID` as Algolia will create one automatically, but by creating our own we have a guarantee that subsequent invocations of this upload script will overwrite existing records instead of creating numerous duplicate records with differing IDs.
|
||||
- Algolia has typo tolerance. Try spelling something wrong and see what you get!
|
||||
- Algolia has lots of controls for customizing each index, so we can add weights to certain attributes and create rules like "title is more important than body", etc. But it works pretty well as-is without any configuration.
|
||||
- Algolia has support for "advanced query syntax" for exact matching of quoted expressions and exclusion of words preceded by a `-` sign. This is off by default but we have it enabled in our browser client. This and many other settings can be configured in Algolia.com web interface. The settings in the web interface can be overridden by the search endpoint. See [middleware/search.js]([middleware/search.js).
|
||||
- It's not strictly necessary to set an `objectID` as the search index will create one automatically, but by creating our own we have a guarantee that subsequent invocations of this upload script will overwrite existing records instead of creating numerous duplicate records with differing IDs.
|
||||
- Our search querying has typo tolerance. Try spelling something wrong and see what you get!
|
||||
- Our search querying has lots of controls for customizing each index, so we can add weights to certain attributes and create rules like "title is more important than body", etc. But it works pretty well as-is without any configuration.
|
||||
- Our search querying has support for "advanced query syntax" for exact matching of quoted expressions and exclusion of words preceded by a `-` sign. This is off by default but we have it enabled in our browser client. The settings in the web interface can be overridden by the search endpoint. See [middleware/search.js]([middleware/search.js).
|
||||
|
|
|
|||
|
|
@ -3,7 +3,7 @@ export const maxContentLength = 5000
|
|||
export const namePrefix = 'github-docs'
|
||||
|
||||
export default {
|
||||
// records must be truncated to avoid going over Algolia's 10K limit
|
||||
// records must be truncated to avoid going over 10K limit
|
||||
maxRecordLength,
|
||||
maxContentLength,
|
||||
namePrefix,
|
||||
|
|
|
|||
|
|
@ -47,7 +47,7 @@ export default function csp(req, res, next) {
|
|||
|
||||
const { requestedVersion } = isArchivedVersion(req)
|
||||
|
||||
// Exception for Algolia instantsearch in deprecated Enterprise docs (Node.js era)
|
||||
// Exception for deprecated Enterprise docs (Node.js era)
|
||||
if (
|
||||
versionSatisfiesRange(requestedVersion, '<=2.19') &&
|
||||
versionSatisfiesRange(requestedVersion, '>2.12')
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
#!/usr/bin/env node
|
||||
// This module accepts an Algolia search record object as input and
|
||||
// This module accepts a search record object as input and
|
||||
// returns a ranking score which influences how results are sorted.
|
||||
|
||||
// higher in this list == higher search ranking
|
||||
|
|
|
|||
|
|
@ -4,7 +4,7 @@ import { namePrefix } from '../../lib/search/config.js'
|
|||
import remoteIndexNames from '../../lib/search/cached-index-names.json'
|
||||
const languageCodes = Object.keys(xLanguages)
|
||||
|
||||
describe('algolia', () => {
|
||||
describe('search', () => {
|
||||
test('has remote indexNames in every language for every supported GHE version', () => {
|
||||
expect(supported.length).toBeGreaterThan(1)
|
||||
supported.forEach((version) => {
|
||||
Загрузка…
Ссылка в новой задаче