зеркало из https://github.com/github/docs.git
Update REST pagination guide (#33197)
Co-authored-by: Jess Hosman <1183847+jhosman@users.noreply.github.com>
This commit is contained in:
Sarah Edwards
GitHub
Родитель
d630294778
Коммит
f97c163101
|
|
@ -117,11 +117,11 @@ For more information about the audit log REST API, see "[Enterprise administrati
|
|||
|
||||
### Example 1: All events in an enterprise, for a specific date, with pagination
|
||||
|
||||
You can use page-based pagination or cursor based pagination. For more information, see "[Traversing with Pagination](/rest/guides/traversing-with-pagination)."
|
||||
You can use page-based pagination or cursor based pagination. For more information about pagination, see "[Using pagination in the REST API](/rest/guides/using-pagination-in-the-rest-api)."
|
||||
|
||||
#### Example with page-based pagination
|
||||
|
||||
The query below searches for audit log events created on Jan 1st, 2022 in the `avocado-corp` enterprise, and return the first page with a maximum of 100 items per page using [REST API pagination](/rest/overview/resources-in-the-rest-api#pagination):
|
||||
The query below searches for audit log events created on Jan 1st, 2022 in the `avocado-corp` enterprise, and return the first page with a maximum of 100 items per page using pagination. For more information about pagination, see "[Using pagination in the REST API](/rest/guides/using-pagination-in-the-rest-api)."
|
||||
|
||||
```shell
|
||||
curl -H "Authorization: Bearer TOKEN" \
|
||||
|
|
@ -131,7 +131,7 @@ curl -H "Authorization: Bearer TOKEN" \
|
|||
|
||||
#### Example with cursor-based pagination
|
||||
|
||||
The query below searches for audit log events created on Jan 1st, 2022 in the `avocado-corp` enterprise, and returns the first page with a maximum of 100 items per page using [REST API pagination](/rest/overview/resources-in-the-rest-api#pagination). The `--include` flag causes the headers to be returned along with the response.
|
||||
The query below searches for audit log events created on Jan 1st, 2022 in the `avocado-corp` enterprise, and returns the first page with a maximum of 100 items per page using pagination. For more information about pagination, see "[Using pagination in the REST API](/rest/guides/using-pagination-in-the-rest-api)." The `--include` flag causes the headers to be returned along with the response.
|
||||
|
||||
```
|
||||
curl --include -H "Authorization: Bearer TOKEN" \
|
||||
|
|
|
|||
|
|
@ -61,7 +61,7 @@ Often, API responses contain data in the form of URLs. For example, when request
|
|||
|
||||
For the stability of your app, you shouldn't try to parse this data or try to guess and construct the format of future URLs. Your app is liable to break if we decide to change the URL.
|
||||
|
||||
For example, when working with paginated results, it's often tempting to construct URLs that append `?page=<number>` to the end. Avoid that temptation. [Our guide on pagination](/guides/traversing-with-pagination) offers some safe tips on dependably following paginated results.
|
||||
For example, when working with paginated results, it's often tempting to construct URLs that append `?page=<number>` to the end. Avoid that temptation. For more information about dependably following paginated results, see "[Using pagination in the REST API](/rest/guides/using-pagination-in-the-rest-api)."
|
||||
|
||||
## Check the event type and action before processing the event
|
||||
|
||||
|
|
|
|||
|
|
@ -30,7 +30,7 @@ In addition to having their own personal repositories, a user may be a collabora
|
|||
|
||||
[OAuth scopes][scopes] and [organization application policies][oap] determine which of those repositories your app can access for a user. Use the workflow below to discover those repositories.
|
||||
|
||||
As always, first we'll require [GitHub's Octokit.rb][octokit.rb] Ruby library. Then we'll configure Octokit.rb to automatically handle [pagination][pagination] for us.
|
||||
As always, first we'll require [GitHub's Octokit.rb][octokit.rb] Ruby library. Then we'll configure Octokit.rb to automatically handle pagination for us. For more information about pagination, see "[Using pagination in the REST API](/rest/guides/using-pagination-in-the-rest-api)."
|
||||
|
||||
``` ruby
|
||||
require 'octokit'
|
||||
|
|
@ -67,7 +67,7 @@ end
|
|||
|
||||
Applications can perform all sorts of organization-related tasks for a user. To perform these tasks, the app needs an [OAuth authorization][scopes] with sufficient permission. For example, the `read:org` scope allows you to [list teams][list-teams], and the `user` scope lets you [publicize the user’s organization membership][publicize-membership]. Once a user has granted one or more of these scopes to your app, you're ready to fetch the user’s organizations.
|
||||
|
||||
Just as we did when discovering repositories above, we'll start by requiring [GitHub's Octokit.rb][octokit.rb] Ruby library and configuring it to take care of [pagination][pagination] for us:
|
||||
Just as we did when discovering repositories above, we'll start by requiring [GitHub's Octokit.rb][octokit.rb] Ruby library and configuring it to take care of pagination for us. For more information about pagination, see "[Using pagination in the REST API](/rest/guides/using-pagination-in-the-rest-api)."
|
||||
|
||||
``` ruby
|
||||
require 'octokit'
|
||||
|
|
@ -105,7 +105,6 @@ As an application, you typically want all of the user's organizations that your
|
|||
[make-authenticated-request-for-user]: /rest/guides/basics-of-authentication#making-authenticated-requests
|
||||
[oap]: https://developer.github.com/changes/2015-01-19-an-integrators-guide-to-organization-application-policies/
|
||||
[octokit.rb]: https://github.com/octokit/octokit.rb
|
||||
[pagination]: /rest#pagination
|
||||
[platform samples]: https://github.com/github/platform-samples/tree/master/api/ruby/discovering-resources-for-a-user
|
||||
[publicize-membership]: /rest/reference/orgs#set-public-organization-membership-for-the-authenticated-user
|
||||
[register-oauth-app]: /rest/guides/basics-of-authentication#registering-your-app
|
||||
|
|
|
|||
|
|
@ -18,7 +18,7 @@ children:
|
|||
- /delivering-deployments
|
||||
- /rendering-data-as-graphs
|
||||
- /working-with-comments
|
||||
- /traversing-with-pagination
|
||||
- /using-pagination-in-the-rest-api
|
||||
- /building-a-ci-server
|
||||
- /best-practices-for-integrators
|
||||
- /getting-started-with-the-git-database-api
|
||||
|
|
|
|||
|
|
@ -1,350 +0,0 @@
|
|||
---
|
||||
title: Traversing with pagination
|
||||
intro: Explore how to use pagination to manage your responses with some examples using the Search API.
|
||||
redirect_from:
|
||||
- /guides/traversing-with-pagination
|
||||
- /v3/guides/traversing-with-pagination
|
||||
versions:
|
||||
fpt: '*'
|
||||
ghes: '*'
|
||||
ghae: '*'
|
||||
ghec: '*'
|
||||
topics:
|
||||
- API
|
||||
shortTitle: Traverse with pagination
|
||||
miniTocMaxHeadingLevel: 3
|
||||
---
|
||||
|
||||
The {% ifversion fpt or ghec %}{% data variables.product.prodname_dotcom %}{% else %}{% data variables.product.product_name %}{% endif %} API provides a vast wealth of information for developers to consume.
|
||||
Most of the time, you might even find that you're asking for _too much_ information,
|
||||
and in order to keep our servers happy, the API will automatically [paginate the requested items](/rest/overview/resources-in-the-rest-api#pagination).
|
||||
|
||||
In this guide, we'll make some calls to the Search API, and iterate over
|
||||
the results using pagination. You can find the complete source code for this project
|
||||
in the [platform-samples][platform samples] repository.
|
||||
|
||||
{% data reusables.rest-api.dotcom-only-guide-note %}
|
||||
|
||||
|
||||
|
||||
## Basics of Pagination
|
||||
|
||||
To start with, it's important to know a few facts about receiving paginated items:
|
||||
|
||||
|
||||
1. Different API calls respond with different defaults. For example, a call to
|
||||
[List public repositories](/rest/reference/repos#list-public-repositories)
|
||||
provides paginated items in sets of 30, whereas a call to the GitHub Search API
|
||||
provides items in sets of 100
|
||||
2. You can specify how many items to receive (up to a maximum of 100); but,
|
||||
3. For technical reasons, not every endpoint behaves the same. For example,
|
||||
[events](/rest/reference/activity#events) won't let you set a maximum for items to receive.
|
||||
Be sure to read the documentation on how to handle paginated results for specific endpoints.
|
||||
|
||||
{% note %}
|
||||
|
||||
**Note**: You should always rely on URLs included in the link header. Don't try to guess or construct your own URLs.
|
||||
|
||||
{% endnote %}
|
||||
|
||||
|
||||
### Link header
|
||||
|
||||
The response header includes information about pagination. For more information about headers, see "[Getting started with the REST API](/rest/guides/getting-started-with-the-rest-api#about-the-response-code-and-headers)." To get the response header, include the `-I` flag in your request. For example:
|
||||
|
||||
```shell
|
||||
$ curl -I -H "Accept: application/vnd.github+json" -H "Authorization: Bearer YOUR_TOKEN" https://api.github.com/enterprises/advacado-corp/audit-log
|
||||
|
||||
```
|
||||
|
||||
The `-I` flag returns only the response header. If the response is paginated, the response header will include a `link` header. The header will look something like this:
|
||||
|
||||
```
|
||||
link: <https://api.github.com/enterprises/13827/audit-log?after=MS42NjQzODM5MTkzNDdlKzEyfDM0MkI6NDdBNDo4RTFGMEM6NUIyQkZCMzo2MzM0N0JBRg%3D%3D&before=>; rel="next"
|
||||
```
|
||||
|
||||
or
|
||||
|
||||
```
|
||||
link: <https://api.github.com/repositories/1300192/issues?page=2>; rel="next", <https://api.github.com/repositories/1300192/issues?page=511>; rel="last"
|
||||
```
|
||||
### Types of pagination
|
||||
|
||||
{% data variables.product.company_short %}'s API uses two pagination methods: page-based pagination and cursor-based pagination. If the `link` header includes `page`, then the operation uses page-based pagination. If the `link` header includes `before` and `after`, then the operation uses cursor-based pagination.
|
||||
|
||||
|
||||
#### Page based pagination
|
||||
|
||||
The link header for page-based pagination will tell you information about the previous, next, first, and last pages. If you did not request a specific page, then the response will default to the first page and information about the first and previous pages will be omitted.
|
||||
|
||||
For example, for a request that did not specify a page, this header states that the next page is `2` and the last page is `511`.
|
||||
|
||||
```
|
||||
link: <https://api.github.com/repositories/1300192/issues?page=2>; rel="next", <https://api.github.com/repositories/1300192/issues?page=511>; rel="last"
|
||||
```
|
||||
|
||||
For example, for a request that specified page 5, this header states that the previous page is `4`, the next page is `6`, the last page is `511`, and the first page is `1`.
|
||||
|
||||
```
|
||||
link: <https://api.github.com/repositories/1300192/issues?page=4>; rel="prev", <https://api.github.com/repositories/1300192/issues?page=6>; rel="next", <https://api.github.com/repositories/1300192/issues?page=511>; rel="last", <https://api.github.com/repositories/1300192/issues?page=1>; rel="first"
|
||||
```
|
||||
|
||||
#### Cursor based pagination
|
||||
|
||||
Cursor pagination uses terms `before` and `after` in order to navigate through pages. `rel="next"` and `rel="prev"` this mark the cursor point in the data set and provides a reference for traveling to the page `before` and `after` the current page.
|
||||
|
||||
```
|
||||
link: <https://api.github.com/enterprises/13827/audit-log?after=MS42NjQzODMzMzk2MzZlKzEyfFdxSzIxdGU0MlBWNUp5UzhBWDF6LWc%3D&before=>; rel="next",
|
||||
<https://api.github.com/enterprises/13827/audit-log?after=&before=>; rel="first",
|
||||
<https://api.github.com/enterprises/13827/audit-log?after=&before=MS42NjQzODM5MTcyMjllKzEyfDI4NDE6NEVFNDoxODBDRkM5OjY5REE0MzI6NjMzNDdCQUQ%3D>; rel="prev"
|
||||
```
|
||||
|
||||
In this example, `rel=next` says that the next page is located at:
|
||||
|
||||
```
|
||||
after=MS42NjQzODM5MTkzNDdlKzEyfDM0MkI6NDdBNDo4RTFGMEM6NUIyQkZCMzo2MzM0N0JBRg%3D%3D&before=
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
### Using pagination
|
||||
|
||||
#### Cursor based pagination
|
||||
|
||||
Using cursor based pagination requires you to use the terms `before` and `after`. To navigate using `before` and `after`, copy the link header generated above into your curl request:
|
||||
|
||||
```shell
|
||||
$ curl -I -H "Accept: application/vnd.github+json" -H "Authorization: Bearer YOUR_TOKEN" https://api.github.com/enterprises/13827/audit-log?after=MS42NjQzODM5MTkzNDdlKzEyfDM0MkI6NDdBNDo4RTFGMEM6NUIyQkZCMzo2MzM0N0JBRg%3D%3D&before=
|
||||
```
|
||||
|
||||
The above example will generate a page of results and new header information that you can use to make the next request. `rel="next"` provides the next page of results. `rel="prev"` provides the previous page of results. The important part of the output here is the link header needs to be generated rather than manually imputed. Copy the entire link from the following output.
|
||||
|
||||
```
|
||||
link: <https://api.github.com/enterprises/13827/audit-log?after=MS42NjQzODMzMzk2MzZlKzEyfFdxSzIxdGU0MlBWNUp5UzhBWDF6LWc%3D&before=>; rel="next",
|
||||
<https://api.github.com/enterprises/13827/audit-log?after=&before=>; rel="first",
|
||||
<https://api.github.com/enterprises/13827/audit-log?after=&before=MS42NjQzODM5MTcyMjllKzEyfDI4NDE6NEVFNDoxODBDRkM5OjY5REE0MzI6NjMzNDdCQUQ%3D>; rel="prev"
|
||||
```
|
||||
|
||||
Unlike page-based pagination, the results will not return the last page number in the response.
|
||||
|
||||
link: <https://api.github.com/enterprises/13827/audit-log?after=MS42NjQzODMzMzk2MzZlKzEyfFdxSzIxdGU0MlBWNUp5UzhBWDF6LWc%3D&before=>; rel="next",
|
||||
<https://api.github.com/enterprises/13827/audit-log?after=&before=>; rel="first",
|
||||
<https://api.github.com/enterprises/13827/audit-log?after=&before=MS42NjQzODM5MTcyMjllKzEyfDI4NDE6NEVFNDoxODBDRkM5OjY5REE0MzI6NjMzNDdCQUQ%3D>; rel="prev"
|
||||
|
||||
Because cursor based pagination creates a reference point in the data set, it cannot calculate the total number of results.
|
||||
|
||||
|
||||
#### Page based pagination
|
||||
|
||||
To navigate using page based pagination pass in a `page`
|
||||
parameter. By default, `page` always starts at `1`. In the following example, we have made a curl request to the search API Mozilla projects use the phrase `addClass`. Instead of starting at 1, lets jump to page 14.
|
||||
|
||||
```shell
|
||||
$ curl -I "https://api.github.com/search/code?q=addClass+user:mozilla&page=14"
|
||||
```
|
||||
|
||||
Here's an except of the link header in the HTTP request:
|
||||
|
||||
Link: <https://api.github.com/search/code?q=addClass+user%3Amozilla&page=15>; rel="next",
|
||||
<https://api.github.com/search/code?q=addClass+user%3Amozilla&page=34>; rel="last",
|
||||
<https://api.github.com/search/code?q=addClass+user%3Amozilla&page=1>; rel="first",
|
||||
<https://api.github.com/search/code?q=addClass+user%3Amozilla&page=13>; rel="prev"
|
||||
|
||||
In this example, `rel="next"` is at 15, and `rel="last"` is 34. But now we've
|
||||
got some more information: `rel="first"` indicates the URL for the _first_ page,
|
||||
and more importantly, `rel="prev"` lets you know the page number of the previous
|
||||
page. Using this information, you could construct some UI that lets users jump
|
||||
between the first, previous, next, or last list of results in an API call.
|
||||
|
||||
|
||||
### Changing the number of items received
|
||||
|
||||
#### Page based pagination
|
||||
|
||||
By passing the `per_page` parameter, you can specify how many items you want
|
||||
each page to return, up to 100 items. Let's try asking for 50 items about `addClass`:
|
||||
|
||||
```shell
|
||||
$ curl -I "https://api.github.com/search/code?q=addClass+user:mozilla&per_page=50"
|
||||
```
|
||||
|
||||
Notice what it does to the header response:
|
||||
|
||||
Link: <https://api.github.com/search/code?q=addClass+user%3Amozilla&per_page=50&page=2>; rel="next",
|
||||
<https://api.github.com/search/code?q=addClass+user%3Amozilla&per_page=50&page=20>; rel="last"
|
||||
|
||||
As you might have guessed, the `rel="last"` information says that the last page
|
||||
is now 20. This is because we are asking for more information per page about
|
||||
our results.
|
||||
|
||||
#### Cursor based pagination
|
||||
|
||||
You can also pass the `per_page` parameter for cursor-based pagination.
|
||||
|
||||
```shell
|
||||
$ curl -I -H "Accept: application/vnd.github+json" -H "Authorization: Bearer YOUR_TOKEN" https://api.github.com/enterprises/13827/audit-log?after=MS42NjQzODM5MTkzNDdlKzEyfDM0MkI6NDdBNDo4RTFGMEM6NUIyQkZCMzo2MzM0N0JBRg%3D%3D&before=&per_page=50
|
||||
```
|
||||
|
||||
## Consuming the information
|
||||
|
||||
You don't want to be making low-level curl calls just to be able to work with
|
||||
pagination, so let's write a little Ruby script that does everything we've
|
||||
just described above.
|
||||
|
||||
As always, first we'll require [GitHub's Octokit.rb][octokit.rb] Ruby library, and
|
||||
pass in our [{% data variables.product.pat_generic %}][personal token]:
|
||||
|
||||
``` ruby
|
||||
require 'octokit'
|
||||
|
||||
# !!! DO NOT EVER USE HARD-CODED VALUES IN A REAL APP !!!
|
||||
# Instead, set and test environment variables, like below
|
||||
client = Octokit::Client.new :access_token => ENV['MY_PERSONAL_TOKEN']
|
||||
```
|
||||
|
||||
Next, we'll execute the search, using Octokit's `search_code` method. Unlike
|
||||
using `curl`, we can also immediately retrieve the number of results, so let's
|
||||
do that:
|
||||
|
||||
``` ruby
|
||||
results = client.search_code('addClass user:mozilla')
|
||||
total_count = results.total_count
|
||||
```
|
||||
|
||||
Now, let's grab the number of the last page, similar to `page=34>; rel="last"`
|
||||
information in the link header. Octokit.rb support pagination information through
|
||||
an implementation called "[Hypermedia link relations][hypermedia-relations]."
|
||||
We won't go into detail about what that is, but, suffice to say, each element
|
||||
in the `results` variable has a hash called `rels`, which can contain information
|
||||
about `:next`, `:last`, `:first`, and `:prev`, depending on which result you're
|
||||
on. These relations also contain information about the resulting URL, by calling
|
||||
`rels[:last].href`.
|
||||
|
||||
Knowing this, let's grab the page number of the last result, and present all
|
||||
this information to the user:
|
||||
|
||||
``` ruby
|
||||
last_response = client.last_response
|
||||
number_of_pages = last_response.rels[:last].href.match(/page=(\d+).*$/)[1]
|
||||
|
||||
puts "There are #{total_count} results, on #{number_of_pages} pages!"
|
||||
```
|
||||
|
||||
Finally, let's iterate through the results. You could do this with a loop `for i in 1..number_of_pages.to_i`,
|
||||
but instead, let's follow the `rels[:next]` headers to retrieve information from
|
||||
each page. For the sake of simplicity, let's just grab the file path of the first
|
||||
result from each page. To do this, we'll need a loop; and at the end of every loop,
|
||||
we'll retrieve the data set for the next page by following the `rels[:next]` information.
|
||||
The loop will finish when there is no `rels[:next]` information to consume (in other
|
||||
words, we are at `rels[:last]`). It might look something like this:
|
||||
|
||||
``` ruby
|
||||
puts last_response.data.items.first.path
|
||||
until last_response.rels[:next].nil?
|
||||
last_response = last_response.rels[:next].get
|
||||
puts last_response.data.items.first.path
|
||||
end
|
||||
```
|
||||
|
||||
Changing the number of items per page is extremely simple with Octokit.rb. Simply
|
||||
pass a `per_page` options hash to the initial client construction. After that,
|
||||
your code should remain intact:
|
||||
|
||||
``` ruby
|
||||
require 'octokit'
|
||||
|
||||
# !!! DO NOT EVER USE HARD-CODED VALUES IN A REAL APP !!!
|
||||
# Instead, set and test environment variables, like below
|
||||
client = Octokit::Client.new :access_token => ENV['MY_PERSONAL_TOKEN']
|
||||
|
||||
results = client.search_code('addClass user:mozilla', :per_page => 100)
|
||||
total_count = results.total_count
|
||||
|
||||
last_response = client.last_response
|
||||
number_of_pages = last_response.rels[:last].href.match(/page=(\d+).*$/)[1]
|
||||
|
||||
puts last_response.rels[:last].href
|
||||
puts "There are #{total_count} results, on #{number_of_pages} pages!"
|
||||
|
||||
puts "And here's the first path for every set"
|
||||
|
||||
puts last_response.data.items.first.path
|
||||
until last_response.rels[:next].nil?
|
||||
last_response = last_response.rels[:next].get
|
||||
puts last_response.data.items.first.path
|
||||
end
|
||||
```
|
||||
|
||||
## Constructing Pagination Links
|
||||
|
||||
Normally, with pagination, your goal isn't to concatenate all of the possible
|
||||
results, but rather, to produce a set of navigation, like this:
|
||||
|
||||
|
||||
|
||||
Let's sketch out a micro-version of what that might entail.
|
||||
|
||||
From the code above, we already know we can get the `number_of_pages` in the
|
||||
paginated results from the first call:
|
||||
|
||||
``` ruby
|
||||
require 'octokit'
|
||||
|
||||
# !!! DO NOT EVER USE HARD-CODED VALUES IN A REAL APP !!!
|
||||
# Instead, set and test environment variables, like below
|
||||
client = Octokit::Client.new :access_token => ENV['MY_PERSONAL_TOKEN']
|
||||
|
||||
results = client.search_code('addClass user:mozilla')
|
||||
total_count = results.total_count
|
||||
|
||||
last_response = client.last_response
|
||||
number_of_pages = last_response.rels[:last].href.match(/page=(\d+).*$/)[1]
|
||||
|
||||
puts last_response.rels[:last].href
|
||||
puts "There are #{total_count} results, on #{number_of_pages} pages!"
|
||||
```
|
||||
|
||||
From there, we can construct a beautiful ASCII representation of the number boxes:
|
||||
``` ruby
|
||||
numbers = ""
|
||||
for i in 1..number_of_pages.to_i
|
||||
numbers << "[#{i}] "
|
||||
end
|
||||
puts numbers
|
||||
```
|
||||
|
||||
Let's simulate a user clicking on one of these boxes, by constructing a random
|
||||
number:
|
||||
|
||||
``` ruby
|
||||
random_page = Random.new
|
||||
random_page = random_page.rand(1..number_of_pages.to_i)
|
||||
|
||||
puts "A User appeared, and clicked number #{random_page}!"
|
||||
```
|
||||
|
||||
Now that we have a page number, we can use Octokit to explicitly retrieve that
|
||||
individual page, by passing the `:page` option:
|
||||
|
||||
``` ruby
|
||||
clicked_results = client.search_code('addClass user:mozilla', :page => random_page)
|
||||
```
|
||||
|
||||
If we wanted to get fancy, we could also grab the previous and next pages, in
|
||||
order to generate links for back (`<<`) and forward (`>>`) elements:
|
||||
|
||||
``` ruby
|
||||
prev_page_href = client.last_response.rels[:prev] ? client.last_response.rels[:prev].href : "(none)"
|
||||
next_page_href = client.last_response.rels[:next] ? client.last_response.rels[:next].href : "(none)"
|
||||
|
||||
puts "The prev page link is #{prev_page_href}"
|
||||
puts "The next page link is #{next_page_href}"
|
||||
```
|
||||
|
||||
[pagination]: /rest#pagination
|
||||
[platform samples]: https://github.com/github/platform-samples/tree/master/api/ruby/traversing-with-pagination
|
||||
[octokit.rb]: https://github.com/octokit/octokit.rb
|
||||
[personal token]: /articles/creating-an-access-token-for-command-line-use
|
||||
[hypermedia-relations]: https://github.com/octokit/octokit.rb#pagination
|
||||
[listing commits]: /rest/reference/commits#list-commits
|
||||
|
|
@ -0,0 +1,180 @@
|
|||
---
|
||||
title: Using pagination in the REST API
|
||||
intro: Learn how to navigate through paginated responses from the REST API.
|
||||
redirect_from:
|
||||
- /guides/traversing-with-pagination
|
||||
- /v3/guides/traversing-with-pagination
|
||||
- /rest/guides/traversing-with-pagination
|
||||
versions:
|
||||
fpt: '*'
|
||||
ghes: '*'
|
||||
ghae: '*'
|
||||
ghec: '*'
|
||||
topics:
|
||||
- API
|
||||
shortTitle: Pagination
|
||||
miniTocMaxHeadingLevel: 3
|
||||
---
|
||||
|
||||
## About pagination
|
||||
|
||||
When a response from the REST API would include many results, {% data variables.product.company_short %} will paginate the results and return a subset of the results. For example, `GET /repos/octocat/Spoon-Knife/issues` will only return 30 issues from the `octocat/Spoon-Knife` repository even though the repository includes over 1600 open issues. This makes the response easier to handle for servers and for people.
|
||||
|
||||
This guide demonstrates how to request additional pages of results for paginated responses, how to change the number of results returned on each page, and how to write a script to fetch multiple pages of results.
|
||||
|
||||
## Using link headers
|
||||
|
||||
When a response is paginated, the response headers will include a link header. The link header will be omitted if the endpoint does not support pagination or if all results fit on a single page. The link header contains URLs that you can used to fetch additional pages of results. To see the response headers if you are using curl or {% data variables.product.prodname_cli %}, pass the `--include` flag with your request. To see the response headers if you are using a library to make requests, follow the documentation for that library. For example:
|
||||
|
||||
```shell
|
||||
curl --include --request GET \
|
||||
--url "https://api.github.com/repos/octocat/Spoon-Knife/issues" \
|
||||
--header "Accept: application/vnd.github+json"
|
||||
```
|
||||
|
||||
If the response is paginated, the link header will look something like this:
|
||||
|
||||
```
|
||||
link: <https://api.github.com/repositories/1300192/issues?page=2>; rel="prev", <https://api.github.com/repositories/1300192/issues?page=4>; rel="next", <https://api.github.com/repositories/1300192/issues?page=515>; rel="last", <https://api.github.com/repositories/1300192/issues?page=1>; rel="first"
|
||||
```
|
||||
|
||||
The link header provides the URL for the previous, next, first, and last page of results:
|
||||
|
||||
- The URL for the previous page is followed by `rel="prev"`.
|
||||
- The URL for the next page is followed by `rel="next"`.
|
||||
- The URL for the last page is followed by `rel="last"`.
|
||||
- The URL for the first page is followed by `rel="first"`.
|
||||
|
||||
In some cases, only a subset of these links are available. For example, the link to the previous page won't be included if you are on the first page of results, and the link to the last page won't be included if it can't be calculated.
|
||||
|
||||
You can use the URLs from the link header to request another page of results. For example, to request the last page of results based on the previous example:
|
||||
|
||||
```shell
|
||||
curl --include --request GET \
|
||||
--url "https://api.github.com/repositories/1300192/issues?page=515" \
|
||||
--header "Accept: application/vnd.github+json"
|
||||
```
|
||||
|
||||
The URLs in the link header use query parameters to indicate what page of results to return. The query parameters in the link URLs may differ between endpoints: each paginated endpoint will use the `page`, `before`/`after`, or `since` query parameters. (Some endpoints use the `since` parameter for something other than pagination.) In all cases, you can use the URLs in the link header to fetch additional pages of results. For more information about query parameters see "[Getting started with the REST API](/rest/guides/getting-started-with-the-rest-api#using-query-parameters)."
|
||||
|
||||
## Changing the number of items per page
|
||||
|
||||
If an endpoint supports the `per_page` query parameter, then you can control how many results are returned on a page. For more information about query parameters see "[Getting started with the REST API](/rest/guides/getting-started-with-the-rest-api#using-query-parameters)."
|
||||
|
||||
For example, this request uses the `per_page` query parameter to return two items per page:
|
||||
|
||||
```shell
|
||||
curl --include --request GET \
|
||||
--url "https://api.github.com/repos/octocat/Spoon-Knife/issues?per_page=2" \
|
||||
--header "Accept: application/vnd.github+json"
|
||||
```
|
||||
|
||||
The `per_page` parameter will automatically be included in the link header. For example:
|
||||
|
||||
```
|
||||
link: <https://api.github.com/repositories/1300192/issues?per_page=2&page=2>; rel="next", <https://api.github.com/repositories/1300192/issues?per_page=2&page=7715>; rel="last"
|
||||
```
|
||||
|
||||
## Scripting with pagination
|
||||
|
||||
Instead of manually copying URLs from the link header, you can write a script to fetch multiple pages of results.
|
||||
|
||||
The following examples use JavaScript and {% data variables.product.company_short %}'s Octokit.js library. For more information about Octokit.js, see "[Getting started with the REST API](/rest/guides/getting-started-with-the-rest-api?tool=javascript)" and [the Octokit.js README](https://github.com/octokit/octokit.js/#readme).
|
||||
|
||||
### Example using the Octokit.js pagination method
|
||||
|
||||
To fetch paginated results with Octokit.js, you can use `octokit.paginate()`. `octokit.paginate()` will fetch the next page of results until it reaches the last page and then return all of the results as a single array. A few endpoints return paginated results as array in an object, as opposed to returning the paginated results as an array. `octokit.paginate()` always returns an array of items even if the raw result was an object. If the results are not paginated, `octokit.paginate()` will behave like `octokit.request()`.
|
||||
|
||||
For example, this script gets all of the issues from the `octocat/Spoon-Knife` repository. Although it requests 100 issues at a time, the function won't return until the last page of data is reached.
|
||||
|
||||
```javascript{:copy}
|
||||
import { Octokit } from "octokit";
|
||||
|
||||
const octokit = new Octokit({ {% ifversion ghes or ghae %}
|
||||
baseUrl: "{% data variables.product.api_url_code %}",
|
||||
{% endif %}});
|
||||
|
||||
const data = await octokit.paginate("GET /repos/{owner}/{repo}/issues", {
|
||||
owner: "octocat",
|
||||
repo: "Spoon-Knife",
|
||||
per_page: 100,{% ifversion api-date-versioning %}
|
||||
headers: {
|
||||
"X-GitHub-Api-Version": "{{ allVersions[currentVersion].latestApiVersion }}",
|
||||
},{% endif %}
|
||||
});
|
||||
|
||||
console.log(data)
|
||||
```
|
||||
|
||||
You can pass an optional map function to `octokit.paginate()` to end pagination before the last page is reached or to reduce memory usage by keeping only a subset of the response. You can also use `octokit.paginate.iterator()` to iterate through a single page at a time instead of requesting every page. For more information, see [the Octokit.js documentation](https://github.com/octokit/octokit.js#pagination).
|
||||
|
||||
### Example creating a pagination method
|
||||
|
||||
If you are using another language or library that doesn't have a pagination method, you can build your own pagination method. This example still uses the Octokit.js library to make requests, but does not rely on `octokit.paginate()`.
|
||||
|
||||
The `getPaginatedData` function makes a request to an endpoint with `octokit.request()`. The data from the response is processed by `parseData`, which handles cases where no data is returned or cases where the data that is returned is an object instead of an array. The processed data is then appended to a list that contains all of the paginated data collected so far. If the response includes a link header and if the link header includes a link for the next page, then the function uses a RegEx pattern (`nextPattern`) to get the URL for the next page. The function then repeats the previous steps, now using this new URL. Once the link header no longer includes a link to the next page, all of the results are returned.
|
||||
|
||||
```javascript{:copy}
|
||||
import { Octokit } from "octokit";
|
||||
|
||||
const octokit = new Octokit({ {% ifversion ghes or ghae %}
|
||||
baseUrl: "{% data variables.product.api_url_code %}",
|
||||
{% endif %}});
|
||||
|
||||
async function getPaginatedData(url) {
|
||||
const nextPattern = /(?<=<)([\S]*)(?=>; rel="Next")/i;
|
||||
let pagesRemaining = true;
|
||||
let data = [];
|
||||
|
||||
while (pagesRemaining) {
|
||||
const response = await octokit.request(`GET ${url}`, {
|
||||
per_page: 100,{% ifversion api-date-versioning %}
|
||||
headers: {
|
||||
"X-GitHub-Api-Version":
|
||||
"{{ allVersions[currentVersion].latestApiVersion }}",
|
||||
},{% endif %}
|
||||
});
|
||||
|
||||
const parsedData = parseData(response.data)
|
||||
data = [...data, ...parsedData];
|
||||
|
||||
const linkHeader = response.headers.link;
|
||||
|
||||
pagesRemaining = linkHeader && linkHeader.includes(`rel=\"next\"`);
|
||||
|
||||
if (pagesRemaining) {
|
||||
url = linkHeader.match(nextPattern)[0];
|
||||
}
|
||||
}
|
||||
|
||||
return data;
|
||||
}
|
||||
|
||||
function parseData(data) {
|
||||
// If the data is an array, return that
|
||||
if (Array.isArray(data)) {
|
||||
return data
|
||||
}
|
||||
|
||||
// Some endpoints respond with 204 No Content instead of empty array
|
||||
// when there is no data. In that case, return an empty array.
|
||||
if (!data) {
|
||||
return []
|
||||
}
|
||||
|
||||
// Otherwise, the array of items that we want is in an object
|
||||
// Delete keys that don't include the array of items
|
||||
delete data.incomplete_results;
|
||||
delete data.repository_selection;
|
||||
delete data.total_count;
|
||||
// Pull out the array of items
|
||||
const namespaceKey = Object.keys(data)[0];
|
||||
data = data[namespaceKey];
|
||||
|
||||
return data;
|
||||
}
|
||||
|
||||
const data = await getPaginatedData("/repos/octocat/Spoon-Knife/issues");
|
||||
|
||||
console.log(data);
|
||||
```
|
||||
|
|
@ -10,6 +10,7 @@ featuredLinks:
|
|||
- /rest/guides/getting-started-with-the-rest-api
|
||||
- /rest/guides/basics-of-authentication
|
||||
- /rest/guides/best-practices-for-integrators
|
||||
- /rest/guides/using-pagination-in-the-rest-api
|
||||
popular:
|
||||
- /rest/overview/resources-in-the-rest-api
|
||||
- /rest/overview/api-versions
|
||||
|
|
|
|||
|
|
@ -303,54 +303,7 @@ gem:
|
|||
|
||||
## Pagination
|
||||
|
||||
Requests that return multiple items will be paginated to 30 items by
|
||||
default. You can specify further pages with the `page` parameter. For some
|
||||
resources, you can also set a custom page size up to 100 with the `per_page` parameter.
|
||||
Note that for technical reasons not all endpoints respect the `per_page` parameter,
|
||||
see [events](/rest/reference/activity#events) for example.
|
||||
|
||||
```shell
|
||||
$ curl '{% data variables.product.api_url_pre %}/user/repos?page=2&per_page=100'
|
||||
```
|
||||
|
||||
Note that page numbering is 1-based and that omitting the `page`
|
||||
parameter will return the first page.
|
||||
|
||||
Some endpoints use cursor-based pagination. A cursor is a string that points to a location in the result set.
|
||||
With cursor-based pagination, there is no fixed concept of "pages" in the result set, so you can't navigate to a specific page.
|
||||
Instead, you can traverse the results by using the `before` or `after` parameters.
|
||||
|
||||
For more information on pagination, check out our guide on [Traversing with Pagination][pagination-guide].
|
||||
|
||||
### Link header
|
||||
|
||||
{% note %}
|
||||
|
||||
**Note:** It's important to form calls with Link header values instead of constructing your own URLs.
|
||||
|
||||
{% endnote %}
|
||||
|
||||
The [Link header](https://datatracker.ietf.org/doc/html/rfc5988) includes pagination information. For example:
|
||||
|
||||
Link: <{% data variables.product.api_url_code %}/user/repos?page=3&per_page=100>; rel="next",
|
||||
<{% data variables.product.api_url_code %}/user/repos?page=50&per_page=100>; rel="last"
|
||||
|
||||
_The example includes a line break for readability._
|
||||
|
||||
Or, if the endpoint uses cursor-based pagination:
|
||||
|
||||
Link: <{% data variables.product.api_url_code %}/orgs/ORG/audit-log?after=MTYwMTkxOTU5NjQxM3xZbGI4VE5EZ1dvZTlla09uWjhoZFpR&before=>; rel="next",
|
||||
|
||||
This `Link` response header contains one or more [Hypermedia](/rest#hypermedia) link relations, some of which may require expansion as [URI templates](https://datatracker.ietf.org/doc/html/rfc6570).
|
||||
|
||||
The possible `rel` values are:
|
||||
|
||||
Name | Description
|
||||
-----------|-----------|
|
||||
`next` |The link relation for the immediate next page of results.
|
||||
`last` |The link relation for the last page of results.
|
||||
`first` |The link relation for the first page of results.
|
||||
`prev` |The link relation for the immediate previous page of results.
|
||||
When a response from the REST API would include many results, {% data variables.product.company_short %} will paginate the results and return a subset of the results. You can use the link header from the response to request additional pages of data. If an endpoint supports the `per_page` query parameter, then you can control how many results are returned on a page. For more information about pagination, see "[Using pagination in the REST API](/rest/guides/using-pagination-in-the-rest-api)."
|
||||
|
||||
## Timeouts
|
||||
|
||||
|
|
@ -723,5 +676,3 @@ If no `Time-Zone` header is specified and you make an authenticated call to the
|
|||
### Defaulting to UTC without other timezone information
|
||||
|
||||
If the steps above don't result in any information, we use UTC as the timezone to create the git commit.
|
||||
|
||||
[pagination-guide]: /guides/traversing-with-pagination
|
||||
|
|
|
|||
|
|
@ -51,7 +51,7 @@ in order to get more results.
|
|||
|
||||
It's important to *not* try and guess the format of the pagination URL. Not every
|
||||
API call uses the same structure. Instead, extract the pagination information from
|
||||
[the Link Header](/rest#pagination), which is sent with every request.
|
||||
the link header, which is returned with every request. For more information about pagination, see "[Using pagination in the REST API](/rest/guides/using-pagination-in-the-rest-api)."
|
||||
|
||||
[oap-guide]: https://developer.github.com/changes/2015-01-19-an-integrators-guide-to-organization-application-policies/
|
||||
|
||||
|
|
|
|||
|
|
@ -1,13 +0,0 @@
|
|||
{% ifversion not fpt or ghec %}
|
||||
|
||||
{% note %}
|
||||
|
||||
**Note**: The following guide uses the REST API for {% data variables.product.prodname_dotcom_the_website %}.
|
||||
|
||||
- Use <code>{% data variables.product.api_url_pre %}</code> to access the API for {% data variables.product.product_name %}.
|
||||
|
||||
- The guide specifies usernames and repositories that may not exist on {% data variables.location.product_location %}. You may need to use different names to see similar output.
|
||||
|
||||
{% endnote %}
|
||||
|
||||
{% endif %}
|
||||
Загрузка…
Ссылка в новой задаче