Clarify pagination requirements and recommended page size (#981)

agency/README.md

@@ -161,7 +161,7 @@ _Path Parameters:_
 | ------------ | ---- | ----------------- | ------------------------------------------- |
 | `device_id`  | UUID | Optional          | If provided, retrieve the specified vehicle |
 
-If `device_id` is specified, `GET` will return an array with a single vehicle record, otherwise it will be a list of vehicle records with pagination details per the [JSON API](https://jsonapi.org/format/#fetching-pagination) spec:
+If `device_id` is specified, `GET` will return an array with a single vehicle record, otherwise it will be a list of vehicle records with pagination details (see [Pagination][pagination]):
 
 ```json
 {
@@ -204,7 +204,7 @@ _Path Parameters:_
 | ------------ | ---- | ----------------- | ------------------------------------------- |
 | `device_id`  | UUID | Optional          | If provided, retrieve the specified vehicle |
 
-If `device_id` is specified, `GET` will return an array with a vehicle status record, otherwise it will be a list of vehicle records with pagination details per the [JSON API](https://jsonapi.org/format/#fetching-pagination) spec:
+If `device_id` is specified, `GET` will return an array with a vehicle status record, otherwise it will be a list of vehicle records with pagination details (see [Pagination][pagination]):
 
 ```json
 {
@@ -540,6 +540,7 @@ See [Responses][responses], [Bulk Responses][bulk-responses], and [schema][schem
 [hdop]: https://en.wikipedia.org/wiki/Dilution_of_precision_(navigation)
 [iana]: https://www.iana.org/assignments/http-status-codes/http-status-codes.xhtml
 [modes]: /modes/README.md
+[pagination]: /general-information.md#pagination
 [propulsion-types]: /data-types.md#propulsion-types
 [reports]: /data-types.md#reports
 [responses]: /general-information.md#responses

general-information.md

@@ -18,6 +18,7 @@ This document contains specifications that are shared between the various MDS [A
 - [Geographic Data](#geographic-data)
   - [Intersection Operation](#intersection-operation)
 - [Geography-Driven Events](#geography-driven-events)
+- [Pagination](#pagination)
 - [Responses](#responses)
   - [Error Messages](#error-messages)
   - [Bulk Responses](#bulk-responses)
@@ -187,6 +188,50 @@ Agencies that wish to use Geography-Driven Events do so by requiring a new `even
 
 [Top][toc]
 
+## Pagination
+
+Several MDS endpoints return collections of records that may be split across multiple pages (for example, the [Provider](/provider/README.md) and [Agency](/agency/README.md) `/vehicles` and `/vehicles/status` endpoints, and the [Policy](/policy/README.md) endpoint). When an endpoint uses pagination, it must comply with the page-based pagination convention defined by the [JSON:API][json-api-pagination] specification.
+
+Clients can request a page using the JSON:API page-based query parameters:
+
+| Query Parameter | Type    | Default            | Description                                  |
+| --------------- | ------- | ------------------ | -------------------------------------------- |
+| `page[number]`  | Integer | 1                  | 1-based index of the page to return.         |
+| `page[size]`    | Integer | 1000 (recommended) | Maximum number of records returned per page. |
+
+The following keys must be used for pagination links, in a top-level `links` object:
+
+| Key     | Description                        |
+| ------- | ---------------------------------- |
+| `first` | URL to the first page of data.     |
+| `last`  | URL to the last page of data.      |
+| `prev`  | URL to the previous page of data.  |
+| `next`  | URL to the next page of data.      |
+
+At a minimum, a paginated payload must include a `next` key, which must be set to `null` to indicate the last page of data.
+
+```jsonc
+{
+    "version": "x.y.z",
+    // ...the endpoint's data array, e.g. "vehicles", "vehicles_status", "policies"...
+    "links": {
+        "first": "https://...",
+        "last": "https://...",
+        "prev": "https://...",
+        "next": "https://..."
+    }
+}
+```
+
+To keep polling of large fleets efficient while remaining a non-breaking change, the following are **recommended** (not required):
+
+- A provider or agency **should** support a `page[size]` of at least `500`, so consumers can always request that many records per page.
+- The maximum supported `page[size]` is bound by the provider's or agency's infrastructure and **should** be documented in their API documentation.
+
+These recommendations apply to the [Agency](/agency/README.md), [Provider](/provider/README.md), and [Policy](/policy/README.md) APIs.
+
+[Top][toc]
+
 ## Responses
 
 - **200:** OK: operation successful.
@@ -338,6 +383,7 @@ If an unsupported or invalid version is requested, the API must respond with a s
 [Top][toc]
 
 [decimal-degrees]: https://en.wikipedia.org/wiki/Decimal_degrees
+[json-api-pagination]: https://jsonapi.org/format/#fetching-pagination
 [st-intersects]: https://postgis.net/docs/ST_Intersects.html
 [toc]: #table-of-contents
 [wgs84]: https://en.wikipedia.org/wiki/World_Geodetic_System

policy/README.md

@@ -181,7 +181,7 @@ _Query Parameters:_
 
 `start_date` and `end_date` are only considered when no `id` parameter is provided. They should return any policy whose effectiveness overlaps with or is contained with this range. Suppose there's a policy with a `start_date` of 1/1/21 and `end_date` of 1/31/21. Assuming an `end_date` that is null, 12/1/20 and 1/5/21 `start_dates` will return the policy, but 2/10/21 wouldn't. Assuming a `start_date` parameter of say, 11/1/20, then an `end_date` of 12/1/20 wouldn't return the policy, but 1/5/21 and 2/10/21 would. Lastly, a `start_date` of 1/5/21 and `end_date` of 1/6/21 would also return the policy. Please note also that while dates in the format MM:DD:YY are being used here, `start_date` and `end_date` must be numbers representing milliseconds since the Unix epoch time.
 
-Policies will be returned in order of effective date (see schema below), with pagination as in the `agency` and `provider` specs.
+Policies will be returned in order of effective date (see schema below). Pagination is optional for this endpoint, and policies are typically returned in a single response. If pagination is used, it follows the same convention as the `agency` and `provider` specs (see [Pagination][pagination]).
 
 `provider_id` is an implicit parameter and will be encoded in the authentication mechanism, or a complete list of policies should be produced. If the Agency decides that Provider-specific policy documents should not be shared with other Providers (e.g. punitive policy in response to violations), an Agency should filter policy objects before serving them via this endpoint.
 
@@ -883,6 +883,7 @@ You may also show which APIs, endpoints, and fields your agency is serving to pr
 [json-schema]: #json-schema
 [modes]: /modes/README.md
 [muni-boundary]: /provider/README.md#municipality-boundary
+[pagination]: /general-information.md#pagination
 [propulsion-types]: /general-information.md#propulsion-types
 [responses]: /general-information.md#responses
 [schema]: /schema/

provider/README.md

@@ -125,32 +125,7 @@ See the [Endpoints](#endpoints) below for information on their specific schema,
 
 The `/trips` and `/events/historical` endpoints must not use pagination.
 
-If Providers choose to use pagination for either of the `/events` or `/vehicles` endpoints, the pagination must comply with the [JSON API][json-api-pagination] specification.
-
-The following keys must be used for pagination links:
-
-* `first`: url to the first page of data
-* `last`: url to the last page of data
-* `prev`: url to the previous page of data
-* `next`: url to the next page of data
-
-At a minimum, paginated payloads must include a `next` key, which must be set to `null` to indicate the last page of data.
-
-```json
-{
-    "version": "x.y.z",
-    "trips": [{
-        "provider_id": "...",
-        "trip_id": "...",
-    }],
-    "links": {
-        "first": "https://...",
-        "last": "https://...",
-        "prev": "https://...",
-        "next": "https://..."
-    }
-}
-```
+If Providers choose to use pagination for the `/events`, `/vehicles`, or `/vehicles/status` endpoints, it must follow the page-based pagination convention described in [Pagination][pagination] in General Information, including the page-based query parameters (`page[number]`, `page[size]`), the `links` object, and the recommended minimum and default page sizes.
 
 [Top][toc]
 
@@ -205,7 +180,7 @@ _Path Parameters:_
 | ------------ | ---- | ----------------- | ------------------------------------------- |
 | `device_id`  | UUID | Optional          | If provided, retrieve the specified vehicle |
 
-If `device_id` is specified, `GET` will return an array with a single vehicle record, otherwise it will be a list of vehicle records with pagination details per the [JSON API](https://jsonapi.org/format/#fetching-pagination) spec:
+If `device_id` is specified, `GET` will return an array with a single vehicle record, otherwise it will be a list of vehicle records with pagination details (see [Pagination][pagination]):
 
 ```json
 {
@@ -248,7 +223,7 @@ The `/vehicles` POST endpoint takes a JSON body of `device_ids` UUIDs, and retur
 **Method:** `POST`  
 **Payload:** An array of [Vehicles](/data-types.md#vehicles)  
 
-Returned will be a list of vehicle records with pagination details per the [JSON API](https://jsonapi.org/format/#fetching-pagination) spec (recommmend 1,000 devices or less per page):
+Returned will be a list of vehicle records with pagination details (see [Pagination][pagination]):
 
 ```json
 {
@@ -307,7 +282,7 @@ _Path Parameters:_
 | ------------ | ---- | ----------------- | ------------------------------------------- |
 | `device_id`  | UUID | Optional          | If provided, retrieve the specified vehicle |
 
-If `device_id` is specified, `GET` will return an array with a vehicle status record, otherwise it will be a list of vehicle records with pagination details per the [JSON API](https://jsonapi.org/format/#fetching-pagination) spec:
+If `device_id` is specified, `GET` will return an array with a vehicle status record, otherwise it will be a list of vehicle records with pagination details (see [Pagination][pagination]):
 
 ```json
 {
@@ -664,10 +639,10 @@ See [Provider examples](examples.md#reports).
 [incidents]: /data-types.md#incidents
 [intersection]: /general-information.md#intersection-operation
 [iso4217]: https://en.wikipedia.org/wiki/ISO_4217#Active_codes
-[json-api-pagination]: http://jsonapi.org/format/#fetching-pagination
 [json-schema]: https://json-schema.org
 [muni-boundary]: #municipality-boundary
 [mode]: /modes/README.md
+[pagination]: /general-information.md#pagination
 [point-geo]: /data-types.md#gps-data
 [propulsion-types]: /general-information.md#propulsion-types
 [responses]: /general-information.md#responses