Key takeaway
Counted rows establish what arrived. Endpoint-specific continuation evidence establishes whether the export followed every available page within its stated scope.
Identify the endpoint’s stopping rule
An API returns a valid response with the requested number of records. The exporter saves the file and labels the job complete. That proves one response was processed, not that the endpoint had nothing more to return. The acceptance question is whether the extraction followed the actual continuation mechanism until its documented terminal condition.
GitHub’s REST documentation describes pagination through Link response headers, including a next relationship when another page is available. Zendesk documents cursor and offset approaches with different response fields and endpoint-specific exceptions. These are concrete implementations to inspect, not a universal recipe that every API follows. Start with the exact endpoint, access scope and API version.
Complete a three-response ledger
This hypothetical cursor endpoint returns the invented chain below. The first response contains two records and a continuation C2. The second contains one record and a continuation C3. The third contains two records and the documented terminal flag. The fixture intentionally shows that a short intermediate page is not the stopping rule for this fictional endpoint.
The accepted traversal retains five returned records and evidence for all three responses. Stopping after the second would retain three records while leaving C3 unvisited. This is a property of the stated fixture, not a claim that GitHub or every Zendesk endpoint produces short nonterminal pages. An actual endpoint needs its own documented and observed rule.
| Fictional request | Returned identities | Continuation evidence | Completed decision |
|---|---|---|---|
| Initial request | A, B | Next cursor C2 | Continue |
| Request using C2 | C | Next cursor C3 | Continue despite short page |
| Request using C3 | D, E | Documented terminal flag | Traversal complete for this fixture |
| Stop after C2 | A, B, C retained | C3 remains unvisited | Hold completeness claim |
Keep the response chain, not just the last count
A useful internal ledger records request sequence, endpoint/version, approved filter set, response status, returned record count, continuation received, continuation used and final stopping evidence. Bind it to the resulting package identity. Record the permission or account scope that constrained the source view without exposing credentials or opaque continuation tokens in public materials.
Verify whether page size was accepted or merely requested. GitHub notes that limits can constrain per-page requests; requesting a large page is not proof that every matching row fit. Its documentation also explains that some Link relationships may be absent, such as a last page that cannot be calculated. The receiver must not require a field the endpoint never promises.
Zendesk’s documented cursor flow uses meta.has_more and links.next, while its offset responses use next_page. Some endpoints use different pagination patterns. Record the selected method instead of combining convenient fields from separate examples. A hard offset limit can be an extraction limit rather than evidence that the source population ended.
Test retries and repeated continuations
Exercise the proposed extractor against synthetic responses with multiple pages, a short nonterminal page, an empty terminal response if supported, an interrupted request and a repeated continuation. Define how it detects no progress. Repeatedly saving one successful response can inflate the output count while leaving later records unreachable.
Separate retry handling from record identity. A safe restart may revisit a page, so retain the intended deduplication or checkpoint policy and test it with stable fictional identities. A repeated business value is not automatically the same record. If a continuation expires or a required page fails, mark the extraction partial and retain the last confirmed boundary rather than declaring completion from the available row count.
State what a completed traversal does not prove
Even a correctly followed chain may reflect a source changing during the run. Pagination completion does not establish that all pages shared one committed snapshot. Nor does it prove that the authorized account could see every company record, that deleted material remained accessible or that the endpoint exposed all relevant object types. Those are separate coverage and consistency questions.
For the fictional export, the bounded conclusion is five identities across the complete declared response chain. A real owner should retain the endpoint’s visibility limits and unresolved objects with that conclusion. The due-diligence request can ask for this evidence before a receiving party is offered any sample; a traversal record is not disclosure permission.