Operational article · published
How to Evaluate AI Vendor Data Portability Before You Switch
Test AI vendor data portability before renewal: verify export completeness, run a bounded switch drill, estimate rebuild cost, and document offboarding terms.
Reviewed 2026-09-21 · National guidance, Austin proofThe task and the failure mode
Built for: Procurement, RevOps, IT, security, data, and AI owners deciding whether a vendor can be replaced without losing required records, workflow logic, evidence, or operating continuity.
An AI vendor saying “you can export your data” does not establish portability. Buyers need to test two separate things: whether required records and files can be extracted in reusable form, and whether vendor-dependent behavior can be recreated from owned configuration, integrations, evaluation evidence, and documentation. Run that test before renewal or before the system becomes expensive to replace.
Decision brief
Separate data portability from workload portability. Data portability covers records, files, metadata, logs, and other owned material. Workload portability covers the workflow behavior, integrations, retrieval setup, permissions, and evaluation process that a replacement must recreate.
Treat “API available” as unproven until a bounded export preserves the fields, identifiers, relationships, attachments, timestamps, permissions, and history the replacement needs. Record rate limits, manual steps, vendor help, and any objects that remain vendor-native.
A less-portable tool can still fit a low-risk, reversible use case. Require stronger tested portability when the vendor becomes a system of record, holds critical workflow logic, or sits inside a revenue, security, or regulated process where exit delay would be costly.
Questions to answer before changing the system
- 01How do I evaluate data portability if I switch AI vendors later?
- 02Which records, files, metadata, workflow rules, logs, and evaluation artifacts would the replacement need?
- 03Can the documented export path move those assets at practical volume without losing required relationships or permissions?
- 04Which dependencies are tested portable, which need transformation or rebuild, and which remain vendor-native or unknown?
- 05What manual work, vendor assistance, parallel-run time, and stated offboarding fees make up the one-time exit cost?
Workflow
- 01Build a portability inventory. Include business records, conversation history, files, prompts and instructions, agent or workflow configuration, integrations, logs, evaluation datasets and results, retrieval source material, custom schemas, relationship keys, roles, permissions, and retention or deletion evidence. Mark each asset TESTED PORTABLE, TRANSFORM/REBUILD, or VENDOR-NATIVE / UNKNOWN.
- 02Write acceptance criteria before requesting an export. Define the fields, relationships, attachments, timestamps, permissions, history, and workflow behavior that must survive. Select a bounded sample with ordinary records plus linked objects, files, historical activity, permissions, and at least one known exception.
- 03Export through the documented customer path. Record the method, timestamp, schema or product version when available, limits, and vendor help required. Preserve a raw copy, record object and file counts, and version or hash the package when practical so later transformations remain distinguishable.
- 04Load the raw export into a neutral staging layer. Normalize formats without silently dropping unknown fields, keep mappings from source identifiers to staged identifiers, and document every transformation. Reissue credentials in the replacement environment instead of copying secrets into the export package.
- 05Recreate the minimum replacement workflow. Compare required fields, stable IDs, linked objects, timestamps, attachments, permissions, and exceptions. Record every field, relationship, integration, prompt, retrieval dependency, or configuration element that cannot be reconstructed from the export plus owned documentation.
- 06Rerun a fixed evaluation set when behavior matters. Compare task-level acceptance criteria written before migration rather than requiring identical model wording. Save the evaluation dataset, scoring rule, results, service version when available, and material exceptions so another reviewer can repeat the comparison.
- 07Estimate the actual exit. One-time exit cost equals extraction plus transformation and mapping plus integration or workflow rebuild plus validation and evaluation plus parallel run or cutover plus stated offboarding fees. Keep recurring replacement licensing separate. Test post-termination retention and deletion as a separate control.
Evidence to retain
- A portability inventory with a tested state for every required asset class.
- The raw export with object and file counts, method, timestamp, limits, and vendor help recorded.
- A field and relationship map showing stable identifiers, transformed values, and missing dependencies.
- Owned documentation for prompts, workflow rules, integrations, retrieval setup, roles, and permissions needed to rebuild.
- A fixed evaluation dataset and acceptance criteria used to compare incumbent and replacement behavior.
- An exit-cost worksheet separating extraction, transformation, rebuild, validation, cutover, and stated offboarding charges.
- Current contract or vendor documentation covering export scope, termination access, retention, deletion, and support.
Worked exit test: what counts as portable?
- Business records
- PASS when required fields, stable IDs, timestamps, and relationships survive. WARN when a documented transform is needed. FAIL when required records or links cannot be reconstructed.
- Attachments
- PASS when files keep deterministic links to parent records. WARN when a separate relinking step is bounded. FAIL when files or parent mappings are missing.
- Workflow logic
- PASS when prompts, rules, and configuration are exportable or reconstructable from owned documentation. WARN for bounded manual rebuild. FAIL when critical logic is inaccessible.
- Logs and evaluations
- PASS when the history needed for replacement comparison is retrievable. WARN for a known retention limit. FAIL when required evaluation evidence cannot be retained.
- Deletion
- PASS when retention and deletion steps plus completion evidence are documented. WARN when manual support is required. FAIL when residual-data handling after termination is unknown.
AI vendor portability checks before signing or renewal
- Required records and historical data are named in the export scope.
- Bulk export or API access can move the needed volume in a practical window.
- Stable IDs, relationships, timestamps, attachments, and required metadata survive the export.
- Critical prompts, workflow rules, integrations, and custom objects can be exported or reconstructed.
- Evaluation datasets, results, and audit evidence needed for replacement comparison can be retained.
- Offboarding support, fees, response expectations, and the post-termination export window are documented.
- Retention and deletion handling is documented separately from account closure.
- Material API, schema, or export constraints are known before renewal.
- Credentials and secrets are reissued rather than copied with the migration package.
- A bounded switch drill is complete and every unknown dependency has an owner.
What to measure—and what it does not prove
- Track the share of required asset classes that are TESTED PORTABLE, not merely described as exportable.
- Count required fields, relationships, files, permissions, or workflow components that fail the bounded migration.
- Record manual rebuild hours, vendor assistance, and elapsed time from export request to an accepted replacement test.
- Report evaluation exceptions and deletion evidence separately from export success; a complete extract proves neither behavioral parity nor deletion.
Boundaries and caveats
Portability depends on the product, plan, contract, architecture, data model, and use case. Recheck current vendor documentation before a procurement decision.
A technically complete export can still create legal, privacy, security, contractual, or operational obligations. This is an engineering and procurement test, not legal advice.
A bounded drill reduces uncertainty but does not prove every production edge case. Record what the sample excluded and which dependencies were not exercised.
Do not infer switching cost from page count, API availability, or vendor marketing. Measure the assets and dependencies your organization would actually move.
Primary sources
- NIST: Cloud Computing Standards Roadmapwww.nist.gov
- NIST SP 500-291 Rev. 2: Cloud Computing Standards Roadmapnvlpubs.nist.gov
- NIST: AI Risk Management Frameworkwww.nist.gov
- NIST: AI RMF Development and Generative AI Profilewww.nist.gov
Start with one bounded case
Before signing or renewing an AI vendor, choose one representative workflow and run a bounded exit drill. Export the records and evidence it needs, stage them outside the vendor, recreate the minimum replacement path, rerun the fixed evaluation set, and record every dependency that could not be moved or independently rebuilt. Use those receipts, not an “export available” claim, to decide whether the switching risk is acceptable.