Capability records, perspective 6: Lifecycle, adoption, and operations
Part of the SEI AI Adoption Framework (AIAF).
6.1 Feasibility and organizational readiness
Purpose
Prevent initiatives from starting or scaling on the assumption that they are feasible, when data, skills, workflow, provider support, cost, governance, or adoption readiness is missing.
Intended outcome
Decisions reflect technical feasibility, data, skills, workflow, provider, cost, governance, support, and adoption readiness.
Decisions enabled
- whether to proceed, prepare first, narrow the scope, or stop;
- which gaps must close before commitment;
- what the organization must change to be ready; and
- which assumptions need testing.
Applicability
Workloads: All
Delivery modes: All
Typical state: Required
Key activities
- assess technical feasibility with the actual data and workflow, not a demonstration;
- assess readiness in data, skills, process, provider, cost, governance, support, and adoption;
- record assumptions, risks, and the evidence for each;
- run small tests of the riskiest assumptions;
- decide with named owners and conditions; and
- revisit the assessment when facts change.
Required evidence
- feasibility and readiness assessment;
- assumptions and evidence;
- results of tests of key assumptions;
- decision with conditions; and
- revisit records.
Common failure patterns
- feasibility judged from a vendor demonstration;
- readiness limited to technology;
- assumptions never tested;
- gaps noted and then ignored; and
- assessment repeated only after failure.
Capability levels
| Level | Observable condition |
|---|---|
| Emerging | Feasibility is judged by sponsors or builders from experience and demonstrations. |
| Defined | A feasibility and readiness assessment with assumptions exists for in-scope initiatives. |
| Managed | Key assumptions are tested, gaps drive decisions, and assessments are revisited on change. |
| Adaptive | Assessment methods improve through comparison of predictions with outcomes, and readiness is built ahead of the portfolio's needs. |
Review and exit triggers
- a key assumption fails a test;
- scope, data, provider, or workflow changes;
- a readiness gap persists past its date;
- cost or timing departs from the assessment; or
- sponsors or priorities change.
6.2 Requirements and quality attributes
Purpose
Prevent AI capabilities from being judged by impressions because usefulness and qualities such as safety, security, accessibility, privacy, performance, and cost were never stated in testable form.
Intended outcome
The capability has testable requirements for usefulness and qualities such as reliability, safety, security, accessibility, privacy, performance, cost, and maintainability.
Decisions enabled
- what the capability must achieve and under what limits;
- which qualities matter and at what level;
- what counts as acceptable for release; and
- whether a requirement has been met.
Applicability
Workloads: All
Delivery modes: All
Typical state: Required
Key activities
- describe the work, users, and context of use;
- state usefulness requirements that can be tested;
- state quality requirements for reliability, safety, security, accessibility, privacy, performance, cost, and maintainability;
- set acceptance criteria and thresholds by consequence;
- include requirements from affected people and from obligations; and
- keep requirements under change control.
Required evidence
- requirements and quality measures;
- acceptance criteria;
- trace between requirements and tests;
- sources of requirements; and
- change records.
Common failure patterns
- requirements that say the AI should be helpful;
- qualities treated as someone else's concern;
- thresholds set after results are known;
- accessibility and affected users omitted; and
- requirements that drift without record.
Capability levels
| Level | Observable condition |
|---|---|
| Emerging | Requirements are expressed as goals or examples, and quality is judged by impression. |
| Defined | Testable usefulness and quality requirements with acceptance criteria exist for in-scope capabilities. |
| Managed | Requirements trace to tests and obligations, thresholds reflect consequence, and changes are controlled. |
| Adaptive | Requirements evolve based on operating evidence, and the organization improves its specification practice from escapes and complaints. |
Review and exit triggers
- a defect or complaint shows an unstated requirement;
- obligations or user needs change;
- the AI is used for different work;
- thresholds prove too lax or too strict; or
- a quality attribute cannot be measured.
6.3 Evaluation methodology
Full record: worked example 3 of 5 in Part 3, Capability model. The catalog entry in Part 4 gives the intended outcome, workloads, delivery modes, and condition.
6.4 Deployment and authority patterns
Purpose
Prevent AI from being given more autonomy, tools, access, and authority than evidence and consequence justify.
Intended outcome
AI receives only the autonomy, tools, access, and authority justified by evidence and consequence.
Decisions enabled
- which deployment pattern applies;
- what authority level the AI holds;
- what evidence is needed to promote it to a higher level; and
- when to restrict or withdraw authority.
Applicability
Workloads: All
Delivery modes: All
Typical state: Required
Key activities
- define patterns from assistance to supervised action to delegated action;
- assign tools, data, and authority per pattern, using the least needed;
- set controls suited to each pattern, including approval and limits;
- define promotion criteria that rely on evidence from operation;
- define demotion triggers; and
- record every change in pattern or authority with its approver.
Required evidence
- pattern decision and authority level;
- controls by pattern;
- promotion criteria and evidence;
- demotion triggers; and
- records of changes.
Common failure patterns
- authority granted because it was technically possible;
- a pilot pattern becoming permanent;
- promotion based on enthusiasm;
- no mechanism for demotion; and
- tools added without reviewing authority.
Capability levels
| Level | Observable condition |
|---|---|
| Emerging | Autonomy and access are decided by builders as convenient. |
| Defined | Patterns and authority levels are defined, with approval for each in-scope deployment. |
| Managed | Authority is based on evidence, promotion and demotion criteria are used, and changes are recorded. |
| Adaptive | Patterns adapt to results across deployments, and the organization extends authority only where evidence justifies and withdraws it quickly when not. |
Review and exit triggers
- an error shows authority exceeded competence;
- tools or access are added;
- evidence for promotion is weakened;
- consequence of the use rises; or
- demotion criteria are met.
6.5 Staged deployment and adoption
Purpose
Prevent wide exposure of AI before it has been shown to work for real people, by expanding in controlled stages with the means to stop.
Intended outcome
Exposure expands in controlled stages with monitoring, support, feedback, rollback, and explicit promotion decisions.
Decisions enabled
- who receives access at each stage;
- what evidence promotes to the next stage;
- when to hold, roll back, or stop; and
- what support each stage needs.
Applicability
Workloads: All
Delivery modes: A, B, C
Typical state: Required
Key activities
- define stages, cohorts, and entry criteria;
- monitor behavior, outcomes, support demand, and feedback in each stage;
- set explicit promotion decisions with owners;
- prepare rollback and support before each stage;
- include cohorts that represent the range of users and conditions; and
- record decisions and lessons.
Required evidence
- cohort and stage plan;
- monitoring results per stage;
- promotion and rollback records;
- support and feedback logs; and
- lessons recorded.
Common failure patterns
- stages defined by calendar rather than evidence;
- early cohorts of enthusiasts only;
- rollback not rehearsed;
- promotion by default; and
- support unprepared for the next stage.
Capability levels
| Level | Observable condition |
|---|---|
| Emerging | Release goes to whoever asks, and expansion follows enthusiasm or deadline. |
| Defined | Stages, cohorts, and promotion owners are defined for in-scope deployments. |
| Managed | Promotion depends on evidence, rollback is ready, and decisions and lessons are recorded. |
| Adaptive | Staging practice adapts to what earlier releases revealed, and the organization sizes stages by risk and evidence. |
Review and exit triggers
- monitoring crosses a hold or rollback threshold;
- support demand exceeds capacity;
- a cohort differs from the expected population;
- a promotion decision lacks evidence; or
- the plan is changed under schedule pressure.
6.6 Continuous outcome, behavior, and change evaluation
Purpose
Prevent AI that was good at release from quietly declining in value, drifting in behavior, or changing underneath its users.
Intended outcome
Operation detects declining value, behavior change, provider change, drift, misuse, unexpected effects, and changing assumptions.
Decisions enabled
- whether the AI still delivers its intended outcome;
- whether behavior has changed in ways that matter;
- whether to retrain, reconfigure, restrict, or retire; and
- whether assumptions made at approval still hold.
Applicability
Workloads: All
Delivery modes: All
Typical state: Required
Key activities
- measure outcomes, not only technical performance;
- set behavior thresholds and test sets that repeat on a schedule and after change;
- watch for drift in data, users, provider behavior, and context;
- detect misuse and unexpected effects;
- compare current results with the baseline and expected value; and
- decide and act on the findings.
Required evidence
- outcome reviews;
- behavior thresholds and test results;
- change and drift alerts;
- misuse and unexpected-effect findings; and
- decisions taken.
Common failure patterns
- monitoring uptime while quality falls;
- reviews that happen only after complaints;
- provider model updates that go unnoticed;
- thresholds that never trigger; and
- findings with no decision.
Capability levels
| Level | Observable condition |
|---|---|
| Emerging | Problems are noticed when users complain or when someone happens to compare results. |
| Defined | Outcome and behavior measures with thresholds and owners exist for in-scope uses. |
| Managed | Reviews repeat on schedule and after change, drift and misuse are detected, and findings lead to recorded decisions. |
| Adaptive | Evaluation adapts to the changing context, and the organization detects declining value early and redirects or retires capabilities on evidence. |
Review and exit triggers
- outcomes or behavior cross a threshold;
- the provider changes the model or terms;
- data or user population shifts;
- misuse or unexpected effects appear; or
- assumptions in the approval no longer hold.
6.7 Configuration, version, and decision lineage
Purpose
Prevent consequential outputs and actions from being impossible to explain, reproduce, or dispute because nobody can tell which model, data, prompt, tools, policies, and configuration produced them.
Intended outcome
Consequential outputs and actions can be connected to the relevant model, data, prompt, tools, policies, integrations, identity, and approved configuration.
Decisions enabled
- whether a result can be explained and defended;
- which configuration was in force at a given time;
- whether a change caused a problem; and
- how much lineage a use needs.
Applicability
Workloads: All
Delivery modes: All
Condition: Applied proportionately to consequence
Typical state: Required, with depth based on consequence
Key activities
- decide the lineage needed in proportion to consequence;
- record versions of models, prompts, data, tools, policies, integrations, and identities;
- link outputs and actions to the configuration in force;
- keep approved configurations and detect departures;
- protect records from alteration; and
- test that a past decision can be reconstructed.
Required evidence
- configuration records and versions;
- decision and action lineage;
- approved-configuration baseline;
- departure detection results; and
- reconstruction tests.
Common failure patterns
- prompts and settings changed without a record;
- provider model updates not captured;
- lineage kept for all uses equally or for none;
- records that cannot be matched to a decision; and
- reconstruction never tested.
Capability levels
| Level | Observable condition |
|---|---|
| Emerging | Configuration is known to the people who set it, and past results cannot be reconstructed. |
| Defined | Versions of key components are recorded for uses where consequence calls for it. |
| Managed | Outputs and actions link to configuration, departures are detected, and reconstruction is tested. |
| Adaptive | Lineage practice adapts to consequence and dispute patterns, and the organization reconstructs decisions quickly and with confidence. |
Review and exit triggers
- a result cannot be reconstructed;
- configuration changes without approval;
- consequence of a use rises;
- a dispute or inquiry requires evidence; or
- a provider changes components not versioned by the organization.
6.8 Provider and capability change monitoring
Purpose
Prevent provider changes in terms, prices, data use, safeguards, features, interfaces, locations, or availability from taking effect unnoticed after selection.
Intended outcome
Provider terms, prices, data use, safeguards, features, interfaces, locations, and availability are reviewed for continued fitness.
Decisions enabled
- whether a change is acceptable;
- what mitigation or negotiation is needed;
- whether to continue, restrict, or exit; and
- who must be informed or consulted.
Applicability
Workloads: All
Delivery modes: All
Condition: Modes B, C, and D directly; Mode A when it depends on a provider
Typical state: Shared
Key activities
- identify the change channels for each provider, including notices, release notes, and status pages;
- assign an owner who reads and records changes;
- assess each change for effect on risk, cost, obligations, and outcomes;
- test effects where behavior may have changed;
- decide to accept, mitigate, or exit, and record the decision; and
- feed findings to sourcing, portability, and risk decisions.
Required evidence
- change notices received;
- impact reviews;
- acceptance, mitigation, or exit decisions;
- results of tests after change; and
- link to provider risk reviews.
Provider evidence
Ask the provider for:
- advance notice of material changes;
- release notes and documentation of behavior changes;
- notice of terms, price, or location changes; and
- availability and incident reports.
Common failure patterns
- notices unread because they arrive by email to a shared inbox;
- behavior changes found by users;
- price changes accepted by default;
- no owner; and
- changes assessed without testing.
Capability levels
| Level | Observable condition |
|---|---|
| Emerging | Provider changes are noticed by chance, and effects are discovered through use. |
| Defined | Owners and channels for provider change are defined, and notable changes are recorded. |
| Managed | Changes are assessed and tested, decisions are recorded, and findings reach related decisions. |
| Adaptive | Monitoring adapts to provider behavior, and the organization anticipates changes and negotiates or exits in good time. |
Review and exit triggers
- a provider issues a material notice;
- behavior changes without notice;
- a change conflicts with an obligation or risk limit;
- the provider's own status changes; or
- no review has occurred by the due date.
Boundary
Watches what changes after selection, such as terms, prices, features, and locations. 1.5 chooses, 5.4 preserves the ability to switch, and 7.8 assesses provider risk at selection and renewal.
6.9 Rollback, replacement, and retirement
Full record: worked example 5 of 5 in Part 3, Capability model. The catalog entry in Part 4 gives the intended outcome, workloads, delivery modes, and condition.
6.10 Continuity and exit planning
Purpose
Prevent important work from stopping when an AI capability or provider becomes unavailable, unacceptable, uneconomic, or obsolete.
Intended outcome
Important work can continue when an AI capability or provider becomes unavailable, unacceptable, uneconomic, or obsolete.
Decisions enabled
- which work must continue and for how long;
- what alternatives are ready;
- how much to invest in preparation; and
- when to invoke the plan.
Applicability
Workloads: All
Delivery modes: All
Condition: Applied proportionately to consequence
Typical state: Required, with depth based on consequence
Key activities
- identify work that depends on each AI capability and its tolerance for interruption;
- define alternatives, such as manual procedures, other tools, or other providers;
- set recovery objectives suited to consequence;
- prepare data, access, and agreements that make alternatives usable;
- exercise continuity and exit plans; and
- update plans after changes and exercises.
Required evidence
- continuity plan and dependency map;
- alternatives and their readiness;
- recovery objectives;
- exercise results; and
- update records.
Common failure patterns
- manual fallback assumed but nobody remembers how;
- backups of data with no way to use it elsewhere;
- plans that cover provider outage but not provider exit;
- exercises skipped; and
- dependence discovered during the failure.
Capability levels
| Level | Observable condition |
|---|---|
| Emerging | Continuity relies on improvisation, and dependence on AI is not mapped. |
| Defined | Continuity plans and alternatives exist for the uses where consequence calls for them. |
| Managed | Plans are exercised, alternatives are ready, and lessons update the plans. |
| Adaptive | Continuity design adapts to dependence, provider change, and exercise findings, and the organization shifts work between alternatives with little disruption. |
Review and exit triggers
- an exercise fails to meet recovery objectives;
- a provider fails, changes, or announces withdrawal;
- dependence on the capability grows;
- alternatives become unavailable; or
- consequence of interruption rises.
Boundary
Keeps standing continuity and alternatives ready so work survives unavailability. 6.9 carries out the rollback, replacement, or retirement.
