Readiness evaluation, full detail
How scoping and the readiness evaluation work
The complete scoping agenda, what is recorded, the boundaries that stay in force, and how responsibilities are split. If you only need the summary, start on the readiness evaluation overview.
Request a Readiness Evaluation Scoping ConversationStep 1, entry conversation
Readiness Evaluation Scoping Conversation
A 30-minute Readiness Evaluation Scoping Conversation defines one named AI use case, intended users and workflow, decision owner, decision deadline, existing evidence, required documentation fields, the known data pathway, the unresolved question, and preliminary stopping conditions. It ends with an evaluation-fit determination.
The scoping review ends in one of four evaluation-fit determinations: Proceed, Modify, Pause, Stop. Nothing about data movement, analysis environment, or fees is assumed before the scope is understood.
- 30-minute scoping and evidence-review meeting
- One named AI use case
- Intended user and workflow
- Decision owner and deadline
- Existing evidence inventory
- Minimum data specification
- Preliminary feasibility and gap determination
- Determination of whether VitaSignal would add nonduplicative value
- Written next-step recommendation
- Named AI use case
- Intended user and workflow
- Decision owner and deadline
- Systems and fields in scope
- Evidence already available
- Unresolved gaps
- Minimum evidence needed
- Whether a bounded review is justified
- Which AI use case is being considered?
- Which local documentation fields and workflows support it?
- What decision must leadership make?
- Who owns the decision?
- When is the decision due?
- What evidence already exists?
- What remains unresolved?
Scoping starts with the organization's current alternatives. Healthcare organizations may already answer part of the question through EHR reports, help desk data, staff feedback, vendor materials, and internal analyses.
- What concern or decision started this review?
- What have nurses, managers, or workflow owners reported?
- Which EHR reports, help desk data, surveys, vendor materials, and internal analyses already exist?
- What does each source answer?
- What remains unresolved?
- Who performs the review work today?
- Who owns the decision?
- Who controls the budget?
- Who can approve or block the data pathway?
- What event or deadline makes the decision urgent?
A VitaSignal engagement proceeds only when a specific, consequential evidence gap remains after current EHR reporting, vendor evidence, and internal capabilities are considered.
Stop is a valid scoping outcome, including when a study is not justified:
- Existing evidence already answers the question
- No named decision or deadline exists
- No approved data pathway is feasible
- The requested use conflicts with prohibited clinical or workforce uses
- VitaSignal would duplicate existing internal work
Step 2, flagship commercial offer
Clinical AI Documentation Readiness Evaluation
A bounded retrospective or approved read-only readiness evaluation for one named AI use case. The evaluation runs on a retrospective extract or an approved read-only pathway without producing clinician-facing output. Any later observational phase requires separate written authorization and prespecified evidence gates. A use-case-specific readiness evaluation that includes only evidence-matched components supported by the available fields, ground truth, workflow definitions, and written scope.
The retrospective or approved read-only evaluation is designed to avoid clinician-facing tasks and added charting. Site review, approvals, data preparation, interpretation, and governance participation may still require staff time and must be recorded as study-created burden.
Scope, data pathway, deliverables, and fees are documented in a site-specific statement of work, and appropriate agreements are completed before any real organizational data moves. Final timing depends on site approvals, data readiness, and the written Statement of Work.
- Retrospective extract or approved read-only pathway
- No EHR write-back
- No bedside alerts
- No staffing decisions
- No individual nurse scoring or surveillance
- No employee performance evaluation
- No patient-care decisioning
- No diagnosis or treatment
- No autonomous prediction
- Human-owned decisions and stopping rules
- The scoping review identified a specific unresolved documentation question
- The named use case has a decision owner and a decision deadline
- An approved retrospective or read-only data pathway is available
- Required agreements and site determinations are in place
- VitaSignal would add nonduplicative value rather than repeat existing internal work
- Stopping rules are agreed in writing before any analysis begins
The governance package records, for each review question, which public-safe VitaSignal capability was applied and what it could and could not support.
- Review question
- Public-safe VitaSignal capability used
- Why the capability was selected
- Data required and actually available
- Evidence level
- Finding
- Limitation
- Prohibited conclusion
- Effect on the governance decision
Formulas, thresholds, weights, source code, protected method detail, confidential mappings, and unpublished mechanics are not included in customer-facing material.
The public product demonstration uses synthetic data. Retrospective evidence is component-specific and separate from synthetic demonstration output. Broader VitaSignal technology lanes remain research work rather than available products. See Evidence and Evidence.
- The named AI use case under consideration
- The decision owner and the decision deadline
- Intended users, workflow, and department
- An informatics or data contact for the field-level discussion
- Evidence already gathered internally or from the AI vendor
- Approved subgroup variables where subgroup review is in scope
- Site determination for research, quality improvement, or evaluation where required
- A structured scoping and evidence-review conversation
- A minimum data specification for the named use case
- A documentation pathway review covering timing, availability, missingness, duplication, provenance, and mapping consistency
- An explicit statement of what cannot be evaluated with the available information
- A governance decision record with owners and stopping rules
- A written next-step recommendation, including a recommendation to stop where that is the honest answer
When a study is justified, and when it is not
A study should exist only to resolve a question that matters and remains open. The fit check happens before any work is scoped.
A study may be justified when
- One named clinical AI use case with an intended use, an accountable decision owner, and a decision that has to be made.
- A consequential question about the documentation pathway that existing reports, feedback, vendor materials, internal analysis, and governance review have not resolved.
- Approved retrospective or read-only access to the minimum fields required to examine that question.
- Willingness to accept a written finding that the evidence does not support proceeding.
A study is not justified when
- Current teams and tools already answer the question.
- The decision has already been made and the study would only document it after the fact.
- The required fields, field meanings, or approvals are not available, so the inference cannot be supported.
- The expected use includes clinician-facing output, staffing action, individual performance review, or anything outside read-only evaluation.
Synthetic worked example
What the Governance Decision Package looks like
The VitaSignal Interactive Product Tour illustrates how a bounded readiness evaluation examines a documentation pathway and records a governance decision. Tour data is generated and contains no PHI. It does not represent the full research portfolio or establish clinical, operational, or prospective performance.
Every value below is synthetic. Synthetic results are not empirical clinical evidence and are not used to claim real-world performance.
- Question under review
- Can shift-end documentation timing be measured reliably enough at this site to inform a governance decision about an AI-assisted documentation workflow?
- Data route
- Retrospective de-identified extract. No write-back, no bedside alert, and no clinician-facing task during the evaluation.
- Field availability finding
- Synthetic example: required timing fields present for most encounters, handoff events not consistently recorded, local shift boundaries unverified.
- Reliability and calibration review
- Synthetic example: component behavior stable across the sampled period, with subgroup review flagged as incomplete pending verified local definitions.
- Study-created burden
- Site review, approvals, data preparation, interpretation, and governance participation required staff time and are recorded as study-created burden.
- Recommended decision
- Synthetic example: Modify. Verify local shift boundaries and handoff capture before any further evaluation phase.
Nursing workflow is part of the evidence
Documentation data do not explain a nursing shift by themselves. Where the scope allows, VitaSignal includes structured nursing workflow review so the organization can ask whether the interpretation matches actual practice, what context is missing, and which conclusions would be unfair or unsupported.
- No individual nurse ranking
- No employee performance evaluation
- No staffing allocation
- No added documentation tasks during the read-only review
- No bedside alerts or patient-care decisions
- No conclusion from timestamps alone about workload, burnout, safety, fairness, or causality
How VitaSignal reviews the evidence
VitaSignal maintains a protected and proprietary method library spanning documentation patterns, reliability, subgroup monitoring, calibration, provenance, and governance evidence. A pilot does not automatically use every method. The written scope identifies which capabilities apply, what data they require, what question they address, and what limitations remain.
Documentation Pathway Review
Examines required fields, timing, availability, missingness, duplication, and workflow context where supported.
Reliability and Provenance Review
Examines source meaning, recorded time, event time, mapping consistency, and known data-lineage limitations.
Subgroup and Model Safety Review
Supports prespecified subgroup, calibration, and local-evidence review only when approved fields, adequate data, and appropriate ground truth are available.
Governance Decision Support
Organizes findings, unknowns, prohibited conclusions, accountable owners, stopping rules, and next evidence requirements.
These capability descriptions do not disclose formulas, thresholds, scoring weights, claim language, source code, or confidential implementation mechanics.
One decision, several roles
Different people feel the problem, use the evidence, approve the data pathway, and fund the work.
- Problem owner and likely champion
- CNIO or senior nursing informatics leader
- Likely daily users of the evidence
- Nursing informatics specialists, clinical informatics reviewers, quality, safety, and AI governance staff
- Affected stakeholders
- Frontline nurses and nurse managers
- Possible economic buyer
- CNIO, CNO, CIO, CFO, CMIO, quality leader, innovation office, or AI governance sponsor
- Gatekeepers
- Data, EHR integration, privacy, security, legal, procurement, finance, and governance teams
The champion, user, budget owner, approver, and affected stakeholder may be different people. VitaSignal confirms these roles during scoping rather than assuming one universal buyer.
Documentation-Pathway Intelligence
A structured evaluation of how documentation timing, provenance, missingness, duplication, workflow conditions, and subgroup visibility affect the evidence pathway supporting one defined clinical AI use case.
This is internal working language for the evaluation described above. It is not a recognized, validated, owned, or customer-established category, and it does not describe a product that is available for clinical use.
What the evidence does and does not show
VitaSignal is a pre-market research prototype. Components have been evaluated retrospectively using de-identified critical-care datasets. Those analyses do not establish local organizational validity, clinical benefit, workflow improvement, deployment readiness, regulatory clearance, or prospective performance.
- Synthetic examples demonstrate structure only.
- Retrospective component evidence is not product validation.
- Local field meaning, data quality, workflow fit, and subgroup analysis require site approval.
- Null, negative, adverse, underpowered, and blocked findings are valid outcomes.
- A study may conclude that VitaSignal is not needed or that the available data cannot support the inference.
Common questions
How does an engagement begin?
Every engagement begins with a Readiness Evaluation Scoping Conversation, a 30-minute qualification and scope conversation with no data commitment. It ends in one of four determinations, using the same taxonomy as the engagement itself: Proceed, Modify, Pause, or Stop.
How long does the evaluation take?
There is no fixed duration. The engagement is bounded and time-limited, and scope and duration are set in the written Statement of Work based on site approvals, data readiness, and the named use case. The evaluation is conducted on a retrospective extract or an approved read-only pathway. Silent-mode observation is not part of the evaluation; it is a possible later phase only under separate written authorization once all applicable technical, privacy, security, governance, contractual, and evidence gates are met.
Does every evaluation include the same analytical components?
No. An evaluation is a use-case-specific retrospective or approved read-only evaluation that includes only evidence-matched components supported by the available fields, ground truth, workflow definitions, and written scope. No component is automatically included in every evaluation.
Do clinicians use anything during the evaluation?
The retrospective or approved read-only evaluation is designed to avoid clinician-facing tasks and added charting. Site review, approvals, data preparation, interpretation, and governance participation may still require staff time and must be recorded as study-created burden. It does not generate bedside alerts, assign tasks, write back to the EHR, or guide patient-care decisions. Any prospective or clinician-facing workflow would require a separate governance decision.
What does the data extract look like?
The preferred extract includes encounter metadata, unit and shift information, documentation events, timestamps, author-role categories where approved, and approved subgroup variables for governance review. Depending on the pilot scope, data may be provided through a site-approved export pathway. FHIR R4 and HL7 v2 are future targets that would require written agreement and technical review; no connector is implemented today.
Do you need identifiable patient data?
No identifiable patient data is required for the retrospective evaluation. The evaluation is designed to use de-identified, limited, synthetic, or otherwise site-approved data. Any data transfer follows site policy and applicable agreements.
What if our results do not support continued evaluation?
That is an acceptable evaluation outcome. The Governance Decision Package supports a human-owned decision to Proceed, Modify, Pause, or Stop. If data quality, field availability, workflow differences, or subgroup concerns limit interpretation, Stop is a legitimate recorded outcome. VitaSignal does not make the decision.
Can the results be published?
Publication can be discussed if both parties agree, the site permits publication, and all required approvals are in place. The evaluation itself does not require publication.
VitaSignal is research-stage governance decision-support technology. It is not FDA cleared or approved and is not available for clinical use. Intended use, product classification, and the applicable regulatory pathway remain under evaluation. Prospective validation and intended-use determination remain required.