World Sleep Industry ENCYCLOPEDIA

Sleep Vertical Foundation Models

4.4.2

Definition, scope, and system boundary

Sleep vertical foundation models are foundation-model systems trained, continually pretrained, instruction-tuned, or knowledge-augmented for sleep medicine, sleep health, sleep research, or related operational settings. They may process text alone or connect to polysomnography signals, wearable time series, questionnaires, clinical records, audio, images, and environmental data. Possible tasks include knowledge retrieval, literature synthesis, report drafting, coding support, research-cohort screening, signal interpretation, and reusable representations for several downstream tasks. “Vertical” means that domain data, language, tasks, and governance have been deliberately adapted. “Large” has no universal parameter threshold across suppliers and does not show that the model is clinically better than a smaller task-specific system.

The subject is broader than a chatbot. A deployed system normally combines a base model with domain adaptation, a retrieval collection, tool calls, access control, prompt templates, output validation, and an application interface. Question answering is only one possible application. A generic model connected to a few sleep documents may be better described as a sleep knowledge application than as a validated domain foundation model. Similarly, a model that can summarize a sleep report has not thereby demonstrated an ability to interpret raw signals or make a safe clinical recommendation.

Placement under WSIC 4.4.2 identifies an intelligent-engine category only. It does not confer medical-device registration, diagnostic authority, clinical validity, or regulatory approval. Regulatory status depends on the intended use, target users, input data, claims, autonomy, and how an output affects a health decision in the relevant jurisdiction. The same underlying model can support a low-risk administrative drafting function and a much higher-risk patient-specific decision function. Each configured use case therefore requires its own boundary, evidence, and controls rather than inheriting an unrestricted claim from the base model.

Architecture, data, and domain adaptation

Development routes include training from scratch, continued pretraining of a general foundation model on domain material, supervised instruction tuning, parameter-efficient tuning, preference optimization, retrieval-augmented generation, and connection to external analytical tools. Continued pretraining may improve sleep terminology and document style, but it may also amplify source bias or reduce capabilities learned earlier. Retrieval can provide current and traceable material, yet retrieval failure, stale documents, poor chunking, incorrect ranking, or instructions embedded in a retrieved file can still yield an unsafe answer. A tool can perform calculations, database queries, or signal analysis, but it also introduces permission, parameter, availability, and cascading-error risks.

These components require separate configuration records. A meaningful release identifies the base-model build, tokenizer, adaptation data, tuning method, system prompt, inference settings, retrieval index and date, embedding and ranking components, connected tools, guardrails, and user interface. A single marketing version can conceal a consequential change in any of them. Replacing the base model while keeping the application name, refreshing the knowledge store, changing a prompt, quantizing weights, or updating an external application programming interface may alter behavior enough to require new testing.

Sleep data differ markedly across time, equipment, and setting. Polysomnography can include electroencephalography, electrooculography, electromyography, airflow, respiratory effort, oxygen saturation, electrocardiography, position, and other channels. A wearable may provide only movement, heart rate, or an optical signal. Questionnaires and clinical notes occupy different semantic levels again. Documentation should state sampling rates, sensors and placements, hardware and firmware, time synchronization, filtering, artifact handling, missing channels, scoring rules, label provenance, scorer qualifications, and inter-rater agreement. A model should not silently treat consumer-device estimates as interchangeable with laboratory measurements.

Data leakage is a central evaluation threat. Different nights from one person, epochs derived from the same recording, duplicated reports, or a report and the signals it interprets must not be distributed across development and test sets in a way that exposes the answer. Automatically generated labels should not be presented as independent expert ground truth. Patient-level separation is a minimum; site, device, geography, and calendar-time separation may also be needed to estimate transportability. Randomly splitting millions of short epochs can create an impressive test score while measuring recognition of the same people, devices, or recording conditions rather than general sleep-domain performance.

Task design and development lifecycle

Development begins with a defined use context, not with a model in search of a problem. Knowledge retrieval, patient-education drafts, clinic-note summarization, sleep staging, candidate respiratory-event marking, risk stratification, and treatment suggestions have different hazards and need different data, metrics, and human controls. Each task specification should define eligible inputs, intended users, permitted outputs, prohibited outputs, foreseeable misuse, consequences of failure, and an escalation path. A model serving the public, researchers, technicians, and clinicians should use role-appropriate workflows and permissions; the same sentence can have very different consequences for an untrained user and a specialist reviewing primary evidence.

Data governance covers acquisition, licensing, consent or another lawful basis, inclusion and exclusion rules, cleaning, de-duplication, personal-information handling, labeling, adjudication, versioning, and retirement. Dataset documentation should disclose where data came from, what populations and settings are absent, how labels were created, and which preprocessing could affect interpretation. Training corpora, tuning sets, development sets, locked tests, and retrieval collections need controlled separation. Benchmark questions and their solutions can appear in public pretraining data, so a high score without contamination analysis may reflect memorization rather than domain reasoning.

A reproducible deployment package freezes or records model weights, tokenizer, system instructions, retrieval snapshot, tool contracts, inference parameters, post-processing, and safety policies. It also records the software and hardware environment needed to reproduce relevant outputs. Release criteria should include intended performance, safety tests, unresolved limitations, rollback readiness, and responsible sign-off. Change control should classify an update by potential impact instead of assuming it is harmless. New source material, preference tuning, compression, longer context, a changed refusal policy, or a new tool can create regression even when average benchmark performance improves.

The lifecycle includes decommissioning. A supplier or deploying organization needs a plan for version support, knowledge expiration, export of audit records, migration to another system, revocation of credentials, and deletion or lawful retention of data. If a model is withdrawn, users should know which prior outputs remain in the record and how those outputs will be interpreted. A foundation model is not a static publication: operational controls, documentation, monitoring, and retirement are part of the product, particularly when people may act on health-related output.

Evaluation, clinical validation, and evidence interpretation

Evaluation must follow the specified use. Classification and prediction tasks may require sensitivity, specificity, positive and negative predictive values, receiver-operating and precision-recall characteristics, calibration, decision utility, and error costs. A single average accuracy hides class imbalance and can look favorable when the clinically important event is rare. Thresholds should be selected for the intended workflow and validated rather than tuned on the final test set. Confidence estimates need calibration; an interface should not present an uncalibrated model score as a reliable probability.

Generative tasks require additional tests for factual correctness, completeness, whether cited sources actually support each assertion, temporal currency, uncertainty communication, appropriate refusal, crisis escalation, prompt robustness, and consistency across paraphrases and languages. Exact word overlap is usually an inadequate safety measure for a clinical summary. Human evaluation should use predefined rubrics, representative and adversarial cases, masking where feasible, more than one qualified reviewer, and agreement analysis. Hand-picked demonstrations, medical-exam scores, or favorable conversations chosen by the developer do not substitute for a locked evaluation protocol.

Internal testing should be followed by external validation across sites, devices, populations, and time. Performance should be reported for relevant age, sex, racial or ethnic, language, comorbidity, disability, device, and care-setting subgroups, with uncertainty intervals and sample sizes. A subgroup claim based on a handful of cases is not reassuring. Poor data quality may be distributed unevenly: sensors may fail differently across skin characteristics, body habitus, movement patterns, home environments, or device placements. Fairness assessment therefore examines data capture, labels, model errors, and downstream consequences, not merely equal representation in a table.

Agreement with an expert on sleep stages or candidate events does not prove improved patient outcomes. A fluent report is not necessarily a safe recommendation. For clinical decision support, the evidence chain should address a valid clinical association, analytical or technical validation, clinical performance, and the effect in the real workflow. Studies should compare clinicians or technicians without the model, the model alone where ethically appropriate, and the human-model combination. They should measure time, automation bias, correction behavior, missed events, unnecessary follow-up, user comprehension, and patient-relevant outcomes rather than only model speed.

Independent review is most credible when evaluators can access the relevant protocol, model and data version, failure examples, and prespecified analysis. Retrospective data can support early validation but may not predict prospective use, where users adapt their behavior to the system. A silent or shadow deployment can reveal integration problems before outputs influence care. Prospective evidence may be required when the system changes decisions or manages a serious condition. Claims should remain no broader than the populations, sites, devices, languages, and tasks actually evaluated.

Deployment, human oversight, and operational safety

The interface should identify output as model-generated, show the source and time of supporting information, state the intended scope, and make important uncertainty visible. Users should be able to inspect original signals, records, or references rather than receiving only a polished conclusion. Citations must resolve to the cited version and support the associated claim; the presence of a reference list is not evidence that the answer is grounded. Where the model transforms a record, the interface should distinguish observed facts, model inferences, and user-entered corrections.

“Human in the loop” is a real control only if the reviewer has appropriate expertise, sufficient evidence and time, authority to reject the output, and a clear responsibility boundary. Excessive alerts, one-click copying, productivity pressure, or repeated exposure to apparently accurate output can turn review into a ceremonial signature. High-consequence outputs may need deterministic safety rules, a second review, restricted permissions, or escalation to an appropriate service. A model should not independently handle suspected breathing pauses, severe sleepiness with driving risk, medication changes, self-harm content, or another urgent condition merely because its response sounds cautious.

Runtime monitoring should cover input distributions, missing channels, out-of-range values, latency, failures, invalid citations, refusal and escalation rates, human overrides, subgroup performance, complaints, and safety incidents. Data drift can follow a new device, altered referral patterns, revised scoring criteria, new clinical guidance, or a change in how users phrase requests. Pre-release validation becomes stale if these changes are not detected. Monitoring thresholds should lead to defined actions such as investigation, restricted use, rollback, retraining, or suspension rather than producing dashboards with no owner.

Appropriate audit records identify model and knowledge versions, relevant input and output, tool calls, post-processing, user action, and corrections while observing data minimization and retention limits. Serious errors require a practiced response: contain the function, preserve evidence, assess affected people, communicate where necessary, determine root cause, validate remediation, and update risk controls. A static model card is useful but cannot replace this operating system. Continuity planning should also cover service outage, network isolation, dependency failure, and the safe return to a manual workflow.

Privacy, security, equity, and accountable governance

Sleep data can reveal health conditions, daily schedules, location, household relationships, and work patterns and should be treated as highly sensitive where applicable. Collection and use need a defined purpose and lawful basis, minimum necessary fields, role-based access, retention limits, cross-border review, and controls on secondary use. Removing a name does not necessarily anonymize a record. Timestamps, rare conditions, free text, voice, facial imagery, device identifiers, and linkage to other datasets can make a person identifiable. Training, retrieval, logging, evaluation, and user-feedback data require distinct decisions; clinical input should not be reused for training by default.

People should be told where their data go, which processors or subprocessors can access them, whether a remote model service retains prompts, and how access, correction, deletion, or objection requests are handled where those rights apply. Contracts should address incident notice, government requests, location of processing, deletion verification, model-training restrictions, and data return at termination. Local deployment can reduce some transfers but does not itself provide privacy if permissions, logs, backups, or administrator access are poorly controlled.

Security testing should include direct and indirect prompt injection, poisoned retrieval content, excessive tool authority, malicious files, model extraction, membership inference, memorized-data reproduction, vulnerable dependencies, denial of service, and misuse by an authorized account. Instructions inside a retrieved report or web page must not outrank the system policy. Tools need least privilege, allowlisted operations, parameter validation, output constraints, network segmentation, and confirmation for consequential actions. Content filters alone do not protect a system that can query a record, send a message, or modify a workflow.

Equity requires examining who is missing, whose data are noisier, whose labels reflect unequal access to diagnosis, and who bears the cost of false reassurance or unnecessary referral. Translation quality and health literacy matter as much as literal grammatical fluency. Governance should include sleep-domain professionals, data science, privacy and security, legal and ethics expertise, frontline users, and patient or community representation. Named owners should be accountable for the product, clinical safety, data, cybersecurity, incident handling, and supplier management. Accountability cannot be delegated to “the model,” and a vendor contract does not remove the deploying organization's duties.

Procurement, regulation, and sustainable development

Procurement should request a model card, data documentation, precise intended and prohibited uses, external validation, subgroup results, known failures, update policy, service levels, data-flow diagrams, security testing, incident notification, audit access, exit assistance, and data portability. Fluent demonstrations, parameter counts, broad leaderboards, or medical-examination scores do not establish local sleep-task performance. A trial should first compare the existing workflow in silent or shadow mode, define success and stopping criteria in advance, and then proceed to limited deployment if justified. Total cost includes review time, false positives and false negatives, integration, monitoring, interface maintenance, compute, energy, staff training, and vendor lock-in, not just the price per call.

Regulatory classification follows intended use rather than the words “foundation model.” Internal literature search or administrative drafting generally differs from a system that analyzes physiological signals, provides patient-specific diagnosis, or drives treatment. The latter may fall within medical-device software, clinical decision-support, or high-risk AI regimes. Requirements vary by jurisdiction and can cover quality management, technical documentation, clinical evaluation, cybersecurity, human oversight, transparency, post-market monitoring, and controlled changes. A general model's availability does not authorize every medical configuration built on it.

Public generative services may also trigger rules on lawful model and data sources, intellectual property, personal information, content governance, generated-content labeling, security assessment, or algorithm filing. A filing number, cybersecurity certificate, health-IT certification, or quality-system certificate supports only its stated scope; none is automatically evidence of clinical effectiveness. Buyers should verify the exact model, version, operator, region, and function to which a credential applies.

Sustainable development does not mean replacing an expert system with one opaque model. It means maintaining a modular infrastructure in which validated signal models, curated knowledge, deterministic calculations, and generative interfaces are used only for tasks they can support. Outputs remain traceable, changes testable, failures reversible, and accountable people remain in control. A sleep vertical foundation model is most valuable when it reduces well-defined burdens while preserving clinical judgment, patient autonomy, privacy, and the ability to stop using the system when evidence no longer supports it.