Project dashboards, requirements databases, sprint boards, and status reports attempt to answer this question. Yet in complex Software as a Medical Device development, these tools frequently describe the state of the project rather than the state of the product.
A requirement may be marked as implemented even though its associated risk control has not been fully integrated. A software story may be closed because the core application logic is complete, while the user workflow, data handling, exception management, or verification evidence remains unfinished. Conversely, functionality may already exist in the codebase but remain disconnected from requirements, tests, or planning artifacts.
These discrepancies are often discovered during system integration, formal verification, or design reviews – precisely when correcting them becomes more expensive.
Large language models (LLMs) integrated into development environments create an opportunity to reduce this gap. Their most important contribution to regulated software development is not just generating code faster. It may be helping engineering teams understand what has actually been built, what remains incomplete, and where implementation has drifted from documented intent.
The Difference Between Reported Progress and Product Readiness
Most planning tools rely on human-reported status. Developers update work items. Systems engineers maintain requirements. Test engineers record verification results. Project leaders consolidate these inputs into reports and dashboards. Each activity is necessary, but the resulting view is only as accurate as the synchronization between the underlying artifacts.
In a large SaMD program, relevant information may be distributed across:
- Requirements and application lifecycle management systems
- Source-code repositories
- Architecture and design documents
- Risk management files
- Automated test frameworks
- Defect-tracking systems
- Continuous integration pipelines
- Agile planning tools
No single artifact provides a complete picture of product readiness. The source code may show that a capability exists, but not whether it was implemented in accordance with its requirements. The requirements system may show that a requirement is approved, but not whether every supporting code path is complete. A test-management system may show that test cases passed, but not whether recent code changes invalidated the assumptions behind them. As the product becomes more complex, manually reconciling these views becomes increasingly difficult.
Moving From Code Generation to Implementation Awareness
Current discussions about generative AI in software development frequently center on productivity: generating code, explaining functions, creating unit tests, or accelerating debugging.
Those applications are useful, but they represent only one layer of potential value. LLMs that can analyze repositories, interfaces, dependencies, commit histories, test results, and linked work items may also help teams answer higher-level questions:
- Which parts of a clinical workflow are implemented?
- Which intended behaviors are only partially supported?
- Are there placeholder functions, mocked integrations, or incomplete error paths?
- Has a risk control been implemented across every relevant execution pathway?
- Does the verification evidence correspond to the latest code state?
- Which requirements appear to have code changes without updated tests?
- Which closed work items still have unresolved implementation dependencies?
This capability can be described as implementation awareness: the ability to evaluate the actual software state relative to intended system behavior.
It does not replace formal traceability, verification, or engineering judgment. Instead, it provides another source of evidence that can help teams identify inconsistencies earlier.
A SaMD Example: When “Done” Is Not Complete
Consider a hypothetical remote cardiac-monitoring application designed to analyze electrocardiogram data and notify clinicians when a potentially significant arrhythmia is detected. At the planning level, the capability may appear as a single feature: Detect a suspected arrhythmia and present the event to a clinician for review.
In implementation, however, that capability may depend on multiple elements:
- Receiving ECG data from a connected monitoring device
- Confirming data completeness and signal quality
- Preprocessing the signal before analysis
- Running an arrhythmia-detection algorithm
- Assigning an event classification or confidence score
- Applying alert-prioritization rules
- Presenting the event in the clinician dashboard
- Allowing the clinician to review, annotate, or dismiss the event
- Handling delayed, duplicated, or corrupted data
- Recording the detection and review history
- Verifying that safety-related controls operate as intended
Suppose the signal-processing pipeline, algorithm, and primary dashboard are implemented. Most development tasks are closed, and the feature appears nearly complete in the project plan.
An LLM-assisted engineering agent examining the code, interfaces, test evidence, and linked requirements might reveal a different picture.
For example, it could identify that:
- Signal-quality checks are performed during data ingestion but are bypassed during manual file reprocessing
- The clinician interface displays a confidence score, but one code path still uses a default value
- A failure in the notification service is logged but is not surfaced to the user
- Automated tests cover nominal arrhythmia detection but not degraded-signal conditions
- A risk-control requirement addressing delayed clinician notification is linked to the feature but not to a completed verification test
The capability is not merely “80% complete.” More importantly, it contains specific gaps that affect safety, traceability, and clinical usability. This is the type of distinction implementation awareness can provide.
Why Vertical Slicing Makes AI-Based Assessment More Useful
LLM-based analysis becomes more meaningful when work is structured around coherent system capabilities rather than isolated technical components.
Traditional decomposition often separates work into categories such as frontend, backend, database, algorithm, and testing. This may make resource assignment easier, but it can obscure whether a usable, safe, and testable capability exists across the full system.
Vertical slicing offers a different structure. A vertical slice represents an end-to-end unit of functionality that spans the layers required to deliver observable product behavior. In SaMD, that may include:
- The user or clinical workflow
- Application logic
- Algorithms
- Data management
- External interfaces
- Error handling
- Risk controls
- Verification activities
When planning artifacts are organized around vertical slices, an AI system has a clearer unit of intent against which to evaluate the implementation.
Instead of determining whether individual components contain code, the system can assess whether the complete capability exists across the necessary layers.
In the cardiac-monitoring example, the relevant slice is not simply “arrhythmia algorithm” or “clinician dashboard.” It is the full pathway from receipt of ECG data through detection, prioritization, presentation, clinician review, and event recording.
This structure also creates a stronger connection between product planning and systems engineering. A vertical slice can link clinical intent, software implementation, risk controls, and verification evidence within a common functional boundary.
What an AI-Assisted Workflow Could Look Like
A practical implementation could begin with a limited, advisory workflow rather than allowing an AI agent to modify controlled records automatically.
First, the organization defines a functional slice and its associated evidence. The slice might include requirements, acceptance criteria, architecture elements, risk controls, repositories, interfaces, and verification tests.
The AI system then examines the available implementation artifacts and builds a map between intent and evidence. It may identify relevant source files, pull requests, tests, interface definitions, and traceability links.
Rather than assigning a single completion percentage, the system produces an evidence-based assessment. For example:
- Core detection logic appears implemented
- Data-quality handling is inconsistent across ingestion pathways
- One alert-failure scenario lacks corresponding user feedback
- A linked risk control has no completed verification result
- Automated tests exist but do not cover the latest algorithm change
- The end-to-end clinical workflow cannot yet be demonstrated under degraded-network conditions
The system could then post these findings to the planning environment, generate a review summary, or flag the slice for human evaluation.
Improving Design Reviews and Integration Readiness
Implementation awareness could strengthen several existing SaMD activities.
Design reviews
Before a formal review, an AI-assisted analysis could identify discrepancies between approved requirements, implementation artifacts, risk controls, and verification status. This would allow reviewers to focus on unresolved technical questions rather than spending review time locating missing information.
Integration planning
Teams could identify features that appear locally complete but remain disconnected from upstream or downstream components. This is particularly valuable when data ingestion, algorithms, cloud services, user interfaces, and external notifications are developed by different teams.
Risk-control implementation
An AI system could help locate the code paths associated with a risk control, identify whether the control is consistently applied, and highlight execution paths where the expected behavior is absent. Any conclusion would still require review by qualified engineers.
Verification readiness
Before formal testing begins, the system could compare the current implementation with test coverage, requirements, and known code changes. This may reveal missing test conditions, obsolete test assumptions, or requirements without corresponding evidence.
Change-impact analysis
When a developer modifies an interface, shared service, signal-processing module, or clinical algorithm, the system could identify affected capabilities, tests, requirements, and risk controls. This would not replace formal change assessment, but it could make the initial analysis more comprehensive.
Planning Tools Should Reflect Evidence, Not Replace It
An AI agent could eventually help keep project plans synchronized with the implementation. However, allowing it to automatically move work items to “done” would be an oversimplification.
In SaMD, completion is multidimensional.
Code may be complete while verification remains open. Verification may be complete while documentation updates are pending. A risk control may be implemented but not yet reviewed. An individual component may function correctly while the end-to-end workflow remains unavailable.
A more useful model would allow the system to display separate indicators for:
- Implementation completeness
- Integration completeness
- Requirements traceability
- Risk-control coverage
- Verification readiness
- Documentation status
This provides engineering and program leaders with a more honest view than a single status field or completion percentage.
From Static Traceability to a Living System View
Traditional traceability establishes relationships among requirements, design, implementation, risks, and tests. Those relationships remain foundational to regulated development.
However, traceability is often evaluated as a static structure: the links exist, the records are present, and the required fields are complete.
Implementation-aware AI creates the possibility of a more dynamic view.
The system could continuously ask whether the linked artifacts remain consistent with one another:
- Does the current code still implement the requirement?
- Does the test still exercise the relevant behavior?
- Has an architectural change introduced a new dependency?
- Is the documented risk control present in every relevant code path?
This moves organizations from maintaining a traceability network toward maintaining a living representation of the product state.
Looking Ahead
The value of AI in SaMD development should not be measured only by how quickly it generates code. For complex and safety-critical software, a more consequential application may be helping teams understand the relationship between clinical intent, implementation, risk, and verification.
The long-term opportunity is not a planning system that updates itself without human involvement. It is an engineering environment that continuously compares what the organization intended to build with what the evidence shows has actually been built—and helps teams resolve the difference before it becomes a late-stage product risk.
The post Beyond the Status Report: Using LLMs to Reveal the True State of SaMD Development appeared first on MedTech Intelligence.
