Copyright Notice: 2026 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

This work has been supported by the SNS JU project 6G-DISAC under the EU's Horizon Europe research and innovation program under Grant Agreement No 101139130.

Abstract

We present a methodology that places a 3D digital twin (DT) of the environment as the main enabler behind the development of radio sensing at scale. The DT acts as a world model, providing geometry, materials, and transmitter/receiver placements to a ray-tracing engine that generates time-indexed channel impulse responses (CIRs) for large numbers of plausible scenes (moving people and objects, layout variants, seasonal/weather conditions, etc). From these synthetic sequences, we train a sequential neural network that maps CIR time series to spatial occupancy estimates, enabling device-free localization (DFL) without instrumented targets. We posit that sensing is best approached as an environment-conditioned learning problem: rather than seeking a single global model, we advocate training or fine-tuning local models specialized to a site-specific DT. As a first experiment, we introduce a novel State Space Model architecture, trained and evaluated across multiple room geometries. The localization performances obtained demonstrate the potential of the approach.

1 Introduction

Telecommunication networks continuously illuminate the environment with radio waves. Beyond their primary role of carrying information, these waves bear rich signatures of geometry, materials, motion, and occupancy imprinted by their propagation through the environment. Integrated sensing and communication (ISAC) exploits this duality to enable sensing tasks, such as device-free localization (DFL), human activity monitoring, and scene interpretation, without deploying dedicated radar hardware or requiring targets to carry active devices [1]. For network operators with already-deployed sub-6 GHz infrastructure, ISAC offers the prospect of new services at marginal cost [2].

In modern systems (Wi‑Fi, 5G), channel estimates are commonly available in the form of channel state information (CSI) or channel impulse responses (CIR). While traditional statistical channel models used in simulation describe average or aggregate propagation behavior, the realized channel in a given deployment is a fingerprint of its environment: multipath components arising from reflection, scattering, and diffraction depend on object placement, materials, and motion. Therefore, leveraging CSI/CIR for radio sensing to extract local environmental information is a natural and broadly applicable strategy.

Early device-free systems used received signal strength indicator (RSSI) to detect presence and track intruders in Wi‑Fi environments, demonstrating feasibility but suffering from low spatial resolution and sensitivity to environmental changes [3]. Subsequent work shifted to finer channel descriptors CSI/CIR to capture per‑subcarrier amplitude/phase, enabling subtle motions detection, such as respiration, and through‑wall sensing [4], [5]. More recently, radio frequency (RF) sensing has been formulated as a data-driven learning problem on CSI/CIR, with deep models, autoencoders, and attention-augmented hybrids, achieving strong performance in activity/gesture recognition and fingerprinting/localization, with surveys reporting improved robustness and cross-environment transfer [6], [7], [8], [9]. However, most of these approaches face practical limitations preventing scalable deployment:

• Analytical models (e.g., Fresnel zones, line-of-sight/free-space assumptions) overly simplify actual multipath propagation and degrade in cluttered, dynamic environments, necessitating data-driven approaches.

• Data-driven models require training data that varies significantly across environments due to differences in local geometry, materials, and scenarios. CSI/CIR statistics are strongly conditioned on these factors, making broad out-of-domain generalization unlikely and rendering a single global model that works “everywhere" elusive, thus demanding site-specific training or fine-tuning.

• Data collection through site-specific measurement campaigns, as in fingerprinting approaches, becomes prohibitively labor-intensive when scaling across multiple sites, and require repeated data collection and retraining as layouts and usage patterns evolve.

Consequently, we advocate for a digital-twin (DT) in-the-loop methodology: for each site of interest, instantiate a 3D DT with the known static geometry and associated materials; render physics-based radio propagation to synthesize CIR sequences under diverse, plausible scenarios (occupancy, trajectories, furniture layouts, seasonal factors, etc.); and train sequential models that map CIR time series to spatial occupancy. The DT yields de facto labeled data at scale and preserves the site-specific multipath structure. One approach would be to train a coarse general model across multiple DTs, then fine-tune it locally for each specific site.

Related DT‑in‑the‑loop sensing approaches place the RF simulator itself inside a differentiable inference loop, using inverse or differentiable ray tracing to backpropagate through the DT and directly optimize scene or pose parameters from CSI/CIR. For example, InverTwin addresses path‑space discontinuities and RF phase periodicity via tailored differentiation and surrogate models, enabling stable gradient‑based updates of DT parameters, while RayLoc performs fully differentiable wireless ray tracing for indoor localization with loss smoothing to mitigate sparse or unstable gradients [10], [11]. In spirit, these methods and ours share the idea of keeping a site‑specific DT in the loop; however, our approach is learning‑first: we use the DT primarily as a generator of labeled, site‑specific CIR sequences and train DT‑conditioned sequential models that exploit temporal consistency and environmental context to infer probabilistic occupancy and positions, rather than optimizing the DT or performing gradient‑based inverse rendering at test time.

The remainder of the paper details this methodology and presents a partial proof-of-concept (PoC) application, with emphasis on synthetic data generation and model training, while deferring real-world deployment evaluation to future work. In Section 2, we present the complete DT-in-the-loop methodology for site-specific sensing. In Section 3, we propose our PoC where we detail the DT-based synthetic data generation pipeline, as well as model architecture and training aspects. In Section 4, we evaluate the trained models on the proposed scenarios. Section 5 discusses limitations and future deployment considerations.

2 General Digital-Twin-in-the-Loop Framework

We approach radio sensing as an environment-conditioned sequence learning problem: a site-specific DT, containing the 3D geometry and known RF infrastructure (e.g., access points (APs), localized user equipments (UEs) and their antenna poses/patterns), is paired with a radio sensing model based on machine learning. During training, the model consumes synthetic, time-indexed CIRs rendered from the DT; during deployment, it consumes measured CIRs associated with the current DT state. The model maps CIR sequences to probabilistic occupancy/position fields, thereby updating the DT’s dynamic state. This leverages structure beyond fundamental resolution limits: although frequency and bandwidth bound theoretical resolution, targets move continuously (no “teleportation”) and follow place-specific constraints (e.g., pedestrians on sidewalks, vehicles on roads). Processing environment-conditioned, continuous-time sequences enables the model to exploit the DT site-specific context and spatial constraints.

This framework is now practically feasible due to three key technological enablers: (i) mature 3D tooling and content sources for site DTs (3D mobile LIDAR capture, OpenStreetMap, Building Information Modeling, etc.), (ii) GPU-accelerated ray-tracing frameworks for large-scale CIR synthesis, and (iii) commodity hardware/software for training deep sequence models. Together, these make it practical to author site DTs, generate labeled datasets at scale, and deploy per-site models.

2.1 Training: Synthetic Data Generation using a Digital Twin

During training, the DT is initialized with static geometry and RF infrastructure. Plausible dynamics (e.g., trajectories of people and vehicles, door schedules, layout variants, seasonal foliage) are sampled, and the corresponding CIR streams are rendered by an RF simulator (e.g., ray tracing). A sequence model is trained to map CIR time series to probabilistic outputs (e.g., 2D/3D occupancy heatmaps). Ground-truth occupancy/positions are read directly from the DT state, avoiding manual annotation. The same framework can be extended to predict additional attributes (e.g., material or object type). Because learning is conditioned on a site’s channel and occupancy distributions, broad zero-shot transfer to new environments is not expected, and few-shot fine-tuning is likely needed. This approach can be extended to 3D scene modeling (voxel occupancy) to feed and refine the DT [12].

2.2 Deployment: Continuous Sensing Loop

Per-site models run with their DTs for continuous monitoring. The DT maintains both static and dynamic occupancies: tracks are updated in real time from sensing, while the DT also supplies priors for sensing. Dedicated anchor nodes (UEs with known pose in the DT) can be deployed to improve sensing performance in specific locations and can also be used during training. As targets approach DT borders, predicted positions hand over to neighboring tiles, and spatial priors help maintain consistency at scale [13]. Crucially, we view the DT not as a one-time offline training resource but as a continuously updated world model: real-world sensing refines the DT, which in turn improves future predictions.

3 Proof-of-Concept Implementation

Building on the methodology outlined above, this section details our PoC implementation: we describe the considered DT environments, formalize the CIR‑based occupancy inference problem, and present the dataset construction, model design, and evaluation metrics. Evaluation on real-world deployment CIR data will be addressed in follow-up studies.

3.1 Digital Twin Environments

Two synthetic environments, presented in Figure 1, are considered. They are purpose‑built test rooms rather than reconstructions of specific real spaces. The rooms are described as 3D meshes with coarse material tags (e.g., concrete, wood).

Per room, we define a fixed couple of transmitter and receiver. During training, the receiver observes sequential complex baseband CIR of length \(K\) taps, producing a sequence of length \(T\) CIRs, \(\mathcal{H}_{1:T} = [h_1,\dots,h_T]\) with \(h_t \in \mathbb{C}^{K}\). To isolate the learning problem, we adopt idealized measurements: no additive noise is injected, so the CIRs are the direct outputs of the Sionna Ray‑Tracing (Sionna RT) engine [14].

In the current study, the task is to localize characters moving in the room by mapping motion‑induced multipath perturbations to occupancy. From each CIR time sequence, a model \(I_\varphi\) predicts a probabilistic occupancy heatmap on a uniform \(H\times W\) discrete floor grid at time \(t\), \(\tilde{y}_t \in [0,1]^{H\times W}\), allowing multiple simultaneous occupants. Ground‑truth maps are rasterized from the DT state at time \(t\), yielding the binary ground-truth heatmap \(b_t \in \left\{0,1\right\}^{H \times W}\). Formally, \[I_\varphi : \mathbb{C}^{K \times T} \rightarrow [0,1]^{H \times W \times T}, \quad \tilde{\mathbf{y}} = I_\varphi\bigl(\mathcal{H}_{1:T}\bigr),\]

where the parameters \(\varphi\) are learned by gradient descent to minimize a loss between \(\tilde{y}_t\) and \(b_t\).

3.2 Dataset and Preprocessing

We generate sequences of complex baseband CIRs together with synchronized 2D occupancy maps. The left panel of Figure 2 shows an example ground-truth heatmap for the square room. Characters’ starting locations are random in the room, and they move along straight paths at constant speed. Their positions labels are subsequently rasterized.

To stabilize learning and reduce spurious correlations, we apply two preprocessing steps.

3.2.1 Baseline Removal

For each time step \(t\) and tap \(k\), we subtract the empty-room CIR to suppress static multipath tied to room geometry. This subtraction substantially decreases spatial correlation between CIRs acquired with different occupant positions. Similarly, using sequences rather than single snapshots helps resolve position ambiguities that arise when instantaneous CIRs are highly correlated across different configurations.

3.2.2 RSSI-based Label Blurring

Motivated by the intuition that the model should be less penalized where the object is intrinsically harder to detect (e.g., in low-RSSI regions), we implement RSSI-adaptive label blurring. Specifically, we soften targets with a Gaussian kernel whose width decreases with the received power at the true location (computed as empty-room RSSI), reflecting higher localization confidence where the link is stronger: \[y_t \;=\; b_t * G_{\sigma}, \quad G_{\sigma}[i,j] \;=\; \frac{1}{2\pi\sigma^2}\exp\!\left(-\frac{i^2+j^2}{2\sigma^2}\right),\]

where \(*\) denotes 2D convolution. So high RSSI yields a narrow spread and low RSSI a wider one. Finally, we renormalize to keep targets in \([0,1]\). The empty square room RSSI and a resulting blurred label are shown in Figure 2. The blurred heatmaps are normalized such that \(\sum_{i,j}y_t[i,j]=1\).

3.3 Proposed Model: Structured State-Space Approach

State-space models (SSMs) capture the evolution of a latent state that summarizes past information and drives future predictions. In their classical form, they couple a state update with an observation map. Modern neural SSMs parameterize these operators to learn long-range temporal dependencies efficiently while remaining streamable and memory-frugal. This bridges control-theoretic formulations with deep sequence modeling, enabling stable and scalable processing of long sequences and offering a perspective closely related to, yet often more efficient than, transformers [15].

Our model is a structured state-space sequence model that predicts a spatial heatmap at each time step from radio measurements, optionally conditioned on a global prior (e.g., an RSSI coverage map). As illustrated in Figure 3, two encoder networks map each normalized CIR snapshot \(h_t\) and constant RSSI prior to latent representations, then provided to an attention mechanism. A learned SSM backbone updates the latent state \(s_t\) from \(s_{t-1}\) to modulate the contribution of current evidence while preserving long-range memory. The updated state is decoded by a mixture-of-experts: multiple MLP decoders produce candidate heatmaps, and a mixer blend them into a final predicted distribution over the \(H\times W\) grid.

3.4 Loss and Metrics

3.4.1 Training loss

At each time step \(t\), the model outputs a heatmap \(\tilde{y}_t\) normalized such that \(\sum_{i,j}\tilde{y}_t[i,j]=1\). The target \(y_t\) is the Gaussian-blurred probability map. We minimize the regularized Kullback–Leibler divergence: \[D_{\mathrm{KL}}\!\bigl(\mathbf{y}\| \tilde{\mathbf{y}}\bigr)+\left\|\varphi\right\|_2^2 = \sum_{t=1}^T \sum_{i,j} y_t[i,j]\;\log\!\frac{y_t[i,j]}{\tilde{y}_t[i,j]}+\left\|\varphi\right\|_2^2. \label{eq:loss}\]

3.4.2 Evaluation metrics

3.4.2.1 Cosine Similarity

To assess spatial agreement between the predicted and target heatmaps at timestep \(t\), we use the Cosine Similarity (CS): \[\mathrm{CS}(\tilde{y}_t, y_t) = \frac{\left\langle \tilde{y}_t,\; y_t \right\rangle} {\left\|\tilde{y}_t \right\|_2 \;\left\|y_t\right\|_2}.\]

CS is effective at capturing overall spatial pattern similarity. However, CS is highly sensitive to small spatial misalignments: even a one-grid-cell shift of a sharp peak can severely degrade the score, making it overly harsh for evaluating localization accuracy where small position errors may be practically acceptable.

3.4.2.2 Confidence Circles

Let \(P_t=\{(i_k,j_k): b_t[i_k,j_k]=1\}\) be the set of active cells corresponding to binary ground‑truth heatmap \(b_t\). For each active cell \(p_k=(i_k,j_k)\), we define a circle of radius \(\rho_k\) centered at \(p_k\). The radius \(\rho_k\) is inversely proportional to RSSI at \(p_k\). The union mask is \(M_t[i,j]=\mathbb{1}\bigl(\exists k:\|(i,j)-p_k\|_2\le \rho_k\bigr)\): it equals 1 if there exists at least one circle covering cell \((i,j)\), otherwise 0. The score is the fraction of predicted mass lying inside the union of circles: \[\mathrm{CC}(\tilde{y}_t,M_t)\;=\;\sum_{i,j}\tilde{y}_t[i,j]\; M_t[i,j].\]

Complementary to the CS, this metric measures how predictions concentrate near ground-truth locations, with RSSI adaptive tolerance.

We emphasize that the task considered here is not direct detection or coordinate-level localization, but occupancy heatmap prediction. The predicted heatmap is intended as an intermediate probabilistic representation of spatial occupancy, which could serve as input to subsequent detection or localization stages in a complete sensing pipeline. Consequently, the proposed loss and evaluation metrics have been designed accordingly.

4 Results and Analysis

4.1 Experimental Setup

We compare our SSM model with three baselines of comparable parameter counts (\(\approx10^6\) to \(10^7\) parameters): a Multi-Layer Perceptron (MLP), a Convolutional Neural Network (CNN), and a Vision Transformer (ViT). The MLP consists of fully-connected layers with ReLU activations. The CNN uses 1D convolutions with max-pooling. The ViT employs a two-stage architecture: spatial processing per timestep followed by temporal attention across the sequence.

Each model is trained and evaluated with k-fold cross-validation across two recording sessions, using five folds per session. Each fold is trained for 30 epochs with batch size 128, and the checkpoint with the best validation score is retained. Training samples are sequences of 10 time steps, and we consider scenarios with 1 to 3 moving persons in the room. The combinatorial space of configurations (16×16 positions, 1 to 3 occupants, continuous coordinates, random trajectories) vastly exceeds our total dataset size of 2000 sequences. Successful generalization therefore requires learning temporal dynamics and target-clutter separation rather than memorizing input-output pairs (overfitting).

4.2 Performance Summary

Figure 4 illustrates an example in the U‑shaped room at convergence. Each CIR frame is represented as a two‑channel input (amplitude and phase in \(\mathbb{R}^2\)). The SSM processes a sequence of CIRs and outputs a probability heatmap per time step (right panel). The ground‑truth occupancy (middle panel) indicates three subjects present. As the sequence unfolds, the SSM’s heatmaps become progressively sharper, reflecting the integration of temporal evidence across time steps. This highlights the central role of sequence modeling in our approach.

Table 1 summarizes the results. All scores are computed on the last heatmap. Across both environments, the SSM consistently achieves the best performance for both metrics, followed by the ViT, the CNN and the MLP. This gap suggests that explicit temporal modeling of CIR sequences adds information that purely spatial or shallow temporal models do not fully capture. On the U‑shaped room, scores generally decrease due to increased multipath and non‑LOS conditions, but the SSM maintains its relative advantage. Although larger studies are needed to confirm these findings across a broader range of environments and datasets, the present results identify the SSM and the ViT as particularly promising candidate architectures.

5 Conclusion and Perspectives

We presented a DT-in-the-loop methodology for scalable radio sensing and studied its feasibility through a PoC for DFL. Trained purely on synthetic CIR sequences generated via Sionna RT from 3D environment models, our SSM outperforms MLP, CNN, and ViT baselines across two geometries. While this study is simulation-only and restricted to two synthetic rooms, these results validate the core hypothesis: site-specific models trained on DT-generated data can learn to map CIR temporal sequences to probabilistic occupancy.

The main limitation is that the study remains simulation-only. The key next step is therefore real-world validation, to assess how DT fidelity (geometry precision, material accuracy, antenna modeling, ray tracing quality, hardware impairments) affects sim-to-real transfer. Beyond this, we plan to expand datasets to diverse geometries, occupancy scenarios, and environmental conditions; develop decision layers to convert probabilistic outputs into discrete detections, enabling standard detection metrics (false alarms, missed detections, etc.); produce sensing-accuracy and coverage maps to optimize transmitter placement jointly for communication and sensing objectives; and assess whether models can generalize across environments or require per-site fine-tuning.