Excess outage isolates the pointing penalty
The paper's headline numbers are not total link downtime. The authors first define an outage as any moment when the received signal-to-noise ratio drops below 10 dB, then subtract the outage duration a perfectly aligned system would still suffer at the same depth. What remains is the excess outage: the fraction of simulation time where connectivity is lost exclusively because the beam missed the receiver aperture.
That distinction matters because depth-dependent attenuation, turbulence and noise degrade the link regardless of how well the transmitter aims. By benchmarking each episode against perfect tracking, the metric attributes only the avoidable, pointing-driven portion of the loss to the learning agent. Under current-free conditions the mean excess outage is 8.5%, which the authors translate into roughly 0.8 s of pointing-caused signal loss inside each 10-second episode. With current-induced drift, the same metric rises to 16.2%.
A depth-resolved 3D channel replaces flat assumptions
Earlier reinforcement learning work on underwater optical pointing largely used 2D spatial models or treated the water column as homogeneous, which cannot represent how attenuation, turbulence and noise change with depth. This paper builds a 3D Cartesian downlink between a transmitter AUV near the surface and a mobile receiver AUV, with depth measured positively downward and the receiver following discrete waypoints.
The channel loss is decomposed into attenuation, turbulence, geometric spreading and pointing loss. Attenuation follows the Beer-Lambert law with absorption and scattering coefficients from the Haltrin model, driven by a Gaussian vertical chlorophyll profile that can represent pure sea water, clear ocean, coastal or harbor conditions. Turbulence is modeled as a cascade of discrete depth layers, each with its own scintillation index, combined into a single lognormal fading coefficient. Pointing and geometric loss are computed by tracing a set of discrete rays that reproduce a Gaussian intensity profile and counting how many land inside the receiver aperture.
The agent tunes three angles across 27 discrete actions
The controller does not choose a continuous beam direction. It selects from 27 discrete actions, each a combination of increasing, holding or decreasing three parameters: the polar steering angle, the azimuthal steering angle and the divergence angle. Steering steps are 10 mrad and divergence steps are 1 mrad, so the policy trades coarse pointing moves against finer beam-width adjustments.
The reward function is the ratio of instantaneous SNR to the maximum achievable SNR at that depth, which removes the depth-dependent attenuation that the agent cannot control. If the misalignment distance exceeds the sum of the beam radius and the receiver aperture radius, the receiver captures no optical power and the agent receives a penalty of −1. Because the aperture radius is fixed hardware while the beam radius is agent-controlled, widening the beam is the policy's fallback for keeping the link alive.
Without currents, tracking stays close to the planned path
Training used 15,000 episodes for the current-free scenario and 30,000 episodes for the current-induced case, with each 10-second episode split into 100 decision steps of 0.1 s. Receiver trajectories were generated as interpolated 3D random walks bounded between 1 and 50 m in depth, with simulated AUV velocities from 0.6 to 1.2 m/s.
In a representative current-free test episode, the beam-steered path closely follows the receiver trajectory, with minor oscillations attributed to the discrete-step decision process. Across 1,500 unseen episodes, the excess outage histogram is positively skewed with its main peak at the lower bound, meaning many episodes incur no tracking-induced signal loss at all. The 8.5% mean is therefore an average over a distribution that includes both near-perfect and worse-than-average episodes, not a typical per-episode value.
Under drift, the agent widens the beam instead of chasing precisely
When ocean currents perturb the receiver, its actual path deviates from the planned waypoints. The agent still tracks the drifting target, which the authors present as evidence that environmental data was successfully integrated into the policy. Because continuous drift progressively pushes the beam off the aperture, the agent autonomously increases beam divergence, producing a more diffuse beam over time.
A third test episode shows the opposite behavior when alignment is good. At 2.5 s the misalignment stays well inside the receiver aperture, and the agent narrows the beam to maximize received power, reaching −21.82 dBm. After about 3.5 s, tracking stability deteriorates and the agent switches back to widening divergence to preserve the link. The policy is thus not a fixed beam width but a conditional trade-off between concentration and coverage.
Environmental history is what separates the two scenarios
Both scenarios share the same action space, reward function and channel model; they differ only in the state vector. The current-free agent observes 11 tracking variables, including receiver position at the current and previous step in spherical coordinates, received optical power at both steps, and its own beam parameters.
For the drift scenario the observation space expands to 24 continuous variables by adding 13 environmental components: receiver depth, sea surface elevation, cyclical day and hour encodings, water velocity from 3 and 24 hours earlier, water temperature and salinity from 3 hours prior, and the episode time step. These come from Hybrid Coordinate Ocean Model data. The two branches are processed by separate towers of a two-tower multilayer perceptron, which the authors describe as decoupling beam dynamics from environmental uncertainty. The jump from 8.5% to 16.2% excess outage reflects this harder task: the agent must simultaneously control pointing, estimate current-induced drift and predict where the receiver actually is.
The 8.5% and 16.2% figures are simulation results
Both excess outage values come from simulated episodes, not from at-sea trials. The evaluation covers 1,500 unseen trajectories generated by 3D random walks, with ocean current effects derived from real oceanographic data but the link itself modeled rather than measured. The scenario assumes both AUVs know their starting positions and planned waypoints, each carries an inertial measurement unit for position and velocity estimation, and the vehicles exchange state information over a bidirectional optical link.
The channel configuration reported in the shared simulation parameters uses a chlorophyll concentration of 0.005, a 530 nm wavelength and a 0.2 m receiver aperture, which corresponds to the pure sea water setting rather than the clear ocean, coastal or harbor profiles the model can also represent. Readers should treat the two outage percentages as bounds for this simulated configuration and these tracking conditions, not as field-validated performance guarantees for arbitrary water types or vehicle speeds.
The gap between 8.5% and 16.2% is the real finding
The most decision-relevant result is not either number alone but the difference between them. Adding unknown current-induced drift roughly doubles the pointing-attributable outage, from 8.5% to 16.2%, even though the agent gains 13 environmental observations and a dedicated processing branch to cope with it. That gap quantifies how much of the alignment problem remains unsolved once the environment stops cooperating, and it is the figure to watch when comparing this framework against alternatives.
The paper's own framing supports reading the result as a baseline policy rather than a solved problem: the authors state that the agent learned a baseline policy to estimate oceanic drift and preserve link reliability by simultaneously controlling beam direction and divergence. For readers weighing beam-control approaches, the practical threshold is therefore whether a method can hold excess outage near the current-free 8.5% level when drift is present, since that is where this framework's margin is thinnest.
Sources used in this answer
Reinforcement Learning-Based 3D Beam Adaptation for Underwater Wireless Optical Communication with AUVs
A deep reinforcement learning agent jointly controlling beam steering and divergence limited mean excess outage to 8.5% under current-free tracking and 16.2% under current-induced drift, measured against a perfect-alignment baseline across 1,500 unseen simulated episodes.