Skip to content

Soapy block refactoring — ABI fix, API alignment, TX support#754

Draft
RalphSteinhagen wants to merge 8 commits intomainfrom
soapyRefactoring
Draft

Soapy block refactoring — ABI fix, API alignment, TX support#754
RalphSteinhagen wants to merge 8 commits intomainfrom
soapyRefactoring

Conversation

@RalphSteinhagen
Copy link
Copy Markdown
Member

@RalphSteinhagen RalphSteinhagen commented Mar 23, 2026 •
edited
Loading

Incrementally refactors SoapyBlock to match RTL2832Source quality and patterns. Single PR, independent bisectable commits.

Commits:

  • ABI fix — remove SoapySDR/Device.hpp, C-API only, Device::make() → std::expected, all setters return std::expected, fix GCC argument evaluation order bug, null device segfault, LimeSDR null optionNames crash
  • Namespace + naming — move soapy → sdr, SigMF-aligned field renames (frequency, bandwidth, gain, antenna, channels), Doc<> tags, remove dead code/Doxygen boilerplate, Tensor → std::vector
  • IO thread + error handling — async ioReadLoop() (mirrors RTL2832Source), overflow → tag, stream restart recovery, atomic counters, scheduler progress notification
  • API alignment — 3-param settingsChanged with forwardSettings, timing tags (TRIGGER_NAME/TIME/OFFSET), clk_in port, optional DC blocker + ppm estimator (disabled by default), tag-triggered retune
  • Burst taper algorithm — BurstTaper.hpp in algorithm/: None, Linear, RaisedCosine, Tukey, Gaussian, Mushroom (zero-integral, C1), MushroomSine (zero-integral, C∞)
    image
  • Loopback test infrastructure — vendor SoapyLoopback (~850 SLOC), enhanced for timing/overflow/burst-ACK simulation, rewrite qa_Soapy.cpp for CI without hardware
  • Device registry — process-global SoapyDeviceRegistry with ref-counted shared_ptr for RX/TX handle sharing
  • TX support — SoapySinkBlock with writeStream, timed bursts (HAS_TIME/END_BURST/BURST_ACK), built-in taper, wrapper extensions (clock source, master clock rate, sensors)

Out of scope (separate PR)

  • WebSocket proxy for WASM (tunnel SoapyRemote TCP over WebSocket)

Known external issues (so far)

  • 168-byte leak in SoapySDR module loader (LSAN suppressed)
  • RTL-SDR driver segfaults on PLL lock failure after rapid reopen (mitigated with USB reset)
  • LimeSDR driver returns null optionNames in SoapySDRArgInfo (worked around)
  • TSAN: data race and SEGV inside LimeSuite/LimeSDR driver:
    • data race in lime::LMS64CProtocol::TransferPacket + libusb-1.0.so
    • SEGV in __strlen_avx2 triggered by LimeSuite's internal threading

@RalphSteinhagen RalphSteinhagen force-pushed the soapyRefactoring branch 10 times, most recently from f8df0bf to d710f37 Compare March 30, 2026 20:36
@RalphSteinhagen RalphSteinhagen force-pushed the soapyRefactoring branch 2 times, most recently from 67ea8a3 to 3cc2aaf Compare March 31, 2026 09:02
@gretel
Copy link
Copy Markdown
Contributor

gretel commented Mar 31, 2026

Noticed during hardware testing with singleThreadedBlocking scheduler + USRP B210:

SoapySource::work() always returns {requestedWork, 0, OK} — zero performed work, status OK. The scheduler interprets this as "block is alive, keep polling" and spins through timeout_inactivity_count empty block-list traversals before sleeping. Each time a downstream block makes progress (incrementing the shared progress counter), the inactivity counter resets.
Net effect: the scheduler thread never sleeps, causing ~4x CPU load vs the old synchronous processBulk model where readStream() blocking was the natural throttle.

@gretel
Copy link
Copy Markdown
Contributor

gretel commented Mar 31, 2026 •
edited
Loading

SoapySink IO write thread idles when no TX data is available (normal for burst-mode TX). The scheduler watchdog prints trigger watchdog update N of M every second once timeout_inactivity_count is exceeded, with no upper bound on warnings. There's no watchdog_max_warnings knob to suppress this.

@gretel
Copy link
Copy Markdown
Contributor

gretel commented Mar 31, 2026 •
edited
Loading

With DeviceRegistry, whichever block calls Device::make() first creates the device. In full-duplex TX/RX, SoapySink may create the device before SoapySource. On B210, clock_source and master_clock_rate must be set before setSampleRate() / setupStream() — otherwise the AD9361 auto-selects MCR and a subsequent MCR change disrupts the already-configured stream.
SoapySink currently has no master_clock_rate setting, so it can't configure the shared device correctly if it initializes first.

@RalphSteinhagen
Copy link
Copy Markdown
Member Author

RalphSteinhagen commented Apr 2, 2026

SoapySource::work() always returns {requestedWork, 0, OK} — zero performed work, status OK. The scheduler interprets this as "block is alive, keep polling" and spins ...

Correct, should be insufficient input samples rather than OK. The progress is reported anyway via the IO thread. Will adjust. This is the same issue for source and sink. The actual write to the HW should block if there are no samples... will investigate.

@RalphSteinhagen
fix: remove SoapySDR C++ API dependency, use C-API only for ABI safety
b656ab0 
- replace SoapySDR::Kwargs with own typedefs (no #include <SoapySDR/Device.hpp>)
- Device() = default (null handle), Device::make() -> std::expected factory
- all wrapper setters/activate/deactivate/setupStream return std::expected
- block uses emitErrorMessage + requestStop instead of throw
- fix argument evaluation order bug (GCC evaluates right-to-left) in 5 call sites
  that caused empty device/module lists on GCC-14/GCC-15
- fix null optionNames assertion crash with LimeSDR driver
- fix null device segfault in settingsChanged during emplaceBlock
- fix RAII deleters throwing in destructors (log to stderr instead)
- test: expect() on all std::expected returns (41 assertions), USB reset between suites

Known external issues (SoapySDR / driver bugs, not fixable on our side):
- 168-byte memory leak in SoapySDR's module loader (LSAN suppressed in CTest)
- RTL-SDR driver segfaults in readStream when PLL fails to lock after rapid
  close/reopen — mitigated by USB device reset + settle delay between suites
- RTL-SDR PLL lock failure is non-deterministic and hardware-dependent
- LimeSDR driver returns null optionNames in SoapySDRArgInfo (worked around)

Verified with RTL-SDR + LimeSDR on GCC-14, GCC-15, Clang-20 (Debug/Release, ASAN).

Signed-off-by: Ralph J. Steinhagen <r.steinhagen@gsi.de>
@RalphSteinhagen
refactor: move Soapy[Source] to gr::blocks::sdr, SigMF-aligned field …
ba1df2d 
…names

- absorbed into gr-sdr library (gr-soapy removed)
- wrapper namespace: gr::blocks::sdr::soapy
- SigMF-aligned renames (sigmf-schema.json v1.2.6)
- direction-prefixed per-channel fields: rx_gains, rx_bandwidths, rx_antennae
- Annotated<> used directly (removed using A = ... alias), Doc<> on all settings
- Tensor<double> → std::vector<double> for per-channel settings
- removed dead getMockDeviceSettingInfo(), added applyBandwidth() to reinitDevice()
- max_fragment_count added to GR_MAKE_REFLECTABLE

Signed-off-by: Ralph J. Steinhagen <r.steinhagen@gsi.de>
@RalphSteinhagen
refactor: SoapySource IO thread model, atomic overflow, disconnect_on…
2241c2b 
…_done

- migrate SoapySource from synchronous processBulk to async IO thread
  (ioReadLoop on thread_pool::defaultIoPool, same pattern as RTL2832Source)
- single-port and multi-port paths with cached writer references
- overflow counter now std::atomic, with verbose_overflow setting
- stream restart recovery on overflow (deactivate + reactivate)
- fix disconnect_on_done for IO thread sources (both SoapySource and
  RTL2832Source): check hasNoDownStreamConnectedChildren() in work()
  so the scheduler stops when CountingSink returns DONE

Signed-off-by: Ralph J. Steinhagen <r.steinhagen@gsi.de>
@RalphSteinhagen
feature: extend SoapySource for timing tags, forwardSettings, clk_in,…
4af25bc 
… DC blocker, ppm estimator

- 3-param settingsChanged with forwardSettings for frequency and sample_rate
- timing tags: TRIGGER_NAME, TRIGGER_TIME, TRIGGER_OFFSET per chunk
- emit_timing_tags, emit_meta_info, tag_interval, trigger_name settings
- emitChangedParams in TRIGGER_META_INFO (sample_rate, frequency, clock_source)
- optional clk_in port for external timing (GPS/PPS) with clock-offset interpolation
- drainClockInput mirrors RTL2832Source pattern
- optional DC blocker (disabled by default): dc_blocker_enabled, dc_blocker_cutoff
  using existing iir::designFilter<float>(HIGHPASS, Butterworth), complex<float> only
- optional sample rate estimator (disabled by default): ppm_estimator_cutoff
  using existing algorithm::SampleRateEstimator
- ppm_error, corrected sample_rate and frequency emitted in timing tags when enabled

Signed-off-by: Ralph J. Steinhagen <r.steinhagen@gsi.de>
@RalphSteinhagen
feature: add BurstTaper<T, N, Allocator> for pulsed RF burst shaping
5e86841 
implements stateless coefficient generation and stateful real-time envelope for burst TX shaping.
Target-driven state machine (Off↔RampUp↔On↔RampDown) with pre-computed lookup-table, physical-unit configuration, and glitch-free mid-ramp reversal. Template storage follows HistoryBuffer pattern (dynamic vector / fixed array / PMR).

Taper types: None, Linear, RaisedCosine, Tukey, Gaussian, Mushroom,
MushroomSine. Mushroom/MushroomSine: zero-integral for RF cavity phase
preservation (J. Tückmantel, R.J. Steinhagen, CERN).

Signed-off-by: Ralph J. Steinhagen <r.steinhagen@gsi.de>
@RalphSteinhagen
feat(soapy): added LoopbackDevice harness, RAII wrapper extensions, r…
d9611de 
…eview fixes

- LoopbackDevice: SoapySDR-compatible loopback with pluggable ChannelModel,
  DeviceRegistry for instance pairing, CF32/CS16/CU8 format conversion
- DeviceMode enum: Loopback (TX→RX), RxOnly (tone→model→RX), TxOnly (null sink)
- SoapyRaiiWrapper: ~30 new C-API wrappers (sensors, GPIO, registers, clock,
  frontend corrections), error returns as gr::Error
- fix: writeStream returns actual count (was always numElems), TIMEOUT on backpressure
- fix: null-check 8 C-API string getters, string_view→string for C null-termination
- fix: per-instance stream sentinels, intentional-leak DeviceRegistry statics,
  assert model-set-before-activate contract
- fix: getHardwareTime keeps uint64_t return with explicit cast, init 2 lengths
- tests: 99 tests (4666 assertions) — concurrent TX+RX threads, backpressure,
  CS16/CU8 boundary values, RxOnly/TxOnly modes, full SoapySDR API coverage

Signed-off-by: Ralph J. Steinhagen <r.steinhagen@gsi.de>
@RalphSteinhagen
feat(soapy): GR4 integration test for SoapySource + loopback device
f73cc9a 
- qa_SoapyIntegration.cpp: 7 tests (36 assertions) wiring SoapySource
  through gr::scheduler::Simple with the loopback device in rxOnly mode
- tests: single/2-channel sample delivery, tag propagation, clk_in with
  ClockSource→GPS_PPS forwarding, DC blocker on DC tone, tag_interval
  emission, graceful degradation (loopback without TX)

Signed-off-by: Ralph J. Steinhagen <r.steinhagen@gsi.de>
@RalphSteinhagen
feat(soapy): SoapySink block, shared DeviceRegistry, Stream::writeStream
21f4f54 
- SoapySink<T, nPorts>: IO-thread TX block mirroring SoapySource pattern
- soapy::DeviceRegistry: process-global shared_ptr<SoapySDRDevice> keyed
  by Kwargs — Source and Sink transparently share the same device handle
- Stream::writeStream: TX wrapper with variadic + buffer-list overloads
- plugin registration: SoapySink<1/2/4> × {uint8_t, int16_t, CF32}
- fix: Device constructor routes through registry (was bypassing shared handle)
- fix: registry deleter no longer locks mutex (was deadlock risk)
- fix: work() returns 0UZ (IO thread updates progress directly)
- shared parseKwargsString in soapy:: namespace (was duplicated)
- tests: TX→RX round-trip, 2-channel, standalone sink, registry cleanup

Signed-off-by: Ralph J. Steinhagen <r.steinhagen@gsi.de>
@sonarqubecloud
Copy link
Copy Markdown

sonarqubecloud bot commented Apr 3, 2026

Quality Gate Failed Quality Gate failed

Failed conditions
56.6% Coverage on New Code (required ≥ 80%)
3.6% Duplication on New Code (required ≤ 3%)

See analysis details on SonarQube Cloud

@RalphSteinhagen
Copy link
Copy Markdown
Member Author

RalphSteinhagen commented Apr 3, 2026

N.B. Was the missing 'progress' increase and notify_all in the SoapySink in the end. Added some other reasonably common Soapy-access methods in addition to the (master/reference) clock settings, merged the duplicate commit, rebased and pushed again.

@gretel
Copy link
Copy Markdown
Contributor

gretel commented Apr 3, 2026 •
edited
Loading

Correct, should be insufficient input samples rather than OK. The progress is reported anyway via the IO thread. Will adjust. This is the same issue for source and sink. The actual write to the HW should block if there are no samples... will investigate.

sure, maybe this PR can help during investigations.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@RalphSteinhagen @gretel