Commit 2e548c1
feat(openapi): upgrade OpenAPI FDW to v0.2.0 with modular architecture (#573)
* feat(openapi): upgrade to v0.2.0 with modular architecture
Replace v0.1.4 monolithic codebase with v0.2.0 refactored modules:
config, request, response, pagination, column_matching, spec, schema.
New features: POST-for-read endpoints, spec_json inline specs,
LIMIT-to-page_size pushdown, api_key_location (query/cookie),
debug mode, max_pages/max_response_bytes safety limits, OpenAPI 3.1
support.
Includes 518 unit tests, benchmarks, 5 real-world examples (NWS,
CarAPI, PokeAPI, GitHub, Threads), Docker-based integration test
infrastructure with 113 assertions, and performance analysis docs.
* docs(openapi): rewrite README and fix catalog docs
Rewrite README with clearer features list, honest performance section
comparing FDW vs pg_http (DX tradeoff with SQL examples and end-to-end
benchmarks), and move limitations up for visibility. Consolidate
PERFORMANCE.md into README. Update benchmark script to measure full
read-to-write lifecycle (INSERT INTO) instead of PERFORM. Fix tabbed
content indentation in catalog docs for pymdownx.tabbed rendering.
* chore(openapi): revert workspace release profile and update README
Remove [profile.release] (strip, lto) from the shared wasm-wrappers
workspace Cargo.toml — these affect all wasm FDWs, not just openapi.
Revert Cargo.lock to match main. Minor README updates.
* chore(openapi): remove criterion micro-benchmarks
Remove benches/fdw_benchmarks.rs and the criterion dev-dependency.
These benchmarks tested re-implemented copies of FDW logic rather than
actual code, added ~38 transitive dependencies, and caused build errors
on wasm targets. The SQL-level benchmark script (test/benchmark.sh)
provides meaningful end-to-end performance analysis.
* fix(openapi): use native target for test and clippy in Makefile
Unit tests and clippy can't run on wasm32-unknown-unknown since there's
no runtime to execute the binary. Auto-detect the host target via rustc
so make test and make clippy work out of the box on any platform.
* feat(openapi): add YAML spec support, review fixes, and example improvements
Add YAML spec parsing via serde_yaml_ng so spec_url accepts both JSON
and YAML OpenAPI specs. Many APIs only publish YAML, so this makes the
FDW work out of the box with more APIs.
Also addresses PR review items:
- Replace deprecated serde_yaml with serde_yaml_ng
- debug_assert! -> assert! in this_mut() for release safety
- Header deduplication prevents duplicate content-type/authorization
- Empty/whitespace credentials filtered with warning
- Retry on 502/503 in addition to 429, with status-specific hints
- RowsOut stats now count rows consumed by PG, not just fetched
- Validate max_pages >= 1
- base_url validation for spec-derived server URLs
- Improved error messages (show both JSON and YAML parse errors)
Example updates:
- All 5 examples get IMPORT FOREIGN SCHEMA as section 1
- New import servers with spec_url (or spec_json for Threads)
- Threads example shows CREATE SERVER with inline spec_json
- PokeAPI highlights YAML spec support
* fix(openapi): address PR #573 review feedback from Copilot and CodeRabbit
- Fix pagination URL resolution for parameterized endpoints by storing
resolved_endpoint after path param substitution
- Fix absolute-path pagination to use origin-only base to avoid
duplicating path prefixes (e.g. /v1/v1/items)
- Map time and byte/binary formats to text (WIT TypeOid has no
time/bytea variants)
- Fix .env.example copy instructions and README test counts
- Replace {checksum} placeholder with descriptive text in catalog docs
* docs(openapi): consolidate feature tables and add missing import server blocks
Move per-example feature tables into a single Feature Coverage comparison
on the main examples README. Add the missing CREATE SERVER blocks for
spec_url/spec_json import servers to carapi, pokeapi, github, and nws
examples. Rename NWS references to Weather.gov.
* docs(openapi): fix attrs column description to match actual behavior
The attrs column returns the full JSON response object, not just
unmapped fields. Updated all example READMEs to accurately describe
this, consistent with the catalog docs and every other wasm FDW.
* minor code improvements
* add default user agent header
---------
Co-authored-by: Cody Bromley <codybrom@users.noreply.github.com>
Co-authored-by: Bo Lu <lv.patrick@gmail.com>1 parent 57e2c42 commit 2e548c1
File tree
42 files changed
+21963
-1455
lines changed- docs/catalog
- wasm-wrappers/fdw
- openapi_fdw
- examples
- carapi
- github
- nws
- pokeapi
- threads
- src
- test
- wit
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
42 files changed
+21963
-1455
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
12 | 12 | | |
13 | 13 | | |
14 | 14 | | |
15 | | - | |
| 15 | + | |
16 | 16 | | |
17 | 17 | | |
18 | 18 | | |
19 | 19 | | |
20 | 20 | | |
21 | | - | |
| 21 | + | |
| 22 | + | |
22 | 23 | | |
23 | 24 | | |
24 | 25 | | |
| |||
94 | 95 | | |
95 | 96 | | |
96 | 97 | | |
97 | | - | |
98 | | - | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
99 | 101 | | |
100 | 102 | | |
101 | 103 | | |
102 | 104 | | |
| 105 | + | |
103 | 106 | | |
104 | 107 | | |
105 | 108 | | |
| |||
109 | 112 | | |
110 | 113 | | |
111 | 114 | | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
112 | 118 | | |
113 | 119 | | |
114 | 120 | | |
| |||
151 | 157 | | |
152 | 158 | | |
153 | 159 | | |
| 160 | + | |
| 161 | + | |
154 | 162 | | |
155 | 163 | | |
156 | 164 | | |
157 | | - | |
| 165 | + | |
158 | 166 | | |
159 | 167 | | |
160 | 168 | | |
| |||
244 | 252 | | |
245 | 253 | | |
246 | 254 | | |
| 255 | + | |
| 256 | + | |
| 257 | + | |
| 258 | + | |
| 259 | + | |
| 260 | + | |
| 261 | + | |
| 262 | + | |
| 263 | + | |
| 264 | + | |
| 265 | + | |
| 266 | + | |
| 267 | + | |
| 268 | + | |
| 269 | + | |
| 270 | + | |
| 271 | + | |
| 272 | + | |
| 273 | + | |
| 274 | + | |
| 275 | + | |
| 276 | + | |
| 277 | + | |
| 278 | + | |
| 279 | + | |
| 280 | + | |
| 281 | + | |
| 282 | + | |
| 283 | + | |
| 284 | + | |
| 285 | + | |
| 286 | + | |
| 287 | + | |
| 288 | + | |
| 289 | + | |
| 290 | + | |
| 291 | + | |
| 292 | + | |
| 293 | + | |
| 294 | + | |
| 295 | + | |
| 296 | + | |
| 297 | + | |
| 298 | + | |
| 299 | + | |
| 300 | + | |
| 301 | + | |
| 302 | + | |
| 303 | + | |
| 304 | + | |
| 305 | + | |
| 306 | + | |
| 307 | + | |
| 308 | + | |
247 | 309 | | |
248 | 310 | | |
249 | 311 | | |
250 | 312 | | |
251 | 313 | | |
252 | 314 | | |
| 315 | + | |
253 | 316 | | |
254 | 317 | | |
255 | 318 | | |
| |||
306 | 369 | | |
307 | 370 | | |
308 | 371 | | |
309 | | - | |
| 372 | + | |
310 | 373 | | |
311 | 374 | | |
312 | 375 | | |
313 | 376 | | |
314 | | - | |
| 377 | + | |
315 | 378 | | |
316 | | - | |
| 379 | + | |
317 | 380 | | |
318 | 381 | | |
319 | 382 | | |
320 | 383 | | |
| 384 | + | |
| 385 | + | |
321 | 386 | | |
322 | 387 | | |
323 | 388 | | |
| |||
339 | 404 | | |
340 | 405 | | |
341 | 406 | | |
342 | | - | |
| 407 | + | |
343 | 408 | | |
344 | | - | |
| 409 | + | |
345 | 410 | | |
346 | | - | |
| 411 | + | |
347 | 412 | | |
348 | 413 | | |
349 | 414 | | |
350 | 415 | | |
351 | 416 | | |
352 | 417 | | |
353 | 418 | | |
| 419 | + | |
| 420 | + | |
354 | 421 | | |
355 | 422 | | |
356 | 423 | | |
357 | 424 | | |
358 | 425 | | |
359 | 426 | | |
360 | 427 | | |
361 | | - | |
| 428 | + | |
362 | 429 | | |
363 | | - | |
364 | | - | |
| 430 | + | |
| 431 | + | |
365 | 432 | | |
366 | 433 | | |
367 | 434 | | |
| |||
395 | 462 | | |
396 | 463 | | |
397 | 464 | | |
| 465 | + | |
| 466 | + | |
| 467 | + | |
| 468 | + | |
| 469 | + | |
| 470 | + | |
| 471 | + | |
| 472 | + | |
| 473 | + | |
| 474 | + | |
| 475 | + | |
| 476 | + | |
| 477 | + | |
| 478 | + | |
| 479 | + | |
| 480 | + | |
| 481 | + | |
| 482 | + | |
| 483 | + | |
| 484 | + | |
398 | 485 | | |
399 | 486 | | |
400 | 487 | | |
| |||
413 | 500 | | |
414 | 501 | | |
415 | 502 | | |
| 503 | + | |
| 504 | + | |
| 505 | + | |
| 506 | + | |
| 507 | + | |
| 508 | + | |
| 509 | + | |
| 510 | + | |
| 511 | + | |
| 512 | + | |
| 513 | + | |
| 514 | + | |
| 515 | + | |
| 516 | + | |
| 517 | + | |
| 518 | + | |
| 519 | + | |
| 520 | + | |
416 | 521 | | |
417 | 522 | | |
418 | 523 | | |
| |||
0 commit comments