Skip to content

Fix missing optimize/fp16/external-data handling in PIR exporter#1640

Open
982945902 wants to merge 1 commit intoPaddlePaddle:developfrom
982945902:fix/pir-external-data-export
Open

Fix missing optimize/fp16/external-data handling in PIR exporter#1640
982945902 wants to merge 1 commit intoPaddlePaddle:developfrom
982945902:fix/pir-external-data-export

Conversation

@982945902
Copy link
Copy Markdown

Summary

ModelExporter::Run(const PaddlePirParser& ...) currently diverges from the legacy PaddleParser path in a critical way: it builds the ONNX graph and immediately calls SerializeToString(&out) without applying:

  • enable_optimize
  • export_fp16_model
  • big-model external-data handling via SaveExternalData(...)

For large PIR-exported models, that means Paddle2ONNX still tries to serialize a single in-memory ModelProto, which can exceed the protobuf 2GB limit and produce an unusable ONNX export.

This change brings the PIR exporter path back in line with the legacy exporter path by applying the same post-export steps before checker/serialization.

What this patch changes

Inside ModelExporter::Run(const PaddlePirParser& ...), this PR adds:

  1. ONNX optimization when enable_optimize is enabled
  2. FP16 conversion when export_fp16_model is enabled
  3. external-data emission when onnx_model_.ByteSizeLong() > INT_MAX

The logic mirrors the existing implementation already used in ModelExporter::Run(const PaddleParser& ...).

Why this matters

Without this parity, PIR-exported large models can fail even though the codebase already has a supported external-data flow for large ONNX graphs.

In local validation with a real PIR-exported uie-m-large model:

  • before this patch, export hit the >2GB protobuf limit and produced an unusable result
  • after this patch, export produced a small model.onnx plus model.onnx.data
  • the saved model passed path-based onnx.checker.check_model(model_path) validation

Scope

This PR intentionally keeps the change minimal:

  • no API changes
  • no behavior changes for the legacy parser path
  • no new flags or new codepaths beyond restoring parity for the PIR exporter

If helpful, I can follow up with a focused regression test once there is an agreed-upon large-model fixture strategy for CI.

@CLAassistant
Copy link
Copy Markdown

CLAassistant commented Apr 15, 2026

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@982945902 982945902 force-pushed the fix/pir-external-data-export branch from 3c711be to 3f96025 Compare April 15, 2026 10:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants