Shall a non canonical purl be a valid input ?

[TG1999]

https://github.com/package-url/purl-spec/blob/main/schemas/purl-test.schema-0.1.json

We also need to define what kind of inputs a "base" and "advanced" tests can take ?

[johnmhoran]

And for the base and advanced tests, for build, parse, and roundtrip,

• when is a non-canonical PURL allowed as an input?
• how are the various non-canonical characteristics to be handled (unencoded slash in a qualifiers value being just one of many way an incoming PURL can be non-canonical and must be handled somehow)?
• when must a test fail, i.e., expected_failure is true, and when must the expected_output be null (presumably in coordination with expected_failure is true)?

[johnmhoran]

@TG1999 Thank you for opening this issue. I added a few closely-related questions ^ and also added the issue to Upgrade PURL test suite (view).

[TG1999]

@johnmhoran thanks 🙇, also thanks for bringing this issue up, I hope resolving this issue brings more clarity on valid inputs and the expected behaviours.

[johnmhoran]

Thanks @TG1999 . While I'm at it ;-)

• why does the test schema seem to require a roundtrip input be canonical and prohibit a null value in expected_output?

[petergardfjall]

Referencing a discussion relevant to this ticket:
#644 (comment)

An excerpt from that comment:

I had a discussion on qualifiers with @matt-phylum here. In this comment he clarified a thing to me -- the procedure for parsing a PURL accepts a wider range of PURLs than the procedure for producing a canonical (output) purl.

In other words, the way the spec is currently formulated it does NOT require input purls to be canonical.

[johnmhoran]

@petergardfjall Please note that the how-to-build and how-to-parse sections (not currently part of the 1.0 standard submitted to Ecma) have not yet been updated to accurately reflect the 1.0 submission.

[mjherzog]

We do not yet have good documentation for the use case(s) for a 'roundtrip' test_type, but a relevant use case seems to be testing for data deserialization/serialization: i.e., parsing an object (PURL string) into PURL components and then re-building it from the PURL components.

If this is the use case then:

• the input PURL string for a 'roundtrip' test should be in canonical form to match the output
• a 'roundtrip' test is effectively the combination of a 'parse' test with a 'build' test

Jan Kowalleck proposed the use case (#738 (comment)) as:

I used it as "i get some PURL string input, and want to re-use it with all normalizations and cleaning applied."
For example, qualifiers are sorted, qualifiers with empty values are removed, all non-alphanumeric are percent-encoded, etc ...
I don't care whether the implementation needs to parse and build the purl for this, or does it in another way.

This use case seems closer to the proposed 'validation" test_type. See:

• Schema change / purl-test: Add validation test type #692

Is there any logical use case for a 'roundtrip' test to accept a non-canonical input PURL to test that a tool can produce a matching non-canonical PURL?

On a separate note, round-trip-correctness seems to be an important test methodology for LLMs - e.g.
https://community.sap.com/t5/technology-blog-posts-by-sap/round-trip-correctness-a-new-metric-for-generative-ai-based-process/ba-p/14091979#:~:text=In%20conclusion%2C%20Round%2DTrip%20Correctness,and%20see%20what%20it%20reveals!

See also:

• Correct spelling of "roundtrip" test_group #801