Multi-line text fields corrupted when using custom marshalers

[wrouesnel]

https://go.dev/play/p/zEajxshUE_y

I have a reliable bug from the following YAML snippet, which is output of a custom encoder:

container:
  type: generic
  config:
    message: |-
    some test text
    second line of the text

Note the corrupted text. The inner struct, if marshaled properly should produce output like:

container:
  type: generic
  config:
    message: |-
      some test text
      second line of the text

When it hits this line:

go-yaml/encode.go

Line 911 in 92bc79c

if e.isMapNode(encoded) {

and e.AddColumn is called, the multiline text field indentation is corrupted.

This only happens when a custom encoder is embedding the output into the key of another struct. The following program reproduces the problem:

package main

import (
	"bytes"
	"fmt"

	"github.com/goccy/go-yaml"
)

type D struct {
	Message string
}

type T struct {
	Type   string
	Config interface{}
}

type O struct {
	Container interface{}
}

func main() {
	marshaler := func(t D) ([]byte, error) {
		return yaml.Marshal(&T{
			"generic",
			t,
		})
	}

	testStruct := &O{&D{
		Message: "some test text\nsecond line of the text",
	}}

	b := bytes.NewBuffer(nil)
	err := yaml.NewEncoder(b, yaml.CustomMarshaler(marshaler)).Encode(testStruct)
	fmt.Println(err)
	fmt.Println(b.String())
}

EDIT:

Further investigation seems to indicate this is a general problem with embedded multiline text in version v1.19.2 from any marshalers - https://go.dev/play/p/sH6LhtCIzAB

package main

import (
	"bytes"
	"fmt"

	"github.com/goccy/go-yaml"
)

type D struct {
	Message string
}

type X struct {
	Type   string
	Config interface{}
}

func (x *X) MarshalYAML() ([]byte, error) {
	return yaml.Marshal(map[string]interface{}{
		"type":   "generic",
		"config": x.Config,
	})
}

type O struct {
	Container interface{}
}

func main() {
	testStruct3 := &O{
		&X{
			"generic",
			&D{"some test text\nsecond line of the text"},
		},
	}

	b3 := bytes.NewBuffer(nil)
	err := yaml.NewEncoder(b3).Encode(testStruct3)
	fmt.Println(err)
	fmt.Println(b3.String())

}

container:
  config:
    message: |-
    some test text
    second line of the text
  type: generic

EDIT 2: Extremizing the example makes it clear what's happening it looks like -

https://go.dev/play/p/KsNas3IMiVw

package main

import (
	"bytes"
	"fmt"

	"github.com/goccy/go-yaml"
)

type D struct {
	Message string
}

type X struct {
	Type   string
	Config interface{}
}

func (x *X) MarshalYAML() ([]byte, error) {
	return yaml.Marshal(map[string]interface{}{
		"type":   "generic",
		"config": x.Config,
	})
}

type O struct {
	Container interface{}
}

func main() {
	message := "some test text\nsecond line of the text"
	testStruct3 := &O{&O{&O{
		&O{&O{
			&X{
				"generic",
				&D{message},
			},
		},
		}}},
	}

	b3 := bytes.NewBuffer(nil)
	yaml.NewEncoder(b3).Encode(testStruct3)
	fmt.Println(b3.String())

}

container:
  container:
    container:
      container:
        container:
          config:
            message: |-
    some test text
    second line of the text
          type: generic

[aeijdenberg]

Looking at the design of yaml.CustomMarshaler, it's not clear to me that this could ever work as expected, since the custom marshaler doesn't know (since there's no method signature to tell it) how deep it should be when rendering.

Implementing yaml.InterfaceMarshaler looks like it will get the desired output, e.g. in your last example, change the MarshalYAML() to:

func (x *X) MarshalYAML() (any, error) {
	return map[string]interface{}{
		"type":   "generic",
		"config": x.Config,
	}, nil
}

and it then outputs:

container:
  container:
    container:
      container:
        container:
          config:
            message: |-
              some test text
              second line of the text
          type: generic

I'm guess that carries over from (https://pkg.go.dev/gopkg.in/yaml.v2#Marshaler):

The Marshaler interface may be implemented by types to customize their behavior when being marshaled into a YAML document. The returned value is marshaled in place of the original value implementing Marshaler.