Saving new measurement

[dlanci]

Dear mantainers,

Apologies in advance if what I'm about to ask is detailed in some guide but I couldn't find it anywhere. I would like to create a new measurement instance modifying one of the existing ones in flavio. For example, for the relevant observables I want to create an arbitrary Likelihood:

obs_space=np.linspace(0.5,1.5,1001)
logL=normal_logpdf(obs_space, 1., 0.04)
plt.plot(obs_space,np.exp(logL-logL.max()))

Then from the file measurements_test.yml that contains a dictionary that is the copy of the observable I want to edit I do:

with open('/path/to/flavio/data/measurements_test.yml') as file:

    test = yaml.load(file, Loader=yaml.FullLoader)

test2=copy.deepcopy(test)
test2['observable']['values'][0]['value']['y']=np.exp(logL)

When I want to save I do:

flavio.measurements.write_file('/path/to/flavio/data/measurements_test_1.yml',test2)

And I obtain a key error

KeyError: 'observable'

What am I missing in the definition of the measurement instance with these steps?

Thanks

[DavidMStraub]

Hi,

can you try whether it works when you use flavio.measurements.read_file for loading the measurements? BTW, when you use ```python in front of your code blocks, you get nice syntax highlighting.

[dlanci]

If I run

flavio.measurements.read_file('/path/to/flavio/data/measurements_test.yml')

It outputs a list with the name of the observable as only item

['observable']

[DavidMStraub]

Right, that's the return value, but it also instantiates all the Measurement instances.

You can pass the list of names to write_file and it should work.

[dlanci]

Ok, but then how can I edit/update the measured Likelihood for the relevant observable with the values I want to assign? i.e. how can I edit the dictionary and dump it back to the file like in:

flavio.measurements.read_file('/path/to/flavio/data/load_measurements_to_edit.yml') #this instantiate the measurements

dict['values'][0]['value']['y']=np.exp(logL) #this step would be what I'd need

flavio.measurements.write_file('/path/to/flavio/data/dump_for_edited_measurements.yml', dict)

[DavidMStraub]

Ah OK, you want to edit the YAML before instantiating the Measurements? Then you can just do the low-level operations yourself, see here https://github.com/flav-io/flavio/blob/master/flavio/measurements.py#L99

[Measurement.from_yaml_dict(dict, pname='observables') for m in measurements]

[DavidMStraub]

I suspect you were missing the pname. It's a subtlety needed because the same method is inherited by Measurement and ParameterConstraints, but the key names are different.

[DavidMStraub]

(Arguably, this is not an elegant implementation. It would make more sense to make pname a class attribute with value "parameters" on the base class and "observables" on the measurement class. @peterstangl?)

[dlanci]

Thanks. So what I now do is:

#get the measurement to edit
with open('/path/to/flavio/data/measurements_test.yml') as file:

    test = yaml.load(file, Loader=yaml.FullLoader)

#edit the likelihood y
test2=copy.deepcopy(test)
test2['observable']['values'][0]['value']['y']=np.exp(logL)

#create a measurement list  with the edited yaml dict
from flavio.classes import Measurement
measurements = [Measurement.from_yaml_dict(test2, pname='observables')]

And I obtain

KeyError: 'constraints'

I attach for clarity the measurements_test.yml I'm trying to edit

LHCb RK Example:
  experiment: LHCb
  values:
    - name: <Rmue>(B+->Kll)
      q2min: 1.1
      q2max: 6.0
      value:
        distribution: numerical
        x: [0.5  , 1.5] #dummy values
        y: [1.0  , 1.0]

[DavidMStraub]

Ah OK. This module is a mess. The reason is that the YAML serialization/deserialization was added at a later point and the measurements.yaml uses a different, older format, like your example. This can only be parsed with the measurements._load method that expects a YAML stream. The format that works with from_yaml_dict is the same you get when you dump a Measurement with the get_yaml_dict method.

[dlanci]

It's ok and actually with a simple edit I could make it work. For now I just dump the modified measurement with yaml and edited this line https://github.com/flav-io/flavio/blob/master/flavio/measurements.py#L14 to:

measurements = yaml.load(obj, Loader=yaml.Loader)

And it works just fine. Thank you!

[peterstangl]

(Arguably, this is not an elegant implementation. It would make more sense to make pname a class attribute with value "parameters" on the base class and "observables" on the measurement class. @peterstangl?)

Yes, good point! I think in general loading and saving of measurements can be improved and I will keep this issue open as a reminder for that.