from_json and transparent elements in extensible NAS IEs

[burrodesancho]

I'm having 2 problems when assigning values to objects using from_json method.

1. Cannot set values with default transparency set to True

For the following code:

snssai_dict = {'SNSSAI': [{'SST': 1}, {'SD': 7}]}
snssai_obj = pycrate_mobile.TS24501_FGMM.SNSSAI()
print("default:", snssai_obj)
snssai_obj.from_json(json.dumps(snssai_dict))
print("after from_json:", snssai_obj)

I get the output:

default: <SNSSAI : <SST : 0><SD [transparent] : 0x><MappedHPLMNSST [transparent] : 0><MappedHPLMNSD [transparent] : 0x>>
after from_json: <SNSSAI : <SST : 1 (eMBB)><SD [transparent] : 0x><MappedHPLMNSST [transparent] : 0><MappedHPLMNSD [transparent] : 0x>>

The SD element is transparent by default and it is not defined even though it's value is provided in the dictionary used in from_json method.

2. Values with default transparency set to False are represented in byte/hex string

For the following code:

fgmmcap_dict = {'5GMMCap': [{'SGC': 0}, {'5G-HC-CP-CIoT': 0}, {'N3Data': 0}, {'5G-CP-CIoT': 0}, {'RestrictEC': 0}, {'LPP': 0}, {'HOAttach': 1}, {'S1Mode': 1}, {'RACS': 0}, {'NSSAA': 1}, {'5G-LCS': 0}, {'V2XCNPC5': 0}, {'V2XCEPC5': 0}, {'V2X': 0}, {'5G-UP-CIoT': 0}, {'5GSRVCC': 0}]}
fgmmcap_obj = pycrate_mobile.TS24501_FGMM.FGMMCap()
fgmmcap_obj.from_json(json.dumps(fgmmcap_dict))

I get the following error: EltErr: 5GMMCap [_from_jval]: missing elements, 5G-EHC-CP-CIoT ...

The default FGMMCap object has following elements:

<5GMMCap : <SGC : 0><5G-HC-CP-CIoT : 0><N3Data : 0><5G-CP-CIoT : 0><RestrictEC : 0><LPP : 0><HOAttach : 0><S1Mode : 0><RACS : 0><NSSAA : 0><5G-LCS : 0><V2XCNPC5 : 0><V2XCEPC5 : 0><V2X : 0><5G-UP-CIoT : 0><5GSRVCC : 0><spare : 0><5G-EHC-CP-CIoT : 0><MultipleUP : 0><WUSA : 0><CAG : 0><PR : 0><RPR : 0><PIV : 0><NCR : 0><NR-PSSI : 0><5G-ProSe-l3rmt : 0><5G-ProSe-l2rmt : 0><5G-ProSe-l3relay : 0><spare : 0><UAS : 0><NSAG : 0><Ex-CAG : 0><SSNPNSI : 0><EventNotification : 0><MINT : 0><NSSRG : 0><spare : 0x>>

So, the problem is that in the fgmmcap_dict there are no defined values for spare, 5G-EHC-CP-CIoT, and following elements...

In the second case, I can use set_val method, but at the end I'll get longer byte/hex string because all the nontransparent elements that I have not defined in the fgmmcap_dict will be represented with zeroes and the final string will be longer than the original one.

The code:

fgmmcap_obj = pycrate_mobile.TS24501_FGMM.FGMMCap()
fgmmcap_obj['HOAttach'] = 1
fgmmcap_obj['S1Mode'] = 1
fgmmcap_obj['NSSAA'] = 1

print(fgmmcap_obj.hex())

Outputs 0340000000 but the original value is 0340

Solution

I have overwritten the Envelope's _from_jval method:

def envelope_from_jval(self, val): 
    """Function to overwrite the Envelope._from_jval method.
	Changes are made so we make sure that attributes provided in the json
	are not transparent and those missing from the josn are transparent""" 
	if not isinstance(val, list): 
	    raise ( 
		    EltErr("{0} [_from_jval]: invalid format, {1!r}".format(self._name, val)) 
		) 
	i = 0 
	val_len = len(val) 
	for e in self._content: 
	    # set an attribute as transparent if the index is longer than the list of values 
    	    if i >= val_len and not e.get_trans(): 
    	        e.set_trans(True) 
    		
    	    # check if the element exist in val and undo transparency if needed 
    	    if i < val_len and e.get_trans() and val[i]: 
    	        e.set_trans(False) # undo transparency 
    		
    	    if not e.get_trans(): 
    	        try: 
    		    e._from_jval_wrap(val[i]) 
    		except Exception: 
    		    break 
    		else: 
    		    i += 1 
				
	# ensure all non-transparent elements were set 
	for e in self._content[1 + self._content.index(e) :]: 
	    if not e.get_trans() and e.get_bl(): 
		    raise ( 
		        EltErr( 
		            "{0} [_from_jval]: missing elements, {1} ...".format(
                                self._name, e._name
		            )
		        )
		    )
			
			
# overwrite the original _from_jval method
Envelope._from_jval = envelope_from_jval

With those changes both of my problems are fixed.

Is there something I don't see/know/understand about this and my changes cannot be applied in all cases?
Or my fix is acceptable?

Thank you!

[mitshell]

Thanks for your feedback.

Generally, there is a custom JSON support for IEs in NAS Layer3 messages:

• for general Layer3 and Layer3E messages:

  pycrate/pycrate_mobile/TS24007.py

  Line 212 in fe6a308

  def _from_jval(self, val):

• for IE:

  pycrate/pycrate_mobile/TS24007.py

  Line 449 in fe6a308

  def _from_jval(self, val):

But effectively, for IEs' internal structures, there is no custom JSON encoding. In those cases, a specific JSON encoder should apply to the core Envelope object, here:

pycrate/pycrate_core/elt.py

Line 2273 in fe6a308

def _from_jval(self, val):

I need to further check if integrating such support in the core object does not break any other parts of the lib, before following this path. By the way, your solution to overload the default method is also perfectly fine, in your specific case.

[mitshell]

I checked this more carefully, and I think such a JSON decoding should only apply to NAS IEs. If I integrate this specific JSON decoding behaviour in pycrate's core part, some other modules may break or become unusable with JSON.

Thus, the solution would be to create a dedicated extensible IE class somewhere (e.g. in TS24007.py), that implements this JSON decoding, and then inherit from it for all the required IEs in the various TS24XYZ_IE.py modules.

[burrodesancho]

Thank you for your feedback!