Hi there,
I’m trying to add jinna-v3 embeddings in the MLX format. However, I’ve run into an issue due to the model’s customized architecture — it appears to differ significantly from standard architectures like xlm-roberta.
For example, one of the errors I encountered mentioned that around 400 layers were not found in the standard xlm-roberta model definition.
I’d like to better understand how to approach implementing such customized or non-standard architectures in MLX.
Specifically:
• Is there a recommended approach or best practice for integrating models with custom architectures?
• Are there any guides, cookbooks, or examples that show how to handle non-standard transformer-based models in MLX?
Thanks in advance for your help and guidance! 🙏
Hi there,
I’m trying to add jinna-v3 embeddings in the MLX format. However, I’ve run into an issue due to the model’s customized architecture — it appears to differ significantly from standard architectures like xlm-roberta.
For example, one of the errors I encountered mentioned that around 400 layers were not found in the standard xlm-roberta model definition.
I’d like to better understand how to approach implementing such customized or non-standard architectures in MLX.
Specifically:
• Is there a recommended approach or best practice for integrating models with custom architectures?
• Are there any guides, cookbooks, or examples that show how to handle non-standard transformer-based models in MLX?
Thanks in advance for your help and guidance! 🙏