Spec prefill

[kuanjames]

Do I need to load the model for spec prefill in order to use it?

No right? If I don't load it, what about the settings like context size, temperature and top k? Will the main model know to use those settings for the smaller model to spec fill?

[kuanjames]

I think the answer is no. No need to load the smaller model.