O1 Models Incompatible with ArenaGEval due to response_format Parameter #1820

❗BEFORE YOU BEGIN❗
Are you on discord? 🤗 We'd love to have you asking questions on discord instead: https://discord.com/invite/a3K9c8GRGt

Describe the bug
O1 models (o1, o1-2024-12-17, o3-mini, etc.) fail when used with ArenaGEval because DeepEval internally passes a response_format parameter with type json_schema, which O1 models don't support.

To Reproduce
Steps to reproduce the behavior:

1. Create an O1 model: gpt_judge = GPTModel(model="o1")
2. Create an ArenaGEval metric with the O1 model: arena_metric = ArenaGEval(name="test", criteria="Compare outputs", model=gpt_judge)
3. Create test cases and arena test case with contestants
4. Call await arena_metric.a_measure(arena_test_case)
5. See error: Error code: 400 - {'error': {'message': "Unsupported parameter: 'temperature' is not supported with this model.", 'type': 'invalid_request_error', 'param': 'temperature', 'code': 'u...",

Expected behavior
O1 models should work with ArenaGEval, either by automatically detecting that the model doesn't support structured output and falling back to text parsing, or by providing a configuration option to disable structured output. Please also add support for O3

Desktop (please complete the following information):

• OS: Linux Ubuntu
• DeepEval Version: Latest
• Python Version: 3.11