updated system prompt

[fahamin5149]

This pull request enhances the evaluation pipeline to support challenge-specific guidelines and improves the context and instructions given to LLM judges. The changes ensure that guidelines are consistently passed through the system and integrated into the evaluation prompt, and the prompt instructions are clarified for better scoring and justification.

Support for guidelines in evaluation:

• Added an optional guidelines field to the ChallengeConfig type and ensured it is extracted and included in the challengeConfigMap for each challenge. [1] [2] [3]
• Passed the guidelines field through all levels of the evaluation pipeline, including the call to runJudges, evaluateWithRetry, and callLLM functions. [1] [2] [3] [4] [5]

Prompt and context improvements for LLM evaluation:

• Refactored the LLM prompt instructions to clarify how rubric sub-sections and guidelines should affect scoring, and made explicit the requirement to reference both the problem statement and challenge context in the justification.
• Improved logging for system prompts and LLM responses for easier debugging and verification, including truncating long outputs for readability. [1] [2]

[Copilot]

Pull request overview

This pull request enhances the LLM evaluation pipeline by adding support for challenge-specific guidelines and refactoring the prompt structure to provide clearer context to LLM judges. The changes thread the new guidelines field through the entire evaluation pipeline and restructure how challenge context is presented to the evaluation models.

Changes:

• Added optional guidelines field to ChallengeConfig type and threaded it through the evaluation pipeline
• Refactored the LLM evaluation prompt to move challenge context from system prompt to user prompt with explicit instructions about sub-sections and guidelines
• Simplified logging of system prompts and LLM responses with inline truncation

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 7 comments.

File	Description
server/routes/evaluateAll.ts	Added guidelines field to ChallengeConfig type, extracted guidelines from Firestore, and passed to evaluation functions
server/utils/judgeLlms.ts	Updated function signatures to accept guidelines parameter, refactored prompt structure to include guidelines and challenge context in user prompt, and improved logging

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

This instruction assumes that the CHALLENGE CONTEXT always contains both "Problem Statement and Visual Clues," but based on the code structure, the challenge context and problem statement are separate fields that may or may not exist. The instruction should be more flexible to handle cases where only one or neither is provided, and should refer to "CHALLENGE CONTEXT" rather than assuming it contains visual clues.

[Copilot]

The new guidelines parameter lacks documentation. Adding a JSDoc comment explaining the purpose of this parameter would improve code maintainability and help other developers understand when and how to use it. Consider documenting that it contains challenge-specific binding specifications for evaluation.

[Copilot]

The parameters problemStatement, guidelines, and challengeSystemPrompt are not typed as optional (using ?) in the function signature, but they are passed as undefined when they're not available in the calling code (e.g., line 631-633 in evaluateAll.ts uses ?? undefined). This creates a type mismatch where the function expects non-nullable strings but may receive undefined at runtime. These parameters should be typed as optional: problemStatement?: string, guidelines?: string, challengeSystemPrompt?: string.

Suggested change

	problemStatement: string,
	guidelines: string,
	challengeSystemPrompt: string,
	problemStatement?: string,
	guidelines?: string,
	challengeSystemPrompt?: string,

[Copilot]

The parameters problemStatement, guidelines, and challengeSystemPrompt should be typed as optional (using ?) since they can be undefined at runtime. The function signature should be: problemStatement?: string, guidelines?: string, challengeSystemPrompt?: string. This matches the pattern used in the runJudges function and the actual usage where these values can be undefined.

[Copilot]

When challengeSystemPrompt is undefined, this will result in the string "undefined" being interpolated into the prompt instead of being omitted. This should be conditionally included like problemStatement and guidelines are, for example: ${challengeSystemPrompt ? CHALLENGE CONTEXT (GROUND TRUTH):\n${challengeSystemPrompt} : ''}.

Suggested change

	CHALLENGE CONTEXT (GROUND TRUTH):
	${challengeSystemPrompt}
	${challengeSystemPrompt ? `CHALLENGE CONTEXT (GROUND TRUTH):\n${challengeSystemPrompt}` : ''}

[Copilot]

The description field references specific criterion names (Interpretation, Translation, Prompt Design) that appear to be hardcoded for a specific use case. This makes the prompt less flexible and reusable for challenges with different rubric criteria. Consider making this more generic, such as "each criterion defined in the rubric" instead of listing specific names.

Suggested change

	1. How the submission performed on each criterion (Interpretation, Translation, Prompt Design),
	2. How each rubric sub-section influenced the scoring (e.g., Original Insight, Emotional & Cultural Depth, Meaning over Description),
	1. How the submission performed on each criterion defined in the rubric,
	2. How any rubric sub-sections or sub-criteria influenced the scoring,

[Copilot]

The description field references specific rubric sub-section names (Original Insight, Emotional & Cultural Depth, Meaning over Description) that appear to be hardcoded for a specific challenge. This reduces the reusability of the evaluation prompt for other challenges with different rubric structures. Consider making this more generic to work with any rubric configuration.

Suggested change

	1. How the submission performed on each criterion (Interpretation, Translation, Prompt Design),
	2. How each rubric sub-section influenced the scoring (e.g., Original Insight, Emotional & Cultural Depth, Meaning over Description),
	3. How well the submission responded to both Problem Statement and Visual Clues inside the CHALLENGE CONTEXT string,
	1. How the submission performed on each criterion defined in the rubric,
	2. How each rubric sub-section or detailed guideline influenced the scoring, referencing them by name where helpful,
	3. How well the submission responded to both the Problem Statement (if provided) and the CHALLENGE CONTEXT string,