# Chain-of-Thought Evaluation Guide

**Version:** 1.0 · **domainXpert Annotation Standards**

---

## What Is Chain-of-Thought?

Chain-of-thought (CoT) responses show the model's reasoning steps before arriving at a conclusion. Your job is to evaluate whether those steps are logically valid, factually grounded, and necessary.

---

## Evaluation Criteria

### 1. Logical Validity
Does each step follow from the previous one? Are there unjustified leaps?

### 2. Factual Grounding
Are the premises in each step correct? A valid argument built on false premises is still wrong.

### 3. Relevance
Are all steps necessary? Flag steps that are tangential or that pad the response without adding value.

### 4. Conclusion Alignment
Does the final answer follow from the reasoning shown? Flag cases where the conclusion contradicts or ignores the chain.

### 5. Appropriate Uncertainty
Does the model acknowledge when a step involves an assumption or when evidence is limited?

---

## Scoring Guide

| Score | Meaning |
|---|---|
| 5 | All steps valid, grounded, and necessary; conclusion follows |
| 4 | Minor issues in one step; conclusion still sound |
| 3 | One significant logical or factual error; conclusion partially affected |
| 2 | Multiple errors; conclusion unreliable |
| 1 | Reasoning is fundamentally flawed or fabricated |

---

*For questions, contact your project coordinator.*