Building Evaluation Frameworks for LLM Applications
Design evaluation pipelines for LLM applications: automated metrics, human evaluation protocols, regression tests, prompt versioning and monitoring.
You can't ship an LLM feature without knowing whether it works. Unlike traditional software where a test either passes or fails, LLM outputs exist on a spectrum of quality. The same prompt might produce an excellent response one minute and a mediocre one the next. Without systematic evaluation, you're shipping a feature where quality is a coin flip.
Evaluation frameworks for LLM applications need automated metrics for fast feedback, human evaluation for nuanced quality assessment, and regression testing to catch degradation when prompts or models change.
Defining Evaluation Dimensions
Different applications need different quality dimensions. A summarizer needs faithfulness and conciseness. A code generator needs correctness and security. Define these dimensions before writing a single evaluation function.
// ❌ Single-score evaluation — tells you nothing actionable
function evaluate(output: string): number {
// "Quality score" from 0 to 1
return someBlackBoxScore(output);
}
// Is a 0.7 good? Bad? Good summary but wrong facts?// ✅ Multi-dimensional evaluation framework
interface EvalDimension {
name: string;
description: string;
scorer: (input: EvalInput) => Promise<DimensionScore>;
weight: number;
threshold: number; // Minimum acceptable score
}
interface EvalInput {
prompt: string;
context?: string;
expectedOutput?: string;
actualOutput: string;
metadata: Record<string, unknown>;
}
interface DimensionScore {
dimension: string;
score: number; // 0.0 to 1.0
reasoning: string;
evidence: string[];
}
interface EvalResult {
inputId: string;
scores: DimensionScore[];
overallScore: number;
passed: boolean;
failedDimensions: string[];
}
class EvaluationFramework {
private dimensions: EvalDimension[] = [];
addDimension(dimension: EvalDimension): void {
this.dimensions.push(dimension);
}
async evaluate(input: EvalInput): Promise<EvalResult> {
const scores: DimensionScore[] = [];
const failedDimensions: string[] = [];
for (const dim of this.dimensions) {
const score = await dim.scorer(input);
scores.push(score);
if (score.score < dim.threshold) {
failedDimensions.push(dim.name);
}
}
const totalWeight = this.dimensions.reduce(
(sum, d) => sum + d.weight,
0
);
const overallScore = scores.reduce((sum, score, i) => {
return sum + score.score * this.dimensions[i].weight;
}, 0) / totalWeight;
return {
inputId: input.metadata.id as string ?? "unknown",
scores,
overallScore,
passed: failedDimensions.length === 0,
failedDimensions,
};
}
}Automated Scoring Functions
Some quality dimensions can be evaluated automatically. Factual consistency, format compliance, and toxicity detection don't require human judgment for every evaluation.
// Factual consistency: check if output contradicts the source
async function scoreFactualConsistency(
input: EvalInput
): Promise<DimensionScore> {
if (!input.context) {
return {
dimension: "factual_consistency",
score: 1.0,
reasoning: "No source context to check against",
evidence: [],
};
}
// Extract claims from the output
const claims = extractClaims(input.actualOutput);
const supportedClaims: string[] = [];
const unsupportedClaims: string[] = [];
for (const claim of claims) {
const isSupported = await checkClaimAgainstSource(
claim,
input.context
);
if (isSupported) {
supportedClaims.push(claim);
} else {
unsupportedClaims.push(claim);
}
}
const score = claims.length > 0
? supportedClaims.length / claims.length
: 1.0;
return {
dimension: "factual_consistency",
score,
reasoning:
`${supportedClaims.length}/${claims.length} claims ` +
`supported by source`,
evidence: unsupportedClaims.map(
(c) => `Unsupported: "${c}"`
),
};
}
// Format compliance: check structural requirements
function scoreFormatCompliance(
input: EvalInput
): Promise<DimensionScore> {
const checks: { name: string; passed: boolean }[] = [];
const output = input.actualOutput;
const rules = input.metadata.formatRules as FormatRule[];
for (const rule of rules ?? []) {
switch (rule.type) {
case "max_length":
checks.push({
name: `Max length ${rule.value}`,
passed: output.length <= (rule.value as number),
});
break;
case "contains_section":
checks.push({
name: `Contains "${rule.value}"`,
passed: output.includes(rule.value as string),
});
break;
case "json_valid":
try {
JSON.parse(output);
checks.push({ name: "Valid JSON", passed: true });
} catch {
checks.push({ name: "Valid JSON", passed: false });
}
break;
}
}
const passed = checks.filter((c) => c.passed).length;
const score = checks.length > 0 ? passed / checks.length : 1.0;
return Promise.resolve({
dimension: "format_compliance",
score,
reasoning: `${passed}/${checks.length} format checks passed`,
evidence: checks
.filter((c) => !c.passed)
.map((c) => `Failed: ${c.name}`),
});
}
interface FormatRule {
type: "max_length" | "contains_section" | "json_valid";
value: string | number;
}
function extractClaims(text: string): string[] {
// Split into sentences and filter for factual assertions
return text
.split(/[.!?]+/)
.map((s) => s.trim())
.filter((s) => s.length > 20);
}
async function checkClaimAgainstSource(
claim: string,
source: string
): Promise<boolean> {
// Semantic similarity check
const similarity = computeCosineSimilarity(claim, source);
return similarity > 0.6;
}
function computeCosineSimilarity(a: string, b: string): number {
// Placeholder for embedding-based similarity
return 0.8;
}Human Evaluation Protocol
Automated metrics have blind spots. Human evaluation catches quality issues that no metric can detect—tone, helpfulness, coherence, and whether the response actually answers the question.
interface HumanEvalTask {
id: string;
input: EvalInput;
dimensions: HumanEvalDimension[];
assignedTo: string;
status: "pending" | "in_progress" | "completed";
results?: HumanEvalResult;
}
interface HumanEvalDimension {
name: string;
description: string;
scale: { min: number; max: number; labels: string[] };
}
interface HumanEvalResult {
evaluatorId: string;
scores: Map<string, number>;
freeformFeedback: string;
completedAt: Date;
timeSpentSeconds: number;
}
class HumanEvalPipeline {
private tasks: Map<string, HumanEvalTask> = new Map();
createSampledBatch(
allOutputs: EvalInput[],
sampleSize: number,
strategyFn: (outputs: EvalInput[]) => EvalInput[]
): HumanEvalTask[] {
// Sample strategically — not randomly
const sampled = strategyFn(allOutputs).slice(0, sampleSize);
return sampled.map((input) => {
const task: HumanEvalTask = {
id: `eval-${Date.now()}-${Math.random().toString(36).slice(2, 8)}`,
input,
dimensions: [
{
name: "Helpfulness",
description:
"Does the response address the user's actual need?",
scale: {
min: 1,
max: 5,
labels: [
"Not helpful",
"Slightly helpful",
"Moderately helpful",
"Helpful",
"Very helpful",
],
},
},
{
name: "Accuracy",
description:
"Are the factual claims in the response correct?",
scale: {
min: 1,
max: 5,
labels: [
"Incorrect",
"Mostly incorrect",
"Mixed",
"Mostly correct",
"Fully correct",
],
},
},
],
assignedTo: "",
status: "pending",
};
this.tasks.set(task.id, task);
return task;
});
}
}
// Sampling strategy: focus on edge cases and low-confidence outputs
function edgeCaseSampling(outputs: EvalInput[]): EvalInput[] {
return outputs.sort((a, b) => {
// Prioritize longer outputs (more room for errors),
// outputs with low automated scores, and diverse prompts
const aLength = a.actualOutput.length;
const bLength = b.actualOutput.length;
return bLength - aLength;
});
}Regression Testing Across Prompt Versions
When you change a prompt, you need to know if the change improved or degraded quality across your test set. This requires versioned evaluation.
interface PromptVersion {
version: string;
template: string;
changedAt: Date;
changelog: string;
}
interface RegressionReport {
baselineVersion: string;
candidateVersion: string;
testSetSize: number;
improvements: DimensionComparison[];
regressions: DimensionComparison[];
neutral: DimensionComparison[];
recommendation: "ship" | "investigate" | "rollback";
}
interface DimensionComparison {
dimension: string;
baselineAvg: number;
candidateAvg: number;
delta: number;
significant: boolean;
}
async function runRegressionTest(
baseline: PromptVersion,
candidate: PromptVersion,
testSet: EvalInput[],
framework: EvaluationFramework
): Promise<RegressionReport> {
const baselineResults: EvalResult[] = [];
const candidateResults: EvalResult[] = [];
for (const testCase of testSet) {
const baselineOutput = await generateWithPrompt(
baseline.template,
testCase
);
const candidateOutput = await generateWithPrompt(
candidate.template,
testCase
);
baselineResults.push(
await framework.evaluate({
...testCase,
actualOutput: baselineOutput,
})
);
candidateResults.push(
await framework.evaluate({
...testCase,
actualOutput: candidateOutput,
})
);
}
return compareResults(
baseline.version,
candidate.version,
baselineResults,
candidateResults
);
}
function compareResults(
baselineVersion: string,
candidateVersion: string,
baseline: EvalResult[],
candidate: EvalResult[]
): RegressionReport {
const dimensions = new Set(
baseline.flatMap((r) => r.scores.map((s) => s.dimension))
);
const comparisons: DimensionComparison[] = [];
for (const dim of dimensions) {
const baseScores = baseline.map(
(r) => r.scores.find((s) => s.dimension === dim)?.score ?? 0
);
const candScores = candidate.map(
(r) => r.scores.find((s) => s.dimension === dim)?.score ?? 0
);
const baseAvg =
baseScores.reduce((a, b) => a + b, 0) / baseScores.length;
const candAvg =
candScores.reduce((a, b) => a + b, 0) / candScores.length;
const delta = candAvg - baseAvg;
comparisons.push({
dimension: dim,
baselineAvg: baseAvg,
candidateAvg: candAvg,
delta,
significant: Math.abs(delta) > 0.05,
});
}
const regressions = comparisons.filter(
(c) => c.significant && c.delta < 0
);
const improvements = comparisons.filter(
(c) => c.significant && c.delta > 0
);
return {
baselineVersion,
candidateVersion,
testSetSize: baseline.length,
improvements,
regressions,
neutral: comparisons.filter((c) => !c.significant),
recommendation:
regressions.length > 0
? "investigate"
: improvements.length > 0
? "ship"
: "investigate",
};
}
async function generateWithPrompt(
template: string,
input: EvalInput
): Promise<string> {
// Placeholder for LLM call
return "";
}Key Takeaways
LLM evaluation requires multi-dimensional scoring because a single quality number tells you nothing actionable—break evaluation into specific dimensions like factual consistency, format compliance, helpfulness, and accuracy. Automate what you can: format checks, claim verification against source documents, and toxicity detection provide fast feedback without human effort. Sample strategically for human evaluation—focus on edge cases, long outputs, and low-confidence predictions rather than random sampling. Version your prompts and run regression tests against a fixed test set whenever you change them, comparing dimension-by-dimension scores to catch degradation. Set per-dimension quality thresholds that gate deployment: a response that's well-formatted but factually wrong should not ship. The evaluation framework is not a one-time setup—it evolves as you discover new failure modes in production and encode them as new test cases and scoring dimensions.
