Testing

Vitest

The project uses Vitest for unit testing.

To run the tests:

pnpm test

AI Evals

The project uses Evalite for testing AI behavior. Eval files are located in tests/evals and follow this pattern:

evalite("Test description", {
  // Test data generator
  data: async () => [
    {
      input: "test input",
      expected: "expected output",
    },
  ],
 
  // The task to evaluate
  task: async (input) => {
    // Implementation that uses the AI model
    return result;
  },
 
  // Scoring methods from autoevals
  scorers: [Factuality],
});

To run eval tests:

pnpm eval:dev

Testing

Vitest

AI Evals

On this page