Install & Compatibility
Where this runs
No compatibility data collected yet for this library.
Code
Verified usage
Verified import paths — ran on the pinned version, not inferred.
ConversationEvalCaseSchema
✓ import { ConversationEvalCaseSchema } from 'mongodb-assistant-eval/schema'
✗ import { ConversationEvalCaseSchema } from 'mongodb-assistant-eval'
Schema is exported from the /schema subpath, not the main entry.
getConversationEvalCasesFromYaml
✓ import { getConversationEvalCasesFromYaml } from 'mongodb-assistant-eval/datasets'
✗ import { getConversationEvalCasesFromYaml } from 'mongodb-assistant-eval'
Loaders are in /datasets subpath.
createMongoDbAssistantEvalCli
✓ import { createMongoDbAssistantEvalCli } from 'mongodb-assistant-eval/cli'
✗ import { createMongoDbAssistantEvalCli } from 'mongodb-assistant-eval'
CLI helper is in /cli subpath.
ConversationEvalScorer
✓ import type { ConversationEvalScorer } from 'mongodb-assistant-eval/eval'
✗ import { ConversationEvalScorer } from 'mongodb-assistant-eval/eval'
Scorer types are exported as types from /eval subpath.
Shows how to load eval cases from YAML, define a task, and create a CLI with Slack reporter.
import { ConversationEvalCaseSchema } from 'mongodb-assistant-eval/schema';
import { getConversationEvalCasesFromYaml } from 'mongodb-assistant-eval/datasets';
import { createMongoDbAssistantEvalCli } from 'mongodb-assistant-eval/cli';
import { createSlackReporter } from 'mongodb-assistant-eval/reporters';
import { createEvalJudgeModel } from 'mongodb-assistant-eval/scorers';
import fs from 'fs';
const yaml = fs.readFileSync('./cases.yaml', 'utf8');
const cases = getConversationEvalCasesFromYaml({ yaml });
const cli = createMongoDbAssistantEvalCli({
projectName: 'my-eval-project',
datasets: { default: cases },
models: { gpt4: 'gpt-4' },
tasks: {
chat: async ({ input }) => {
return { messages: [{ role: 'assistant', content: 'Hello!' }] };
},
},
scorers: {},
validators: {},
reporters: {
slack: createSlackReporter({
slackToken: process.env.SLACK_BOT_TOKEN ?? '',
slackConversationId: process.env.SLACK_EVAL_CHANNEL_ID ?? '',
}),
},
});
cli.parse();
Errors
Common errors & fixes
Cannot find module 'mongodb-assistant-eval' or its corresponding type declarations.
Importing from the main package entry instead of subpath like 'mongodb-assistant-eval/schema'.
fixUse correct subpath import: import { ConversationEvalCaseSchema } from 'mongodb-assistant-eval/schema' TypeError: createMongoDbAssistantEvalCli is not a function
Importing from 'mongodb-assistant-eval' instead of 'mongodb-assistant-eval/cli'.
fiximport { createMongoDbAssistantEvalCli } from 'mongodb-assistant-eval/cli' Error: Missing required peer dependencies: braintrust, openai, zod
Required peer deps not installed.
fixnpm install braintrust openai zod
Audit
Dependencies
braintrustrequiredCore eval runner, datasets, results
openairequiredTool call types used by scorers
zodrequiredSchema validation, makeConversationEvalCaseSchema
aioptionalcreateEvalJudgeModel for LLM-as-judge scorers
@ai-sdk/openaioptionalcreateEvalJudgeModel for LLM-as-judge scorers
@ai-sdk/azureoptionalcreateEvalJudgeModel for LLM-as-judge scorers
autoevalsoptionalBuilt-in LLM scorers (Factuality, Faithfulness, etc.)
mongodboptionalgetConversationEvalCasesFromMongoDb loader
@slack/web-apioptionalSlack evaluation reporting
slack-block-builderoptionalSlack evaluation reporting
slackify-markdownoptionalSlack evaluation reporting