Skip to main content
XCurveAI
  • New

AI Evals for Product Owners and Business Analysts

PA-201

How to tell if an AI feature actually works. For product and analysis roles who must decide whether an AI feature is good enough to ship, and prove it to others. Best taken after AI for Product Owners and Business Analysts.

Level
Practitioner
Duration
Two 4-hour sessions
Delivery
Live virtual, Private team
Language
English
Suitable for
IndividualsOrganisations
See upcoming datesTrain your team

Overview

How to tell if an AI feature actually works

For product and analysis roles who must decide whether an AI feature is good enough to ship, and prove it to others. You define what good looks like before the feature is built, then test it with evaluations you can run and read yourself.

You leave with

An evaluation plan and scoring rubric for one of your own AI features.

Coming soon. Best taken after the flagship course, AI for Product Owners and Business Analysts. Join the waitlist to hear when the first cohort is scheduled.

What you will learn

  • Define what good looks like for an AI feature before it is built
  • Write test cases and scoring rubrics for AI outputs
  • Run simple evaluations and read the results with confidence
  • Set release criteria that risk and compliance teams can sign off

Who it is for

Product owners, business analysts and product managers who must decide whether an AI feature is good enough to ship, and prove it to others.

Prerequisites

Best taken after AI for Product Owners and Business Analysts, or equivalent experience writing requirements for AI features.

What's included

  • Evaluation planFor one of your own AI features
  • Scoring rubricReady to use with your team
  • Certificate of completion

Course outline

  1. Defining good before you build2 topics, 1 objective

    Topics

    • What good looks like for an AI feature
    • Acceptance for probabilistic outputs

    Objectives

    • Define what good looks like for an AI feature before it is built
  2. Test cases and scoring rubrics2 topics, 1 objective

    Topics

    • Writing test cases for AI outputs
    • Building a scoring rubric

    Objectives

    • Write test cases and scoring rubrics for AI outputs
  3. Running and reading evaluations2 topics, 1 objective

    Topics

    • Simple evaluations you can run yourself
    • Reading results with confidence

    Objectives

    • Run simple evaluations and read the results with confidence
  4. Release criteria2 topics, 1 objective

    Topics

    • Criteria risk and compliance can sign off
    • Your evaluation plan and rubric

    Objectives

    • Set release criteria that risk and compliance teams can sign off

Questions

Course

Should I take the flagship course first?

Yes, where possible. This course builds on the requirements and backlog work from AI for Product Owners and Business Analysts.