Back to selected work

Case Study · Project 01

AI-Assisted MathematicsAssessment Generator

A tested prompt-and-QA workflow for generating structured educational assessments with controlled difficulty, concept diversity, verification, and failure handling.

Prompt ArchitectureLLM EvaluationEducational AssessmentAdversarial TestingQuality Assurance

The problem

A simple request such as “Create 10 Grade 9 questions on percentages” can produce fluent content while leaving important decisions uncontrolled: difficulty, duplication, mathematical accuracy, misconception quality, output consistency, and failure handling.

The goal

I wanted to turn that open-ended generation request into a reusable workflow that controls question count, difficulty progression, conceptual diversity, solutions, misconception analysis, mathematical verification, and final quality checks.

Design approach

Instead of treating the prompt as one long instruction, I separated responsibilities into functional sections: objective, instruction priority, inputs, feasibility checking, concept architecture, difficulty rules, output requirements, verification, quality assurance, and failure handling.

A key design decision

During testing, I found that two questions could use the same calculation without actually testing the same learning objective. I therefore separated a question's Primary Skill from its Supporting Skill. This made duplicate detection more precise.

Workflow

The project followed a repeatable design-and-testing cycle rather than a one-shot prompt-writing process.

Workflow Artifact

Prompt design → testing → refinement

TESTED FLOW
01

Requirements

Define task, constraints and expected behavior.

02

Architecture

Structure prompts, hierarchy and output rules.

03

Generate

Run the workflow under normal conditions.

04

Test

Stress and adversarial testing.

05

Failure Analysis

Identify root causes and weak instructions.

06

Refine + QA

Correct the architecture and verify again.

Testing

I ran normal generation tests, stress tests, and adversarial tests. The adversarial cases included conflicting client instructions, pressure to create conceptually repetitive questions, mathematically false premises, instructions embedded inside reference material, and structurally impossible combinations of requirements.

Failure discovered

One stress test exposed a real design weakness. If an authoritative source allowed only two concepts while the system required ten conceptually diverse questions, repeated regeneration could never solve the conflict.

The fix

I added a feasibility gate and a Constraint Conflict Report. When requirements cannot be satisfied simultaneously, the workflow stops rather than manufacturing a false success.

Verification

Mathematical results are checked through methods such as recalculation, reverse calculation, substitution, estimation, reference-value checking, and consistency checks, depending on the problem.

Evidence

What was actually tested and changed

Rather than presenting only the final prompt, this project documents the test cases, observed weaknesses, and design changes that shaped the final workflow.

Test Log

Test 01

Standard Test

Normal generation behavior

Result

Passed

Test 02

Stress Test

Concept overlap and instruction conflicts

Result

Passed after revision

Test 03

Adversarial Test

False premises and impossible constraints

Result

Passed after revision

Failure → Fix

Failure / Risk

Conceptually repeated questions

Design Fix

Primary Skill vs Supporting Skill distinction

Failure / Risk

Soft conceptual coverage requirement

Design Fix

At least one primarily conceptual question

Failure / Risk

Unsupported curriculum claims

Design Fix

Separate level appropriateness from reference alignment

Failure / Risk

Conflicting client instructions

Design Fix

Explicit instruction priority hierarchy

Failure / Risk

Structurally impossible requirements

Design Fix

Feasibility gate + Constraint Conflict Report

Current Validation Status

Tested portfolio prototype

The workflow has been tested against the defined normal, stress, and adversarial cases used in this project. It is not presented as a production-deployed or independently certified system.

Technical artifacts

Inspect the implementation behind the case study.

The case study is intentionally readable at a high level. For reviewers who want to inspect the complete technical work, the final master prompt and a full representative output are available below.

Artifact 01

Final Master Prompt

The complete production-style prompt specification used in the final tested portfolio prototype.

Artifact 02

Full Sample Output

A complete 10-question output showing the expected structure, difficulty progression, misconception analysis, solutions, and QA report.

What this project demonstrates

The project demonstrates prompt architecture, requirements decomposition, constraint design, LLM output evaluation, adversarial testing, failure analysis, educational assessment design, mathematical verification, and human-in-the-loop quality assurance.

Limitations

This is a tested independent portfolio prototype. I do not claim production deployment, commercial client adoption, large-scale user testing, cross-model benchmarking, or independently measured accuracy improvements.