Key takeaways
- Choose a recurring task with a recognizable output and a way to assess quality.
- Compare process changes and existing software with an AI option.
- Measure staff review and failures, and examine differences between users and case types.
A team looking for an AI use case can start with a recurring task that has a recognizable output: an answered customer request, a classified document or a completed report. Employees who perform it can identify where time goes and what makes a result acceptable. That gives a trial something to measure.
Results can differ substantially between people doing the same job. In a 2025 study, Erik Brynjolfsson, Danielle Li and Lindsey Raymond examined an AI assistant used by 5,172 customer-support workers at a business software company. Access increased issues resolved per hour by 15 percent on average. Less experienced workers improved more, while the most experienced and highly skilled workers saw small speed gains and small declines in quality.
The assistant offered suggestions during conversations; staff could edit or ignore them. Those conditions matter when designing a trial elsewhere. A tool sending replies without review would be a different intervention, and a department-wide average could obscure differences among users.
Describe one task and its current difficulty
Ask employees for recent cases that took longer than expected or needed rework. Follow a case through the existing process and distinguish time spent finding information, interpreting it, entering it elsewhere and waiting for approval. Identify which part a proposed tool would handle.
A task needs sufficient detail for someone to test it. Preparing a draft reply from an approved returns policy is more specific than improving customer service. Extracting dates and amounts from a standard form is more specific than automating finance. Keep authority to act separate from authority to prepare a draft.
Examples of bounded trials
| Proposed task | What to compare | Errors to inspect |
|---|---|---|
| Draft replies from an approved policy | Time to produce an accepted reply, including review. | Wrong policy version, unsupported promises or missed customer context. |
| Extract fields from incoming forms | Correct fields and staff correction time. | Missing pages, ambiguous handwriting or values assigned to the wrong field. |
| Classify requests for staff routing | Routing accuracy and time until the right team receives a request. | Unrecognized categories and urgent requests sent to the wrong queue. |
| Prepare a recurring report | Preparation time and agreement with source records. | Incorrect totals, missing periods or unsupported explanations. |
Illustrative trial designs.
Compare the proposed tool with simpler changes
Check whether existing software can remove the troublesome step. A standard template might reduce repeated drafting; a form with required fields might prevent missing information; a direct connection between applications might avoid retyping. Ask an engineer or software administrator what those options would cost alongside an AI trial.
Also check whether employees can legally and practically supply the required material. If a document cannot be shared with the proposed service, identify an approved environment or change the trial scope. Do not assume that removing a person’s name resolves every permission or confidentiality issue.
Compare users and cases separately
Collect a baseline from recent work before introducing the tool. Record completion time, corrections and a quality measure that the process owner can explain. During the trial, include setup and review time. Keep cases that fail or need a human fallback in the results.
Use comparable cases and account for differences in experience. Where practical, assign cases or access in a way that avoids giving only easier work to the AI group. For a small trial, retain case-level notes instead of presenting a precise improvement rate from a handful of examples.
Examine whether new staff and experienced staff benefit differently, whether a particular document type causes problems and whether performance changes under a heavier queue. Use those findings to narrow the proposed deployment or improve it before adding users.
Ask the process owner to review actual examples alongside the measurements. Agree which error must be corrected before release, which group should try the next version and how much another trial can cost. Preserve the unsuccessful cases so they remain part of subsequent testing.
