Xbox researchers gave a new Microsoft AI model more than 10,000 comments gathered from surveys, Steam reviews and posts on X. Their assignment was to sort players’ remarks into categories researchers had already chosen. Microsoft says Decision-1 was competitive in quality with its comparison model while processing the comments faster and at lower cost.
That trial is among the first uses Microsoft has described for Microsoft-Decision-1, introduced on October 9. Built by further training Qwen3.5-9B, it assigns numbers to possible answers supplied by a developer. Microsoft is selling it for jobs including classifying documents, choosing which software should handle a request and grading an AI assistant’s response.
A Microsoft Foundry demonstration shows how those judgments can shape customer service: software checks urgency, chooses a support team and sets a priority. A separate language model can then write a reply. Businesses can use a small, specialized model at several points in that process, reserving a text-generating model for drafting.
OpenRouter lists the price at 4.2 cents per million input tokens, the units AI systems use to process text, with no charge for output. At that rate, a million requests averaging 1,000 input tokens would incur $42 in model charges. This calculation assumes one request per item and excludes other services or labor. Instructions and answer choices also consume input, so a comment’s length alone cannot determine its cost.
Microsoft’s technical documentation describes three formats: a probability for a yes-or-no question, a choice with probabilities for each option, and an ordered rating with probabilities for each level. None includes a written explanation.
Two different judgments can produce the same rating
For ratings, Microsoft averages the level numbers using their probabilities as weights. Its example ranks an incident from cosmetic through minor and major to critical, numbered zero through three. Microsoft advises using these scores for ordering items and setting thresholds, “rather than as absolute, calibrated ratings.”
A simple calculation shows why retaining the probabilities matters. Suppose a model assigns a 50 percent probability to level zero and 50 percent to level three. Its average is 1.5. A different assessment, split evenly between levels one and two, also averages 1.5. These are illustrative distributions, not measurements of Decision-1.
In the first case, the model is divided between the least and most serious classifications. In the second, both possibilities sit in the middle. A dashboard displaying only 1.5 would make them look identical. A support manager could use that difference to decide which incident needs immediate investigation.
Achint Srivastava, a Microsoft vice president who announced the release, wrote that applications use confidence scores “to decide when to act, defer, or ask for review.” Researchers call the match between confidence and accuracy calibration: among many predictions assigned an 80 percent chance of being correct, roughly four in five should be correct. A system can often choose the right answer while assigning probabilities that exaggerate how reliably it does so.
In a 2017 study, Cornell researchers Chuan Guo, Geoff Pleiss, Yu Sun and Kilian Weinberger found that improvements in neural-network accuracy could accompany worse calibration. Their experiments covered image and document classification. Adjusting the models’ confidence scores after training often improved calibration. That research predates Decision-1 and supplies a way to understand its scores, rather than an evaluation of Microsoft’s release.
Microsoft tells developers to compare confidence with observed accuracy on examples from their own applications. It also recommends testing ambiguous cases and measuring processing time under expected demand. Those checks require messages whose correct classifications are already known, allowing a company to compare what the model predicts with decisions made by its reviewers.
The deployment guide suggests including an option to abstain when a forced choice would be inappropriate. It cautions that question wording can affect scores and advises meaningful human review for consequential decisions about people.
Microsoft also plans to rebuild Decision-1 using other underlying models, including its own and OpenAI’s. Its October 9 announcement did not give a release date for those versions.
.jpg){kind=link}