Question 160
At Seagrass Media, support tickets need one of four named routing categories but a free-text model generates dozens of synonyms. Which monitoring step protects the accepted result?
Answer and explanations
Audit confusion patterns and revise category definitions when new subject matter appears. At Seagrass Media, support tickets need one of four named routing categories but a free-text model generates dozens of synonyms. This recurring check guards against the same problem returning.
The Seagrass Media case reports that support tickets need one of four named routing categories but a free-text model generates dozens of synonyms. A check of agent routing between structured and text tools would not follow this failure mode. The relevant continuing control is: Audit confusion patterns and revise category definitions when new subject matter appears.
Monitoring custom agent tool authorization alone misses the source of this case. The immediate Seagrass Media problem is that support tickets need one of four named routing categories but a free-text model generates dozens of synonyms. The suitable recurring safeguard is: Audit confusion patterns and revise category definitions when new subject matter appears.
This measure is focused on end-to-end rag evaluation decomposition. To catch a repeat of the Seagrass Media problem, audit confusion patterns and revise category definitions when new subject matter appears.