Checking what the Bot claims before it is sent

Conversa Labs

Last updated on Oct 13, 2026

Overview

A classic problem with AI support is the Bot saying it did something it did not do: "I've booked you for tomorrow at 2pm", when the calendar refused that slot β€” or when the calendar was never even called. The customer walks away trusting something that does not exist, and the team only finds out later, when the customer shows up at the door.

The platform now runs a final check before the reply is delivered. It works like this:

  1. The Bot writes the reply.
  2. Before sending, the platform compares what the reply claims with what that turn actually executed (which actions ran, and whether they succeeded).
  3. If the reply claims a result that does not hold up, that claim is removed and nothing is written in its place.
  4. The rest of the message is preserved β€” the conversation continues normally. If nothing usable is left, nothing is sent: the turn is not redone and the conversation is handed over to a human, with a private note explaining why.

Two other things changed alongside it:

  • Failures of internal actions become a private note in the conversation, so the team can see what broke.
  • The Bot's content filters now apply to every output β€” text, audio and buttons β€” not just the written message.

Read the limits section before you tell the team about this. It does not make the Bot infallible. It can still be wrong; what the platform does is take down the claim that does not hold up.

Prerequisites

  • Maestro enabled and a Bot configured on the inbox handling the conversation.
  • The per-module tools the Bot uses (calendar, tasks, payments, catalog, contact records…) must be enabled β€” their result is the reference the check compares against.
  • Administrator permission to review the Bot configuration and the content filters.
  • Someone responsible for the conversation (assigned agent or team) to receive and act on the private note.
  • There is no switch to flip: the check is automatic and runs on every Bot reply.

Step by step

The check happens on its own. The steps below are for you to observe and validate that it is working in your operation.

  1. Open a conversation handled by the Bot that involves a concrete action (booking, charging, creating a task, registering an order).
  2. Read the message delivered to the customer. If the Bot tried to claim a result that did not happen, the "done" sentence will not be there β€” and nothing was written in its place.
  3. Open the conversation's private notes. Failures of internal actions are recorded there, with an indication of what did not work.
  4. Fix the cause. It is usually one of these: a disabled tool, a credential/integration that is down, a required piece of data missing (time, amount, document), or a module rule blocking the operation.
  5. Redo the action manually if the customer is waiting, and reply confirming it.
  6. If the same point keeps failing, adjust the Bot's instructions or enable/fix the corresponding tool.

Settings & options

What Where it lives Note
Reply check Automatic Always on, for every Bot reply. Nothing to configure
Unsupported claim Automatic Deleted from the reply, with nothing written in its place. If nothing usable is left, nothing is sent and the conversation is handed over to a human
Failure private note Conversation private notes Never delivered to the customer
Bot content filters Bot configuration Now applied to text, audio and buttons
Autonomy mode Bot configuration Complementary: defines what needs human approval before it happens

About the private note: it is internal by nature. Customers never see private notes on any channel β€” not on WhatsApp, not by email, not in the website chat. It shows up only for your team, inside the conversation.

About the content filters: if you configured the Bot to avoid certain terms, promises or formats, that rule is now applied to the audio reply and to button/list options as well. Before, a button option could slip past a filter that only covered text.

Use cases

  • Calendar: the Bot tries to book, the slot is already taken, and the reply would claim "booked". The claim goes away and nothing takes its place: the customer does not get a confirmation for a slot that does not exist.
  • Payment: the charge was not created because of a missing tax detail. The customer does not get a "link sent" that never arrived: the sentence is removed and nothing takes its place.
  • Task / human handover: the Bot would say "I've escalated you to a specialist", but the handover failed. Without the check, nobody would know β€” with it, there is a trail in the conversation.
  • Catalog / order: the item could not be added. The sentence claiming the order is removed; check the order before retrying.
  • Auditing: when reviewing conversations, failure private notes show where automation breaks most often.

Tips, limits & best practices

This is the most important part of the article. Set expectations with your team using exactly these terms β€” promising more than the platform delivers destroys trust at the first exception.

The check recognizes the most common shapes of a claim, not all of them. It spots the usual ways of saying "I did it" β€” "booked", "created", "registered", "sent", "cancelled" and close variations. A sentence written in an unusual, indirect, very long or roundabout way may slip through.

It does not stop the Bot from being wrong. The check does not fix reasoning, does not validate real-world information and does not guarantee the answer is correct. What it does is remove the claim of a result that does not hold up against what that turn executed; a promise to come back with nothing to keep it follows another rule, described in the turn verification article.

Do not describe it as "the Bot can no longer make things up". That is not true. The Bot can still state something incorrect and have the sentence go through. Describe it as: "the platform removes the most common claims of a completed action when the action did not happen".

Other limits, equally honest:

  • The check looks at the current turn. A claim made in an earlier reply is not reviewed retroactively.
  • The reference is what the turn's tools did β€” not the overall state of the system. If the action happened through another path (an agent did it by hand at the same moment, or a slow process only finished afterwards), the Bot may still be corrected "unnecessarily".
  • The removal is surgical: the claim goes out and nothing comes in. The resulting text can read slightly dry β€” which is better than a false promise.
  • The private note does not replace monitoring. It is a trail inside the conversation, not an alert dashboard.
  • Content filter and reply check are different things: the filter deals with what must not be said; the check deals with what must not be claimed as done.

Best practices:

  • Treat private notes as a work queue: a recorded failure means a customer is waiting.
  • Actually enable the tools the Bot needs to use. A disabled tool produces exactly the scenario the check corrects β€” and frustrates the customer all the same.
  • For sensitive actions (money, cancellation, contact data), combine it with human approval. The check is the last net; approval prevents the problem earlier.
  • Review failure private notes weekly to find the point that breaks the most.

Troubleshooting

  • "The Bot could not confirm, but the action worked": check the result in the relevant module before repeating the action. Record the example for review and reply to the customer with the confirmed result.

  • "The Bot claimed something that did not happen and the sentence went through": this is a known limit β€” the wording fell outside the recognized shapes. Save the excerpt, adjust the Bot's instructions to be more direct at that point and, if possible, turn that step into an action with approval.

  • "The customer saw an internal remark": a private note is never delivered on any channel. If the customer received the text, it was sent as a normal message, not as a note β€” review who replied and how.

  • "The audio said something different from the text": filters now cover audio too. If you find a difference, record the example and review the Bot's instructions and filters.

  • "A button option I had banned showed up": filters now apply to buttons/lists. Confirm the term is actually in the Bot's filter configuration, and not only in the written instruction.

  • "I cannot find the private note": confirm you are looking at the right conversation and that your profile can see internal notes on that inbox.

A price stays in the reply only when the same amount appears, as a monetary value, in the result of some tool that succeeded in that turn: catalog, payment or any other. The check compares the amount only. It does not validate the commercial origin of the number, the product, the variation or the currency, so an amount returned by another tool (a CRM note, for example) also counts as support. With no result carrying that amount, the sentence with the price is removed; the platform does not run a new catalog lookup and does not ask the Bot for another reply. If nothing usable is left, nothing is sent to the customer. In that case the Bot is paused on that conversation and it follows the handoff to a human configured on the Bot, so the team answers, unless the turn has already handed the conversation off or the Bot is in silent mode. If part of the text is left and the verification policy is set to hold unverified replies, that remainder, already without the price, waits for approval, and approving sends the held text without generating a new reply. Before giving a price to the customer, check the product, variation and currency in the evidence. A failure does not erase another successful call in the same turn.

See also