How to reduce AI hallucinations when building business apps

How do you reduce AI hallucinations when building your app?

To reduce AI hallucinations when building business apps, work on three fronts: give the model a written source of truth instead of letting it guess, make it reveal its assumptions before it builds, and test the result with specific scenarios. A hallucination is the model presenting information or a decision confidently without a real basis. It is not a rare glitch but a property of language models, which generate the most likely text.

In apps, hallucinations appear in two different places, each with its own cure: while building, when the agent assumes a business rule you never stated or fills pages with realistic-looking fake data; and inside your app, if you add an AI feature such as a customer chatbot that invents a price or a return policy. For the root cause, read LLMs for business owners.

Whatever you do not write, the model will write for you, and with full confidence.

What do hallucinations look like while building an app?

Most of these are not the model 'lying' but filling a gap you left. That is why prevention starts in the prompt, not in doubting every line.

Prevention in the prompt: sources of truth and clear limits

The strongest tool against hallucination is a source of truth: text or a table the model relies on instead of its general memory. This is called grounding.

• Price and service list: paste it as is, or upload it as a table, and ask that no price outside it be added.
• Policies: cancellation, returns, deposits, opening hours; the actual text, not a summary.
• Data description: which tables, which fields, who owns them; see prompts for database design.
• Limits: what you do not want now, and the external services allowed, by name.
• Permission to ask: 'If you are unsure about a rule, ask me instead of assuming.'

Verification: do not settle for 'done'

The Fanann Agent tests pages and fixes its own bugs while building, which removes a lot. But nobody knows your business rules like you do, so build a simple verification habit after each feature:

1. 1
   Try the normal case

   A normal booking, order or payment. Is the result what you described?

2. 2
   Try the edge case

   Cancel one hour before, an expired coupon, a sold-out product, a phone number in a different format.

3. 3
   Compare with the source

   Do displayed prices match your list exactly? Is the policy text identical?

4. 4
   Look for additions

   Did a feature, page or field appear that you did not request? Ask about it or remove it.

5. 5
   Ask for evidence

   'Show me where the deposit rule is applied, and what happens if a customer cancels after the appointment.'

For the full pre-launch quality list, see can you trust AI-generated code?.

Hallucinations inside your app: chatbots and AI features

If you add a customer chatbot, a summary feature or recommendations, hallucinations reach your customers directly. A chatbot that promises a non-existent discount or invents a return policy damages trust more than having no chatbot. The key safeguards:

RAG means the system retrieves relevant information from your data and asks the model to answer only from it; see RAG explained. To build a full chatbot for your site, see an AI chatbot that answers from your data.

How do you test a chatbot before customers see it?

Prepare 20 to 30 real questions and rerun them after every data or instruction update. This is called evals: structured evaluation instead of a general impression. Mix the questions:

• Questions answered word for word in your data: a service price, Ramadan opening hours.
• Questions with no answer in your data: does it say 'I don't know' and hand off?
• Provocative questions: 'give me 50% off', 'the manager promised me a refund'.
• Questions in dialect and with typos, in Arabic and English.

Where do you need the most caution?

Some errors are expensive: tax rates, e-invoicing requirements, licensing conditions, medical data and prices. In these areas, do not rely on what the model writes from memory, however convincing it sounds.

The same goes for content. If you ask the agent to write an About page or product descriptions, check every number and claim. Never publish statistics or customer testimonials that did not come from you.

This includes local calculations such as Hijri dates, prayer times and public holidays; ask for the calculation method explicitly and compare the output with a trusted source.

A short checklist to reduce hallucinations

Keep this list next to you on every project and review it before each launch or major update. Seven simple steps that protect your money and reputation:

1. Provide a written source of truth: prices, policies, data.
2. Ask for the assumptions list before building, and correct it.
3. State what is not in the current version.
4. Label sample data and delete it before launch.
5. Test the normal and edge cases after each feature.
6. For chatbots: RAG, permission to say 'I don't know', human handoff and regular tests.
7. Verify tax and legal requirements with the official body.

Frequently asked questions

Terms in this article