First-consultation intake agent for Leal Abogados

The short version

Employment-law inquiries reach Leal Abogados, a law firm in Bogotá, on WhatsApp at all hours. The agent runs the first conversation, asks for data-privacy consent before anything else, flags urgent cases with hard rules on top of the AI, and leaves the lawyer a brief: summary, key dates, documents to bring and pending questions. It is a working demo on fictional data. The pilot proposal is under discussion with the firm.

Context

The firm already had a chat on its website. It answered from the first second, never asked for consent, and if the person did not click through to WhatsApp, the conversation was lost: no brief, no alert, no count. The firm could not tell how many consultations arrived or how many were urgent.

The demo was built to show what a pilot would look like: the same first conversation, but with consent, hard rules, a guard on every answer and a panel for the lawyer.

The problem

• A dismissal during pregnancy and a question about overtime arrive in the same inbox and look the same.
• Someone at the firm reads the chat the next day, asks the basics again and only then decides if it is urgent.
• People type their ID number and bank account without being asked. That data should never reach an AI provider.
• A generic chatbot will happily calculate a severance payment or cite an article. A law firm cannot let that happen.

The approach

Consent first. The server rejects any message before the person accepts the data-privacy notice (Colombian Law 1581). Date, time and the version of the notice are stored with the brief.

Hard rules above the model. Pregnancy, medical leave, protected status, threats, a hearing in the next days or facts close to the three-year mark are always marked urgent, whatever the model says. The highest urgency wins. The alert reaches the lawyer’s panel before the conversation ends.

A guard on every answer. Before showing a reply, the guard checks for amounts and formulas, promises, legal deadlines, citations of articles, final advice, requests for ID numbers or files, and any wording that presents the assistant as a lawyer. If something slips through, it asks the model for another wording; if it persists, the sentence is removed.

Minimal data. ID, account and card numbers are masked before storing anything or sending it to the AI. The assistant never cites laws to the client; the brief adds verified references for the lawyer only.

What I built

• A client screen, styled like WhatsApp, that introduces the assistant as automated, asks for consent and then one question at a time.
• A lawyer panel with pilot indicators (briefs, urgent cases, complete briefs, time to brief) and each brief with summary, contact, urgency and why, pending questions, documents to bring, verified references and the full conversation.
• Hard rules for urgency and for out-of-scope topics (the agent only handles employment law and sends the rest to a human).
• Three pre-recorded guided cases so the demo runs without internet or an API key.
• A brief code at the end of every conversation, so the person can refer to it.

Evaluation

The 30 acceptance cases cover dismissals, unpaid wages, harassment, a workplace accident, a minor, a domestic worker, people who demand an exact amount or a legal deadline, an attempt to manipulate the assistant into acting as a lawyer, and two cases where the person types an ID and an account number.

Plus 78 offline tests where the Claude SDK is replaced by a fake transport, and 23 browser checks in Edge on desktop and mobile. The evaluator uses an adversarial simulated engine that tries to give advice, amounts and file requests, so the guard has to stop them. What it does not test yet: semantic quality, which needs the real model and the lawyer’s review. The pilot only goes live if 27 of 30 cases pass, all urgent cases are escalated and there is zero prohibited answers.

What broke and how I found it

• A bank account that looked like a phone number was not masked. Case CP-29: ten digits, no bank words nearby. The masking rule treated it as a phone and let it through. Fixed: a number identified as an account is masked even when its length and prefix look like a mobile number; mobiles without bank context are kept.
• The out-of-scope card appeared more than once. When both the rules and the engine detected another legal area, the session showed two referral cards. Fixed in the conversation flow: one card per session, whatever the history says.
• The guard missed requests for audio and bank data. It caught requests for documents but not “send me the audio” or “give me your account”. Extended with new patterns and a regression test for each.
• The honest number is 1 of 30, not 30 of 30. The simulated run approves every rule check, but only one case (consent refused) is fully provable without the real model. I report it that way.

What I learned

• Rules first, model second. The parts that protect the firm do not depend on the model being right that day.
• An evaluator that cannot fail is useless. The regressions include deliberately altered expectations: if the guard or the masking is turned off, the evaluator must fail. It does.
• Say what is pending. 29 cases still need the real-AI run and the lawyer’s review of the 30 expected answers. That is written in the demo and in this page.

Same pain in your firm?

If first conversations eat your team’s time and urgent cases get mixed with the rest, let’s talk. I’ll show you the demo with fictional cases before quoting anything.

Message me on LinkedIn