Site icon Kevin Waites Consulting Services

Making notes stick: why your AI keeps forgetting your rules

If you had to remind a new hire how to format a report three times, you’d start to wonder about their attention to detail.

When the same thing happens with AI, most people blame the AI. I’d look at the process first.

We all picked up a bad habit in the chatbot days. We treat AI like a calculator. Paste in some background, get some text back, close the tab. Tomorrow we start from a blank slate and explain it all again.

That’s annoying in a chat window. Once you’re working with AI agents that take on multi-step work, like logging donor activity or pulling a board report together, it gets expensive.

Why AI forgets

Large language models are stateless by default. Every message arrives with no idea who you are or what you said ten minutes ago.

A chat feels like it remembers because the app quietly sends your whole conversation back to the model with every new message. The model’s memory is really just the transcript.

That’s fine for a single chat. It falls apart when the work spans several steps or tools. When an agent opens a separate workspace to run some code, or reaches into your CRM through a connector, your chat history doesn’t come along. The rule you typed three days ago isn’t there anymore.

The fix is to keep two things apart:

In a well-built agent system, when you say “remember this,” the rule gets saved against the specific workflow it belongs to, like donor thank-yous or monthly board reports. The next time that workflow starts, the system loads the rule before the AI does anything. Your rules end up in one place you can read and check, instead of buried in an old chat.

How to make your rules stick

Here’s the rule of thumb I use.

  1. Use the three-strike rule. If you’ve corrected the AI or given it the same background more than twice, stop re-typing it. Tell it to make a permanent note of the rule for that piece of work. In an everyday chatbot that usually means the custom instructions or a project’s instructions. In an agent setup, it’s the agent’s memory for that workflow.
  2. Tie the rule to a trigger. “Remember our tone” is vague. “Whenever you draft a donor email, use first names and keep it under 150 words” is a rule the AI can actually follow.
  3. Write in the boring follow-up steps. This is where the real time savings are. Take a donor thank-you letter. Your team drafts a lovely, personal letter, you sign off, and it goes out. Then someone has to log into the CRM, record that it was sent, and set a reminder to follow up in two months. That’s the part that gets dropped. So make it part of the rule: “Whenever a donor thank-you letter is approved, log it in our CRM, work out the date 60 days from today, and set the donor’s next action to a follow-up on that date.”
  4. Add the checks people skip when they’re busy. Every team has a final 10% of the checklist that gets missed. For a communications lead, that might be: “Every time you draft a social campaign, check our shared content calendar for clashes before anything gets scheduled.” If you have a tech or data person, the same idea works for them: “Every time a change goes live, update the documentation.”
  5. Read your rules back now and then. Ask the AI to list the rules it’s holding for a workflow. Delete the ones that are out of date.

For leaders: build the checklist into the system

When a social post double-books over a big announcement, or a donor never gets that follow-up call, it’s usually because a busy person skipped the tedious last step. An agent that has the step written into its memory won’t skip it.

The fix is cheaper than a new project management tool. Write your rules down in a place the AI reads every time.

A few questions to take to your next team meeting:

Your people still set the strategy, and the follow-through runs in the background the same way every time.

Making memory reliable is a big part of what I’ve been working on with Cosly, the AI staff team I’m building for small nonprofits.

Exit mobile version