A few weeks ago I posted the short version of this: stop relearning the same lesson, and put what you learned somewhere people and agents can actually find it. Someone in the comments asked how that fits with GitHub, pull requests, and the day to day of building software with AI in the mix. Fair question, so here's the longer answer.
Why companies keep relearning the same lesson
Most of us are decent at teaching outward. You hit a wall, you write up what happened, and maybe the next person doesn't step on the same rake. What I find strange is how rarely companies do the same thing for themselves.
They keep shipping code and turning on AI agents, but the actual lessons still live in a Slack thread, in one senior engineer's head, and on a Confluence page nobody has opened since the last reorg. So a new hire (or a new model) spends a week rediscovering how login and permissions work in your stack. Somebody learns the expensive way that clicking "approve" in a vendor's admin portal is not the same as getting the change deployed to production. An AI coding agent "helps" by switching a date or number column to a string so Azure Data Factory stops complaining, and you spend the afternoon undoing a fix that only hid the problem. You already paid tuition on all of it.
Agents make that tax bigger. Picture a well-meaning intern who works all night, doesn't remember yesterday's review, and keeps re-proposing the fix you already rejected. Output goes up, but so does how often you pay for the same mistake.
What worked for us was simpler than a big documentation project, and more important than humans writing notes after the fact.
Skills and wiki, on purpose
We have two places the lessons live, and they overlap on purpose.
The skills repo is the set of SOPs the agents start from: short playbooks in version control for how to do a job and what to refuse. Before you edit an OAuth app registration, list every redirect URL that still has to work. Deploy to non-production and verify there before prod. If Terraform wants to destroy or replace a database, a Key Vault, or a storage account, stop and get a person.
The company wiki is the growing pool of knowledge agents and people can dig into: context, history, ownership, and the "why" behind a rule. It covers things like why we don't let a model turn a typed SQL column into a string just to quiet a pipeline, who is allowed to approve a new permission in Microsoft Entra or Google or GitHub, and which environments can go near production. Skills are the operating base. The wiki is the expanding reference library. Plenty of lessons show up in both.
Make capture part of the agent work
The part that actually compounds is this: updating those two places is part of the agent work itself, not a separate homework assignment for humans.
Take our data-ingestion health pipeline. It finds issues, proposes and applies fixes, and as part of that same run it updates the wiki and the skills when it learns something durable. The next run of that pipeline starts with a wider and deeper set of patterns than the last one. Breadth and depth go up over time.
We do the same with interactive agent chats, not only the batch pipelines. When a human and an agent work through a real problem in the IDE or chat, the useful bits do not stay trapped in that thread. We harvest durable lessons into the wiki and skills, and when something applies everywhere we promote it into global AI rules the next agent session has to follow. The point isn't prettier documentation. The point is agents helping themselves, so humans spend less time intervening and correcting the same mistake.
You still don't have to document everything, and you definitely do not paste every chat transcript into Confluence. If something hurts you twice, it belongs in the curriculum. If it only happened once, it might have been bad luck. The critical piece is that capture is wired into the work, whether that work is a scheduled pipeline or a one-off agent session, not a separate process that runs after the fact.
Four lessons that stuck
These are the first four lessons that went into the wiki and skills:
Login, OAuth, and admin-console handoffs. An agent can't finish work that only a human is allowed to consent to inside a vendor portal: admin consent in Microsoft Entra, adding test users to a Google OAuth app, approving GitHub app permissions. Write down who holds that access and what they have to click, or the work quietly stalls at the last mile.
Deployment and infrastructure guard rails. Creating and updating infrastructure resources can be fully autonomous through Terraform or another infrastructure-as-code tool. Destroying or replacing a resource requires human review. That one sentence has saved us more grief than any amount of prompt tuning.
Data-pipeline type rules. When Azure Data Factory complained about a typed column, agents kept proposing the same thing: make it a string and the error goes away. It does, and it also lies to the type system, so bad values land in the warehouse looking perfectly fine. The rule we wrote down is that SQL column types are the contract, and stringifying a column is never the fix. Go repair whatever is emitting the wrong value.
Auth and browser-cache edges. You change how login works, it tests clean locally, and then a stale service worker keeps serving an old page or old cookies to anyone who visited last week. It looks like a bug in your new code. It isn't.
A wiki on its own becomes another graveyard if the agents that do the work never read it or write to it. Somebody still has to decide which lessons are load-bearing. That is not every chat session. It is the handful that should change the next pipeline run, the next interactive session, or a global rule.
Governance: hard stops vs soft guidance
The boring half is governance, and that's the part that saves you money, time, and sanity. Hard rejects in the build pipeline beat polite model opinions. "Never stringify a typed SQL column to make Data Factory happy" belongs in automated review or a CI check, not buried in a prompt where it reads as a suggestion. Tunable guidance goes on the wiki. Hard stops go in code. Humans still own git and the dangerous changes: agents edit files, automation opens the pull request, a person approves anything irreversible, like destroying a database or a private key.
What moved the numbers
The numbers moved for us once that loop was real. One of our data-pipeline remediation agents used to send about half of its work items to a human for review. After the pipeline kept feeding lessons back into the wiki and skills, and the automated rejects stopped the same bad fix from coming back, that dropped under 10%. Most of what's left gets swept up in a later catch-all pass instead of aging in someone's queue. That wasn't a model upgrade story. It was an org memory story, written by the process that does the work.
Make the knowledge usable
That loop can still fail if the knowledge stores are unusable. A Confluence (or Notion, or SharePoint) page that is technically correct and 4,000 words long is a museum exhibit, not something an agent or a tired human will follow on a Friday night. A skill that only says "be careful with production" is a slogan, not an SOP. The useful version is concrete: "if the plan includes destroy or replace on Postgres, Key Vault, or storage, stop and require human approval." Docs that only describe the happy path fail when reality strays from that path. The page that actually gets used starts with what went wrong, why it did, and what we won't do again. And if Slack search is still the system of record, you don't have a knowledge base. You have a scavenger hunt, and the next agent run can't compound on anything solid.
Models are getting cheaper and more interchangeable. The lessons your company already paid for are not. Put those into a wiki and skills that agents keep updating, and you are not just collecting docs. You are building a process that grows your edge and needs less human cleanup each time around. That has been my biggest takeaway this past year.
Where do your lessons live?
If you are doing anything like this, I would like to hear how it works for you. Where do lessons learned live in your company today? A wiki? A skills or playbook repo? Slack? People's heads? When an agent fixes something in a pipeline or in chat, does that same work update the docs and rules, or does someone have to remember to write it down later?
Originally published on LinkedIn.
