AI-native software engineering is building software that uses a model while it runs. The product sends requests to a model, retrieves documents for it, gives it tools, keeps memory for it, chooses between models, and checks what comes back before anyone sees it. The model is a component, called every time someone uses the feature.
The layer page covers what the discipline costs as a whole: evaluation, a dependency that moves, spend that scales with use, a new security surface, and explaining a wrong answer. This page goes through the parts a feature is usually built from, because what a feature costs you depends on which of them it uses.
What it is for
- An assistant inside a product, answering questions about the product or about the user's own data.
- Answers grounded in an organisation's documents rather than in whatever a model happened to learn.
- An agent that reads a request and acts on it, inside limits you set.
- A pipeline where one model drafts, another checks, and ordinary code decides what happens next.
Who it suits
Engineers building product features on top of models, in a team that can name an owner for evaluation and an owner for security before the feature ships. It is usually a team adding a capability to a product, rather than one person changing how they work.
How to tell this is your category
- A model call sits in your production code path, not only in your editor.
- Your product behaves differently when a provider changes a model.
- You can list which parts your feature uses: a prompt, retrieval, tools, memory, more than one model. Each part on that list carries one of the costs below.
What it costs you later
- The prompt is code without a compiler. Changing a prompt changes the product's behaviour, and nothing fails to build when the change is wrong. Keep prompts in the repository, review changes to them as you would code, and run them against a set of saved cases before they ship. A prompt edited live in a dashboard is a production change nobody reviewed.
- Retrieval answers from what it finds. An answer grounded in your documents is only as good as the documents retrieval returns, and a stale document produces a confident stale answer. Retrieval that ignores permissions is worse: it can put one user's document into another user's answer. Filter by who is asking before anything reaches the model, not afterwards.
- The model is not where permissions live. A model with tools will act on what it reads, and what it reads can be written by anyone. A model allowed to change account details will act on a confident enough request unless something outside the model checks who is asking. Check permissions in ordinary code, against the signed-in user, before any tool runs, whatever the model asked for. For actions that touch access, recovery or money, put a real identity check or a person in front of them.
- Memory and logs are stored personal data. Whatever the product remembers about a user, it stores. The record that lets you explain a wrong answer, the input, what was retrieved and what each tool returned, is also a store of everything your users typed. Choose on purpose what you keep, for how long, and who can read it, and treat both as part of what you hold about each user.
- A fallback is a second product. Decide per feature whether it switches to another provider or degrades. Switching sounds safer, but a prompt tuned for one model behaves differently on another, so a fallback needs its own evaluation, or it is untested behaviour that runs only during an outage. Degrading, turning the model-backed part off and keeping the rest of the product working, is often the more honest choice.
- Routing is a quality decision as well as a cost one. Sending simpler requests to a cheaper model saves money and changes answers. Each routing rule needs the same saved cases and scores as any other change, or the saving is paid for in quality nobody measured.
Where to start
- Write the cases before the feature: a set of real requests, each with what a good answer looks like. Run them on every change to a prompt, a model, a routing rule or the retrieved content.
- Put permission checks in code, outside the model, before any tool runs and before any document is retrieved.
- Decide each feature's fallback before launch, and test it.
- Decide what you log, how long you keep it, and who can read it.
Evaluation is the largest of these and the least settled. It deserves a guide of its own, and this page does not pretend to be one.