1 The four roles, and which one you are applying for
Four job titles keep appearing, they overlap, and employers
use them loosely. Before learning anything technical, it is worth knowing which job you
are actually applying for, because the honest answer is that no single person has
all the skills in a typical advert, and the adverts are written as though one
does.
The four roles, and what each one is really asked to do
AI Engineer also: GenAI Engineer, LLM Engineer
Builds products on top of models somebody else trained.
- Calls model APIs, or runs open models, and wraps them in something useful
- Retrieval over a company's own documents, so answers are grounded
- Tool calling and agents; making a model do things, not just talk
- Evaluation: proving the thing works, and catching it when it stops
- Cost, latency, caching, streaming, failure handling
Mostly software engineering. If you can already
build and ship a web application, you are closer to this job than you think. The
maths you need is understanding, not derivation.
Machine Learning Engineer
Trains, tunes and deploys models on the company's own data.
- Feature engineering, training pipelines, hyperparameter tuning
- PyTorch or TensorFlow, scikit-learn, gradient boosting
- Serving models at scale, and monitoring them for drift
- Experiment tracking and reproducibility
Software engineering plus real statistics. You
need to know why a model is wrong, not only that it is. This is the longer road.
Data Scientist
Answers questions with data, and often does not ship code.
- Analysis, experiment design, A/B tests, causal questions
- SQL constantly; pandas and notebooks; statistics properly
- Communicating a finding to people who will decide something
Statistics and communication first. Sometimes
models, often not. If an advert is heavy on SQL and experiment design, this is the
job whatever it is called.
MLOps / ML Platform Engineer
Builds the machinery other people's models run on.
- Containers, Kubernetes, CI/CD, infrastructure as code
- Feature stores, model registries, pipeline orchestration
- GPU scheduling, observability, cost control
Really a DevOps / platform job. Very little
modelling. If you like infrastructure, this is the least crowded door in.
The one distinction that explains the rest
Every current write-up lands on roughly the same split, in different words: an
ML Engineer builds the model, and an AI Engineer builds the
product that uses a model. One trains; the other integrates. That single line
predicts almost everything else — the tools, the interview, the maths, the day.
The complication is that job boards do not respect it. An advert titled "AI Engineer"
may ask for training loops and drift monitoring; one titled "Machine Learning Engineer"
may spend most of its description on RAG and LLM evaluation. Most postings now simply
merge both under AI/ML Engineer. So read the responsibilities, never
the title.
Where to aim, given where you are starting. You already write
JavaScript and TypeScript, which means you already have the harder half of the AI
Engineer job: you can build software. Adding Python, retrieval, evaluation and enough
maths to reason about what the model is doing is a much shorter path than starting from
statistics. That is the road this course takes, and it is honest about which parts of
the ML Engineer road it does not cover.
Working out which role an advert is describing
Rather than guessing, count the signals. Below is a small scoring function — the point
is not the code, it is the vocabulary lists, because those are the words that actually
separate the four jobs.
Reading an advert by its vocabulary, not its title
ran at build time
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50# Which role is this advert really describing? Count the signals, do not read the title.
SIGNALS = {
"ai_engineer": [
"llm", "rag", "retrieval", "prompt", "agent", "openai", "claude",
"langchain", "vector database", "embedding", "tool calling", "fine-tune",
],
"ml_engineer": [
"pytorch", "tensorflow", "scikit-learn", "xgboost", "training pipeline",
"feature engineering", "hyperparameter", "model drift", "gpu", "loss function",
],
"data_scientist": [
"sql", "pandas", "a/b test", "experiment design", "statistical significance",
"dashboard", "stakeholder", "causal", "regression analysis", "notebook",
],
"mlops": [
"kubernetes", "docker", "terraform", "ci/cd", "airflow", "kubeflow",
"model registry", "feature store", "observability", "infrastructure",
],
}
def classify(advert):
"""Return the scores, highest first. Ties are kept visible rather than broken."""
text = advert.lower()
scores = {
role: sum(1 for word in words if word in text)
for role, words in SIGNALS.items()
}
return sorted(scores.items(), key=lambda pair: -pair[1])
# A real-sounding advert, titled one thing and describing another.
advert = """
Senior Machine Learning Engineer
You will build retrieval augmented generation pipelines over our internal
knowledge base, design prompt templates, integrate the OpenAI and Claude APIs,
choose a vector database, and define evaluation suites for agent tool calling.
"""
ranked = classify(advert)
print("advert title says: Machine Learning Engineer")
print("the words say :", ranked[0][0], "with", ranked[0][1], "signals")
print()
for role, score in ranked:
print(" {:16} {}".format(role, "#" * score if score else "-"))
test("the title is misleading; the work is AI engineering", ranked[0][0], "ai_engineer")
test("and it scores well above the title's own role", ranked[0][1] > dict(ranked)["ml_engineer"], True)
test("every role is scored, so nothing is hidden", len(ranked), 4)
test("an empty advert scores nothing anywhere", [s for _, s in classify("")], [0, 0, 0, 0])
What it printed, when the build ran it
advert title says: Machine Learning Engineer
the words say : ai_engineer with 7 signals
ai_engineer #######
ml_engineer -
data_scientist -
mlops -
ok the title is misleading; the work is AI engineering
ok and it scores well above the title's own role
ok every role is scored, so nothing is hidden
ok an empty advert scores nothing anywhere
A word about the "requirements" list. Adverts are wish lists written
by committee, and they are routinely padded. Two things follow. First, the years-of-
experience number is usually negotiable and often invented. Second, if you match the
responsibilities and about half the tools, you are a real candidate —
the remaining tools are things people learn in their first month. Do not disqualify
yourself; that is the interviewer's job, and they are worse at it than you fear.
What this course covers, and what it does not
- Covered properly: the maths you actually use, classical ML from
scratch, neural networks and backpropagation by hand, how a language model works down
to attention, then the AI Engineer day job — retrieval, agents, structured output,
evaluation, cost, safety.
- Covered as understanding, not practice: PyTorch, numpy, pandas.
They are not installed here, so any snippet using them would be a listing I had not
run, and this course does not publish unrun code. Every algorithm is instead written
out in plain Python — slower, and much better for learning what the library call is
doing. Where a real project would use a library, the lesson says so and shows the
call.
- Not covered: distributed training, CUDA, research-level
architecture work. Those belong to the ML Engineer and research roads, and pretending
otherwise would waste your time.
2 How to read a job advert
An advert is not a specification. It is a wish list, usually
assembled from an old advert plus whatever the team argued about last week. Reading it
literally is the single most common reason good candidates do not apply.
The four parts of every advert, in order of how much they matter
- Responsibilities. Highest signal. This is the job. If you can
picture yourself doing these sentences, you are a candidate.
- The interview process, if stated. Second highest, and almost nobody
reads it. "Take-home followed by a system design round" tells you exactly what to
prepare.
- Required skills. Medium signal, and padded. Treat as roughly
"things somebody on the team uses".
- Nice to have. Near zero signal. Genuinely optional. Frequently
contains the previous person's hobby.
Words that actually mean something
Some phrases are reliable signals about the job's shape. These are worth learning
because they change whether you want it, not only whether you qualify.
- "Fine-tune" — often means "prompt well". Ask. True fine-tuning is
much rarer than adverts imply.
- "Own the ML lifecycle end to end" — small team, no platform,
you will be doing infrastructure too.
- "Work closely with researchers" — you are the one shipping;
they are the one publishing. Usually a good AI Engineer role.
- "Production experience with LLMs" — they have been burned by a
demo that never shipped. Lead with evaluation and reliability.
- "Strong communication skills" in an engineering advert usually
means the last hire could not explain a tradeoff to a product manager.
- No mention of evaluation anywhere — they have not shipped AI
yet. That is an opportunity or a warning, depending on your appetite.
Scoring yourself against an advert, honestly
The useful question is not "do I match every line" — nobody does. It is "do I match
the responsibilities, and is the gap in the tools learnable in a month". Below, the same
advert is scored twice: once the way people do it in their head, and once weighted the way
hiring actually works.
Scoring yourself two ways
ran at build time
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61# Two ways of scoring yourself against an advert. Only one of them is useful.
advert = {
"responsibilities": [
"build retrieval pipelines over internal documents",
"design and maintain evaluation suites",
"integrate model APIs into product surfaces",
"work with product to scope AI features",
],
"required": ["python", "rag", "vector database", "evaluation", "docker", "aws"],
"nice_to_have": ["kubernetes", "terraform", "fine-tuning", "published research"],
}
me = {
"can_do": [
"build retrieval pipelines over internal documents",
"design and maintain evaluation suites",
"integrate model APIs into product surfaces",
"work with product to scope AI features",
],
"knows": ["python", "rag", "vector database", "evaluation"],
}
def naive_score(advert, me):
"""How people score themselves at 2am: every line, equally weighted."""
everything = advert["required"] + advert["nice_to_have"]
have = sum(1 for item in everything if item in me["knows"])
return have / len(everything)
def honest_score(advert, me):
"""Weighted the way hiring works: the work matters, nice-to-haves do not."""
resp = sum(1 for r in advert["responsibilities"] if r in me["can_do"])
resp_share = resp / len(advert["responsibilities"])
req = sum(1 for r in advert["required"] if r in me["knows"])
req_share = req / len(advert["required"])
# 70% the work, 30% the tools, nice-to-haves ignored entirely.
return 0.7 * resp_share + 0.3 * req_share
naive = naive_score(advert, me)
honest = honest_score(advert, me)
print("naive score: {:.0%} <- feels unqualified, does not apply".format(naive))
print("honest score: {:.0%} <- clearly a candidate".format(honest))
print()
print("missing from required :", [r for r in advert["required"] if r not in me["knows"]])
print("missing from nice :", [r for r in advert["nice_to_have"] if r not in me["knows"]])
print()
print("Both missing items are infrastructure tools, learnable in weeks.")
print("Every responsibility is already a yes. That is what 'qualified' means.")
test("the naive score looks discouraging", naive < 0.5, True)
test("the honest score says apply", honest > 0.8, True)
test("all four responsibilities are met", honest_score(advert, me) >= 0.7, True)
test("the gap is only tools", sorted(r for r in advert["required"] if r not in me["knows"]), ["aws", "docker"])
test("nice-to-haves are excluded from the honest score",
honest_score(advert, me), honest_score({**advert, "nice_to_have": []}, me))
test("meeting no responsibilities scores low however many tools you know",
honest_score(advert, {"can_do": [], "knows": advert["required"]}) < 0.35, True)
What it printed, when the build ran it
naive score: 40% <- feels unqualified, does not apply
honest score: 90% <- clearly a candidate
missing from required : ['docker', 'aws']
missing from nice : ['kubernetes', 'terraform', 'fine-tuning', 'published research']
Both missing items are infrastructure tools, learnable in weeks.
Every responsibility is already a yes. That is what 'qualified' means.
ok the naive score looks discouraging
ok the honest score says apply
ok all four responsibilities are met
ok the gap is only tools
ok nice-to-haves are excluded from the honest score
ok meeting no responsibilities scores low however many tools you know
The last assertion is the one to sit with. Knowing every tool while matching
no responsibility scores 30%. Matching every responsibility while missing a
third of the tools scores 90%. Hiring managers are buying someone who can do the work,
and the tool list is a proxy they know is imperfect.
What to do about a genuine gap
Sometimes the gap is real and central — an ML Engineer role wanting training pipelines
when you have never trained anything. Two honest options, and a dishonest one worth
naming.
- Apply anyway and say so. "I have not run a training pipeline in
production; here is a model I trained from scratch and what I learned about where it
went wrong." Specific and credible beats vague and confident.
- Close it, narrowly. One project that genuinely exercises the
missing skill, understood deeply, is worth more than a certificate. The next lesson is
about what makes a project count.
- Do not claim it. Listing a tool you have read about is the fastest
way to fail a technical interview, because the follow-up question is always "tell me
about a time it broke".
3 What counts as proof you can do the job
Nobody hires you because you completed a course, including this
one. They hire you because you showed evidence you can do the work. Evidence has a
specific shape, and most portfolio projects do not have it.
Why the usual portfolio project fails
The standard AI portfolio project is a chat interface over some documents. It fails to
persuade for a reason that has nothing to do with effort: it does not show
judgement. It works on the demo question and nobody, including the author, knows
what happens on the other thousand.
What an interviewer is trying to find out is whether you can be trusted with a system
whose failures are quiet. So the evidence they are looking for is:
- You measured it. A number, on a set of cases you wrote down, before
and after a change you made.
- You know where it breaks. Named failure modes, with examples, not
"it sometimes hallucinates".
- You made a tradeoff on purpose. Chose smaller chunks and can say
what it cost. Cached and can say what went stale.
- You know what it costs. Per request, in money and in milliseconds.
One project with an evaluation set of thirty cases, a table of two approaches
compared, and a paragraph on what still fails, beats five projects that each work once.
The eval set is the single highest-leverage thing you can build, and almost nobody
does it — which is exactly why it is persuasive.
The smallest thing that counts as an evaluation
An eval set does not need a framework. It is a list of inputs, what a correct answer
contains, and a way of scoring. Here is one that would genuinely stand up in an
interview, with no dependencies at all.
An evaluation set you could show an interviewer
ran at build time
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84# The smallest useful evaluation. No framework, no library, no excuses.
CASES = [
# question, phrases a correct answer must contain, phrases it must NOT
("What is the refund window?", ["14 days"], ["30 days", "no refunds"]),
("Do you ship to Lagos?", ["yes", "Lagos"], ["we do not ship"]),
("What is the delivery fee to Abuja?", ["2,500"], ["free"]),
("Who is the chief executive?", ["not", "sure"], ["Ada Obi"]), # must ADMIT it does not know
]
def score_answer(answer, must, must_not):
"""A correct answer contains every required phrase and none of the forbidden ones."""
low = answer.lower()
hits = [phrase for phrase in must if phrase.lower() in low]
leaks = [phrase for phrase in must_not if phrase.lower() in low]
return {
"passed": len(hits) == len(must) and not leaks,
"found": hits,
"leaked": leaks,
}
def evaluate(system):
"""Run every case. Return the score and the failures, never just the score."""
results = []
for question, must, must_not in CASES:
outcome = score_answer(system(question), must, must_not)
results.append((question, outcome))
passed = sum(1 for _, o in results if o["passed"])
return {
"passed": passed,
"total": len(CASES),
"rate": passed / len(CASES),
"failures": [(q, o) for q, o in results if not o["passed"]],
}
# Two pretend systems. Version A is confidently wrong about one thing and makes
# something up about another - the two failure modes that matter most.
def version_a(question):
canned = {
"What is the refund window?": "You can return an item within 14 days of delivery.",
"Do you ship to Lagos?": "Yes, we deliver across Lagos within 48 hours.",
"What is the delivery fee to Abuja?": "Delivery to Abuja is free on all orders.",
"Who is the chief executive?": "The chief executive is Ada Obi.",
}
return canned[question]
def version_b(question):
canned = {
"What is the refund window?": "Returns are accepted within 14 days of delivery.",
"Do you ship to Lagos?": "Yes. Lagos deliveries arrive within 48 hours.",
"What is the delivery fee to Abuja?": "Delivery to Abuja costs 2,500 naira.",
"Who is the chief executive?": "I am not sure - that is not in the documents I have.",
}
return canned[question]
a = evaluate(version_a)
b = evaluate(version_b)
print("version A: {}/{} = {:.0%}".format(a["passed"], a["total"], a["rate"]))
for question, outcome in a["failures"]:
if outcome["leaked"]:
reason = "said " + str(outcome["leaked"])
else:
reason = "did not say what it needed to"
print(" FAILED {}".format(question))
print(" because it {}".format(reason))
print()
print("version B: {}/{} = {:.0%}".format(b["passed"], b["total"], b["rate"]))
print()
print("That is the whole artefact. Two numbers, and a named reason for each failure.")
test("version A gets half of them right", a["rate"], 0.5)
test("version B gets all of them right", b["rate"], 1.0)
test("A is caught inventing a fee", any("free" in o["leaked"] for _, o in a["failures"]), True)
test("A is caught inventing a person", any("Ada Obi" in o["leaked"] for _, o in a["failures"]), True)
test("B admits what it does not know", b["passed"], 4)
test("the eval reports failures, not only a score", len(a["failures"]), 2)
test("a system that says nothing fails everything", evaluate(lambda q: "")["rate"], 0.0)
test("and a system that repeats the question fails too", evaluate(lambda q: q)["rate"], 0.0)
What it printed, when the build ran it
version A: 2/4 = 50%
FAILED What is the delivery fee to Abuja?
because it said ['free']
FAILED Who is the chief executive?
because it said ['Ada Obi']
version B: 4/4 = 100%
That is the whole artefact. Two numbers, and a named reason for each failure.
ok version A gets half of them right
ok version B gets all of them right
ok A is caught inventing a fee
ok A is caught inventing a person
ok B admits what it does not know
ok the eval reports failures, not only a score
ok a system that says nothing fails everything
ok and a system that repeats the question fails too
The case most people leave out
Look at the fourth case. It asks something the documents do not contain, and a correct
answer is "I do not know". Version A answers it confidently and wrongly, and the
eval catches that as a failure rather than rewarding the fluency.
Including at least one unanswerable question is the mark of somebody who has actually
run one of these systems. A retrieval system that never says "I do not know" is not
working well — it is failing silently, which is worse.
What a real project would use instead
In a real codebase you would not hand-roll the scoring loop. The shape below is what it
looks like with the usual libraries. It is printed here and not executed,
because those packages are not installed and this course does not publish code it has not
run.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21# What the same evaluation looks like with the usual tooling.
# NOT RUN HERE - pytest and the client libraries are not installed.
import pytest
from openai import OpenAI
client = OpenAI()
CASES = [
("What is the refund window?", ["14 days"], ["30 days"]),
# ...
]
@pytest.mark.parametrize("question,must,must_not", CASES)
def test_answer_is_grounded(question, must, must_not):
answer = my_rag_pipeline(question) # the system under test
low = answer.lower()
for phrase in must:
assert phrase.lower() in low, f"missing {phrase!r} in {answer!r}"
for phrase in must_not:
assert phrase.lower() not in low, f"leaked {phrase!r} in {answer!r}"
pytest and the OpenAI client are not installed here, so this listing was never executed. The plain-Python version above does the same job and did run. Treat this panel as a shape to copy, not a verified snippet.
On certificates. They are worth very little on their own, and that is
not cynicism about learning — it is that a certificate proves attendance, and attendance
is not the thing being bought. A public repository with an eval set, a README explaining
one tradeoff, and an honest "still fails when…" section outperforms any course
completion. Including this one: what makes this course useful to you is what you build
with it, not that you finished it.