Skip to content
Job preparation

AI Engineer · 3-5 Years

Real implementation, debugging, tools, logs, edge cases, and measurable fixes.

Try each answer before revealing the suggested coaching answer.

← All AI Engineer levels

25 questions

01You are working on a production project and AI vs ML vs deep learning starts causing issues. How would you diagnose and fix it as an AI Engineer?

A production answer

Say this first: AI vs ML vs deep learning is a choice between approaches with different strengths. The useful answer is the decision rule, not a dictionary definition.

Use a real scenario

Imagine an internal support assistant that answers from approved policy documents. The team must decide how to apply AI vs ML vs deep learning, verify the result, and explain the user impact. For an AI Engineer, attach the explanation to an evaluation set and retrieval trace.

Show judgment

  • describe the implementation path, the main trade-off, and the evidence you would collect.
  • Choose the option that fits the workload and constraints; do not present one option as universally superior.
  • Call out prompt injection and unsupported answers and the control that reduces it.

Evidence to mention

Track grounded-answer rate and p95 response time. Say what baseline you compared against, what would trigger a rollback or escalation, and who owns the follow-up.

Practice prompt: Use an internal support assistant that answers from approved policy documents as the example and show where you would stop a risky rollout.

Link to question 1
02How have you implemented supervised vs unsupervised learning in a real AI / GenAI project?

A production answer

Say this first: supervised vs unsupervised learning is a choice between approaches with different strengths. The useful answer is the decision rule, not a dictionary definition.

Use a real scenario

Imagine an internal support assistant that answers from approved policy documents. The team must decide how to apply supervised vs unsupervised learning, verify the result, and explain the user impact. For an AI Engineer, attach the explanation to an evaluation set and retrieval trace.

Show judgment

  • describe the implementation path, the main trade-off, and the evidence you would collect.
  • Choose the option that fits the workload and constraints; do not present one option as universally superior.
  • Call out prompt injection and unsupported answers and the control that reduces it.

Evidence to mention

Track grounded-answer rate and p95 response time. Say what baseline you compared against, what would trigger a rollback or escalation, and who owns the follow-up.

Practice prompt: Use an internal support assistant that answers from approved policy documents as the example and show where you would stop a risky rollout.

Link to question 2
03A release is blocked because of a problem related to classification vs regression. What steps would you take?

A production answer

Say this first: classification vs regression is a choice between approaches with different strengths. The useful answer is the decision rule, not a dictionary definition.

Use a real scenario

Imagine an internal support assistant that answers from approved policy documents. The team must decide how to apply classification vs regression, verify the result, and explain the user impact. For an AI Engineer, attach the explanation to an evaluation set and retrieval trace.

Show judgment

  • describe the implementation path, the main trade-off, and the evidence you would collect.
  • Choose the option that fits the workload and constraints; do not present one option as universally superior.
  • Call out prompt injection and unsupported answers and the control that reduces it.

Evidence to mention

Track grounded-answer rate and p95 response time. Say what baseline you compared against, what would trigger a rollback or escalation, and who owns the follow-up.

Practice prompt: Use an internal support assistant that answers from approved policy documents as the example and show where you would stop a risky rollout.

Link to question 3
04What logs, metrics, or artifacts would you check while troubleshooting overfitting and underfitting?

A production answer

Say this first: The important point about overfitting and underfitting is how an engineer recognizes the unsafe path early and prevents it from becoming customer impact.

Use a real scenario

Imagine an internal support assistant that answers from approved policy documents. The team must decide how to apply overfitting and underfitting, verify the result, and explain the user impact. For an AI Engineer, attach the explanation to an evaluation set and retrieval trace.

Show judgment

  • describe the implementation path, the main trade-off, and the evidence you would collect.
  • Start with containment and evidence. Changing several variables at once makes the incident harder to understand.
  • Call out prompt injection and unsupported answers and the control that reduces it.

Evidence to mention

Track grounded-answer rate and p95 response time. Say what baseline you compared against, what would trigger a rollback or escalation, and who owns the follow-up.

Practice prompt: Use an internal support assistant that answers from approved policy documents as the example and show where you would stop a risky rollout.

Link to question 4
05How would you make train validation test split reliable enough for day-to-day production use?

A production answer

Say this first: Training data fits the model, validation data guides choices during development, and a held-out test set gives the final unbiased check.

Use a real scenario

Imagine an internal support assistant that answers from approved policy documents. The team must decide how to apply train validation test split, verify the result, and explain the user impact. For an AI Engineer, attach the explanation to an evaluation set and retrieval trace.

Show judgment

  • describe the implementation path, the main trade-off, and the evidence you would collect.
  • State the constraint that could change your decision, such as scale, data sensitivity, recovery target, or team ownership.
  • Call out prompt injection and unsupported answers and the control that reduces it.

Concrete check

npm test -- --runInBand

Evidence to mention

Track grounded-answer rate and p95 response time. Say what baseline you compared against, what would trigger a rollback or escalation, and who owns the follow-up.

Practice prompt: Use an internal support assistant that answers from approved policy documents as the example and show where you would stop a risky rollout.

Link to question 5
06You are working on a production project and precision recall F1 and ROC-AUC starts causing issues. How would you diagnose and fix it as an AI Engineer?

A production answer

Say this first: Precision asks whether positive predictions were correct; recall asks whether the system found the positives it should find. The right balance depends on the cost of each error.

Use a real scenario

Imagine an internal support assistant that answers from approved policy documents. The team must decide how to apply precision recall F1 and ROC-AUC, verify the result, and explain the user impact. For an AI Engineer, attach the explanation to an evaluation set and retrieval trace.

Show judgment

  • describe the implementation path, the main trade-off, and the evidence you would collect.
  • State the constraint that could change your decision, such as scale, data sensitivity, recovery target, or team ownership.
  • Call out prompt injection and unsupported answers and the control that reduces it.

Evidence to mention

Track grounded-answer rate and p95 response time. Say what baseline you compared against, what would trigger a rollback or escalation, and who owns the follow-up.

Practice prompt: Use an internal support assistant that answers from approved policy documents as the example and show where you would stop a risky rollout.

Link to question 6
07How have you implemented feature engineering and embeddings in a real AI / GenAI project?

A production answer

Say this first: The important point about feature engineering and embeddings is how an engineer recognizes the unsafe path early and prevents it from becoming customer impact.

Use a real scenario

Imagine an internal support assistant that answers from approved policy documents. The team must decide how to apply feature engineering and embeddings, verify the result, and explain the user impact. For an AI Engineer, attach the explanation to an evaluation set and retrieval trace.

Show judgment

  • describe the implementation path, the main trade-off, and the evidence you would collect.
  • Start with containment and evidence. Changing several variables at once makes the incident harder to understand.
  • Call out prompt injection and unsupported answers and the control that reduces it.

Evidence to mention

Track grounded-answer rate and p95 response time. Say what baseline you compared against, what would trigger a rollback or escalation, and who owns the follow-up.

Practice prompt: Use an internal support assistant that answers from approved policy documents as the example and show where you would stop a risky rollout.

Link to question 7
08A release is blocked because of a problem related to tokenization and context window. What steps would you take?

A production answer

Say this first: tokenization and context window should be explained through its purpose, the boundary where it applies, and the evidence that shows it is working.

Use a real scenario

Imagine an internal support assistant that answers from approved policy documents. The team must decide how to apply tokenization and context window, verify the result, and explain the user impact. For an AI Engineer, attach the explanation to an evaluation set and retrieval trace.

Show judgment

  • describe the implementation path, the main trade-off, and the evidence you would collect.
  • State the constraint that could change your decision, such as scale, data sensitivity, recovery target, or team ownership.
  • Call out prompt injection and unsupported answers and the control that reduces it.

Evidence to mention

Track grounded-answer rate and p95 response time. Say what baseline you compared against, what would trigger a rollback or escalation, and who owns the follow-up.

Practice prompt: Use an internal support assistant that answers from approved policy documents as the example and show where you would stop a risky rollout.

Link to question 8
09What logs, metrics, or artifacts would you check while troubleshooting prompt engineering basics?

A production answer

Say this first: The important point about prompt engineering basics is how an engineer recognizes the unsafe path early and prevents it from becoming customer impact.

Use a real scenario

Imagine an internal support assistant that answers from approved policy documents. The team must decide how to apply prompt engineering basics, verify the result, and explain the user impact. For an AI Engineer, attach the explanation to an evaluation set and retrieval trace.

Show judgment

  • describe the implementation path, the main trade-off, and the evidence you would collect.
  • Start with containment and evidence. Changing several variables at once makes the incident harder to understand.
  • Call out prompt injection and unsupported answers and the control that reduces it.

Evidence to mention

Track grounded-answer rate and p95 response time. Say what baseline you compared against, what would trigger a rollback or escalation, and who owns the follow-up.

Practice prompt: Use an internal support assistant that answers from approved policy documents as the example and show where you would stop a risky rollout.

Link to question 9
10How would you make RAG pipeline fundamentals reliable enough for day-to-day production use?

A production answer

Say this first: Retrieval-augmented generation fetches relevant, approved context at answer time so a model can ground its response in current source material.

Use a real scenario

Imagine an internal support assistant that answers from approved policy documents. The team must decide how to apply RAG pipeline fundamentals, verify the result, and explain the user impact. For an AI Engineer, attach the explanation to an evaluation set and retrieval trace.

Show judgment

  • describe the implementation path, the main trade-off, and the evidence you would collect.
  • State the constraint that could change your decision, such as scale, data sensitivity, recovery target, or team ownership.
  • Call out prompt injection and unsupported answers and the control that reduces it.

Concrete check

SELECT COUNT(*) AS rows, MAX(loaded_at) AS freshest FROM <table>;

Evidence to mention

Track grounded-answer rate and p95 response time. Say what baseline you compared against, what would trigger a rollback or escalation, and who owns the follow-up.

Practice prompt: Use an internal support assistant that answers from approved policy documents as the example and show where you would stop a risky rollout.

Link to question 10
11You are working on a production project and chunking strategy starts causing issues. How would you diagnose and fix it as an AI Engineer?

A production answer

Say this first: Chunking splits source material into retrieval units. The boundary and size affect whether a retrieved passage contains enough context to support an answer.

Use a real scenario

Imagine an internal support assistant that answers from approved policy documents. The team must decide how to apply chunking strategy, verify the result, and explain the user impact. For an AI Engineer, attach the explanation to an evaluation set and retrieval trace.

Show judgment

  • describe the implementation path, the main trade-off, and the evidence you would collect.
  • State the constraint that could change your decision, such as scale, data sensitivity, recovery target, or team ownership.
  • Call out prompt injection and unsupported answers and the control that reduces it.

Evidence to mention

Track grounded-answer rate and p95 response time. Say what baseline you compared against, what would trigger a rollback or escalation, and who owns the follow-up.

request or change → guardrail / validation → chunking strategy → observable result → owner review

Practice prompt: Use an internal support assistant that answers from approved policy documents as the example and show where you would stop a risky rollout.

Link to question 11
12How have you implemented vector database and similarity search in a real AI / GenAI project?

A production answer

Say this first: The important point about vector database and similarity search is how an engineer recognizes the unsafe path early and prevents it from becoming customer impact.

Use a real scenario

Imagine an internal support assistant that answers from approved policy documents. The team must decide how to apply vector database and similarity search, verify the result, and explain the user impact. For an AI Engineer, attach the explanation to an evaluation set and retrieval trace.

Show judgment

  • describe the implementation path, the main trade-off, and the evidence you would collect.
  • Start with containment and evidence. Changing several variables at once makes the incident harder to understand.
  • Call out prompt injection and unsupported answers and the control that reduces it.

Concrete check

SELECT COUNT(*) AS rows, MAX(loaded_at) AS freshest FROM <table>;

Evidence to mention

Track grounded-answer rate and p95 response time. Say what baseline you compared against, what would trigger a rollback or escalation, and who owns the follow-up.

Practice prompt: Use an internal support assistant that answers from approved policy documents as the example and show where you would stop a risky rollout.

Link to question 12
13A release is blocked because of a problem related to reranking and hybrid search. What steps would you take?

A production answer

Say this first: reranking and hybrid search should be explained through its purpose, the boundary where it applies, and the evidence that shows it is working.

Use a real scenario

Imagine an internal support assistant that answers from approved policy documents. The team must decide how to apply reranking and hybrid search, verify the result, and explain the user impact. For an AI Engineer, attach the explanation to an evaluation set and retrieval trace.

Show judgment

  • describe the implementation path, the main trade-off, and the evidence you would collect.
  • State the constraint that could change your decision, such as scale, data sensitivity, recovery target, or team ownership.
  • Call out prompt injection and unsupported answers and the control that reduces it.

Evidence to mention

Track grounded-answer rate and p95 response time. Say what baseline you compared against, what would trigger a rollback or escalation, and who owns the follow-up.

Practice prompt: Use an internal support assistant that answers from approved policy documents as the example and show where you would stop a risky rollout.

Link to question 13
14What logs, metrics, or artifacts would you check while troubleshooting fine-tuning vs RAG?

A production answer

Say this first: fine-tuning vs RAG is a choice between approaches with different strengths. The useful answer is the decision rule, not a dictionary definition.

Use a real scenario

Imagine an internal support assistant that answers from approved policy documents. The team must decide how to apply fine-tuning vs RAG, verify the result, and explain the user impact. For an AI Engineer, attach the explanation to an evaluation set and retrieval trace.

Show judgment

  • describe the implementation path, the main trade-off, and the evidence you would collect.
  • Choose the option that fits the workload and constraints; do not present one option as universally superior.
  • Call out prompt injection and unsupported answers and the control that reduces it.

Evidence to mention

Track grounded-answer rate and p95 response time. Say what baseline you compared against, what would trigger a rollback or escalation, and who owns the follow-up.

Practice prompt: Use an internal support assistant that answers from approved policy documents as the example and show where you would stop a risky rollout.

Link to question 14
15How would you make LLM hallucination control reliable enough for day-to-day production use?

A production answer

Say this first: LLM hallucination control should be explained through its purpose, the boundary where it applies, and the evidence that shows it is working.

Use a real scenario

Imagine an internal support assistant that answers from approved policy documents. The team must decide how to apply LLM hallucination control, verify the result, and explain the user impact. For an AI Engineer, attach the explanation to an evaluation set and retrieval trace.

Show judgment

  • describe the implementation path, the main trade-off, and the evidence you would collect.
  • State the constraint that could change your decision, such as scale, data sensitivity, recovery target, or team ownership.
  • Call out prompt injection and unsupported answers and the control that reduces it.

Evidence to mention

Track grounded-answer rate and p95 response time. Say what baseline you compared against, what would trigger a rollback or escalation, and who owns the follow-up.

Practice prompt: Use an internal support assistant that answers from approved policy documents as the example and show where you would stop a risky rollout.

Link to question 15
16You are working on a production project and prompt injection defense starts causing issues. How would you diagnose and fix it as an AI Engineer?

A production answer

Say this first: prompt injection defense should be explained through its purpose, the boundary where it applies, and the evidence that shows it is working.

Use a real scenario

Imagine an internal support assistant that answers from approved policy documents. The team must decide how to apply prompt injection defense, verify the result, and explain the user impact. For an AI Engineer, attach the explanation to an evaluation set and retrieval trace.

Show judgment

  • describe the implementation path, the main trade-off, and the evidence you would collect.
  • State the constraint that could change your decision, such as scale, data sensitivity, recovery target, or team ownership.
  • Call out prompt injection and unsupported answers and the control that reduces it.

Evidence to mention

Track grounded-answer rate and p95 response time. Say what baseline you compared against, what would trigger a rollback or escalation, and who owns the follow-up.

Practice prompt: Use an internal support assistant that answers from approved policy documents as the example and show where you would stop a risky rollout.

Link to question 16
17How have you implemented batching and inference latency in a real AI / GenAI project?

A production answer

Say this first: The important point about batching and inference latency is how an engineer recognizes the unsafe path early and prevents it from becoming customer impact.

Use a real scenario

Imagine an internal support assistant that answers from approved policy documents. The team must decide how to apply batching and inference latency, verify the result, and explain the user impact. For an AI Engineer, attach the explanation to an evaluation set and retrieval trace.

Show judgment

  • describe the implementation path, the main trade-off, and the evidence you would collect.
  • Start with containment and evidence. Changing several variables at once makes the incident harder to understand.
  • Call out prompt injection and unsupported answers and the control that reduces it.

Evidence to mention

Track grounded-answer rate and p95 response time. Say what baseline you compared against, what would trigger a rollback or escalation, and who owns the follow-up.

Practice prompt: Use an internal support assistant that answers from approved policy documents as the example and show where you would stop a risky rollout.

Link to question 17
18A release is blocked because of a problem related to model API integration. What steps would you take?

A production answer

Say this first: model API integration should be explained through its purpose, the boundary where it applies, and the evidence that shows it is working.

Use a real scenario

Imagine an internal support assistant that answers from approved policy documents. The team must decide how to apply model API integration, verify the result, and explain the user impact. For an AI Engineer, attach the explanation to an evaluation set and retrieval trace.

Show judgment

  • describe the implementation path, the main trade-off, and the evidence you would collect.
  • State the constraint that could change your decision, such as scale, data sensitivity, recovery target, or team ownership.
  • Call out prompt injection and unsupported answers and the control that reduces it.

Evidence to mention

Track grounded-answer rate and p95 response time. Say what baseline you compared against, what would trigger a rollback or escalation, and who owns the follow-up.

request or change → guardrail / validation → model API integration → observable result → owner review

Practice prompt: Use an internal support assistant that answers from approved policy documents as the example and show where you would stop a risky rollout.

Link to question 18
19What logs, metrics, or artifacts would you check while troubleshooting model evaluation framework?

A production answer

Say this first: The important point about model evaluation framework is how an engineer recognizes the unsafe path early and prevents it from becoming customer impact.

Use a real scenario

Imagine an internal support assistant that answers from approved policy documents. The team must decide how to apply model evaluation framework, verify the result, and explain the user impact. For an AI Engineer, attach the explanation to an evaluation set and retrieval trace.

Show judgment

  • describe the implementation path, the main trade-off, and the evidence you would collect.
  • Start with containment and evidence. Changing several variables at once makes the incident harder to understand.
  • Call out prompt injection and unsupported answers and the control that reduces it.

Evidence to mention

Track grounded-answer rate and p95 response time. Say what baseline you compared against, what would trigger a rollback or escalation, and who owns the follow-up.

Practice prompt: Use an internal support assistant that answers from approved policy documents as the example and show where you would stop a risky rollout.

Link to question 19
20How would you make A/B testing AI features reliable enough for day-to-day production use?

A production answer

Say this first: A/B testing AI features should be explained through its purpose, the boundary where it applies, and the evidence that shows it is working.

Use a real scenario

Imagine an internal support assistant that answers from approved policy documents. The team must decide how to apply A/B testing AI features, verify the result, and explain the user impact. For an AI Engineer, attach the explanation to an evaluation set and retrieval trace.

Show judgment

  • describe the implementation path, the main trade-off, and the evidence you would collect.
  • State the constraint that could change your decision, such as scale, data sensitivity, recovery target, or team ownership.
  • Call out prompt injection and unsupported answers and the control that reduces it.

Concrete check

npm test -- --runInBand

Evidence to mention

Track grounded-answer rate and p95 response time. Say what baseline you compared against, what would trigger a rollback or escalation, and who owns the follow-up.

Practice prompt: Use an internal support assistant that answers from approved policy documents as the example and show where you would stop a risky rollout.

Link to question 20
21You are working on a production project and model monitoring and drift starts causing issues. How would you diagnose and fix it as an AI Engineer?

A production answer

Say this first: model monitoring and drift should be explained through its purpose, the boundary where it applies, and the evidence that shows it is working.

Use a real scenario

Imagine an internal support assistant that answers from approved policy documents. The team must decide how to apply model monitoring and drift, verify the result, and explain the user impact. For an AI Engineer, attach the explanation to an evaluation set and retrieval trace.

Show judgment

  • describe the implementation path, the main trade-off, and the evidence you would collect.
  • State the constraint that could change your decision, such as scale, data sensitivity, recovery target, or team ownership.
  • Call out prompt injection and unsupported answers and the control that reduces it.

Evidence to mention

Track grounded-answer rate and p95 response time. Say what baseline you compared against, what would trigger a rollback or escalation, and who owns the follow-up.

Practice prompt: Use an internal support assistant that answers from approved policy documents as the example and show where you would stop a risky rollout.

Link to question 21
22How have you implemented GPU serving cost optimization in a real AI / GenAI project?

A production answer

Say this first: The important point about GPU serving cost optimization is how an engineer recognizes the unsafe path early and prevents it from becoming customer impact.

Use a real scenario

Imagine an internal support assistant that answers from approved policy documents. The team must decide how to apply GPU serving cost optimization, verify the result, and explain the user impact. For an AI Engineer, attach the explanation to an evaluation set and retrieval trace.

Show judgment

  • describe the implementation path, the main trade-off, and the evidence you would collect.
  • Start with containment and evidence. Changing several variables at once makes the incident harder to understand.
  • Call out prompt injection and unsupported answers and the control that reduces it.

Evidence to mention

Track grounded-answer rate and p95 response time. Say what baseline you compared against, what would trigger a rollback or escalation, and who owns the follow-up.

Practice prompt: Use an internal support assistant that answers from approved policy documents as the example and show where you would stop a risky rollout.

Link to question 22
23A release is blocked because of a problem related to privacy and PII handling. What steps would you take?

A production answer

Say this first: privacy and PII handling should be explained through its purpose, the boundary where it applies, and the evidence that shows it is working.

Use a real scenario

Imagine an internal support assistant that answers from approved policy documents. The team must decide how to apply privacy and PII handling, verify the result, and explain the user impact. For an AI Engineer, attach the explanation to an evaluation set and retrieval trace.

Show judgment

  • describe the implementation path, the main trade-off, and the evidence you would collect.
  • State the constraint that could change your decision, such as scale, data sensitivity, recovery target, or team ownership.
  • Call out prompt injection and unsupported answers and the control that reduces it.

Evidence to mention

Track grounded-answer rate and p95 response time. Say what baseline you compared against, what would trigger a rollback or escalation, and who owns the follow-up.

Practice prompt: Use an internal support assistant that answers from approved policy documents as the example and show where you would stop a risky rollout.

Link to question 23
24What logs, metrics, or artifacts would you check while troubleshooting responsible AI and bias?

A production answer

Say this first: The important point about responsible AI and bias is how an engineer recognizes the unsafe path early and prevents it from becoming customer impact.

Use a real scenario

Imagine an internal support assistant that answers from approved policy documents. The team must decide how to apply responsible AI and bias, verify the result, and explain the user impact. For an AI Engineer, attach the explanation to an evaluation set and retrieval trace.

Show judgment

  • describe the implementation path, the main trade-off, and the evidence you would collect.
  • Start with containment and evidence. Changing several variables at once makes the incident harder to understand.
  • Call out prompt injection and unsupported answers and the control that reduces it.

Evidence to mention

Track grounded-answer rate and p95 response time. Say what baseline you compared against, what would trigger a rollback or escalation, and who owns the follow-up.

Practice prompt: Use an internal support assistant that answers from approved policy documents as the example and show where you would stop a risky rollout.

Link to question 24
25How would you make agent workflow orchestration reliable enough for day-to-day production use?

A production answer

Say this first: agent workflow orchestration should be explained through its purpose, the boundary where it applies, and the evidence that shows it is working.

Use a real scenario

Imagine an internal support assistant that answers from approved policy documents. The team must decide how to apply agent workflow orchestration, verify the result, and explain the user impact. For an AI Engineer, attach the explanation to an evaluation set and retrieval trace.

Show judgment

  • describe the implementation path, the main trade-off, and the evidence you would collect.
  • State the constraint that could change your decision, such as scale, data sensitivity, recovery target, or team ownership.
  • Call out prompt injection and unsupported answers and the control that reduces it.

Evidence to mention

Track grounded-answer rate and p95 response time. Say what baseline you compared against, what would trigger a rollback or escalation, and who owns the follow-up.

Practice prompt: Use an internal support assistant that answers from approved policy documents as the example and show where you would stop a risky rollout.

Link to question 25

Further reading

These are original practice questions and suggested answers. Adapt them to your own work and explain evidence, trade-offs, and limitations.