1. User Application & APIs
Connects web/mobile UI to AI backends via FastAPI, Flask, Node.js or ASP.NET Core with rate limiting & session handling.
2. Prompt Engineering
Designing system instructions, context, few-shot examples, and constraints to guide generative model outputs.
3. Large Language Models (LLMs)
Neural foundation models used via API/SDK/endpoints for text generation, reasoning, coding, and extraction.
4. Tokens & Context Windows
LLMs process text as numerical tokens. Context window limits dictate maximum input/output capacity, latency, and cost.
5. Embeddings
Numerical vector representations that capture semantic meaning of text for similarity search and document retrieval.
6. Vector Databases
Specialized databases (Qdrant, Pinecone, Weaviate, pgvector, Chroma) that index and query high-dimensional embeddings.
7. Retrieval-Augmented Generation (RAG)
Retrieving relevant external private/changing knowledge to ground LLM responses in verifiable facts.
8. Document Chunking
Dividing large documents into optimal semantic or token-based chunks to maximize vector retrieval quality.
9. Reranking
Using a cross-encoder reranker to reorder vector search results by exact relevance before passing to LLM.
10. AI Agents & Tool Calling
Autonomous agent loops that decide actions, execute external functions/APIs/DB queries, observe results, and answer.
11. Machine Learning & Training
Feature engineering, algorithm selection, and validation for tabular predictions (fraud, churn, recommendations).
12. Fine-Tuning vs RAG
RAG provides external knowledge at runtime. Fine-tuning adjusts model style/behavior/pattern using custom dataset training.
13. Model Evaluation
Testing retrieval quality (recall/precision) AND generation quality (faithfulness/groundedness) systematically.
14. Hallucination Mitigation
Reducing unsupported model outputs using RAG, strict system prompts, citations, tool verification, and guardrails.
15. Guardrails & Safety
Input/output validation filters that block prompt injection, PII leaks, toxic content, and unauthorized tool calls.
16. Prompt Injection Protection
Preventing malicious inputs from overriding system instructions using instruction isolation and least-privilege tools.
17. Responsible AI
Ensuring fairness, privacy, security, transparency, accountability, and human-in-the-loop oversight for critical AI actions.
18. Structured Outputs
Enforcing JSON schema output formatting so software applications can parse model outputs reliably without regex.
19. Application Memory
Managing short-term conversation context using recent windows, summaries, or database-backed semantic memory.
20. Caching & Latency Reduction
Storing reusable LLM/embedding query results in Redis/cache to reduce latency and API bill costs.
21. AI Observability
Monitoring token usage, model latency, cost, prompt failures, retrieval accuracy, and safety violations via OpenTelemetry.
22. MLOps / LLMOps
Operational processes for versioning prompts, datasets, models, guardrails, automated testing, and cloud deployments.