What is Data Science & What does a Data Scientist do?
A Data Scientist converts raw customer, business, and operational data into statistical insights, predictions, A/B experiments, and data-driven recommendations.
A Data Scientist converts raw customer, business, and operational data into statistical insights, predictions, A/B experiments, and data-driven recommendations.
Organizations collect large volumes of data from web apps, sales, devices, marketing, and finance. Raw data alone does not create business value — Data Scientists answer critical questions like: Which customers will churn? Which transactions are fraudulent? What will demand look like next month? Which features drive revenue decline?
Data Science leads toward diverse technical roles:
A complete data-science pipeline transforms raw multi-source data into validated models and actionable business decisions:
BUSINESS PROBLEM DEFINITION
│
▼
DATA SOURCES (Databases / APIs / Data Lakes / Files)
│
▼
Data Ingestion & SQL Querying
│
▼
Data Cleaning & Imputation
│
Exploratory Data Analysis (EDA)
┌───────────┴───────────┐
▼ ▼
Descriptive Statistics Data Visualization
│ │
└───────────┬───────────┘
▼
Feature Engineering & Scaling
│
▼
Train / Validation / Test Split
│
▼
Machine Learning (Classification / Regression / Clustering)
│
▼
Model Evaluation & Metrics (Precision/Recall/ROC-AUC/RMSE)
│
┌───────────┼───────────┐
▼ ▼ ▼
Metrics Explainability Bias Check
│ (SHAP) │
└───────────┬───────────┘
▼
Model & Business Insight
│
▼
Business Decision & Actionable Deployment
│
▼
Production Monitoring & Feedback
Defining clear target metrics, querying SQL, understanding structured (tabular) vs unstructured (text/images) data, and handling missing data imputation.
Summarizing distributions, detecting outliers, checking correlations, visualizing relationships with Pandas, Matplotlib, Seaborn, and Plotly.
Mean vs Median (skewed data), Standard Deviation, Normal/Binomial distributions, Covariance, Correlation (≠ Causation), P-Values, and Confidence Intervals.
Null ($H_0$) vs Alternative ($H_1$) hypothesis testing, p-value significance, evaluating A/B variant split conversion lifts.
Deriving new predictive input features, feature scaling (Standardization Z-score vs Normalization Min-Max), and Categorical Encoding (One-hot, Ordinal, Target).
Predicting categorical outcomes (Spam/Fraud/Churn) via Logistic Regression, Decision Trees, Random Forest, XGBoost, LightGBM, and CatBoost.
Predicting continuous numbers (House prices, demand) via Linear Regression, Gradient Boosting, and evaluating MAE, MSE, RMSE, and $R^2$.
Customer segmentation via K-Means clustering, hierarchical clustering, and Principal Component Analysis (PCA) dimensionality reduction.
Train/Val/Test splits, K-Fold Cross-Validation, Bias-Variance tradeoff, Overfitting vs Underfitting, and preventing critical Data Leakage.
Confusion Matrix, Accuracy, Precision (FP-sensitive), Recall (FN-sensitive), F1-Score, ROC-AUC, and handling Class Imbalance (SMOTE, class weighting).
Interpreting model predictions using SHAP (SHapley Additive exPlanations) values, feature importance, fairness audits, and eliminating population bias.
Mastering SQL (JOINs, GROUP BY, Window functions, CTEs) and Python ecosystem (NumPy, Pandas, Scikit-learn, Jupyter Notebooks).
Analyst: Focuses on BI reporting, SQL queries, dashboards, and historical business insights.
Scientist: Focuses on statistics, predictive ML modeling, feature engineering, and experimentation.
Data Scientist: Focuses on business problem formulation, statistical ML models, EDA, and model evaluation.
AI Engineer: Focuses on integrating foundation LLMs, RAG, vector DBs, APIs, and AI agents into software apps.
Data Scientist: Focuses on exploration, hypothesis testing, model building, and business interpretation.
ML Engineer: Focuses on productionizing, scaling, serving, and automating ML pipelines (MLOps).
Python, SQL (PostgreSQL / BigQuery / Snowflake), R, Git, Bash
Pandas, NumPy, Polars, PySpark
Matplotlib, Seaborn, Plotly, Power BI, Tableau
SciPy, Statsmodels, NumPy
Scikit-learn, XGBoost, LightGBM, CatBoost
SHAP, LIME, PyTorch, TensorFlow
Jupyter Notebooks, VS Code, MLflow, Weights & Biases
Azure Machine Learning, AWS SageMaker, Google Vertex AI, Databricks
Snowflake, Databricks, BigQuery, Microsoft Fabric
A telecom enterprise wants to identify which subscription customers are at risk of cancelling in the next 30 days:
Goal: Perform complete EDA, feature engineering, classification model comparison, and business interpretation on a customer dataset.
Technologies: Python, Pandas, NumPy, Matplotlib, Scikit-learn, Jupyter, Git.
Goal: Build a production-grade credit risk classifier with threshold optimization, SHAP explainability, and MLflow experiment tracking.
Technologies: Python, Pandas, Scikit-learn, XGBoost, SHAP, MLflow, FastAPI, Docker.
Goal: Forecast multi-product demand using time-series & tree boosting algorithms and translate predictions into automated inventory replenishment decisions.
Technologies: Python, Pandas, XGBoost, Statsmodels, MLflow, Streamlit / Power BI, Docker.
Practice technical Data Science questions. Connected to our IT Interview Preparation Hub.
Data Science combines statistics, programming, data analysis, domain expertise, and machine learning to discover patterns, answer business questions, and build predictive solutions.
Data Analysts focus on BI reporting, dashboards, and historical business insights. Data Scientists focus on statistical modeling, machine learning algorithms, experimentation, and predictive solutions.
Machine learning where training data contains known target labels (e.g. Fraud/Not Fraud or Price).
Classification predicts categorical labels (e.g. Churn/Stay). Regression predicts continuous numerical values (e.g. Revenue).
When a model fits noise in the training data too closely, producing high training accuracy but poor generalization on test data.
Precision = $TP / (TP + FP)$ (How many predicted positives were correct?). Recall = $TP / (TP + FN)$ (How many actual positives were detected?).
In imbalanced datasets (e.g. 99% legitimate, 1% fraud), a model predicting "Legitimate" for all records gets 99% accuracy but fails completely at fraud detection.
Evaluating model performance across multiple train/test data folds (e.g. 5-Fold CV) to obtain an unbiased estimate of generalization performance.
When information unavailable at real-time prediction time (e.g. post-event features) accidentally leaks into training features, producing artificially high training results.
Inspect missingness mechanism (MCAR, MAR, MNAR); apply mean/median/mode imputation, model-based imputation, or add missing-indicator features depending on business context.
INNER JOIN returns only rows with matching keys in both tables. LEFT JOIN returns all rows from the left table regardless of matches in the right table.
WHERE filters raw individual rows before aggregation occurs. HAVING filters aggregated groups (e.g. HAVING COUNT(*) > 5) after GROUP BY.
Correlation indicates statistical association. Causation means one factor directly produces a change in another. Correlation alone does not establish causation.
Use class-weighted loss functions, SMOTE oversampling, undersampling, threshold tuning, and evaluate using Precision-Recall AUC / F1-Score instead of accuracy.
Use SHAP values and feature importance plots to translate complex model outputs into clear business drivers (e.g. "High support tickets increase churn risk by 35%").
Investigate class imbalance, wrong evaluation metric, incorrect decision threshold, prediction arriving too late for operational action, lack of stakeholder adoption, or data leakage.
Microsoft Learn — Data Scientist Career Path
Covers statistics, data exploration, feature engineering, Scikit-learn, and predictive modeling.
Important Update: Microsoft retired the former DP-100 certification on June 1, 2026. Microsoft now points toward Machine Learning Operations Engineer Associate (AI-300) for Azure ML production pipelines.
Exam: MLA-C02
Validates implementing ML workloads, SageMaker, Bedrock, feature stores, and MLOps on AWS.
Exam: GCP ML Professional
Validates building, productionizing, evaluating, and monitoring ML models on Google Cloud.
Exam: AI-300
Focuses on Azure ML pipelines, model lifecycle, MLOps & GenAIOps infrastructure.
Track your progress against 23 core data science benchmarks:
Recommended next step: Start with Python, SQL & Statistics fundamentals before building machine learning models.