Introduction to Scikit-Learn

What Is Scikit-Learn?

Scikit-Learn is the standard Python library for classical machine learning. It provides clean, consistent implementations of hundreds of algorithms — from linear regression to gradient boosting — plus tools for preprocessing, model selection, and pipeline construction.

It sits on top of NumPy and SciPy, so it’s fast for tabular data. For deep learning, use PyTorch or TensorFlow. For large-scale distributed ML, use Spark MLlib. For everything else — structured tabular data, traditional ML workflows, feature engineering — Scikit-Learn is the default.

The Estimator API

Every model in Scikit-Learn follows the same three-method interface:

Transformers (scalers, encoders) add:

Your First Classifier

from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import accuracy_score, classification_report

# 1. Load a built-in dataset
iris = load_iris()
X = iris.data        # (150, 4) — 150 samples, 4 features
y = iris.target      # (150,) — 0, 1, or 2 (species)

print(f"Features: {iris.feature_names}")
print(f"Classes:  {iris.target_names}")
print(f"X shape:  {X.shape}")

# 2. Split into training and test sets — always do this before touching the data
X_train, X_test, y_train, y_test = train_test_split(
    X, y,
    test_size=0.2,      # 20% for testing
    random_state=42,    # reproducibility
    stratify=y,         # keep class proportions in both splits
)
print(f"Train: {X_train.shape}, Test: {X_test.shape}")

# 3. Initialize and train the model
model = RandomForestClassifier(n_estimators=100, random_state=42)
model.fit(X_train, y_train)

# 4. Evaluate on the held-out test set
y_pred = model.predict(X_test)
print(f"\nAccuracy: {accuracy_score(y_test, y_pred):.4f}")
print("\nClassification Report:")
print(classification_report(y_test, y_pred, target_names=iris.target_names))

Your First Regression Model

from sklearn.datasets import fetch_california_housing
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression
from sklearn.preprocessing import StandardScaler
from sklearn.metrics import mean_absolute_error, r2_score
import numpy as np

# Load dataset — California housing prices
data = fetch_california_housing()
X, y = data.data, data.target   # y = median house value in $100k units

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42
)

# Scale features — linear models require feature scaling
scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)   # fit on train only
X_test_scaled  = scaler.transform(X_test)        # apply same scaling to test

# Train
model = LinearRegression()
model.fit(X_train_scaled, y_train)

# Evaluate
y_pred = model.predict(X_test_scaled)
mae = mean_absolute_error(y_test, y_pred)
r2  = r2_score(y_test, y_pred)

print(f"MAE: ${mae * 100_000:,.0f}")   # error in actual dollars
print(f"R²:  {r2:.4f}")               # 0 = useless, 1 = perfect

# Feature importance — coefficients for linear regression
for name, coef in zip(data.feature_names, model.coef_):
    print(f"  {name:20s}: {coef:+.4f}")

The Train/Test Split Rule

The most important rule in ML: never evaluate on training data. If you fit the model and test it on the same data, you’re measuring memorization — not generalization.

from sklearn.model_selection import train_test_split
import numpy as np

X = np.random.standard_normal((1000, 10))
y = np.random.randint(0, 2, 1000)

# Correct: fit on train, evaluate on test
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)

# Wrong — never do this
# model.fit(X, y)
# score = model.score(X, y)  # artificially inflated score

Built-in Datasets

Scikit-Learn ships with several small standard datasets for learning:

from sklearn import datasets

iris      = datasets.load_iris()            # 150 samples, 4 features, 3 classes
digits    = datasets.load_digits()          # 1797 handwritten digit images
wine      = datasets.load_wine()            # 178 wine samples, 13 chemical features
breast_cancer = datasets.load_breast_cancer()   # 569 tumor samples, binary
housing   = datasets.fetch_california_housing()  # 20k samples, regression target

# Generate synthetic data for experimentation
from sklearn.datasets import make_classification, make_regression

X, y = make_classification(
    n_samples=1000, n_features=20, n_informative=10,
    n_classes=3, random_state=42
)

X_reg, y_reg = make_regression(
    n_samples=1000, n_features=15, noise=0.1, random_state=42
)