Introduction to Machine Learning for Backend Engineers
A practical introduction to machine learning concepts that backend engineers need when integrating ML models into production applications.
You do not need a PhD to integrate machine learning into your applications. You do need to understand what ML models can and cannot do, how they fail, and how to serve them in production. Most backend engineers encounter ML not by training models but by deploying, monitoring, and maintaining them.
The gap between "model works in a Jupyter notebook" and "model works in production" is where backend engineering skills become critical. This is where you come in.
ML Models Are Just Functions
Strip away the jargon and a trained ML model is a function: input data goes in, a prediction comes out. The training process finds the parameters that make this function produce useful outputs.
# A trained model is conceptually this simple
def predict_churn(customer: dict) -> float:
"""Returns probability (0-1) that customer will cancel."""
# Internally: multiply features by learned weights,
# apply activation functions, return result
return model.predict(preprocess(customer))
# In production, you're wrapping this function in an API
@app.post("/predict/churn")
async def churn_endpoint(customer: CustomerInput):
probability = predict_churn(customer.dict())
return {"churn_probability": probability}The complexity lives in three areas: getting data into the right format, serving predictions at acceptable latency, and monitoring whether the model still works.
Feature Engineering: The Real Work
Models consume numbers, not objects. Converting raw application data into the numeric features a model expects is called feature engineering — and it is usually the most fragile part of the pipeline.
# ❌ Passing raw data directly — model can't use this
raw_customer = {
"signup_date": "2020-03-15",
"plan": "premium",
"last_login": "2021-01-20",
"support_tickets": 3,
}
# ✅ Engineered features — numeric values the model expects
def extract_features(customer: dict) -> list[float]:
now = datetime.now()
signup = datetime.fromisoformat(customer["signup_date"])
last_login = datetime.fromisoformat(customer["last_login"])
return [
(now - signup).days, # account_age_days
(now - last_login).days, # days_since_login
customer["support_tickets"], # support_ticket_count
1.0 if customer["plan"] == "premium" else 0.0, # is_premium
customer["support_tickets"] / max((now - signup).days, 1), # tickets_per_day
]The critical rule: the features used in production must match the features used during training, in the same order, with the same transformations. A mismatch here produces predictions that look plausible but are completely wrong.
// Feature pipeline validation
interface FeatureSchema {
name: string;
type: 'numeric' | 'categorical';
range?: [number, number];
required: boolean;
}
const expectedFeatures: FeatureSchema[] = [
{ name: 'account_age_days', type: 'numeric', range: [0, 3650], required: true },
{ name: 'days_since_login', type: 'numeric', range: [0, 365], required: true },
{ name: 'support_tickets', type: 'numeric', range: [0, 100], required: true },
{ name: 'is_premium', type: 'categorical', required: true },
{ name: 'tickets_per_day', type: 'numeric', range: [0, 10], required: true },
];Model Serving Patterns
Three common patterns for serving ML predictions, each with different trade-offs.
Synchronous API
# Direct inference — model loaded in the API process
from fastapi import FastAPI
import joblib
app = FastAPI()
model = joblib.load("models/churn_v3.pkl")
@app.post("/predict")
async def predict(features: FeatureInput):
prediction = model.predict([features.to_array()])
return {"prediction": float(prediction[0])}Simple, but the model consumes memory in every API worker process and inference latency adds directly to response time.
Async with Queue
# Decouple prediction from request/response
import aio_pika
async def request_prediction(customer_id: str, features: list[float]):
message = aio_pika.Message(
body=json.dumps({
"customer_id": customer_id,
"features": features,
"callback_url": "/webhooks/prediction-complete",
}).encode()
)
await channel.default_exchange.publish(
message,
routing_key="predictions",
)
# Worker process — separate from API
async def prediction_worker(message):
data = json.loads(message.body)
prediction = model.predict([data["features"]])
await notify_callback(data["callback_url"], {
"customer_id": data["customer_id"],
"prediction": float(prediction[0]),
})Better for expensive models (>100ms inference) or batch predictions. The API stays responsive while prediction happens asynchronously.
Pre-computed Predictions
-- Batch job runs nightly, stores predictions in database
INSERT INTO churn_predictions (customer_id, probability, computed_at)
SELECT
c.id,
predict_churn(c.features),
NOW()
FROM customers c
ON CONFLICT (customer_id)
DO UPDATE SET probability = EXCLUDED.probability,
computed_at = EXCLUDED.computed_at;// API reads pre-computed predictions — zero inference latency
async function getChurnRisk(customerId: string): Promise<number> {
const result = await db.query(
'SELECT probability FROM churn_predictions WHERE customer_id = $1',
[customerId]
);
return result?.probability ?? 0.5; // default for new customers
}This works when predictions do not need to be real-time. Most recommendation systems, risk scores, and personalization features use pre-computed predictions.
Model Monitoring
A model that was 95% accurate in training can degrade to 60% in production without any code changes. This happens when the real-world data distribution shifts from the training data.
interface PredictionLog {
modelVersion: string;
inputFeatures: Record<string, number>;
prediction: number;
confidence: number;
latencyMs: number;
timestamp: Date;
}
async function logPrediction(log: PredictionLog): Promise<void> {
// Store for monitoring and analysis
await db.insert('prediction_logs', log);
// Alert on anomalies
if (log.confidence < 0.3) {
await alerting.warn('low-confidence-prediction', {
model: log.modelVersion,
confidence: log.confidence,
});
}
if (log.latencyMs > 200) {
await alerting.warn('slow-prediction', {
model: log.modelVersion,
latency: log.latencyMs,
});
}
}Monitor three things:
- Prediction distribution — if the output distribution shifts, the model might be seeing data it was not trained on
- Feature distribution — if input features drift from training ranges, predictions become unreliable
- Latency — model inference time can spike with certain input shapes
A/B Testing Models
Deploying a new model version requires comparing it against the current one with real traffic.
function selectModel(userId: string): ModelVersion {
// Deterministic assignment based on user ID
const hash = hashCode(userId) % 100;
if (hash < 10) {
return 'v4-candidate'; // 10% traffic
}
return 'v3-production'; // 90% traffic
}
async function predict(userId: string, features: number[]): Promise<Prediction> {
const version = selectModel(userId);
const model = models.get(version);
const result = model.predict(features);
await logPrediction({
modelVersion: version,
prediction: result,
userId,
});
return result;
}Compare the candidate model against the production model on business metrics (conversion rate, revenue impact), not just accuracy scores. A model with 2% lower accuracy but 10% higher conversion rate is the better choice.
Key Takeaways
- ML models are functions — input goes in, prediction comes out. Your job is serving that function reliably.
- Feature parity is critical — production features must exactly match training features, or predictions are meaningless
- Choose the serving pattern for your latency requirements — sync API for real-time, queue for expensive models, pre-computed for batch
- Monitor predictions, not just infrastructure — model quality degrades without code changes when data distribution shifts
- A/B test on business metrics — accuracy in isolation does not tell you if the model improves outcomes
