Senior Forward Deployed Engineer.
Fractional Chief AI Officer.

Building Scalable and Cost-Effective Voice Agents: A Platform-Based Blueprint

CVS Health Tech Blog

How to think about voice agents as a platform, separating telephony, LLM, TTS, and orchestration so a fleet of agents can ship without each one re-inventing the runtime.

TrueMem: Building a Model-Agnostic Memory Layer for AI

TrueFoundry engineering blog · byline

How I designed TrueMem, a two-tier (STM + LTM) persistent memory service for LLM applications, and why moving memory outside the model gives you vendor-portability for free.

Transforming Customer Interactions: Evolving IVR Systems for Enhanced Experiences

CVS Health Tech Blog

The migration path from rigid IVR menus to LLM-driven conversational flows, keeping integrations, observability, and compliance posture intact.

My Adventures with LLMs

Leanpub · book

A practitioner's walk-through of building LLMs from scratch in PyTorch, Transformers through DeepSeek, with a companion code repo for every chapter.

Sep 18, 2026

Quantization: Spending a Tight Bit Budget Without Breaking the Model

Every technique so far changed the route the bytes take and left the model bit-for-bit identical. Quantization changes the bytes themselves, storing each number in fewer bits, and it is the first lever in this series that is lossy. This post is the physics: what a number's bits actually buy, why a trained network survives being squeezed, the outlier features that make it hard, and when shrinking the numbers pays off at all.

Sep 16, 2026

Flash Decoding and CUDA Graphs: Filling a Chip One Token at a Time

Every parallelism trick in this series assumed a batch of query rows to hand out. At decode there is exactly one, so a 132-multiprocessor chip receives 32 pieces of work and stands 100 idle. Flash Decoding splits the one dimension nobody had split, the cache itself, and CUDA graphs remove the launch overhead that dominates once each kernel has almost nothing to do.

Sep 16, 2026

PagedAttention: The Cache Was Storing Nothing

Serving systems reserved room for the longest response a request might produce, so only 20 to 38 percent of the KV cache held anything at all. The fix is a 1960s operating systems idea: fixed-size blocks and a table that maps them. Then a second lever, architectural rather than systems, changes how many bytes each token needs to store in the first place.

Sep 16, 2026

FlashAttention 3 and 4: When the Bottleneck Moves

Hopper and Blackwell both made the tensor cores much faster and left their neighbours alone, so the thing in the way kept changing. FlashAttention 3 overlaps loading with computing and drops to 8 bits without losing accuracy. FlashAttention 4 goes after the exponential unit, the rescaling, and the backward pass. The fraction of peak plateaus near three quarters, and that turns out to be the interesting result.

CARL: Cost-Optimized Online Container Placement on VMs Using Adversarial Reinforcement Learning

IEEE Transactions on Cloud Computing · 2025

Adversarial reinforcement learning formulation for online container-to-VM placement, framed as a multi-dimensional vector-bin-packing problem. A learner agent mimics an offline semi-optimal teacher solver while automatically learning a reward function for VM cost reduction, ending up out-performing the teacher it imitates. Evaluated on Google and Alibaba production cluster traces (5k–10k container requests across 2k–8k VMs): ~16% lower VM cost than classic heuristics and SOTA RL methods, ~1,900 placements per second onto ~8,900 candidate VMs, robust to inference-time workload distribution shift.