Nancy Bethala-Frounjian

Reproduced and diagnosed a structured-output failure affecting DeepSeek-R1 reasoning behavior under speculative decoding. Contributed a scheduler-level fix to vLLM core and added validation coverage for the failure path.

"Great analysis and work on a fix! This is a severe issue." — vLLM Maintainer

vLLM · Python · CUDA-aware inference · speculative decoding · structured output · debugging

Built GPU-backed inference infrastructure on AWS EKS using Terraform, Kubernetes, vLLM, NVIDIA Triton Inference Server, FastAPI routing, Prometheus, Grafana, and NVIDIA DCGM telemetry. Focused on backend health, routing behavior, GPU node scheduling, observability, and cost/performance tradeoffs.

AWS EKS · Kubernetes · Terraform · vLLM · Triton · FastAPI · Prometheus · Grafana · NVIDIA DCGM

Built repeatable benchmark workflows to measure how workload shape affects TTFT, ITL/TPOT, throughput, batching behavior, KV-cache pressure, and GPU memory utilization. Used the results to reason about prefill/decode behavior, scheduler pressure, and backend saturation.

vLLM · TTFT · TPOT/ITL · KV cache · throughput analysis · latency decomposition · GPU telemetry

Built a FastAPI-based multi-backend inference router for vLLM workloads with health checks, routing rules, fallback behavior, Redis session affinity, and backend observability. Used the router to study queuing behavior, request placement, and GPU resource coordination under concurrency pressure.

FastAPI · Redis · vLLM · routing · queuing · health checks · backend failover

Building a retrieval/evidence layer focused on document ingestion, chunking, embedding pipelines, vector search, source attribution, retrieval quality evaluation, and isolation of untrusted external content before LLM or agent execution.

RAG · embeddings · pgvector · Qdrant · Milvus patterns · retrieval evaluation · evidence grounding

Designing and implementing a governed agent-runtime lab focused on planner-executor, supervisor-worker, routing, bounded-loop execution, state transitions, tool-calling validation, exception handling, human-in-the-loop checkpoints, and evaluation harnesses for regulated workflows.

agentic system design · state machines · tool validation · bounded loops · eval harnesses · regulated workflows

Building a GPU platform engineering lab covering NVIDIA device-plugin validation, Kubernetes DRA exploration, Kueue admission control, GPU workload placement, GPU failure behavior, observability, and multi-GPU scheduling experiments.

Kubernetes · NVIDIA GPUs · Kueue · DRA · device plugin · workload placement · GPU observability