ML/LLM engineer focused on the parts that actually decide whether ML works in production — evaluation, calibration, thresholds, drift, serving, and agentic systems. 20+ years, ~50 patents.
Bug fixes to production ML/LLM libraries used by millions of developers. All PRs include reproduction case and regression tests.
ResponseInputAudioParam from its Pydantic union, rejecting any audio-bearing message at validation time — while the chat completions endpoint accepted identical payloads. Fix: extended the union with an audio-capable TypedDict variant.gather_examples_from_sets helper filtered for the "augmented" key, which only exists on LLM-bootstrapped demos. Hand-labeled training examples never carry it, so they were dropped silently. The optimizer then generated instructions without ever seeing the user's labeled data._build_partial_object used isinstance(field_type, type) to decide whether to recurse into a nested model. That guard returns False for Optional[NestedModel] (a generic alias, not a type), so those fields were stored as raw dicts — causing AttributeError mid-stream when code accessed chunk.inner.some_field.extract_logprobs() converted these to None, then torch.tensor([None]) raised TypeError. Fix: convert None to nan before tensor construction, use torch.where for neutral importance-sampling weights.inject_adapter_in_model() returned the bare inner nn.Module, stripping the AdaLoraModel wrapper that holds update_and_allocate. Calling it afterward raised AttributeError, silently breaking AdaLoRA's rank-reallocation schedule._update_responses_api_response_id_with_model_id was not idempotent — repeated calls in MCP tool loops applied base64+URL encoding twice, producing ~730-char IDs that are rejected by downstream API calls.reasoning_content. Applications relying on chain-of-thought tokens received empty strings silently.reasoning_tokens: 0 in the usage block regardless of the actual token count returned by the upstream model. Downstream cost attribution and budget tracking silently received wrong values.stream_chunk_builder indexed chunk["choices"] unconditionally, raising KeyError and aborting the entire stream.AIMessageChunk.init_tool_calls treats args="" (stream in progress) identically to args=None (field absent), resolving both to {} and firing the tool call before arguments have arrived. Fix ready; LangChain requires issue assignment before a PR will be accepted.The real decision tree — do you have labeled data, is the task format-sensitive or knowledge-sensitive, have you hit the cost crossover? With the breakeven formula, LoRA vs. RAG tradeoffs, and the data requirements teams consistently underestimate.
SSE framing, proxy buffering, backpressure on slow clients, incomplete JSON from tool calls, reconnection replay, TTFT measurement, and the timeout misconfiguration that kills every long generation.
Token bucket mechanics, full-jitter backoff, prompt caching economics, and a concrete cost model for handling 429s, transient 5xx errors, cost overruns, and cold-start latency spikes at scale.
Why cosine similarity is the wrong objective for retrieval quality. NDCG, MRR, P@K, R@K explained — and how to actually measure whether your retriever is improving your answers.
Most deployed risk models output scores that are monotonically correct but probabilistically wrong. ECE, RMS calibration error, reliability diagrams, Platt scaling, and why your threshold sweep is lying to you.
| tool-loop | Correct agentic tool-use loop: parallel dispatch, error isolation, auto-schema from Python functions |
| mcp-quickserver | MCP server template: tools, resources, and prompts with stdio and SSE transports |
| stream-parse | Parse streaming LLM output: incremental JSON, markdown blocks, tool-call deltas, SSE events |
| agent-scratchpad | Persistent vector memory for agents: embed, store, retrieve by cosine similarity |
| prompt-cache-bench | Benchmark prompt caching: cache hit rate, latency delta, cost savings with real measurements |
| llm-eval-lite | Assertion-based eval harness for LLM/agent outputs; composite checks (AllOf, AnyOf) |
| rag-eval | RAG pipeline evaluation: chunking strategies, retrieval quality, answer faithfulness |
| llm-gateway | Production Anthropic API proxy: token-bucket rate limiting, retry with backoff, cost tracking |
| rag-demo | End-to-end RAG demo: BM25 + tool-loop agent + faithfulness eval + persistent memory |
| ml-eval-report | Binary-classifier eval: metrics, ROC/PR + AUC, threshold sweep, Brier score, ECE |
| calibrate-ml | Probability calibration: Platt scaling, isotonic regression, ECE, reliability diagram |
| thresholdkit | Pick operating thresholds under precision / FPR / cost / expected-value constraints |
| rankeval | NDCG, MRR, AP@K, P@K, R@K — ranking metrics for search, recommendation, RAG |
| featurecheck | Feature drift (PSI/KS/chi-squared) + schema/null/dtype validation |
| idgraph | Identity/entity graphs from shared signals; surface synthetic-identity rings + risk scoring |
| pii-redactor | Detect & redact PII (email, phone, SSN, IP, Luhn-validated cards); custom patterns |
| capture-qa | Image capture-quality gates (sharpness, exposure, resolution) |
| modelcard-gen | Generate Model Card markdown from a JSON config |