Vineeth Reddy Kodakandla / ML engineerIntel Core Ultra 7 155H · Illustration

01 / The engineer

Models on real chips. With the data attached.

I work on inference: how fast and how cheaply models run on real hardware, and whether their output holds up. Every measured result on this site links to the committed file it came from.

02 / Flat-out

Run it flat-out, the NPU looks safer.

On one Intel laptop chip, INT8 segmentation on the Arc iGPU had a 47.0 ms p99 in a back-to-back benchmark. The NPU held 18.0 ms.

Data:live_results.json L3-51

03 / A live camera

Point a camera at it, and the p99 order flips.

On a live webcam loop the iGPU's p99 fell to 13.3 ms and the NPU's rose to 22.3 ms, though the NPU still missed the 33.3 ms frame budget less often.

Data:live_results.json L3-51

In short

Before my M.S. in Computer Science, I worked on LLM serving at EXL and on real-time vision inference at L&T Technology Services. The projects below are public, and each write-up says what its measurement does not show.

Open to ML engineer, inference and software engineering roles. Based in the United States and open to relocating anywhere in the US.

Selected work

Three measurements. Every figure sourced.

Each has a write-up of the setup, the findings and the limits, and every figure links to the committed file it came from.

  1. LLM serving / reproducibility

    bitwise-forensics

    Does a serving engine return the same greedy tokens for a prompt when other requests share its engine steps?

    I measured OpenVINO GenAI's continuous-batching pipeline serving Qwen2.5-Coder-0.5B (INT4 weights) on the CPU and Arc iGPU of an Intel Core Ultra 7 155H: 40 prompts, 8 trials, 7,680 runs. Under each device's default settings, most batched runs produced different tokens from the same prompt served alone. Where tested, the output was a fixed function of the arrival schedule: replaying the eight c8 schedules reproduced all 320 runs on each device. On the CPU the main lever was dynamic activation quantization.

    • Python
    • OpenVINO GenAI
    • Continuous batching
    • Bootstrap statistics
    • pytest
    • GitHub Actions
    320 of 320
    runs at arrival rate c8 reproduced bit for bit when their arrival schedules were replayed, on each device
    Data:readme_stats.json L802-808,readme_stats.json L943-949
    71.2% to 21.2%
    CPU runs at c8 whose tokens changed, before and after turning off dynamic quantization (paired 95% CI for the drop: 34.7 to 64.4 points)
    Data:summary.csv L7-8,paired.csv L3

    CPU · Default options · c8

    Final state

    228of 320 runs changed71.2%by token 128 of 128

    The 320 token strips are drawn with JavaScript. The counts shown are the final measured values.

    Final state with default engine options at arrival rate c8: 228 of 320 CPU runs (71.2%) and 320 of 320 Arc iGPU runs (100%) produced tokens that differ from batch-of-1 within 128 tokens. The median first differing token was at position 29 on the CPU and 12 on the iGPU.

    Each strip is one greedy run of Qwen2.5-Coder-0.5B on an Intel Core Ultra 7 155H at arrival rate c8, turning copper at the first token that differs from the same prompt served alone. Token positions and counts are measured; the streaming speed is illustrative.Data:runs.jsonl.gz,baselines.jsonl.gz,summary.csv L1,summary.csv L7Full figure in the case study
  2. Edge inference / latency

    Meteor Lake Latency Lab

    Which processor on one laptop chip should run live instance segmentation inside a 33.3 ms frame budget: the NPU, the Arc iGPU or the CPU?

    YOLOv8n-seg through OpenVINO on an Intel Core Ultra 7 155H, across three processors and FP32, FP16 and two INT8 builds, with 1,000 timed frames per configuration. For INT8 on the iGPU and the NPU, a flat-out benchmark and a live webcam loop reversed the p99 order, although the NPU missed the 33.3 ms budget less often in both; the CPU was not run live. The iGPU's flat-out tail came from frames where its CPU-side stages stalled, not from inference.

    • Python
    • OpenVINO
    • NNCF INT8
    • NumPy
    • pandas
    47.0 ms to 13.3 ms
    p99 latency of INT8 on the Arc iGPU, flat-out benchmark against a live webcam loop
    Data:live_results.json L30-51
    0.0015 vs 0.0389
    box mAP lost on the CPU when INT8 keeps the detection-head decode in float, against quantizing all of it
    Data:accuracy.json L24-45
    Frame streamReplayingAll 1,000 frames

    The frame strip is drawn with JavaScript. The readout gives the whole run, and every frame is in the linked source file.

    To inspect single frames, press the left and right arrow keys. Page Up and Page Down move 50 frames, Home and End go to the first and the latest frame shown, and Escape ends the inspection. Inspecting pauses the replay.
    Measured frames of the flat-out INT8 benchmark, stacked as preprocess, inference and postprocess; bars taller than 70 ms are cut and marked with a caret. In motion, each bar is one frame and the readout counts the frames shown so far; only the replay speed is illustrative (40 frames a second, not the measured rate). With motion off, all 1,000 frames are drawn at once, and a pixel column that covers several frames shows the tallest.Frames:latency_main.parquetArc iGPU checked against:summary_main.json L146-169,stall_analysis.json L82-94Full figure in the case study

    Whole-run summary, 1,000 frames per processor: Arc iGPU, median 11.30 ms, p99 47.02 ms, 16 frames over 33.3 ms, 17 stall frames. NPU, median 13.93 ms, p99 17.98 ms, 2 frames over 33.3 ms, 2 stall frames. CPU, median 32.63 ms, p99 49.01 ms, 463 frames over 33.3 ms, 1 stall frame. The full figure in the case study lists every stall frame.

  3. LLM evaluation / automation

    LLM Eval Observatory

    Can an open-weight model triage anti-money-laundering alerts without a person checking every case, and can its scores be trusted?

    A GitHub Action runs nightly against open-weight models on Groq and commits the results for a dashboard on Vercel. The main track gives each model 26 hand-written synthetic AML cases, scored by a fixed parser rather than another model; the number that matters is the false clear. Three more tracks cover prompt injection, capability drift and LLM-as-judge reliability, and each track's headline score carries a bootstrap interval.

    • Python
    • GitHub Actions
    • Bootstrap statistics
    • Next.js
    • Groq API
    Nightly run of 15 Nov 2026
    2 of 2 models
    returned results in that run, and 2 finished all of their tracks. 30 of the 30 nights up to it produced results. A model that Groq's free tier rejects or keeps rate-limiting is dropped for the rest of the run.
    Live data, updated nightly:latest.json,history.jsonl

    Snapshot at bf8ad56 · recorded nights, 28 Jul to 15 Sep 2026, 28 Jul to 15 Sep 2026

    Five models returned results on the first night. After 17 Aug, only GPT-OSS 120B and GPT-OSS 20B did.

    GPT-OSS 120B · triage accuracy
    92.3 to 100%
    29 nights, 2 Aug to 15 Sep
    GPT-OSS 120B · false clear rate
    0%
    on all 29 triage nights
    GPT-OSS 120B · injection defense
    93.3 to 96.7%
    31 nights, 28 Jul to 15 Sep
    Llama 3.3 70B · injection defense
    73.3 to 83.3%
    19 nights, 28 Jul to 17 Aug

    Snapshot ends · run 05:05 UTC

    Tue 15 Sep 2026 2 models returned results

    • Returned resultsResults
    • Some tracks onlySome tracks
    • No results recordedNone
    28 Jul4 Aug11 Aug18 Aug25 Aug1 Sep8 Sep15 SepModelResults returned, by nightGPT-OSS 120BGPT-OSS 20BLlama 3.3 70BLlama 3.1 8BQwen3.6 27Bno results 30 Jul to 2 Sepno results after 17 Augno results after 29 Julresults on 28 Jul only15 Sep
    Each mark is a recorded night on which that model returned results; hatched nights have none. Only the replay's pace is illustrative, and it is not a live feed. Data:history.jsonl,latest.json (model names) Full figure in the case study

Also built

Multi-agent pipeline / browser demo

Edith

GoalAgents, in orderShared blackboardEvidenceAnalysisVerdictDraftPolicy“Should we adopt pgvector for our RAG stack?”2 of 8 docs match▍tradeoffs weighed▍PASS▍v1: proceed▍4 of 4 checks pass▍ResearcherAnalystCriticSynthesizerGovernorReleasedrecommendation shown
DiagramHow Edith decides whether to release an answerA replay of the events edith-lab's orchestrator emits in its deterministic mode (no API key) for two demo goals. Timing is illustrative; every verdict, rewrite and gate result is from the recorded trace.
Code:orchestrator.js L8-59,agents.js L49-59,governor.js L4-31,corpus.js L21-29
  1. In-corpus goal: “Should we adopt pgvector for our RAG stack?”. The Researcher finds 2 of 8 documents sharing a keyword with the goal (RAG vs. fine-tuning; pgvector vs. dedicated vector DB). The Analyst weighs tradeoffs. The Critic returns PASS. The Synthesizer writes one draft that proceeds. The Governor's four checks pass and the recommendation is released.
  2. Off-corpus goal: “Should we migrate our monolith to microservices?”. No document in the 8-document corpus shares a keyword with the goal. The Analyst hedges. The Critic returns REVISE, so the Synthesizer rewrites its draft once. The Governor's grounded check fails, so it sends the draft back for one more rewrite and checks again; the grounded check still fails, and the recommendation is labeled blocked for human review. The answer is still shown.

Researcher, Analyst, Critic and Synthesizer agents run in fixed order, deterministically, over an 8-document corpus. A rule-based Governor sends a failing draft back for one rewrite without checking it against the sources, and a goal that matches no document ends labelled blocked, for human review.

  • JavaScript
  • Multi-agent orchestration
  • Rule-based policy gate

Agent memory / browser demo

ANANTA

RetentionR = e−t/S100%50%0%15%0 hMemoryImpS (h)Rt = 132 hrecall 36 hrecall 72 hdecay pass 132 hZoomeval, no recalls5%0%2%3.79%1.45%0.53%0.04%Decay passvisa deadline0.9534.815.0%kept: R ≥ 2%answer style0.8031.21.45%kept: imp ≥ 0.8eval result0.5540.33.79%kept: R ≥ 2%coffee order0.2016.80.04%pruned: R < 2%
DiagramHow ANANTA forgetsAn illustrative scenario on a simulated clock: four seed memories decay, one is recalled, and a decay pass checks each against the 2% prune line. Every retention value is from running the repository's MemoryStore.
Code:memory.js L17-22,memory.js L38-39,memory.js L88-94,memory.js L104-119

Stability S = 24 × (0.5 + importance) × (1 + 0.3 × recalls) hours. Importance 0.9 or more keeps a 15% retention floor. The decay pass prunes a memory below 2% retention unless its importance is 0.8 or more. The inset magnifies 0 to 5% retention over the latest 36 hours.

Retention at the manual decay pass at hour 132. The eval result memory was recalled at hours 36 and 72; without those recalls its retention there would be 0.53% and it would be pruned.
MemoryImportanceRecallsStability (hours)RetentionDecay pass
visa deadline0.95034.815.0%kept: at or above 2%
answer style0.8031.21.45%kept: below 2% but importance at or above 0.8
eval result0.55240.33.79%kept: at or above 2%
coffee order0.2016.80.04%pruned: below 2% and importance below 0.8
eval result, never recalled0.55025.20.53%pruned: below 2% and importance below 0.8

Agent memory that forgets on purpose. Memories decay on an Ebbinghaus curve, recall slows decay, a manual pass prunes faded ones unless important, and a SHA-256 hash chain over text, timestamp and importance exposes edits. A deterministic agent loop escalates a tier on thin retrieval.

  • JavaScript
  • Web Crypto
  • Forgetting-curve decay

Retrieval-augmented chatbot

This site

1 Questionlast 2 user turns2 Embed512 dimsvoyage-3-lite3 Searchcosine, top 5pgvector in Neon4 Answer textknowledge.json5 Claude streamspassages listeddistance →top 5 ids5 of 15 chunksfrom
DiagramHow this site's chatbot finds its answerFrom the chat route and lib/rag.ts. The 512 dimensions, top 5, 4 s timeout and fallback rules are from the code; vector values, distances and tokens are illustrative. With no key or database, the fallback skips embedding.
Code:route.ts L82-93,rag.ts L33-54,rag.ts L56-88,migrate.mjs L22-27
  1. Question: the last two user turns are joined into one query.
  2. Embed: Voyage voyage-3-lite turns the query into a 512-dimension vector. The diagram draws one bar for every 8 dimensions.
  3. Search: pgvector in Neon orders the knowledge-base chunks by cosine distance and returns the 5 nearest ids.
  4. Answer text: the text for those ids always comes from the bundled knowledge base (15 chunks), never from the database rows.
  5. Claude streams the answer from those passages, and the passages are listed under it.
  6. Fallback: all 15 chunks go into the prompt and no passages are listed when there is no Voyage key or database (checked before embedding), when the embedding or the query errors or takes over 4 seconds, when the query returns no rows, or when any returned row differs from the bundle.

Next.js on Vercel with a chatbot that answers from a short knowledge base written to match these pages. It embeds your last two questions together with Voyage, finds the nearest passages with pgvector in Postgres, and Claude answers from them, listing them when retrieval succeeds.

  • Next.js
  • TypeScript
  • pgvector
  • Claude API

Experience

Two roles in inference. Then an M.S.

Work at these companies is not public, so it is described here without figures.

  1. 2023 to 2024

    EXL

    Noida, India

    LLM serving for insurance underwriting and claims work. I worked on the Amazon Bedrock serving path (prompt routing, context-window management and token streaming), on evaluation runs that compared fine-tuned models with commercial models, and on the latency and cost dashboards the team used. I contributed pre-launch engineering to LDS Underwriting Assist, which EXL launched in August 2024.

  2. 2022 to 2023

    L&T Technology Services

    Bengaluru, India

    Real-time inference for endoscopy video, covering image enhancement and polyp detection, on NVIDIA's Holoscan SDK and developer kits. I moved models from PyTorch to ONNX Runtime and TensorRT, enabling FP16 where supported, restructured the input pipeline to cut per-frame latency, and wrote the benchmarking scripts used to compare edge targets. I started there as an intern.

  3. Aug 2024 to May 2026

    Texas A&M University-Corpus Christi

    M.S. in Computer Science

    Graduate research assistant on rule-based multi-agent access control (ABAC and XACML), and teaching assistant for Computer Forensics. Coursework included computer architecture, advanced software engineering, network security and agent-based systems.

Tools and methods

Inference and optimization

  • OpenVINO and OpenVINO GenAI
  • NNCF post-training quantization
  • ONNX Runtime and TensorRT
  • FP16 and INT8 quantization
  • Continuous batching

Measurement

  • Latency distributions and frame budgets
  • Bootstrap confidence intervals
  • Paired comparisons
  • Replay and reproducibility checks
  • Benchmark automation

LLM applications and evaluation

  • Evaluation harnesses
  • Prompt-injection testing
  • LLM-as-judge reliability
  • Retrieval-augmented generation
  • Amazon Bedrock

Engineering

  • Python, TypeScript, C++, SQL
  • NumPy, pandas, PyTorch
  • FastAPI, React, Next.js
  • PostgreSQL and pgvector
  • Docker, Kubernetes, GitHub Actions

Certification

  • Building with the Claude API (Anthropic)

About

Start from a claim. Find out where it holds.

Most of my projects start from a claim that is easy to repeat and then check where it holds: that a benchmark picks the right processor for a camera, that greedy decoding gives the same tokens when requests are batched together, that a model can triage alerts without a person reading each one. The answer is usually that it depends, and the useful part is finding out on what.

I also try to publish what a result does not show, and to drop results that fail a check. bitwise-forensics began as a simulator. When a check across 12 random seeds showed that the simulated load trend changed with the seed, I kept the simulator only as a pipeline test and measured a real engine instead.

Simulator seed check Nearly equal means. The seeds split on direction. bitwise-forensics simulator, not the real engine. For each of 12 seeds, the share of 960 simulated runs (40 prompts x 24 trials) whose tokens changed from the batch-of-1 output, at arrival rates c8 and c32. The means differ by 0.16 points.

c8 is lower than c32 in 6 seeds, higher in 5 seeds and tied in 1 seed (seed 1).

mean of 12 seeds: c8 31.82%, c32 31.98%0−2.9210.002−4.693−3.654−3.545+0.426+4.387−0.318+8.029+1.3510+1.7711−2.71

The animation starts every seed at the two means and moves it to its own values; the tally counts each seed as it lands. Hover a row for its run counts; every value is also in the table below.Every value, with its run count, is in the table below.

Show the numbers as a table

Share of 960 simulated runs per seed whose tokens changed, with the run count in parentheses. The difference is c8 minus c32, in percentage points, computed from the run counts before rounding, so it can be 0.01 away from the rounded c8 minus the rounded c32.

Simulator runs with changed tokens at c8 and c32, by seed
Seedc8c32c8 minus c32c8 vs c32
042.60% (409)45.52% (437)−2.92 ptslower
110.73% (103)10.73% (103)0.00 ptstied
235.73% (343)40.42% (388)−4.69 ptslower
311.77% (113)15.42% (148)−3.65 ptslower
431.35% (301)34.90% (335)−3.54 ptslower
541.56% (399)41.15% (395)+0.42 ptshigher
656.77% (545)52.40% (503)+4.38 ptshigher
733.75% (324)34.06% (327)−0.31 ptslower
846.15% (443)38.12% (366)+8.02 ptshigher
948.23% (463)46.88% (450)+1.35 ptshigher
1011.56% (111)9.79% (94)+1.77 ptshigher
1111.67% (112)14.38% (138)−2.71 ptslower
Mean31.82%31.98%−0.16 pts6 lower, 5 higher, 1 tied
Data:sim_seeds.md L62-77,sim_seeds.md L29,sim_seeds.md L12-18,sim_seeds.json

I like small, inspectable builds as well, like the two browser demos above, and the assistant on this site, which answers questions about my work from a short knowledge base written to match these pages.

Contact

Get in touch. I reply by email.

Work authorization. I am on F-1 OPT and authorized to work in the US now. I will need H-1B sponsorship to continue beyond OPT, and I would rather talk about that early. Roles that require US citizenship or a security clearance are not a fit.

Send a message

Your name, email and message are saved and emailed to me so I can reply.