Sr. Software Development Engineer – AI/ML Networking Disaggregated Inference, Annapurna Labs, Annapurna Labs
Amazon - Cupertino, California, USA
Posted Jul 21, 2026
Benefits
- Parental leave
- Not verified
- Non-birth-parent leave
- Not verified
- Family-building benefits
- Mental health support
- Offered From the posting source checked Nov 16, 2026
- Relocation assistance
- Not verified
- Childcare support
- Offered From the posting source checked Nov 16, 2026
- Learning budget
- Not verified
- Verification
- Not verified checked Nov 20, 2026
- Salary
- $193K-$262K From the posting source checked Nov 16, 2026
- 401(k) match
- Reported from DOL Form 5500 industry filing (not employer-specific)
Was this benefit information wrong? Tell us.
Role
- Seniority
- Senior From the posting source checked Nov 16, 2026
Schedule
- Shift type
- Not verified
- Weekend work
- Not verified
Company
- Equity
- Offered Verified - SEC 10-K source checked Nov 16, 2026
Application
- Cover letter
- Not verified
- Assessment
- Not verified
- Deadline
- Not stated
Where they hire
State eligibility is not yet verified.
About this role
Sr. Software Development Engineer – AI/ML Networking Disaggregated Inference, Annapurna Labs, Annapurna Labs Cupertino, California, USA Every token a large language model generates depends on data reaching the right accelerator at the right moment. As AI models outgrow any single chip, the network between accelerators becomes the bottleneck that decides how fast - and how affordably - the world's largest models can serve real users. That network layer is what our team builds. We're looking for an engineer to work at the frontier of disaggregated inference: splitting LLM serving into separate prefill and decode pools and moving the model's KV cache between them at the absolute limit of what the hardware allows. Get it right and users get answers in milliseconds; get it wrong and the fastest accelerators in the world sit idle waiting on data. You'll own pieces of the high-speed transfer path that make the difference, and you'll measure your success in how close you run to the theoretical peak of the machine. In this role you will: Build and optimize the low-level data-movement software across accelerators, servers, and heterogeneous memory - over AWS's highest-performance network fabric. Push performance to the hardware roofline: profile, find the real bottleneck, and close the gap between "it works" and "it runs as fast as physics permits." Work across the stack - from kernel and network transport up to the inference frameworks - and partner with teams building the chips, runtime, and models. Deliver features that run on our largest clusters, for
Read the full description at www.amazon.jobs. FewerJobs shows a preview and links to the original posting.
Apply link verified; last checked Nov 20, 2026.
What verified means
Verified means a displayed claim has field-level provenance to a source FewerJobs pulled: a government or employer source, or the original job posting. Posting-sourced facts are employer-stated and are labeled separately from government records.
Related jobs
Software Development Engineer – AI/ML Networking Disaggregated Inference, Annapurna Labs , Elastic Collectives
Amazon - Cupertino, California, USA
Sr. AI Inference Platform Engineer
Apple - Seattle, United States of America
Sr. Engineer, AI Platform
Abbott Laboratories - United States - Wisconsin - Madison
Sr. Software Engineer, Training/Inference (C++)
xAI - Palo Alto, CA