Software Development Engineer – AI/ML Networking Disaggregated Inference, Annapurna Labs , Elastic Collectives
Amazon - Cupertino, California, USA
Posted Oct 1, 2026
Benefits
- Parental leave
- Not verified
- Non-birth-parent leave
- Not verified
- Family-building benefits
- Mental health support
- Offered From the posting source checked Nov 16, 2026
- Relocation assistance
- Not verified
- Childcare support
- Offered From the posting source checked Nov 16, 2026
- Learning budget
- Not verified
- Verification
- Not verified checked Nov 20, 2026
- Salary
- $165K-$224K From the posting source checked Nov 16, 2026
- 401(k) match
- Reported from DOL Form 5500 industry filing (not employer-specific)
Was this benefit information wrong? Tell us.
Role
- Seniority
- Mid From the posting source checked Nov 16, 2026
Schedule
- Shift type
- Not verified
- Weekend work
- Not verified
Company
- Equity
- Offered Verified - SEC 10-K source checked Nov 16, 2026
Application
- Cover letter
- Not verified
- Assessment
- Not verified
- Deadline
- Not stated
Where they hire
State eligibility is not yet verified.
About this role
Software Development Engineer – AI/ML Networking Disaggregated Inference, Annapurna Labs , Elastic Collectives Cupertino, California, USA Every token a large language model generates depends on data reaching the right accelerator at the right moment. As AI models outgrow any single chip, the network between accelerators becomes the bottleneck that decides how fast - and how affordably - the world's largest models can serve real users. That network layer is what our team builds. We're looking for an engineer to work at the frontier of disaggregated inference: splitting LLM serving into separate prefill and decode pools and moving the model's KV cache between them at the limit of what the hardware allows. Get it right and users get answers in milliseconds; get it wrong and the fastest accelerators in the world sit idle waiting on data. You'll build components of the high-speed transfer path that make that difference, and you'll learn to measure success in how close we run to the theoretical peak of the machine. In this role you will: Build and optimize the low-level data-movement software that transfers KV cache and activations across accelerators, servers, and heterogeneous memory - over AWS's highest-performance network fabric. Profile real workloads, find the true bottleneck, and close the gap between "it works" and "it runs fast" - pushing components toward the hardware's limit. Work across the stack - from network transport up to the inference frameworks - learning from the teams building the chips, runtime, and models. Deliver features that ship to our
Read the full description at www.amazon.jobs. FewerJobs shows a preview and links to the original posting.
Apply link verified; last checked Nov 20, 2026.
What verified means
Verified means a displayed claim has field-level provenance to a source FewerJobs pulled: a government or employer source, or the original job posting. Posting-sourced facts are employer-stated and are labeled separately from government records.
Related jobs
Sr. Software Development Engineer – AI/ML Networking Disaggregated Inference, Annapurna Labs, Annapurna Labs
Amazon - Cupertino, California, USA
Forward Deployed Engineer, Inference
Cohere - United States, San Francisco, California
Software Engineer - Inference Product
Baseten - San Francisco, California, United States, New York
Software Engineer, ML Accelerators, DeepMind
DeepMind - Mountain View, CA, USA; +1 more