Senior Machine Learning Engineer (vLLM)
Neural Magic - Boston
Posted Nov 3, 2026
Benefits
- Parental leave
- Not verified
- Non-birth-parent leave
- Not verified
- Family-building benefits
-
- Fertility benefits: Not verified
- Adoption assistance: Not verified
- Surrogacy assistance: Not verified
- Mental health support
- Not verified
- Relocation assistance
- Not verified
- Childcare support
- Not verified
- Learning budget
- Not verified
- Verification
- Not verified
- Salary
- $186K-$287K From the posting source checked Nov 17, 2026
- 401(k) match
- Not verified
Was this benefit information wrong? Tell us.
Market context
- U.S. role benchmark (BLS OEWS)
- $116,543 U.S. median for this role
- Projected growth (BLS Employment Projections)
- +9.8% - Much faster than average
103% above the BLS role benchmark for software engineering aggregate.
Matched to SOC 15-1252 - Software Engineering aggregate by role bucket.
Source:U.S. Bureau of Labor Statistics, OEWS, May 2024andEmployment Projections, 2024-2034.
Role
Schedule
- Shift type
- Not verified
- Weekend work
- Not verified
Company
- Equity
- Offered From the posting source checked Nov 17, 2026
Application
- Cover letter
- Not verified
- Assessment
- Not verified
- Deadline
- Not stated
Where they hire
State eligibility is not yet verified.
About this role
Senior Machine Learning Engineer (vLLM) Boston *Telecommuting permitted: work may be performed within normal commuting distance from the Red Hat, LLC office in Boston, MA. Lead the design and implementation of high-performance Python, C++, CUDA, and Triton code for vLLM-based inference systems, GPU kernels, and numerical computing components in production-scale LLM serving environments. What You Will Do: Architect, develop, and optimize advanced LLM inference algorithms, including quantization, sparsification, speculative decoding, paged attention, KV cache optimization, batching, scheduling, and memory management, to significantly improve latency, throughput, scalability, and cost efficiency. Drive deep performance profiling, benchmarking, and modeling on NVIDIA GPUs, using tools such as Nsight Systems, Nsight Compute, CUPTI, PyTorch Profiler, CUDA, Triton, and CUTLASS to identify bottlenecks and deliver measurable system-level improvements. Own the end-to-end optimization of vLLM-based inference pipelines, including kernel-level tuning, runtime optimization, distributed inference, multi-GPU serving, tensor/pipeline parallelism, and large-scale deployment performance. Evaluate, prototype, and integrate state-of-the-art LLM serving techniques from cutting-edge research and open-source communities, including quantization, sparsity, speculative decoding, continuous batching, prefix caching, attention optimization, and model compression. Set engineering standards for performance-critical code, conduct rigorous peer reviews, guide testing and benchmarking strategies, and leverage AI-assisted development tools to maintain high code quality, reliability, and maintainability. Contribute to and maintain upstream open-source ML infrastructure vLLM, including reviewing pull requests, triaging issues, participating in design discussions, and supporting release readiness. Stay at the forefront of LLM systems research and GPU computing advancements, proactively identifying opportunities to improve the product through emerging techniques in serving efficiency, hardware
Read the full description at redhat.wd5.myworkdayjobs.com. FewerJobs shows a preview and links to the original posting.
Apply link verified; last checked Nov 20, 2026.
What verified means
Verified means a displayed claim has field-level provenance to a source FewerJobs pulled: a government or employer source, or the original job posting. Posting-sourced facts are employer-stated and are labeled separately from government records.
Related jobs
Senior Machine Learning Engineer
FloQast - San Jose, California
Senior Machine Learning Engineer - Applied Scientist
Microsoft - Redmond, WA, US
Senior Machine Learning Engineer
Oracle - United States, US
Senior Machine Learning Engineer
Oracle Corporation - United States