Spark jobs

Staff ML Engineer, Fine Tuning - Slack 3 Locations posted: Posted 14 Days Ago

Senior Staff Machine Learning Engineer, (Machine Learning) Remote Canada

Data Architect Warminster, Wiltshire posted: Posted 2 Days Ago

Senior Data Engineer (Databricks) 2 Locations posted: Posted 9 Days Ago

Data Engineer India - Hyderabad posted: Posted Today

Engineering Expert - AI & Data Products, Runtime Platform Country India | Internal Posting Location Bangalore We help the world run better At SAP, we enable you to bring out your best. Our company culture is focused on collaboration and a shared passion to help the world run better. How? We focus every day on building the foundation for tomorrow and creating a workplace that embraces differences, values flexibility, and is aligned to our purpose-driven and future-focused work. We offer a highly collaborative, caring team environment with a strong focus on learning and development, recognition for your individual contributions, and a variety of benefit options for you to choose from. What you'll build As an Engineering Expert, you will be instrumental in the expansion of our Foundation Services team in Bangalore. Your profound expertise in distributed data processing, Spark optimization, data transformation pipelines, data engineering architecture principles, AI Agent development, and LLM-driven engineering will drive scalable data solutions and intelligent automation, reinforcing SAP Data & Analytics' pivotal role in SAP's Data & AI strategy. This is a product expert role requiring deep hands-on expertise across the modern cloud-native stack. Lead the design and execution of optimized data processing solutions at scale. Define and enforce data engineering architecture principles including data lineage, schema evolution, idempotency, fault tolerance, data partitioning strategies, and separation of concerns across processing layers. Spearhead Spark optimization strategies using PySpark

Senior Staff Software Engineer, Backend (Data and Storage Services) Remote US

Senior Data Scientist & Machine Learning Researcher 2 Locations posted: Posted 30+ Days Ago

Computer Vision AI/ML Engineer Huntsville, AL posted: Posted 30+ Days Ago

Media Data Engineer (Databricks) WARSAW PLANT & GO posted: Posted 17 Days Ago

Big Data Engineer, Senior Milwaukee, Wisconsin posted: Posted 11 Days Ago

Software Dev Engineer, Cradle (Accuro) Detroit, Michigan, USA Build exabyte-scale data processing engines with leading-edge technology. If you want to work with Spark, Java, Scala, Python, Flink, Beam, and AWS services (EMR, EKS, Kinesis, DynamoDB, SQS) - processing and transforming data across Amazon's data lakes - this is the role for you. About the Team The BDT team builds the platform that connects millions of businesses to hundreds of millions of customers across Amazon's global marketplaces. We enable native SQL, machine learning, and functional transformations using Apache Spark - executing batch, ML, and streaming workloads over schema'd data in S3, and writing curated datasets to front-end caches (DynamoDB, Redis, ElasticSearch). Our platform lets customers transition seamlessly between Streaming, Batch, Cache, and Analytics to meet demand at Amazon scale. We are a team that relentlessly innovates, keeps customers at the center, and delivers results with velocity while continuously raising the bar. What We're Looking For • Background in distributed systems development • Solid technical ability and passion for building at scale • Good communication skills and a bias for action Motivation to deliver results in a fast-paced, high-impact environment Key job responsibilities • Design & Delivery - Lead the design and delivery of major features and re-architecture of significant technology components • Quality & Operational Excellence - Ensure software maintains a high bar for quality, security, architecture, and operational excellence • Problem Solving - Identify root causes of widespread issues - including those limiting innovation and delivery velocity - across systems

Data Scientist NY - New York, 1407 Broadway Floor 12 posted: Posted Today

Big Data Platform & Distributed Systems (Mid/Senior/Lead/Principal) Washington - Bellevue About the Teams: By applying to this role, you will be considered for two mission-critical teams at the heart of our data strategy. Both teams operate at extreme scale, leveraging Public Cloud (AWS/GCP) and Kubernetes to power the next generation of data-driven intelligence. Pillar 1: Data Engineering & Observability Build and own large-scale data pipelines and observability systems that power metrics, logging, and real-time insights across services. This role focuses on designing reliable telemetry pipelines, improving monitoring and alerting, and ensuring data quality and system visibility at scale. Ideal candidates have strong distributed systems fundamentals, backend development experience (Java or similar), and experience operating high-throughput data or monitoring platforms in cloud or hybrid environments. Build and ship high-quality, production-grade software using modern engineering practices, with AI as a core part of your development workflow by pushing the boundaries of AI development tools to deliver secure, optimized, and high-quality code. Design and orchestrate complex systems where AI agents integrate seamlessly into human workflows, driving efficiency and innovation at scale. Pillar 2: Data Cloud. Big Data Compute platform team. Owns the compute infrastructure that powers large-scale Spark workloads. The team focuses on optimizing core Spark performance, solving distributed systems challenges, and building scalable AI infrastructure, including exploring efficient ways to run smaller language models. Contribute to building and maintaining the shared system context, an explicit repository of system designs, constraints, and standards that enables AI to operate accurately and reliably. Critically evaluate

Staff Engineer - Electrical Systems, EMI and Lightning Lynn posted: Posted Yesterday

Powerhouse Engineer - 2nd shift United States - Ohio - Columbus posted: Posted 4 Days Ago

Social Marketing Manager, YouTube Marketing SPARK (English, Korean) Seoul, South Korea YouTube's mission is to give everyone a voice and show them the world. We believe that everyone deserves to have a voice, and that the world is a better place when we hear, share and build community through our stories. The YouTube Brand and Social Marketing brings this mission to life. You will build brand love, keep YouTube at the center of culture and partner with teams on social and influencer marketing that have the user at its center. As Social Marketing Manager for YouTube, you will be shaping and driving the social presence of one of the most culturally significant brands of our times in a dynamic and growing region. You should have experience driving social and influencer marketing strategies and programs that have created an impact on marketing and business goals. You have a knack for spotting trends before others do, and you are as obsessed about what excites communities as you are about what our products can do for them. You are a constant advocate for our users and creators - your ideas are always deeply rooted in what they care about, what they crave, and what delights them. And you have experience executing and optimizing programs, from influencer marketing, to always-on social editorial, to escalating brand risks and responding swiftly and thoughtfully to topics that are important to our users and creators. You have analytical, strategic and creative capabilities, using data for insights that will

Staff Software Engineer, Batch Processing Platform Seattle, WA, US; Remote, US About Pinterest: Millions of people around the world come to our platform to find creative ideas, dream about new possibilities and plan for memories that will last a lifetime. At Pinterest, we're on a mission to bring everyone the inspiration to create a life they love, and that starts with the people behind the product. Discover a career where you ignite innovation for millions, transform passion into growth opportunities, celebrate each other's unique experiences and embrace the flexibility to do your best work. Creating a career you love? It's Possible. At Pinterest, AI isn't just a feature, it's a powerful partner that augments our creativity and amplifies our impact, and we're looking for candidates who are excited to be a part of that. To get a complete picture of your experience and abilities, we'll explore your foundational skills and how you collaborate with AI. Through our interview process, what matters most is that you can always explain your approach, showing us not just what you know, but how you think. You can read more about our AI interview philosophy and how we use AI in our recruiting process here . We're looking for a Staff Software Engineer to help us build a high-performance, efficient, and easy to use batch processing platform and infrastructure at Pinterest. You'll be working on some of the most exciting big data open source technologies such as Spark, Presto/Trino, Iceberg, etc. at the scale of

Data Engineer Warminster, Wiltshire posted: Posted 30+ Days Ago

Cable Installation Technician, DCC Communities Sparks, Nevada, USA AWS Infrastructure Services owns the design, planning, delivery, and operation of all AWS global infrastructure. In other words, we're the people who keep the cloud running. We support all AWS data centers and all of the servers, storage, networking, power, and cooling equipment that ensure our customers have continual access to the innovation they rely on. We work on the most challenging problems, with thousands of variables impacting the supply chain - and we're looking for talented people who want to help. You'll join a diverse team of software, hardware, and network engineers, supply chain specialists, security experts, operations managers, and other vital roles. You'll collaborate with people across AWS to help us deliver the highest standards for safety and security while providing seemingly infinite capacity at the lowest possible cost for our customers. And you'll experience an inclusive culture that welcomes bold ideas and empowers you to own them to completion. Amazon is looking for an Infrastructure Delivery Installation Technician to join our growing team within infrastructure operations. You will work with minimum supervision in a dynamic environment to drive the stability and sustainability of our next-generation networks and assist in the development of innovative ways to automate and scale our network. To be successful in this role you should possess technical aptitude, written and oral communications skills and the ability to deal effectively with people in a wide variety of situations. In addition, you must possess strong analytical skills with

SAP S/4 HANA Data Lead Analyst (Remote) US-MA-REMOTE posted: Posted Yesterday

Software Development Engineer, AWS Athena Bellevue, Washington, USA Want to help make history in the world of big data and make it easy for customers to efficiently query and analyze petabytes and more of data? Come join the Amazon Athena team in AWS! We are developing innovative new capabilities from the ground up that will transform the way customers analyze their data while also giving them high performance, low costs, and exceptional ease-of-use. We are a small, entrepreneurial team that operates independently, but with access to some of the top engineers in our field across AWS. In this role, you will have the opportunity to collaborate with senior and principal engineers who are domain experts at AWS. You'll develop deep expertise in areas like distributed systems, query engines, open source applications, performance optimization, and operational excellence. You'll also have the opportunity to work with and contribute to the leading open source big data and analytic frameworks like Hive, Spark, and Presto. You will enjoy competitive compensation and your contributions in this role will impact customers and businesses around the world. Key job responsibilities You'll be responsible for designing, developing, testing and deploying some of our most strategic technical projects. You'll deliver large-scale, high performance production systems, design new software systems of distributed technology, and have a significant bottom-line impact on our business results. You will draw from your deep and broad technical expertise to mentor engineers, complete hands-on technical work and provide leadership on complex technical issues. Internal job description

Machine Learning Engineer, Firefly Services 2 Locations posted: Posted 30+ Days Ago

Data Architect Bangalore posted: Posted 30+ Days Ago