Spark jobs

Principal Software Engineer, Machine Learning Simulations United States | Remote About Upstart At Upstart, we're united by a mission that matters: to radically reduce the cost and complexity of borrowing for all Americans. Every day, we bring creativity, experimentation, and advanced AI to reshape access to credit, helping millions move forward financially with clarity and confidence. As the leading AI lending marketplace, we partner with banks and credit unions to expand access to affordable credit through technology that's both radically intelligent and deeply human. Our platform runs over one million predictions per borrower using more than 1,800 signals, powering smarter, fairer decisions for millions of customers. But the numbers only hint at the impact. Every idea, every voice, and every contribution moves us closer to a world where credit never stands between people and their financial progress. We're proudly digital-first, giving most Upstarters the flexibility to do their best work from wherever they thrive, alongside teammates across 80+ cities in the US and Canada. Digital-first doesn't mean distant. We're intentional about in-person connection through team onsites, planning sessions, and moments that spark creativity and trust. And whether you choose to work primarily from home or collaborate in-person from one of our offices in Columbus, Austin, the Bay Area, or New York City (opening Summer 2026), you'll have the support to work in the way that works best for you. If you're energized by tackling meaningful problems, excited to innovate with purpose, and motivated by work that truly matters, we'd love

Director Installations Innovations Remote - Little Rock, Arkansas posted: Posted 9 Days Ago

Senior Security Engineer Sydney, New South Wales, Australia Company Overview At Zuora, we do Modern Business . We're helping people subscribe to new ways of doing business that are better for people, companies and ultimately the planet. It's an approach resulting from the shift to the Subscription Economy that puts customers first by building recurring relationships instead of one-time product sales and focuses on sustainable growth. Through our leading expertise and multi-product suite, we are transforming all industries and working with the world's most innovative companies to monetize new business models, nurture subscriber relationships and optimize their digital experiences. The Team & Role Zuora's Application Security & Security Engineering team partners closely with engineering to embed security into the software development lifecycle through scalable tools, processes, and culture. As a Senior Security Engineer, you'll work hands-on with developers and architects to drive secure design, build security automation, and support critical projects across our cloud-native platform. This role is part of our growing presence in Sydney and offers the opportunity to shape and scale security practices globally. This is a hybrid position, so you'll work both remotely and in the office. Our Tech Stack: Java, Spring, Rest API, Microservices, Kafka, Spark, NodeJS, AWS, Kubernetes, Terraform, AngularJS What you'll do - Collaborate with teams across a global organization to support the adoption and implementation of secure software development practices and tooling. - Contribute hands-on to critical engineering and tooling projects, working closely with technical leads and product owners to ensure security is

Sr. Staff Engineer 2 Locations posted: Posted 23 Days Ago

Enterprise Hunter Account Executive - FSI West Mumbai, India SLSQ227R7 As an Enterprise Account Executive at Databricks, you are an experienced enterprise sales professional specializing in selling to Enterprise accounts (FSI). You know how to sell innovation and change through customer vision expansion and can guide deals forward to compress decision cycles. You love understanding a product in depth and are passionate about communicating its value to Customers and System Integrators. Always seeking new opportunities, you will close new accounts while maintaining existing ones. Along with the chance to close exciting deals and solve meaningful problems. The impact you will have: - Present a territory plan within the first 90 days - Meet with CIOs, IT executives, LOB executives, Program Managers, and other important partners - Close both new accounts and existing accounts - Identify and close quick, small wins while managing longer, complex sales cycles - Exceed activity, pipeline, and revenue targets - Track all customer details, including use case, purchase time frames, next steps, and forecasting in Salesforce - Use a solution-based approach to selling and creating value for customers - Promote Databricks' enterprise cloud data platform powered by Apache Spark - Ensure 100% satisfaction among all customers - Prioritize opportunities and apply appropriate resources What we look for: - Field sales experience within big data and Cloud - Prior customer relationships with CIOs, program managers, and essential decision makers - Simply articulate intricate cloud technologies - 7+ years of Enterprise Sales experience exceeding quotas, covering relevant accounts

Sr. Applied Scientist, Amazon Publisher Monetization - Ads Seattle, Washington, USA Advertising is a complex, multi-sided market with many technologies at play within the industry. The industry is rapidly growing and evolving as viewers are shifting from traditional TV viewing to streaming video and publishers are increasingly adding video content to their online experiences. Amazon's video advertising is a rising competitor in this industry. Amazon's service has differentiated assets in our customer & audience insights, exclusive video content, and associated inventory that position us well as an end-to-end service for advertisers and agencies. We are innovating at the intersection of advertising, e-commerce, and entertainment. Amazon Publisher Monetization (APM) is looking for a a passionate and experienced scientist who is adept at a variety of skills; especially in generative AI, computer vision, and large language models that will accelerate our plans to maximize yield via AI-driven contextual targeting, Ads syndication and more. The ideal candidate will be an inventor at heart, they will provide science expertise, rapidly prototype, iterate, and launch, foster the spirit of collaboration and innovation within our larger sister teams and their scientists, and execute against a compelling product roadmap designed to bring AI-led science innovation to solve one of the most challenging problems in advertising. Key job responsibilities This role is focused on shaping our approach to the solving the trifecta of advertising - serving the right ad to the right viewer at the right moment - delivering engaging ads for viewers, improved performance for advertisers, and

Connect With Us Connect With Us

Software Development Engineer, AWS EMR East Palo Alto, California, USA Do you enjoy working on large-scale distributed systems? Do you want to influence how the world interacts with and analyzes large data sets across a number of industries and use cases? The AWS EMR Control Plane team is looking for smart engineers to work on scaling a heavily used AWS Service. If you want to work with a team solving some of the most interesting challenges in distributed systems, building solutions for large scale problems, please talk to us! Amazon EMR is the industry leading cloud-native big data platform for processing vast amounts of data quickly and cost-effectively at scale. Using open source tools such as Apache Spark, Apache Hive, Apache HBase, Apache Flink, and Presto, coupled with the dynamic scalability of Amazon EC2 and scalable storage of Amazon S3, EMR gives analytical teams the engines and elasticity to run Petabyte-scale analysis for a fraction of the cost of traditional on-premises clusters. We are developing innovative new features including our next-generation cluster and work management system, improvements for real-time processing of big data, and ways to enable customers to more easily interact with their data. In this role, you will have the opportunity to collaborate with and learn from senior and principal engineers who are domain experts at AWS. You will be part of a collaborative and growing team working on complex problems for a successful and long-running product. You will have an opportunity to learn and influence how our

Data Scientist San Francisco, CA Location: San Francisco, CA (Hybrid) What is Verse? Energy markets are more volatile than ever. Rapid electrification and the rise of AI are driving unprecedented demand for power, while energy costs continue to rise across the globe. For the world's largest energy buyers, managing energy has never been more complex or more critical. Verse helps these organizations manage complex power portfolios with confidence by unifying energy data, planning, forecasting, and operations in one tool. Our Energy Cost Intelligence platform, Aria, brings together energy, finance, and operations teams with real-time, finance-ready intelligence-replacing spreadsheets and consultants with precision across the entire energy lifecycle. Built by an expert team of energy buyers, data scientists, and engineers, Verse enables faster, smarter energy decisions that reduce risk and lower energy costs. The Role Verse is seeking a Data Scientist to join our Data Science Team. In this role, you will lead the development and deployment of advanced data-driven solutions across a range of applications, including electricity markets, renewable procurement, and energy risk management. You will shape the machine learning and data modeling foundations that Verse's software is built on. For example, you might spend a cycle deploying electricity market price forecasting pipelines for new regions, developing solar production anomaly detection models, or creating scalable tools for benchmarking energy project financial performance. Key Responsibilities Lead End-to-End Data Science Projects: Own and drive projects from problem definition through scoping, modeling, validation, and production deployment. Translate business problems into scalable, high-impact modeling solutions.

Staff Machine Learning Engineer Remote - US

Technical Writer Dublin The Drivers Documentation team is a hands-on and enthusiastic group of engineers who are passionate about teaching developers how to work with data and build applications. We are hiring a “Technical Writer, Drivers” to help us create world-class documentation for MongoDB's developer-focused products -- in particular, MongoDB Drivers and Connectors. Our team works closely with MongoDB's engineering and product organizations to deliver outstanding developer documentation for everyone, regardless of where they are in their software engineering career. Every month, hundreds of thousands of developers turn to our content to help them build microservices, backend and line of business applications, and other projects that are used by millions of people worldwide. We are looking to speak to candidates who are based in Dublin for our hybrid working model. The team is responsible for the following documentation: - MongoDB Drivers , including Node , Java , Go , .NET/C# , and more! - MongoDB Spark Connector - MongoDB Queryable Encryption and Client-Side Field Level Encryption Candidate Profile: Candidates must have 3+ years of combined experience in one or more of the following roles: technical writer, developer advocate, and/or software engineer. Beyond industry experience, we are looking for someone who is: - Experienced with developing and shipping modern line-of-business applications OR has extensive experience contributing to complex documentation sites, preferably in a docs-as-code environment - Experienced with using great developer documentation and educational materials. You are excited to empower other developers by creating best-in-class documentation - Experienced with and enjoys

Senior Data Engineer, AI and Systems Engineering Remote - Poland Role Description As a Senior Data Engineer on the CMDB and Asset Intelligence platform, you will help build the unified data foundation that powers asset visibility, cost optimization, and security insights across the company. You will design scalable pipelines and data models that bring together sources like ServiceNow, Okta, Oracle, and Jamf into a centralized lakehouse architecture, turning messy, multi-system data into trusted, decision-ready signals. This role is a chance to raise the bar on data quality and governance while building systems that teams actually rely on day to day. You will partner closely with IT, Security, and Finance to define what “good” looks like, deliver high-impact solutions, and shape the long-term direction of the platform. Our Engineering Career Framework is viewable by anyone outside the company and describes what's expected for our engineers at each of our career levels. Check out our blog post on this topic and more here . Responsibilities Design and build scalable data pipelines using Databricks and Spark to ingest, transform, and unify data from multiple enterprise systems Develop and maintain medallion architecture (Bronze, Silver, Gold) data models to create reliable and performant “Golden Record” datasets Implement data normalization, mapping, and entity resolution techniques (e.g., fuzzy matching, XREF tables) to unify asset data across disparate systems Build data workflows to detect and surface Shadow IT across financial, identity, endpoint, and network signals and integrate results into CMDB systems Partner with IT, Security, Finance, Procurement, and

Cloud Services Engineer USA-Atlanta, GA posted: Posted 30 Days Ago

Senior Technical Lead -Power & BatteryLife 2 Locations posted: Posted 30+ Days Ago

Automotive Worker Bath, New York Summary THE CLOSING DATE FOR THIS ANNOUNCEMENT HAS BEEN EXTENDED. IF YOU HAD PREVIOUSLY APPLIED, YOU DO NOT NEED TO REAPPLY BUT YOU CAN MAKE CHANGES TO YOUR APPLICATION. This position is located within the Engineering Department under the direct supervision of the Motor Vehicle Operator Supervisor at the Finger Lakes Health Care System in Bath, NY. The primary duties of the position are to be an Automotive Worker with secondary duties as a Motor Vehicle Operator. Duties Major duties and responsibilities include but are not limited to: Cleaning, removing, adjusting, and replacing defective automotive parts and components. Cleaning, removing, adjusting, and replacing filters, screens, battery cables, and engine components. Replacing sensors, water pumps, spark plugs brake shoes, mufflers, gaskets, hoses, and complete assemblies. Using test equipment and tools such as wrenches, sockets, brake tools, pliers, screwdrivers, etc. Servicing and checking air conditioning systems by purging/recharging systems. Servicing and checking radiators by flushing and replacing anti-freeze. Replacing sensors, water pumps, spark plugs, brake shoes, mufflers, gaskets, hoses, and complete assemblies. Cleaning and replacing filters, screens, battery cables, and engine components. Operating and using basic tools such as wrenches, sockets, brake tools, pliers, screwdrivers, etc., and test equipment, such as test lamps, timing lights, multimeters, tachometers, dwell meters, vehicle on-board computers in conjunction with published references and digital manuals to help diagnose problems, and electronic analyzers to test automotive computer control systems and emission systems. Operating platform, van, stake and dump trucks, passenger shuttle and cargo

Data Center Electrical Engineer Menlo Park, CA Meta's mission is to give people the power to build communities and bring the world closer together. Through our family of apps and services, we're building a different kind of company that connects billions of people around the world, gives them ways to share what matters most to them. Our global teams are constantly iterating, solving problems, and working together to empower people around the world to build communities and connect in meaningful ways. Together, we can help people build stronger communities - we're just getting started. Meta is seeking a Data Center Electrical Engineer experienced in the design and operations of critical facilities to become part of our design engineering team. Our data centers are the foundation upon which our software operates with efficient ease. Building and operating data centers the "right" way from the day they go live is synonymous with ensuring capacity, availability, and capital conservatism. The Meta Data Center Engineering team focuses on improving reliability, safety, and efficiency for the data center industry and improve reliability, safety, and efficiency through innovative solutions, enabling infrastructure teams to excel in achieving their goals. Responsibilities: - Develop concepts for new data center infrastructure solutions focused on electrical power system technologies to reduce cost, improve reliability, efficiency, and speed to market - Lead all phases of design and development life cycle of electrical systems comprising concept design, prototyping and deployments - Develop collaborations with data center technology vendors, and consultants to remain current

Senior Data Engineer Vancouver, BC The Data Engineering team's mission is to ensure high-quality data to enable data-informed decision-making across Asana. You will build data artifacts that are leveraged by Product and Business Data Science teams to optimize our user adoption, growth, and experience. In this role, you will partner with the Infrastructure team to build a self-service analytics platform for the company. This role is based in our Vancouver office with an office-centric hybrid schedule. The standard in-office days are Monday, Tuesday, and Thursday; most Asanas have the option to work from home on Wednesdays. Working from home on Fridays depends on the type of work you do and the teams with which you partner. If you're interviewing for this role, your recruiter will share more about the in-office requirements. What you'll achieve - Design, implement, and scale end-to-end data products that support growing data processing and analytical needs - Transform raw data into actionable insights to drive product strategy and power in-depth analyses and reporting - Leverage AI to build self-serve tools and accelerate Data/GTM workflows - Partner with data scientists, domain experts, and engineering teams to develop a roadmap that aligns with our business goals - Implement systems that guarantee data quality, governance, and availability About you - 5+ years of experience in Data Engineering or Software Engineering - Experience in data modeling and building scalable data pipelines involving complex transformations - Proficiency in data processing and storage technologies like Databricks, AWS/S3, Python/Scala/Java, SQL, Spark, and Airflow

Associate Data Scientist (Data Engineering) India - Hyderabad posted: Posted Yesterday

Senior Software Engineer, Capital Insights United States | Remote About Upstart At Upstart, we're united by a mission that matters: to radically reduce the cost and complexity of borrowing for all Americans. Every day, we bring creativity, experimentation, and advanced AI to reshape access to credit, helping millions move forward financially with clarity and confidence. As the leading AI lending marketplace, we partner with banks and credit unions to expand access to affordable credit through technology that's both radically intelligent and deeply human. Our platform runs over one million predictions per borrower using more than 1,800 signals, powering smarter, fairer decisions for millions of customers. But the numbers only hint at the impact. Every idea, every voice, and every contribution moves us closer to a world where credit never stands between people and their financial progress. We're proudly digital-first, giving most Upstarters the flexibility to do their best work from wherever they thrive, alongside teammates across 80+ cities in the US and Canada. Digital-first doesn't mean distant. We're intentional about in-person connection through team onsites, planning sessions, and moments that spark creativity and trust. And whether you choose to work primarily from home or collaborate in-person from one of our offices in Columbus, Austin, the Bay Area, or New York City (opening Summer 2026), you'll have the support to work in the way that works best for you. If you're energized by tackling meaningful problems, excited to innovate with purpose, and motivated by work that truly matters, we'd love to

Sr. / Staff ML Engineer, FM Training Integration - ML Compute Santa Clara, United States of America We are looking for a ML Engineer to join our ML Compute team to help improve the efficiency, scalability, and reliability of model training and inference workloads in the cloud. In this role, you will lead the integration of large-scale ML workloads with cloud infrastructure, working cross-functionally with ML engineers, infrastructure engineers, and researchers to optimize performance, improve system efficiency, and drive high utilization of accelerator resources. We are a group of engineers to support training foundation models at Apple! We build infrastructure to support training foundation models with general capabilities such as understanding and generation of text, images, speech, videos, and other modalities and apply these models to Apple products. We are looking for engineers who are passionate about building systems that push the frontier of deep learning in terms of scaling, efficiency, and flexibility and delight millions of users in Apple products. Own the integration of large-scale model training workloads with accelerator-based cloud infrastructure, ensuring scalable and reliable execution. Drive performance optimization across the ML stack, including data pipelines, model execution, and distributed systems, to improve throughput, latency, and hardware utilization. Design and run benchmarks to evaluate model performance and infrastructure configurations, using results to guide optimization efforts. Build and improve tooling for observability, profiling, and debugging to increase visibility and reliability of ML workloads. Collaborate cross-functionally with ML engineers, infrastructure engineers, and researchers to improve system efficiency and scalability. Establish

Analytics Engineer — Data Warehouse San Francisco About the Role Together AI is building high-performance inference compute and the software platform around it. We're looking for an early-career Analytics Engineer with strong fundamentals and high growth potential to grow into a technical lead over time. You'll contribute to designing and operating our data warehouse, ETL pipelines and orchestration, work on core data models and metrics, and help raise the bar on data quality and governance across the org - with mentorship and support from experienced engineers. Requirements - 0-4 years of professional experience (or strong internships/projects) working with data warehouses, pipelines, or analytics engineering. - Solid SQL fundamentals - you're comfortable writing queries and have some exposure to window functions or dimensional modeling concepts. - Some hands-on experience with dbt or Airflow, or strong eagerness to learn - coursework and personal projects count. - Basic Python for scripting and data tooling; any exposure to Spark (PySpark/SQL) is a plus. - Familiarity with data modeling concepts like SCD2 or star schemas - even if only from coursework. - Good communication skills: you can ask clarifying questions, explain your reasoning, and work with stakeholders to understand their needs. - High standards for data quality, reliability, and maintainability - you care about getting things right. Responsibilities - Contribute to building and maintaining a medallion/curated data warehouse stack (bronze/silver/gold) for product, usage, billing, and operational data. - Build and maintain Airflow orchestrated pipelines and dbt transformation projects (modular, tested, documented). - Help design analytics-ready

Staff ML Engineer - ML Infrastructure Remote - US Who we are Samsara (NYSE: IOT) is the pioneer of the Connected Operations™ Cloud, which is a platform that enables organizations that depend on physical operations to harness Internet of Things (IoT) data to develop actionable insights and improve their operations. At Samsara, we are helping improve the safety, efficiency and sustainability of the physical operations that power our global economy. Representing more than 40% of global GDP, these industries are the infrastructure of our planet, including agriculture, construction, field services, transportation, and manufacturing - and we are excited to help digitally transform their operations at scale. Working at Samsara means you'll help define the future of physical operations and be on a team that's shaping an exciting array of product solutions, including Video-Based Safety, Vehicle Telematics, Apps and Driver Workflows, and Equipment Monitoring. As part of a recently public company, you'll have the autonomy and support to make an impact as we build for the long term. About the role: Samsara is the industry leader in AI for physical operations. We're hiring a Staff / Senior Staff Machine Learning Infrastructure Engineer to lead the design and evolution of our end-to-end ML platform powering Safety AI and adjacent product areas. This role combines deep platform ownership with direct product impact-enabling teams to build, deploy, and scale ML systems that improve real-world safety outcomes. This is a remote position open to candidates based in the United States. You should apply if: You

Senior Data Platform Engineer Bangalore | OnSite ABOUT ASPORA People on the move deserve a bank that moves with them. Since 2022, Aspora has been building a borderless financial operating system that makes money as mobile and transparent as its users. Backed by influential venture capitalists like Sequoia Capital, Greylock Partners, Hummingbird Ventures, Y Combinator & Global Founders Capital. We're a team of 150+ across India, the UK, the UAE, EU and the US, working with extreme ownership, radical candour, and an obsession with customer impact. We celebrate builders who question assumptions, ship fast, and turn regulatory complexity into elegant solutions. If you're driven to redefine what global banking can be, we'd love to build the future with you. About the Role We're building the data infrastructure that powers decisions across every part of our business - from real-time analytics to large-scale batch computation. As a Senior Data Platform Engineer, you'll own the systems that process billions of events, move data reliably, and make insights fast to produce. You'll work closely with analytics, ML, and product engineering teams - setting the bar for reliability, performance, and data quality across the platform. What You'll Do 1. Big Data Platform & Infrastructure - Design, build, and operate large-scale data processing infrastructure using Spark on Databricks - ensuring reliability, performance, and cost efficiency at scale. - Architect and maintain lakehouse solutions (Delta Lake, Iceberg) including partitioning strategies, Z-ordering, and compaction jobs. - Own cluster management, autoscaling policies, and resource governance across Databricks workspaces.

Staff Computer Vision AI/ML Engineer 2 Locations posted: Posted 2 Days Ago