Spark jobs

Senior Staff Engineer - CFD/Thermal San Jose, California posted: Posted 30+ Days Ago

Data Engineer Santa Barbara, CA, United States Company Description: Our small data science team is changing the way healthcare decisions are made. We're taking advantage of new technologies and the latest in statistics and machine learning to rapidly analyze new kinds of healthcare data. We have one-of-a-kind access to health and behavior datasets, and we're using it to change the world for the better. Our distributed data processing and analysis pipeline is built on top of technologies like Spark, Hadoop, Elasticsearch, Python, and Scala, and we need brilliant engineers to help us scale it and adapt it to new challenges. This is the perfect job if you have a software engineering background and want to get involved in healthcare and big data engineering. We have offices in Menlo Park, CA and Santa Barbara, CA and a Surf Air membership so you can rove between the offices. This position is in Santa Barbara, CA. Job Description: Responsibilities: Own our data ETL pipelines Investigate, prototype, and validate new analytics systems Optimize our analytics systems for performance, cost, and availability Design and implement new data analytics systems using top-notch software engineering practices Work closely with our Data Scientists to deliver dashboards, insights, and data pipelines to clients Qualifications: Requirements: Background in quantitative fields (Computer Science, Engineering, Math, Physics) You're a strong analytical thinker and coder. Think ACM-ICPC, IOI, and IPSC. You're a great software engineer who writes modular, extensible, well-designed systems. You have solid experience with Python and SQL. You value learning, personal

Senior Data Center Engineer Kansas City Senior Data Center Engineer

Senior Software Engineer - Partner Integrations and Origination Platform United States | Remote About Upstart At Upstart, we're united by a mission that matters: to radically reduce the cost and complexity of borrowing for all Americans. Every day, we bring creativity, experimentation, and advanced AI to reshape access to credit, helping millions move forward financially with clarity and confidence. As the leading AI lending marketplace, we partner with banks and credit unions to expand access to affordable credit through technology that's both radically intelligent and deeply human. Our platform runs over one million predictions per borrower using more than 1,800 signals, powering smarter, fairer decisions for millions of customers. But the numbers only hint at the impact. Every idea, every voice, and every contribution moves us closer to a world where credit never stands between people and their financial progress. We're proudly digital-first, giving most Upstarters the flexibility to do their best work from wherever they thrive, alongside teammates across 80+ cities in the US and Canada. Digital-first doesn't mean distant. We're intentional about in-person connection through team onsites, planning sessions, and moments that spark creativity and trust. And whether you choose to work primarily from home or collaborate in-person from one of our offices in Columbus, Austin, the Bay Area, or New York City (opening Summer 2026), you'll have the support to work in the way that works best for you. If you're energized by tackling meaningful problems, excited to innovate with purpose, and motivated by work that truly

Senior Applied Scientist, Ad Measurements Science New York, New York, USA The Ads Measurement Science team in the Measurement, Ad Tech, and Data Science (MADS) team of Amazon Ads serves a centralized role developing solutions for a multitude of performance measurement products. We create solutions which measure the comprehensive impact of advertiser's ad spend, including sales impacts both online and offline and across timescales, and provide actionable insights that enable our advertisers to optimize their media portfolios. We also own the science solutions for AI tools that unlock new insights and automate high-effort customer workflows, such as custom query and report generation based on natural language user requests. We leverage a host of scientific technologies to accomplish this mission, including Generative AI, classical ML, Causal Inference, Natural Language Processing, and Computer Vision. As a Senior Applied Scientist on the team, you will be at the forefront of innovation, developing measurement solutions end-to-end from inception to production. You will set the technical vision and innovate on behalf of our customers. You will propose, design, analyze, and productionize models to provide novel measurement insights to our customers. You will partner with engineering to deploy these solutions into production. You will work with key stakeholders from various business teams to enable advertisers to act upon those metrics. Key job responsibilities * Lead the development of ad measurement models and solutions that address the full spectrum of an advertiser's investment, focusing on scalable and efficient methodologies. * Collaborate closely with cross-functional teams including engineering,

Senior Data Scientist Bangalore Office BLS2 posted: Posted 30+ Days Ago

Technology Lead - Databricks Data Engineering Job Description Today, the corporate landscape is dynamic and the world ahead is full of possibilities! None of the amazing things we do at Infosys would be possible without an equally amazing culture, the environment where ideas can flourish and where you are empowered to move forward as far as your ideas will take you. At Infosys, we assure that your career will never stand still, we will inspire you to build what's next and we will navigate further together. Our journey of learnability, values and trusted relationships with our clients continue to be the cornerstones of our organization and these values are upheld only because of our people. Required Skills (Must Have) - Hands-on experience with Databricks (notebooks, clusters, jobs, Delta Lake). - Strong understanding of ETL/ELT processes . - Experience with Python or SQL . - Good understanding of cloud platforms (Azure, AWS, or GCP). - Knowledge of Delta Lake features (ACID transactions, time travel, schema evolution). - Familiarity with data warehousing concepts . - Ability to work with large datasets and optimize performance. - Strong debugging and problem‑solving skills. Preferred Skills (Good to Have) - Experience with streaming systems (Auto Loader, Structured Streaming). - Knowledge of CI/CD workflows (Git, DevOps pipelines). - Experience with orchestration tools (Azure Data Factory, n8n, etc.). - Understanding of lakehouse architecture. - Exposure to ML workflows or MLOps. Qualifications - 2-6 years of experience in data engineering or analytics. - Certifications in Databricks, Azure, AWS, or

Lead Software Development Engineer - Infrastructure Seattle, WA At Amperity , we're an AI-first company helping the world's leading brands create personalized customer experiences that build loyalty and fuel growth. Our AI-powered Customer Data Cloud, built on multi-patented technology, enables more than 400 global brands , including Alaska Airlines, Wyndham Hotels & Resorts, and DICK'S Sporting Goods, to turn customer data into a competitive advantage. We unlock the full value of customer data with simplicity and speed. AI is at the core of our platform and the way we work - from powering advanced identity resolution and predictive analytics to streamlining internal workflows and decision-making. It's not just a capability; it's part of our DNA. Our team thrives on curiosity, collaboration, and transparency, fostering a culture where everyone can contribute, learn, and grow. We welcome talented individuals from diverse backgrounds to help us remove data bottlenecks, accelerate business impact, and push the boundaries of what AI can do for the world's most innovative companies. With offices in Seattle, New York City, London, and Melbourne, you'll join a fast-growing team tackling critical challenges at the intersection of AI, data, and customer experience. Ready to make an impact? Let's talk. The Role Amperity is looking for a Lead Software Development Engineer to join Team BASS. Team BASS is the "Swiss-army-knife" of our engineering organization, overseeing the critical intersection of developer productivity, shared service platforms, and high-scale Spark infrastructure. You will be the team's strategic anchor, bringing the deep experience necessary to evaluate

Cardiac Sonographer Varied Shifts Banner Staffing Svcs Mesa (525 W Brown Rd) posted: Posted 30+ Days Ago

Solution Engineer - Data Engineering Specialist (FSI) US, Remote, Remote, United States At Snowflake, we are powering the era of the agentic enterprise. To usher in this new era, we seek AI-native thinkers across every function who are energized by the opportunity to reinvent how they work. You don't just use tools; you possess an innate curiosity, treating AI as a high-trust collaborator that is core to how you solve problems and accelerate your impact. We look for low-ego individuals who thrive in dynamic and fast-moving environments and move with an experimental mindset - who rapidly test emerging capabilities to discover simpler, more powerful ways to deliver results. At Snowflake, your role isn't just to execute a function, but to help redefine the future of how work gets done. Snowflake is about empowering enterprises to achieve their full potential - and people too. With a culture that's all in on impact, innovation, and collaboration, Snowflake is the sweet spot for building big, moving fast, and taking technology - and careers - to the next level. Our Solution Engineering organization is seeking a Data Engineering Specialist to join our Applied Field Engineering team supporting the Financial Services industry sector. We are looking for someone who can provide technical leadership in working with both technical and business executives in the design and architecture of the Snowflake Cloud Data Platform as a critical component of their enterprise data architecture and overall ecosystem. In this role you will work directly with the sales team

Data Engineer Data Engineer

Applied Scientist, Agentic Automated Reasoning London, England, GBR The Agentic Automated Reasoning Group is building the next generation of software verification tools combining advances in artificial intelligence, the computational capacity of the cloud, and our deep expertise in the domain. Join us if you want to be a part of this transformational endeavor. The Strata team (https://github.com/strata-org) is seeking an applied scientist with broad interest and expertise in model checking, interactive theorem proving, programming language semantics, and generative AI. You will combine your expertise with that of your coworkers to build new tools that solve code analysis problems previously considered beyond reach. Our application areas span all the way from Infrastructure as Code to high-performance cryptography written in assembly code, while our methods span from interactive theorem proving to automated test generation. Each day, hundreds of thousands of developers make billions of transactions worldwide on AWS. They harness the power of the cloud to enable innovative applications, websites, and businesses. Using automated reasoning technology and mathematical proofs, AWS allows customers to answer questions about security, availability, durability, and functional correctness. We call this provable security, absolute assurance in security of the cloud and in the cloud. https://aws.amazon.com/security/provable-security/ Key job responsibilities - Work with customer teams to understand the nature of their software and the properties they need to establish of it. - Identify tools and methods capable of addressing the verification needs of customers, including any novel analysis capabilities required. - Use techniques spanning property-based testing to model checkers, and

Sr. Solutions Engineer - AI Natives Business Remote - California; Remote - Colorado; Remote - Oregon; Remote - Washington As a Sr. Solutions Engineer on the AI native team supporting the Digital Natives business, you will shape the future of the big data landscape by working with the most sophisticated data engineering and data science teams in the world. Reporting to the Field Engineering Manager, you will collaborate with customers, product teams, and the broader customer-facing team to develop architectures and solutions using our platform and APIs. You will guide customers through the competitive landscape, best practices, and implementation, and develop technical champions along the way. The impact you will have: - You will partner with the sales team and provide technical leadership to help customers understand how Databricks can help solve their business problems. - Consult on Big Data architectures, implement proof of concepts for strategic projects, spanning data engineering, data science and machine learning, and SQL analysis workflows. As well as validating integrations with cloud services, home grown tools, and other 3rd party applications - Collaborate with your fellow Solutions Architects, using your skills to support each other and our users - Become an expert in, promote, and recruit contributors for Databricks inspired open-source projects (Spark, Delta Lake, and MLflow) across the developer community. What we look for: - 3+ years in a data engineering, data science, technical architecture, or similar pre-sales/consulting role - Experience building distributed data systems - Comfortable programming in and debugging Python and SQL

Sr. Technical Marketing Engineer - Lakebase/Apps United States MKTQ227R21 At Databricks, Lakebase and Databricks Apps enable teams to build data-native applications where transactional data, analytics, and AI operate on the same platform. Lakebase provides a managed Postgres transactional layer for application state and operational workloads, while Databricks Apps supports building and deploying secure applications that run close to governed data and models. We are looking for a Senior Technical Marketing Engineer to help developers and data teams design, build, and operate applications using Lakebase and Databricks Apps. This role focuses on building projects end-to-end and translating platform capabilities into clear, practical guidance through demos, reference architectures, and technical content. This role reports to the Director of Technical Marketing and works closely with product, engineering, and field teams. You will work closely with Lakebase and Databricks Apps product and engineering teams to validate application patterns and translate platform capabilities into practical guidance, partner with field engineering to build and scale customer-ready demos and workshops, and support application developers and data engineers as the primary audience, with a focus on Postgres-backed transactional workflows and data-native applications that combine operational data, analytics, and AI. The impact you will have: - Build and maintain end-to-end demo applications using Lakebase and Databricks Apps, covering transactional workflows, application state, analytics, and AI integration. - Design and document reference architectures and application patterns that combine database semantics, analytics, and AI on the same data. - Tell a clear and compelling technical story through the applications you build,

Forward Deployed Engineer - Databricks Location not specified

SDE 2 - DevOps Mumbai FanCode is India's premier digital sports destination, dedicated to delivering a best-in-class experience for fans across live and non-live sports. Launched in March 2019 by industry veterans Yannick Colaco and Prasana Krishnan, FanCode has reached over 160 million users. The platform features a wide array of global and domestic sporting leagues, in partnership with leading sports associations. FanCode offers interactive live streaming through industry-first subscription models, including Match, Bundle, and Tour Passes, as well as monthly and annual plans at accessible price points. Some of the marquee properties include La Liga, Formula 1, MotoGP, cricket leagues from around the globe including CPL, Super Smash amongst others. Dream Sports, India's leading Sports Technology company, is the parent company of FanCode, with brands such as Dream11, and DreamSetGo in its portfolio. Role Overview: Technology @ FanCode The mission is to create a platform for all sports fans, covering Sports Live Video Streaming, Live Scores & Commentary, Video On Demand, Player Analytics, Fantasy Research, News. Our tech stack is hosted on GCP and is built on GKE, GCS, Cloud Run, MediaCDN utilizing a microservices-based architecture with Java, Node.js, Python, Redis, MySQL, Cassandra, and Elasticsearch. They are a data-driven team that also uses Python and big data technologies like Kafka, Spark, and Redshift for Machine Learning and Predictive Analytics. As a data-driven team, we also use Python and other big data technologies for Machine Learning and Predictive Analytics. Besides these, we heavily use Kafka, Spark, Redshift and other cutting

Senior ABAP Platform Engineer - SAP Big Data Fabric Services (BDFS) Country Brazil | Internal Posting Location São Leopoldo We help the world run better At SAP, we keep it simple: you bring your best to us, and we'll bring out the best in you. We're builders touching over 20 industries and 80% of global commerce, and we need your unique talents to help shape what's next. The work is challenging - but it matters. You'll find a place where you can be yourself, prioritize your wellbeing, and truly belong. What's in it for you? Constant learning, skill growth, great benefits, and a team that wants you to grow and succeed. PURPOSE AND OBJECTIVES Join an innovative, collaborative development environment and help build secure data access and connectivity capabilities as part of SAP's Big Data Fabric Services (BDFS).We are looking for a Senior Platform Engineer who can contribute across design, implementation, operability, and technical problem-solving in the SAP Data Accelerator and broader BDFS context.We do not work as data scientists analyzing or extracting data directly. We build the platform capabilities that make this possible.## THE PRODUCTSAP Data Accelerator focuses on secure partner connectivity and access to SAP business data across SAP ERP and LoB applications in on-premise, private cloud, and public cloud landscapes.This work sits within the broader BDFS portfolio, which enables SAP Lines of Business and customers to enter the world of big data more easily. BDFS delivers platform capabilities such as Apache Spark as a service, multi-language data

Manufacturing Team Lead Fremont, California posted: Posted 22 Days Ago

Energy Services Administrator ELLAND, UK posted: Posted 16 Days Ago

Solutions Architect, Hunter - Digital Native Business Remote - California; Remote - Colorado; Remote - Illinois; Remote - New York; Remote - Oregon; Remote - Texas; Remote - Washington As a Hunter Solutions Architect on the Digital Natives team, you will shape the future of the big data landscape by working with the most sophisticated data engineering and data science teams in the world. Reporting to the Field Engineering Manager, you will collaborate with customers, product teams, and the broader customer-facing team to develop architectures and solutions using our platform and APIs. You will guide customers through the competitive landscape, best practices, and implementation; and develop technical champions along the way. The impact you will have: - You will partner with the sales team and provide technical leadership to help customers understand how Databricks can help solve their business problems. - Work directly with prospective, greenfield, customers on their 0 to 1 journey with Databricks - Consult on Big Data architectures, implement proof of concepts for strategic projects, spanning data engineering, data science and machine learning, and SQL analysis workflows. As well as validating integrations with cloud services, home grown tools, and other 3rd party applications - Collaborate with your fellow Solutions Architects, using your skills to support each other and our users - Become an expert in, promote, and recruit contributors for Databricks inspired open-source projects (Spark, Delta Lake, and MLflow) across the developer community. What we look for: - 5+ years in a data engineering, data science, technical

Sr. Software Development Engineer, Measurement, Ad Tech, and Data Science Santa Monica, California, USA Do you want to architect systems that process petabytes of data and power advertising decisions for millions of advertisers worldwide? Are you energized by solving hard distributed systems problems at a scale few companies ever reach? This is your opportunity. Amazon Advertising operates one of the largest big data platforms on the planet - processing trillions of events daily across display, search, native, and video surfaces on every device. We're looking for a Senior Software Development Engineer to lead the design and delivery of our next-generation measurement and reporting infrastructure - the backbone that enables advertisers to understand, attribute, and optimize their spend across Amazon's full advertising suite. You will define the technical direction for critical data systems, driving architectural decisions that balance scale, latency, cost, and extensibility. You'll leverage technologies like Spark, Flink, and Java alongside AWS services - EMR, Kinesis, DynamoDB, Bedrock, and more - to build pipelines and platforms that serve diverse consumers at Amazon scale. You'll own the end-to-end lifecycle: from whiteboarding system designs in ambiguous problem spaces to shipping production systems that handle billions of requests with high availability. The ideal candidate has a proven track record of delivering highly distributed systems at scale, with deep expertise in platform design patterns - including SDKs, libraries, APIs, and versioning strategies that enable teams across the organization to build on your foundations. You are operationally excellent, technically rigorous, and thrive when partnering across

Senior Equipment Engineer Fremont, California posted: Posted 24 Days Ago

Industrial Trainee - Data Engineer/Data Scientist Kulim (Malaysia)

Senior Data Platform Engineer Lisbon About Us Riskified empowers businesses to unleash ecommerce growth by taking risk off the table. Many of the world's biggest brands and publicly traded companies selling online rely on Riskified for guaranteed protection against chargebacks, to fight fraud and policy abuse at scale, and to improve customer retention. Developed and managed by the largest team of ecommerce risk analysts, data scientists and researchers, Riskified's AI-powered fraud and risk intelligence platform analyzes the individual behind each interaction to provide real-time decisions and robust identity-based insights. Riskified is proud to work with incredible companies in virtually all industries including Acer, Gucci, Lorna Jane, GoPro, and many more . We thrive in a collaborative work setting, alongside great people, to build and enhance products that matter. Abundant opportunities to create and contribute provide us with a sense of purpose that extends beyond ourselves, leaving a lasting impact. These sentiments capture why we choose Riskified every day. About the Role We're looking for a talented multi disciplinary Senior Data Platform Engineer with passion for infrastructure, who will be responsible for the performance, scale, and stability of our core data platform products. We are a data-driven organization, ingesting data at high rates, with demands increasing each day. As a Senior Data Platform Engineer , you will be part of the Data Platform group and will be responsible for a large scale production data platform that serves as the core infrastructure of our products. You'll develop self-service tools that support the