Spark jobs

Forward Deployed Engineer San Francisco | OnSite | New York ABOUT POETIC Poetic (formerly known as Forge) is creating the next layer of human-computer interaction - combining the intelligence of AI with the reliability of software. We automate the most complex manual processes for the world's largest enterprises using a new English programming language that blends structured control flow with natural language instructions. Processes that once took hours can now be completed in a fraction of the time at superhuman quality. Poetic is deployed globally, powering critical workflows for Fortune 500 companies with 98%+ quality across over half a million production runs. ABOUT THE ROLE As a Forward Deployed Engineer at Poetic, you re-program entire companies. You will be driving a fundamental shift in how enterprises operate and how all of software is defined. You will own our most strategic customers (some of the largest companies in the world) to build and deploy automation that was previously impossible. These are not incremental improvements; you'll be replacing load-bearing workflows in production that previously consumed thousands of human hours. This role sits at the intersection of engineering, customer partnership, and operational excellence. You'll go deep into complex enterprise workflows, architect robust solutions using our AI-English programming language, and own deployments end-to-end - from initial discovery through global rollout. You'll build lasting relationships inside Fortune 500 organizations, understand the intricacies of their most critical processes, and deliver solutions that fundamentally change how they operate. You will be Poetic to our customers - the

Software Dev Engineer II, Ads AI Core Infra Seattle, Washington, USA Would you like to build highly available, scalable and distributed engineering systems for one of the largest data lakes in Amazon? Does Petabyte scale excite you? The Analytics & Data Management (ADM) team in Ads AI Core Infra owns the central datalake for Advertising unifying Petabytes of data generated across the Ads pipeline such as campaigns, ad-serving, billing, clicks, impressions and more and into a single scalable repository. This is used across the organization to drive hundreds of thousands of complex queries for analysis, measurement and reporting decisions for our customers. The data lake enables customers such as data engineers, business analysts, ML engineers, research scientists, economists and data experts to collect what they need via world-class self-service tools. ADM is building the next version of its data lake for 5x growth. An SDE on the ADM team has a unique opportunity to design and innovate solutions for this scale, delivering robust and scalable microservices built over Java and AWS as well as innovate with big data technologies like Spark, EMR, Athena and more. You will create value that materially impacts the speed and quality of decision making across the organization resulting in tangible business growth. Key job responsibilities - Engage with key decision makers such as Product & Program Managers to understand customer requirements and brainstorm on solutions - Design, code and deploy components and micro-services for the core job management pipeline - Ensure testability, maintanability and low

Senior Software Engineer, Data San Jose, California As a Senior Software Engineer, Data , you will be a key contributor driving the design, implementation, and optimization of FloQast's core data platform. You will build and scale the critical systems that power data ingestion, storage, processing, and access across our product and analytics pipelines. Working closely with cross-functional teams, you will ensure our data infrastructure remains highly reliable, scalable, and secure, enabling engineering and business operations to build on a solid data foundation. What You'll Do What You'll Bring

Senior Principal Network Engineer Bangalore, Karnataka posted: Posted 30+ Days Ago

Senior Engineering Manager Remote | United States, UNITED STATES, United States Company Description: Experian is a global data and technology company, powering opportunities for people and businesses around the world. We operate across a range of markets, from financial services to healthcare, automotive, agribusiness, insurance, and many more. Experian invests in people and new advanced technologies to unlock the power of data. We have an amazing team of 25,200 people in 32 countries. Job Description: Financial Services and Data (FSD) Engineering team is looking for a Senior Manager of Software Development that can excel in a non-traditional environment, working with the latest in the Cloud and Big Data ecosystem. You will be a hands-on leader guiding a Data Engineering team, building data pipelines for batch and streaming. You will work with teams across different time zones in the United States, Costa Rica, and India. You will be reporting to the Director of Software Engineering, and this is a hybrid remote position within the United States. You will: Lead a high-performing team of cloud ETL and Data Engineering, promoting a culture of innovation, collaboration, and knowledge. Architect and oversee the development of scalable, cloud-native data pipelines using technologies like Apache Spark and Airflow across AWS, GCP, or Azure. Collaborate with Product, Architecture, TPM, and Data Governance teams to deliver secure, compliant, and high-quality data products. Improve adoption of best practices in data engineering, including data quality, lineage, cataloging, and governance aligned with CDA and data mesh principles. Champion agile methodologies and

Senior AI/ML Engineer 2 Locations posted: Posted Today

Senior Machine Learning Engineer Warsaw, Masovian Voivodeship, Poland Location: Poland We Are: RTB House is a global company that provides state-of-the-art marketing technologies for top brands and agencies worldwide. Its proprietary ad-buying engine is the first in the world to be powered entirely by Deep Learning algorithms, enabling advertisers to generate outstanding results and reach their goals at every stage of the funnel. Founded in 2012, and now operating in 90+ markets, RTB House has always been private-by-design. It embraces first-party advertising and a relentless approach to innovation. RTB House offers end-to-end Deep Learning-powered AdTech products and solutions to maximize conversion, drive new customer acquisition, create engagement, and fuel long-term demand for a global base of clients. Desired Experience: Expertise in designing and implementing complex IT systems. Ability to develop user-friendly, versatile tools. Proficiency in at least one programming language, such as Python, C++, Java, or Scala, along with expertise in Linux. Strong skills in evaluating and optimizing system performance, from initial design through to production troubleshooting. Deep understanding of algorithms and data structures. Initiative and creativity to improve existing solutions. Ability to work effectively both within and across teams. C1 level in Polish Additional Advantages: Previous experience in the field of Machine Learning is not required, but would be an asset. Solid foundation in mathematics. Experience with GPU programming and Machine Learning frameworks such as Torch, PyTorch, or TensorFlow. Proven experience with distributed systems. Familiarity with Big Data technologies such as Hadoop, Kafka, Storm, Spark, or Flink. Hands-on experience

Lead / Staff Data Engineer - Data Platform

Data Platform Architect USA Walnut Creek, CA - WR posted: Posted Yesterday

Senior Backend Engineer Taipei, Taiwan Realize your potential by joining the leading performance-driven advertising company! As a Senior Backend Engineer on the Backend team in our Taipei Office , you'll play a vital role in building the foundational systems that power content understanding for the entire organization . You will create the robust backbone that enables every team to leverage Generative AI. You will operate with the agility of a startup to deliver powerful, scalable solutions and directly shape how the company innovates with AI. To thrive in this role, you'll need: - 7+ years of hands-on coding experience in Java or similar object oriented programming languages. - Deep understanding of strong Computer Science fundamentals: object-oriented design, data structures, applications programming and multi threading programming. - Experience with SQL and No-SQL, ex: MySQL, Vertica, Cassandra, Google BigQuery. - Experience with common frameworks: Spring, Maven, Kafka, Apache Spark. - Proven experience in building and managing data pipelines and distributed systems. - Thorough understanding of backend integration concepts and technologies such as API, web services, message queuing, and security practices. - Quick and self-motivated learner with strong problem-solving skills and the ability to troubleshoot complex issues. - Excellent communication skills and the ability to work collaboratively in both team and cross-timezone environments. Bonus points if you have: - Well understanding of common design patterns and engineering best practices. - Familiarity with containerization and orchestration technologies like Kubernetes and Helm. - Experience with workflow automation tools like Apache Airflow. - Knowledge of CI/CD

Senior AI Expert Software Engineer – Healthcare Analytics United States, US About the Role At Oracle Health, we're building the future of healthcare - cloud-native Healthcare Solutions with AI at their core, designed to operate at nation-scale. Our mission is to transform how hospitals and physicians work - enabling better patient care while ensuring accurate, timely reimbursement. We are modernizing Electronic Health Record and Revenue Cycle Management systems using LLMs and AI agents, helping clinicians focus more on patients and less on administrative burden. We're looking for highly skilled AI engineers to design and build high-scale, cloud-based data processing pipelines that ingest, transform, and analyze massive volumes of healthcare data with low latency, powering business insights and analytics across EHR and RCM systems. You will leverage LLMs, AI agents, and modern data platforms to solve problems like clinical decision support, revenue optimization, and workflow automation while using AI-assisted development tools to accelerate delivery. Key Responsibilities • Build and enhance data pipelines, ETL workflows, and transformations. • Contribute to LLM/agent-based features and analytics use cases. • Work with EHR/RCM datasets and support KPI/dashboard development. • Learn and apply best practices in cloud, data engineering, and LLMOps. Mandatory Qualifications: • BS/MS in in Computer Science or equivalent. • 5+ years of relevant software engineering experience. • Strong software engineering skills in Python/Java. • Strong knowledge of SQL. • Deep expertise in data engineering: ETL, data transformation, data modeling (Spark, SQL) • Experience building high-scale distributed data systems. • Cloud experience (OCI/AWS/Azure). •

Cloud Architect Warminster, Wiltshire posted: Posted 7 Days Ago

Senior Applied Scientist, Amazon Stores Economics & Science (SEAS) Seattle, Washington, USA Stores Economics and Science (SEAS) is an interdisciplinary science and engineering team in Amazon's Stores organization with a peak-jumping mission: we apply expertise in science and engineering to move from local to global optima in methods, models, and software. We pursue this mission by leveraging frontier science; collaborating with partner teams; and learning from the tools, experience, and perspective of others. We scale by solving problems, first in the small to prove concepts, and then in the large by building scalable solutions. We also help other teams within Amazon scale by hiring and developing the best and embedding them in other business units. We are team of passionate Applied Scientists, Research Scientists, and Economists collaborating to uncover, formulate, and solve big bet issues for broader Stores organization, including marketplace efficiency, seller incentive design, and inventory optimization. We are seeking to add another member to our team who thrives in inter-disciplinary, ambiguous, and high visibility environments. Key job responsibilities The successful candidate will lead large-scale science initiatives from research to production and translate complex business problems into mathematical frameworks. They will design and implement large-scale optimization algorithms for complex supply chain and marketplace problems, and design incentive-compatible mechanisms for marketplace challenges including auction design, matching algorithms, and pricing strategies. The ideal candidate will have a strong publication record in top-tier conferences/journals (INFORMS, EC, WINE, ICML, NeurIPS, etc.) and experience coordinating cross-functional projects. They should have hands-on experience building science

Senior Cloud Engineer Senior Cloud Engineer

Data Center Electrical Engineer The Dalles, OR, USA Our thirst for technology is a part of everything we do. The Data Center Engineering team takes the physical design of our data centers into the future. Our lab mirrors a research and development department -- cutting-edge strategies are born, tested and tested again. Along with a team of great minds, you take on complex topics like how we use power or how to run state-of-the-art, environmentally-friendly facilities. You're a visionary who optimizes for efficiencies and never stops seeking improvements -- even small changes that can make a huge impact. You generate ideas, communicate recommendations to senior-level executives and drive implementation alongside facilities technicians. With your technical expertise, you ensure compliance with codes and standards, develop infrastructure improvements and serve as an expert in your specialty (e.g., cooling, electrical). As an Electrical Engineer, you will join our growing team. In this role, you will play a key part in the design, implementation, and management of electrical systems for our data centers. You will collaborate closely with cross-functional teams, including the Engineer of Record, to ensure our facilities are built to the highest standards of safety, reliability, and efficiency.Behind everything our users see online is the architecture built by the Technical Infrastructure team to keep it running. From developing and maintaining our data centers to building the next generation of Google platforms, we make Google's product portfolio possible. We're proud to be our engineers' engineers and love voiding warranties by taking things apart

Software Development Engineer, Measurement, Ad Tech, and Data Science (MADS) Foundations- Traffic Seattle, Washington, USA The Traffic Applications team, part of Measurement, Ad Tech, and Data Science (MADS), owns the data ingress system for all of Amazon Ads. Traffic ingests hundreds of billions of ad events daily from upstream systems, decorates them with dimensional and identity data, and publishes structured traffic and billing datasets in a standardized format to downstream consumers across the Ads ecosystem. The system operates under strict SLAs for data freshness and accuracy on tier-one advertising metrics, owning three of the five critical metrics used for Ads billing. As the first MADS system to encounter regulatory and identity changes, Traffic's architectural decisions propagate across the entire MADS stack, making design choices here uniquely consequential. You will have the opportunity to work on critical business initiatives that directly impact Ads VP and SVP level goals. These include compliance initiatives such as GDPR and ePrivacy, enabling geo-targeting features, and driving architecture scalability to support spiky traffic from live sports events. Key job responsibilities As a SDE , you will: - Own team architecture and lead design on multi-engineer efforts across our Spark-based EMR pipelines, decoration jobs, and publication systems - Navigate ambiguous technical problems with conflicting constraints - regulatory deadlines, performance requirements, cost targets, and cross-org dependencies - Identify one-way-door decisions, proactively address architectural deficiencies, and ensure Traffic's design doesn't limit what downstream teams can build - Drive adoption of engineering best practices and maintain sound operations - alarms,

Cardiac Sonographer Echocardiology Banner Staffing Svcs Mesa (525 W Brown Rd) posted: Posted 10 Days Ago

Data Automation Engineer EG-Banha, Egypt (EI Corniche Street) Position: Data Automation Engineer Job Description: As a Data Automation Engineer, you will play a crucial role in developing and maintaining automated data processing systems. You will work closely with data analysts, and other stakeholders to design, implement, and optimize data pipelines, ensuring the efficient and accurate flow of data throughout the organization. What You'll Do: Develop and implement automated data pipelines for ingesting, processing, and storing structured and unstructured data. Integrate data from various sources in both real-time and batch mechanisms and perform transformations to ensure consistency, accuracy, and reliability. Implement data quality checks and monitoring mechanisms to identify and address issues proactively. Write code and scripts for automation of data processes using languages such as Python, Java, or Scala. Leverage ETL (Extract, Transform, Load) tools and frameworks [Apache spark, AWS Glue] for efficient data processing. Collaborate with other intelligent data teams. Ensure seamless data flow for machine learning and analytics applications. Implement monitoring systems to track data pipeline performance and identify areas for optimization. Continuously optimize data processes for efficiency and scalability. Document data processes, system architecture, and codebase for knowledge sharing and future reference. Keep documentation up-to-date with any changes or enhancements to the data infrastructure. Implement and adhere to data security and compliance standards. Collaborate with security teams to ensure data protection and privacy. You're a Perfect Fit If You Have: 1+ years of experience in data engineering, automation, or a related field. Strong understanding of data pipelines

Sr. Engineer - Cloud Romania - Bucharest posted: Posted 30+ Days Ago

Digital Native Go-to-Market (GTM) Leader San Francisco, California FEQ327R147 As the Digital Native Go-to-Market Leader , you will drive the global strategy for the world's most innovative high-growth companies-from late-stage startups to "born-in-the-cloud" giants. You will play a pivotal role in accelerating revenue growth by helping these organizations build their next-generation Data and AI platforms on Databricks. You are not just a strategist; you are a technical authority. You have a strong POV on how Data Intelligence platforms transform digital business models, a deep understanding of the open-source ecosystem, and the ability to engage in "architect-to-architect" discussions with CTOs and VPs of Engineering. You will set the vision, build executive relationships, and develop the programmatic motions required to scale this function globally. The impact you will have: - Global Programmatic Scaling: Design and execute repeatable, global GTM motions that allow the field to effectively sell to Digital Natives at scale. - Strategic Sales Partnership: Work closely with sales and pre-sales teams to identify high-growth use cases, create account-level technical strategies, and build comprehensive sales programs. - Growth of Digital Native Business : Directly engage with priority accounts and influence key stakeholders in strategic deals. - Technical Advisory: Act as a hands-on trusted advisor to CTOs and Engineering leaders, guiding them on setting up modern Data and AI platforms. - Open Source Advocacy: Leverage your deep understanding of the open-source ecosystem (Spark, Delta, MLflow, etc.) to align Databricks' value proposition with the "build vs. buy" mindset of Digital Natives. - Ecosystem

Sr Applied Scientist, Digital Ads , Amazon Digital Advertising Seattle, Washington, USA Are you passionate about developing new state-of-the-art measurement approaches at Petabyte scale? Amazon Advertising is one of Amazon's fastest growing businesses, and we are leveraging our unique data, the latest machine learning methods and big data technologies to better understand how Amazon's marketing influences customer behavior. We are looking for a Senior Applied Scientist to develop new systems and methods in the most challenging and data rich areas of marketing. We need an expert in experimental statistics, machine learning or causal inference to design advanced new models with our world class data systems. Dozens of Amazon businesses use Amazon's ad tech for their marketing objectives, driving more than $1B of marketing investments through Ads services and tools. As part of the 1PM team, this role will partner with a dedicated engineering team measuring the impact Amazon's marketing and identifying opportunities for optimization at scale. We drive initiatives to make smarter marketing decisions and improve the relevance of advertising to our customers. We move away from industry standard measurement systems and build sophisticated and insightful decision engines. We enable massive advertising programs, generating billions of impressions decorated with rich representations of customer state. The major challenges we are solving include integrating petabyte-scale distributed retail systems into a singular service to synthesize e-commerce data into measurement and optimization models. The successful candidate will have a causal inference background, a start-up mentality, an appreciation for white-space, and success solving problems with

Staff Data Scientist NerdWallet US | [Remote] | Remote

Business Data Scientist Sr/Lead TX Houston - Tower posted: Posted 10 Days Ago

Senior Data Scientist Barcelona, Catalunya, Spain We power people's progress. At Preply, we're all about creating life-changing learning experiences. We help people discover the magic of the perfect tutor, craft a personalised learning journey, and stay motivated to keep growing. Our approach is human-led, tech-enabled - and it's creating real impact. We've just reached unicorn status with a $150M Series D, accelerating our vision to transform education through human-led, AI-enhanced learning. Today, 100,000+ tutors teach 90+ languages to learners in 180 countries - and we're only getting started. As a category-defining company, we're shaping what the future of learning looks like at global scale. Every Preply lesson sparks change, fuels ambition, and drives progress that matters. Joining Preply means helping define the future of education at global scale, and building something that truly matters for millions of people, every day. Meet the team At Preply, we're growing a talented Data Team to empower top-notch decision-making across our organization. Our team collaborates closely with various other chapters - including product, business, and platform - leveraging analysis, experimentation, causal inference, and machine learning. You'll be embedded in cross-functional teams, partnering with our MarTech Product Manager, Marketeers, Tech Leads, and the Data Strategy team. Check out our Tech Radar to see the tools and technologies we love using at Preply! What you'll be doing - Collaborate with performance marketing teams in coming up with data solutions to support the optimization of marketing campaigns, budgets and results; - Roll out Marketing Mix Modeling (MMM)