Spark jobs

Devops Infra/cloud Engineer Rehovot,ISR posted: Posted 26 Days Ago

Sr. Software Development Engineer, EMR East Palo Alto, California, USA Do you enjoy working on large-scale distributed systems? Do you want to influence how the world interacts with and analyzes large data sets across a number of industries and use cases? Amazon EMR provides a managed framework that makes it easy, fast, and cost-effective to process vast amounts of data across dynamically scalable Amazon EC2 instances using popular distributed Hadoop frameworks such as Apache Spark, HBase, Presto, and Flink, and interact with data in other AWS data stores such as Amazon S3 and Amazon DynamoDB. Our customers in a variety of industries - eg: real estate, travel, finance and gaming - use EMR to process and interact with their vast datasets to fulfill existing and uncover new business needs. We are looking for talented engineers to help us develop innovative features for EMR to enable an interactive, secure and seamless experience for our growing customer base. One such feature is our next-generation cluster and work management system designed with an emphasis on speed, reliability and security. We'll trust you to design and implement critical high performance, large scale services and libraries in collaboration with other internal development teams to solve key business problems with minimal technical supervision. We'll be looking to you to evangelize SDLC best practices across the organization, help refine operational procedures and workflows, and influence architectural decisions to achieve our technical and business goals. In this role, you will have the opportunity to collaborate with and learn

Data Manager – AI Development Remote posted: Posted Yesterday

Oracle OTM/GTM - Senior Engineer Oracle OTM/GTM - Senior Engineer

Customer Service Talent Pool - German Speaker (all genders) Turkey Remote GoStudent is growing fast, and we're building a Talent Pool in Turkey for future German-language roles in Customer Support and Customer Retention. What you might work on Depending on the specific position that opens, your day could include: - Delivering first-class support via live chat, email and basic billing queries. - Retaining at-risk customers through active listening and tailored solutions. - Coordinating with tutors and booking lessons to keep every class on track. - Solving ad-hoc operational challenges across multiple European markets. - Partnering with our Global Customer Care Managers to improve processes and share market feedback. 🟢 If you are native/C2 in German, fluent in English, and legally based in Turkey , upload your CV (English or German). We'll contact you whenever a suitable opening matches your profile-now or in the future. Our way to develop your full potential: - Flexible & focused working: Benefit from our hybrid working model with 2-3 flexible days per week and a Meeting Free Friday for undisturbed productivity. - We grow together: Continue your career with clear development paths and our Spark! Growth & Development program . Get access to 1000s of courses on LinkedIn Learning , our Spark! Mentorship program and monthly learning sessions. Take advantage of 30 free learning sessions to experience GoStudent for yourself - Work Smart, Play Hard: Our regular team and office events ensure a good atmosphere. We believe in team spirit, celebrating together and having lots

Senior Firmware Engineer

Data Scientist Herzliya AppsFlyer is a cutting-edge technology company that specializes in mobile attribution and marketing analytics. We run at a massive scale- t any given moment thousands of servers are consuming 150+ billion mobile app events, crunching our users' data, serving requests and communicating on a massive scale. Fueled by the AppsFlyer platform, Protect360 is the leading anti-fraud solution in the market, protecting advertisers from fraudulent traffic and saving millions of dollars for our customers. We detect fraudulent traffic (fake installs and attribution hijacking attempts) in real time using machine learning, deep learning, and advanced AI techniques applied on massive data streams. We're looking for a data scientist to join our team and further improve fraud detection capabilities. This position involves analyzing evolving fraud patterns, developing and deploying advanced prevention components, and leveraging domain expertise, data tools, and modern AI methods to stay ahead of emerging threats. A partial list of our tech stack: Python (and its related scientific stack), Spark, TensorFlow/PyTorch, Kafka, AWS services, Airflow, Scala, BigQuery, and more What you'll do - Mine massive data sets to uncover fraud patterns and prevention opportunities. - Develop and productionize large-scale ML, DL, and AI models to detect and prevent fraudulent activity. - Translate business and product needs into scientific, explainable, and high-impact solutions. - Design and maintain end-to-end ML pipelines - from data ingestion and feature engineering to serving and monitoring in production. - Collaborate closely with product, engineering, and business teams to drive data-driven decisions. - Stay current

Senior CloudOps Engineer Senior CloudOps Engineer

Sr. Solutions Architect - Strategic AI Native Remote - Arizona; Remote - California; Remote - Colorado; Remote - Oregon; Remote - Washington While candidates in the listed location(s) are encouraged for this role, candidates in other locations will be considered. #LI-REMOTE As a Solutions Architect on the Digital Natives team, you will shape the future of the big data landscape by working with the most sophisticated data engineering and data science teams in the world. Reporting to the Field Engineering Manager, you will collaborate with customer stakeholders, product teams, and the broader customer-facing team to develop architectures and solutions using our platform and APIs. You will guide one of our largest AI native customers through the competitive landscape, best practices, and implementation; and develop technical champions along the way. The impact you will have: - You will partner with the sales team and provide technical leadership to help customers understand how Databricks can help solve their business problems. - Consult on Big Data architectures, implement proof of concepts for strategic projects, spanning data engineering, data science and machine learning, and SQL analysis workflows. As well as validating integrations with cloud services, home grown tools, and other 3rd party applications - Collaborate with your fellow Solutions Architects, using your skills to support each other and our users - Become an expert in, promote, and recruit contributors for Databricks inspired open-source projects (Spark, Delta Lake, and MLflow) across the developer community. What we look for: - 7+ years in a data engineering,

Facility Manager Sparks, Nevada, USA Join our dynamic team and become a critical technical leader who ensures the heartbeat of our global infrastructure remains strong and resilient. In this pivotal role, you'll be the guardian of our data center operations, driving reliability and performance while optimizing operational efficiency across complex technical environments. Data Center Facility Manger role acts as Amazon's front line when it comes to hands-on electrical and mechanical equipment troubleshooting. They will maintain, operate, and troubleshoot mission-critical data center facility equipment including electrical support equipment such as stand-by diesel generators and related fuel systems, 3 phase electrical systems that include but not limited to switchgear, UPS units, PDUs, and wet cell batteries and associated systems. Mechanical equipment includes CRAC units, centrifugal chillers, cooling towers/water chemical systems, air handlers and associated systems, pumps, and motors. Additional support equipment is included in the scope of the role which includes fire suppression systems, building automation systems, and general facilities equipment. AWS Infrastructure Services owns the design, planning, delivery, and operation of all AWS global infrastructure. In other words, we're the people who keep the cloud running. We support all AWS data centers and all the servers, storage, networking, power, and cooling equipment that ensure our customers have continual access to the innovation they rely on. We work on the most challenging problems, with thousands of variables impacting the supply chain - and we're looking for talented people who want to help. You'll join a diverse team of software, hardware, and network

Manufacturing Team Lead Fremont, California posted: Posted 25 Days Ago

Data Warehouse Engineer San Francisco About the Role Together AI is building high-performance inference compute and the software platform around it. We're looking for an early-career Data Warehouse Engineer with strong fundamentals and high growth potential to grow into a technical lead over time. You'll contribute to designing and operating our data warehouse, ETL pipelines and orchestration, work on core data models and metrics, and help raise the bar on data quality and governance across the org - with mentorship and support from experienced engineers. Requirements - 0-4 years of professional experience (or strong internships/projects) working with data warehouses, pipelines, or analytics engineering. - Solid SQL fundamentals - you're comfortable writing queries and have some exposure to window functions or dimensional modeling concepts. - Some hands-on experience with dbt or Airflow, or strong eagerness to learn - coursework and personal projects count. - Basic Python for scripting and data tooling; any exposure to Spark (PySpark/SQL) is a plus. - Familiarity with data modeling concepts like SCD2 or star schemas - even if only from coursework. - Good communication skills: you can ask clarifying questions, explain your reasoning, and work with stakeholders to understand their needs. - High standards for data quality, reliability, and maintainability - you care about getting things right. Responsibilities - Contribute to building and maintaining a medallion/curated data warehouse stack (bronze/silver/gold) for product, usage, billing, and operational data. - Build and maintain Airflow orchestrated pipelines and dbt transformation projects (modular, tested, documented). - Help design analytics-ready models:

Quality Engineer

(Data) Architect Amsterdam posted: Posted 7 Days Ago

Strategic Enterprise Hunter Account Executive, Financial Services Boston, Massachusetts; New Jersey; New York City, New York; Remote - Texas SLSQ327R3 As a Strategic Enterprise Hunter Account Executive at Databricks, you are a strategic sales professional experienced in selling into Financial Services accounts. You know how to sell innovation and change through customer vision expansion and can guide deals forward to compress decision cycles. You love understanding a product in depth and are passionate about communicating its value to Customers and System Integrators. Always looking for new opportunities, you will be asked to close net new logos. Along with the chance to close an exciting deal, we also offer accelerators above 100% quota attainment. The impact you will have: - Meet with CIOs, IT executives, LOB executives, Program Managers, and other important partners - Close both new accounts and existing accounts - Identify and close quick, small wins while managing longer, complex sales cycles - Exceed activity, pipeline, and revenue targets - Track all customer details including use case, purchase time frames, next steps, and forecasting in Salesforce - Use a solution-based approach to selling and creating value for customers - Promote Databricks' enterprise cloud data platform powered by Apache Spark™ - Ensure 100% satisfaction among all customers - Prioritize opportunities and applying appropriate resources - Build a plan for success internally at Databricks and externally with your accounts What we look for: - You have previously worked in an early stage company and you know how to navigate and be

Senior Data Engineer – AI & Analytics Senior Data Engineer - AI & Analytics

Senior Applied Scientist, International Machine Learning Bengaluru, Karnataka, IND Do you want to join an innovative team of scientists who use machine learning and statistical techniques to create state-of-the-art solutions for providing better value to Amazon's customers? Do you want to build and deploy advanced algorithmic systems that help optimize millions of transactions every day? Are you excited by the prospect of analyzing and modeling terabytes of data to solve real world problems? Do you like to own end-to-end business problems/metrics and directly impact the profitability of the company? Do you like to innovate and simplify? If yes, then you may be a great fit to join the Machine Learning and Data Sciences team for India Consumer Businesses. If you have an entrepreneurial spirit, know how to deliver, love to work with data, are deeply technical, highly innovative and long for the opportunity to build solutions to challenging problems that directly impact the company's bottom-line, we want to talk to you. Major responsibilities - Use machine learning and analytical techniques to create scalable solutions for business problems - Analyze and extract relevant information from large amounts of Amazon's historical business data to help automate and optimize key processes - Design, development, evaluate and deploy innovative and highly scalable models for predictive learning - Research and implement novel machine learning and statistical approaches - Work closely with software engineering teams to drive real-time model implementations and new feature creations - Work closely with business owners and operations staff to optimize various

Senior Engineer, NPI Office - Bangalore IT/BT Park posted: Posted 17 Days Ago

Senior Staff ML Engineer Palo Alto, California, United States, San Francisco About the Team: Grindr is an AI-native platform powering how millions of gay people connect globally. With 15M+ monthly users, 130B+ annual messages, and a team of fewer than 200, we move fast, stay lean, and tackle technical problems at a scale few companies ever see. At Grindr, we're at the dawn of an AI-driven evolution, and you'll be a senior technical leader shaping the vision . This is your chance to harness cutting-edge machine learning to redefine how millions of people in the LGBTQ+ community around the world connect -whether it's an entertaining chat, a casual encounter or long-term relationship. We're doubling down on ML as the future of Grindr, and in these early days, you'll drive strategy and execution that leaves a mark on a one-of-a-kind platform at global scale. - Early-Stage Impact: It's still early days for ML at Grindr, so you'll be charting the roadmap and building foundational systems surrounded by high-impact, greenfield work, setting the stage for our long term ML strategy. - Recommendations That Reshape: Architect and scale recommendation platforms that match millions to their next big moment, adapting to a range of intents with impact you can feel. - Conversations That Connect: Use LLMs to uncover insights, and define best practices for conversational AI at Grindr, helping users find their spark with precision and flair. At Grindr, we operate in Grindr Mode. Moderately hardcore day to day, truly hardcore when it counts.

Software Engineer (L5) - Experimentation Platform USA - Remote At Netflix, our mission is to entertain the world. Together, we are writing the next episode - pushing the boundaries of storytelling, global fandom and making the unimaginable a reality. We are a dream team obsessed with the uncomfortable excitement of discovering what happens when you merge creativity, intuition and cutting-edge technology. Come be a part of what's next. At Netflix, our mission is to entertain the world, and experimentation is how we learn, innovate, and continuously improve the product for hundreds of millions of members. The Experience Foundation team in Experimentation Platform builds and operates the core systems that power A/B test allocation, adaptive experimentation, and feature rollouts across Netflix. Our services are on the critical path for product experiences on every device, and must be highly available, low latency, and resilient. We are looking for a Senior Distributed Systems Engineer to design, build, and operate the next generation of experimentation and feature flag infrastructure at Netflix. What You'll Do Build and evolve critical experimentation and feature flag services Design and implement high-scale, low-latency services for experiment allocation and feature flag evaluation Advance core distributed systems for decentralized allocation, rules evaluation, and real-time decisioning Own reliability and performance Participate in on-call, lead incident response, and drive long-term reliability improvements Instrument services with rich observability (metrics, logs, traces) and continuously tune for resilience, performance, and scalability Shape data and integration surfaces Collaborate with teams using technologies like Flink, Spark, Elasticsearch, and

Senior Technical Lead - Data & AI India-Karnataka-Bangalore

Member of Technical Staff - Mid-training Palo Alto, CA ABOUT xAI xAI's mission is to create AI systems that can accurately understand the universe and aid humanity in its pursuit of knowledge. Our team is small, highly motivated, and focused on engineering excellence. This organization is for individuals who appreciate challenging themselves and thrive on curiosity. We operate with a flat organizational structure. All employees are expected to be hands-on and to contribute directly to the company's mission. Leadership is given to those who show initiative and consistently deliver excellence. Work ethic and strong prioritization skills are important. All employees are expected to have strong communication skills. They should be able to concisely and accurately share knowledge with their teammates. RESPONSIBILITIES: - Scale synthetic coding data to trillions of tokens with large-scale docker verification. - Distill the intelligence of flagship models into flash models through synthetic data generation. - Optimize mid-training data mixtures to boost the ceiling for RL. - Engineer long-context data recipes. - Develop robust and diverse evaluation for mid-training checkpoints. BASIC QUALIFICATIONS: - Expertise in ML and large model scaling, with familiarity across all kinds of scaling laws. - Strong ability to design ML experiments. - Familiarity with state-of-the-art techniques for curating AI training data for text, image, audio, and video modalities. - Strong engineering abilities in Spark, Ray, and other frameworks for large-scale data processing. COMPENSATION AND BENEFITS: $180,000 - $440,000 USD Base salary is just one part of our total rewards package at xAI, which

Lead Data Engineer (Platform) North Macedonia - Remote Why Valtech? We're the experience innovation company - a trusted partner to the world's most recognized brands. To our people we offer growth opportunities, a values -driven culture, international careers and the chance to shape the future of experience. The opportunity At Valtech, you'll find an environment designed for continuous learning, meaningful impact, and professional growth. Whether you're pioneering new digital solutions, challenging conventional thinking or building the next generation of customer experiences, your work will help transform industries. We are proud of: - The work we do and the innovation we drive - Our values of share, care a nd dare - A workplace culture that fosters creativity, diversity and autonomy - Our borderless, global framework, which enables seamless collaboration The role As a Lead Data Engineer (Platform), you are passionate about engineering excellence, developer experience, and scalable data platforms. You bring strong technical expertise, a product mindset for platform engineering, and a drive to simplify and accelerate how engineering teams deliver data solutions. You will thrive in this role if you are: - A platform-minded engineer who thinks in systems, not just pipelines - A strong advocate for automation, standardisation and reusable engineering patterns - Excited by improving developer experience across large engineering communities - Comfortable balancing hands-on technical work with technical leadership and enablement - Motivated by continuous improvement and eliminating manual engineering effort Role responsibilities - Design, build, and maintain scalable data engineering frameworks and platform utilities used

Staff Machine Learning Platform Engineer Kitchener-Waterloo, ON; Toronto, ON About Faire Faire is a technology wholesale platform built on the belief that the future is local. Independent retailers around the globe collectively represent a multi-hundred-billion-dollar wholesale market that has historically been fragmented and offline. At Faire, we're using the power of tech, data, and machine learning to connect this thriving community of entrepreneurs across the globe. Picture your favorite boutique in town - we help them discover the best products from around the world to sell in their stores. With the right tools and insights, we believe that we can level the playing field so businesses can grow and local communities can thrive. We're looking for smart, resourceful and passionate people to join us as we power the shop local movement. If you believe in community, come join ours. About this role As a Staff Machine Learning Platform Engineer, you will help design, improve, and operate a scalable ML platform to accelerate model training, deployment, and governance. You are the technical bridge between data science and production engineering. You'll be joining a small but deeply critical team that scales Faire's ability to support tens of thousands of local businesses in a constantly narrowing retail landscape. What You Will Do Design and operate ML infrastructure, including workspaces, clusters, jobs, and workflows Productionize ML workloads using Spark, Delta Lake, MLflow, and Databricks Workflows Teach data scientists how to utilize our ML platform to advance development from notebook to production for our most