Spark jobs

Software Engineer (Graph), Supply Chain Integration Sunnyvale, United States of America At Apple, new ideas quickly transform into groundbreaking products, services, and customer experiences. Bring passion and dedication to your work, and there's no telling what can be accomplished. As part of the Supply Chain Innovation team, you will play a pivotal role in building end-to-end, best-in-class software solutions for Apple's Supply Chain needs, ranging from Supply Planning and Demand Planning to Product Distribution and beyond. You will collaborate with various internal stakeholders to define and implement solutions that optimize Apple's internal business processes. We are seeking an expert Neo4j Graph Database Engineer to design, develop, and optimize graph-based data models and applications that power sophisticated analytics and insights. The ideal candidate will have hands-on expertise with Neo4j, Cypher, and graph modeling, along with a strong understanding of data integration, performance tuning, and API-based access for enterprise-scale systems. Design and implement graph data models and relationship-based schemas using Neo4j. Develop and optimize Cypher queries for high-performance data access and analysis. Integrate Neo4j with Java, Spring Boot, Python, or Node.js applications. Build data pipelines to ingest, transform, and sync data between Neo4j and other systems (e.g., SQL, Kafka, Spark). Collaborate with product, data science, and engineering teams to translate business problems into graph solutions. Implement data visualization and relationship analytics using Neo4j Bloom or Graph Data Science (GDS) library. Ensure data quality, consistency, and performance optimization across graph datasets. Support system scaling, backup/recovery, and deployment automation for Neo4j environments. Contribute to

Senior Cloud Engineer Troy, MI - USA posted: Posted 23 Days Ago

Full‑Stack Machine Learning Engineer UK - London (London Wall) posted: Posted Yesterday

Senior Software Engineer, Machine Learning New York, NY Software Engineer, Machine Learning (MLOps & Data) A Career with Point72's Surveillance Team On the Knowledge Graph Intelligence team, you'll work alongside product managers, engineers, and data scientists to build the next generation of intelligent systems through graph technology. We're a team of experts who experiment and work to discover new ways to harness open-source solutions, modern cloud architectures, and sophisticated Artificial Intelligence (AI) solutions, while embracing enterprise agile methodologies. Our commitment to building and innovating in the AI space provides the framework intended to drive smarter decision-making and enhance how we build and operate our platforms and applications. What you'll do In this data-heavy role, you will design and build mission-critical infrastructure that powers our machine learning lifecycle, from large-scale data processing and feature engineering to model training, real-time deployment, and monitoring. Specifically, you will: - Architect and implement the full lifecycle of ML models, from data ingestion to production inference, contributing to the design of our next-generation, event-driven architecture, using technologies like gRPC, Kafka, and high-performance API frameworks, like FastAPI, Spring WebFlux, and Axum. - Engineer and automate robust, large-scale data processing pipelines (ETL/ELT) using tools like Spark, dbt, and workflow orchestrators, and lead the design and implementation of our Feature Store strategy. - Own the MLOps framework for model training, versioning, and deployment, including CI/CD pipelines, automated workflows, and experiment tracking and evaluation tooling. - Implement sophisticated deployment strategies, including canary, blue-green, shadow, and A/B testing, to ensure safe,

Principal AL/ML Solution Engineer US-NY-New York City-Remote, Remote, New York, United States, US-CA-Remote, US-WA-Remote, US-OR-Remote, US-AZ-Remote At Snowflake, we are powering the era of the agentic enterprise. To usher in this new era, we seek AI-native thinkers across every function who are energized by the opportunity to reinvent how they work. You don't just use tools; you possess an innate curiosity, treating AI as a high-trust collaborator that is core to how you solve problems and accelerate your impact. We look for low-ego individuals who thrive in dynamic and fast-moving environments and move with an experimental mindset - who rapidly test emerging capabilities to discover simpler, more powerful ways to deliver results. At Snowflake, your role isn't just to execute a function, but to help redefine the future of how work gets done. Snowflake is about empowering enterprises to achieve their full potential - and people too. With a culture that's all in on impact, innovation, and collaboration, Snowflake is the sweet spot for building big, moving fast, and taking technology - and careers - to the next level. Our Solution Engineering organization is seeking an AI/ML specialist to join our Solutions Engineering team, while supporting ourHealthcare & Life Sciences sector. We are looking for someone who can provide AI/ML technical guidance, while working with both technical and business executives in the design and architecture of the Snowflake Cloud Data Platform as a critical component of their enterprise data architecture and overall ecosystem. In this role you will work directly

Lakebase Sales Specialist-Manufacturing Remote - Texas; United States SLSQ327R328 Databricks is seeking a Lakebase Sales Specialists to help customers modernize their operational data foundation with Databricks Lakebase, our fully-managed Postgres offering for intelligent applications. This high-impact role sits within the Lakebase Go-To-Market team and partners closely with regional Account Executives within our Retail vertical to drive adoption of Lakebase with platform, application, and data teams. Lakebase gives customers a unified, governed foundation for operational workloads and AI-native applications, helping them move away from a fragmented estate of point databases toward a modern, scalable, serverless Postgres service. If you want to be at the forefront of operational databases for AI and intelligent applications at one of the fastest-growing data and AI companies in the world, this is your opportunity. The impact you will have - Drive new Lakebase revenue by identifying, qualifying, and driving Lakebase activations and consumption within a defined territory, in partnership with regional Account Executives and the broader account team. - Lead with outcomes for key Lakebase personas - including platform teams and developers, data teams, and central IT - articulating how Lakebase helps them ship features faster, simplify operational data architectures, and improve governance and cost efficiency. - Sell the value of fully-managed Postgres for intelligent applications, positioning Lakebase as the optimal choice for operational workloads that power real-time, AI-driven experiences. - Run complex, multi-threaded sales cycles from discovery and value hypothesis through commercial negotiation and close, navigating executive, technical, and line-of-business stakeholders. - Orchestrate proof-of-value and

Sr. Applied Scientist, Applied AI Solutions Seattle, Washington, USA The newest business group in AWS, Applied AI Solutions are built by AWS and AWS Partners to deliver applied AI solutions that leverage Amazon's operational expertise and that businesses love and trust for their day-to-day success. Our ambition is to become a partner which companies can rely on to run their business every day, putting AI to work delivering better customer experience, operational excellence and speed. Are you excited about pushing past single-turn chat into agents that run for hours or days, planning, recovering from failure, and accumulating knowledge as they work? Do you want to design the harnesses, memory systems, and context strategies that turn frontier models into reliablen collaborators on real business problems? Do you enjoy the prospect of solving agent problems that haven't been solved at scale anywhere before, including memory decay, belief revision, fact grounding, long-horizon credit assignment, and evaluation of open-ended trajectories? Along the way, you'll get opportunities to be a disruptor, innovator, and a problem solver, someone who truly enables AI agents to create significant impacts. Key job responsibilities You will partner with cross-functional business and engineering teams to identify and deliver high-impact agentic use cases across the Applied AI Solutions portfolio. You will design, develop, and evaluate long-running agents - including orchestration harnesses, memory architectures (episodic recall, semantic facts, revisable beliefs, principled decay), context compaction strategies, and safe, auditable tool and environment designs. You will define evaluation frameworks for agents whose outputs defy single-answer

Data Engineer 2 Locations posted: Posted 30+ Days Ago

Senior Applied Scientist, Linear Personalization Experience Team (LPEX) Seattle, Washington, USA Prime Video is a first-stop entertainment destination offering customers a vast collection of premium programming in one app available across thousands of devices. Prime members can customize their viewing experience and find their favorite movies, series, documentaries, and live sports - including Amazon MGM Studios-produced series and movies; licensed fan favorites; and programming from Prime Video add-on subscriptions such as Apple TV+, Max, Crunchyroll and MGM+. All customers, regardless of whether they have a Prime membership or not, can rent or buy titles via the Prime Video Store, and can enjoy even more content for free with ads. Are you interested in shaping the future of entertainment? Prime Video's technology teams are creating best-in-class digital video experience. As a Prime Video technologist, you'll have end-to-end ownership of the product, user experience, design, and technology required to deliver state-of-the-art experiences for our customers. You'll get to work on projects that are fast-paced, challenging, and varied. You'll also be able to experiment with new possibilities, take risks, and collaborate with remarkable people. We'll look for you to bring your diverse perspectives, ideas, and skill-sets to make Prime Video even better for our customers. With global opportunities for talented technologists, you can decide where a career Prime Video Tech takes you! Prime Video is disrupting traditional media with an ever-increasing selection of movies, TV shows, Emmy Award-winning original content, add-on subscriptions, and live events like Thursday Night Football. Within this expanding ecosystem,

Senior AI/ML Engineer Amsterdam, Netherlands posted: Posted 30+ Days Ago

Sr. Applied Scientist, WWGS Real Estate & Store Development Seattle, Washington, USA Do you want to help shape the future of Amazon's physical retail presence? Worldwide Grocery Stores (WWGS), Location Strategy and Analytics team is looking for a Sr. Applied Scientist to join us in developing advanced forecasting models, optimization models, and analytical tools to support critical real estate and network planning decisions for Amazon's Worldwide Grocery business, including Whole Foods Market. Our team is responsible for developing predictive models and tools to support Real Estate and Topology analysts in making important decisions regarding our stores-including new store openings, relocations, closures, remodels, design, new formats, and more. We leverage statistical modeling, machine learning, and GenAI to build solutions for store sales forecasting, sales transfer effects, macrospace optimization, store network optimization, store network diffusion planning, and causal effects. As a Sr. Applied Scientist on our team, you will apply your deep technical expertise to tackle complex business problems and develop innovative solutions to improve our forecasting, decision-making capabilities, and MLOps. You will collaborate with a diverse team of scientists, economists, and business partners to identify opportunities, develop hypotheses, build internal products, and translate analytical insights into actionable recommendations for Executive Leadership. Key job responsibilities - Design and implement forecasting models and machine learning solutions to predict store performance and optimize our retail network. - Analyze large datasets to uncover insights and patterns related to store performance, customer behavior, and market dynamics. - Develop and own end-to-end solutions, tools and frameworks to

Principal Quality Engineer Newark, Delaware posted: Posted 30+ Days Ago

Senior Staff Software Engineer, Indexing & Retrieval Platform Remote - United States Reddit is a community of communities. It's built on shared interests, passion, and trust, and is home to the most open and authentic conversations on the internet. Every day, Reddit users submit, vote, and comment on the topics they care most about. With 100,000+ active communities and approximately 126 million daily active unique visitors, Reddit is one of the internet's largest sources of information. For more information, visit www.redditinc.com . Team The ML Indexing & Retrieval Platform team at Reddit is responsible for building and scaling the core infrastructure that powers machine learning driven recommendations. We design and maintain systems for ML data ingestion, low-latency retrieval services, and end-to-end lifecycle management of data. With a focus on performance, reliability, and scalability, we enable real-time access to high-quality data that supports a wide range of applications, including Content Understanding, Semantic, Lexical retrieval & GenAI applications. How You'll Have Impact You'll lead the development of next-generation ML Indexing & Retrieval systems, owning the full lifecycle from ideation to production and going beyond incremental improvements to reimagine core platform capabilities. As part of a high-impact, cross-functional team, you'll solve complex technical challenges to build scalable, reliable platforms that empower developers to efficiently ship critical ML features. Languages: Go, Java, Python, or any object oriented programming language Frameworks: Flink, Airflow, Spark for large scale batch & stream processing Databases: Familiarity with Vector, Lexical & Key-Value Databases Tools: Kubernetes, Docker, AWS, GCP What

Data Engineer India - Hyderabad posted: Posted 2 Days Ago

Sr. Applied Scientist, Alexa International London, England, GBR The Alexa International Science team is looking for a passionate, talented, and inventive Senior Applied Scientist with a strong background in speech models (understanding and generation) and deep learning, to help build industry-leading Generative AI technology with speech-to-speech models and multilingual systems. At this level, you will drive cross-team scientific strategy for speech quality across international locales, influence partner teams, and deliver solutions that have broad impact across Alexa's global products. Key job responsibilities As a Senior Applied Scientist with the Alexa International team, you will work with talented peers to develop novel algorithms and modeling techniques to advance the state of the art in multilingual speech generation, text-to-speech synthesis, and speech-to-speech models. Your work will directly impact customers across multiple languages in the form of natural, expressive, and locale-appropriate voice experiences for Alexa+. You will leverage Amazon's heterogeneous data sources and large-scale computing resources to accelerate advances in speech synthesis, voice quality, and pronunciation accuracy for non-English locales. The ideal candidate possesses a solid understanding of machine learning, speech synthesis (TTS/S2S), multilingual phonetics, modern model architectures, and evaluation methodology. They thrive in a fast-paced environment, tackle complex challenges in low-resource language settings, excel at delivering impactful solutions while iterating based on customer feedback, and are able to influence and align multiple teams around a shared scientific vision for international speech quality. A day in the life You will start your day reviewing experiment results and iterating on model architectures for speech

Senior Manager, Data Platform Enginering IN - India - Bangalore posted: Posted 30+ Days Ago

Staff Software Engineer, Data Ingestion - Slack Virginia - Washington DC Metro - Remote; Washington - Seattle; California - Palo Alto; Georgia - Atlanta; Washington - Bellevue; Texas - Dallas; California - San Francisco; Indiana - Indianapolis About the team Slack is looking for a Staff Software Engineer to join the Data Ingestion Team. As part of the Data Engineering organization, we build and operate the platform that ingests data from various sources into our Lakehouse with both streaming and batch approaches. Currently, our infrastructure handles 180PB+ of storage, processes ~1T records a day, 400+ data pipelines and 1000+ Active Airflow DAGs. As Slack's data grows (along with the number of customers, features and employees), the goal of the team is to build a highly scalable and resilient ingestion platform leveraging modern Streaming and Lakehouse technologies. As a Staff Software Engineer, you will play a key role in driving our streaming and lakehouse strategies while also building highly scalable services and tools to help our customers acquire large volumes of high quality data at scale and making customers' interactions with the ingested data pleasant and productive. You will have deep technical skills, be a self-starter, detail and quality oriented, and passionate about driving data driven decisions and having a huge impact at Slack! What you will be doing: Design and develop highly scalable and resilient streaming and batch pipelines for ingesting and processing structured and unstructured datasets Design and implement Slack's Lakehouse solutions and enable production use cases for customers

Analyst, Energy Storage Optimization & Analytics New York, NY About Daylight Daylight is building a decentralized electric utility. We're building technology that enables homes to generate, store, and share energy with their neighbors, turning passive consumers into active participants in a smarter, more sustainable grid. Our team brings together energy and crypto veterans who believe in a future where communities power themselves. Backed by a16z, Framework, and other industry-leading investors, we're building the infrastructure to make that vision a reality. Learn more at www.godaylight.com The Role In this role at Daylight, you'll play a critical role in executing our Virtual Power Plant (VPP) and demand response program participation across the country. You'll develop strategies to drive revenue growth, optimize asset performance, and collaborate with the development team to build an internal platform that integrates revenue stacking, analytics, and operational efficiencies. We're an early-stage startup, so this role goes beyond traditional asset optimization. You'll support functions related to energy trading, pricing strategies, and risk management while ensuring compliance with program requirements and internal risk metrics. You'll also develop partnerships and build operational capabilities to enhance commercial performance. This role is ideal for someone passionate about energy markets and technology who thrives in a fast-paced environment. You'll shape our analytics platform, integrate key data sources, and develop risk management frameworks-all while having opportunities to grow into a leadership position as we scale. This role is remote with the New York City metro area preferred. Travel to New York City is required for

Cloud Engineer, Health Software San Francisco Bay Area, United States of America Imagine what you could do here. At Apple, new ideas have a way of becoming extraordinary products, services, and customer experiences very quickly. Bring passion and dedication to your job and there's no telling what you could accomplish. The Apple Health team is looking for a hardworking Sr. Cloud Engineer with a background in large scale engineering support. You will enjoy using technology to automate solutions and optimize outcomes focusing on data/backend engineering, continuous integration and continuous deployment in a fast-changing world of software delivery. Apple's Health team is seeking a high-energy Site Reliability Engineer. As a member of our dynamic group, you'll have the rare and rewarding opportunity to craft upcoming products that will delight and inspire millions of Apple's customers every day. Comfortable with weekly on-call rotation Minimum Qualifications: Hands-on experience in engineering, Cloud, or as a software engineering staff member Strong programming and scripting fundamentals, Python and Bash Modern web services architectures, cloud platforms such as AWS, GCP Source control management such as Git, SVN Linux fundamentals, experience with hardening operating system images Hands-on experience with Log mining frameworks like Splunk, Elastic, Prometheus, Loki etc Container services(Docker/Kubernetes), UNIX system administration Grafana or similar for investigating or monitoring problems on systems Release and management of NoSQL, Zookeeper, Spark, Mesos In-depth knowledge of build/release systems and process BS Computer Science or equivalent experience Preferred Qualifications: Passionate about Continuous Build, Integration, Test, and Delivery systems Able to quickly

AV & Event Engineer L4 San Francisco, California

Sr. Applied Scientist, C360 Seattle, Washington, USA Are you a scientist passionate about advancing Information Retrieval, NLP, and Large Language Models? Do you want access to massive datasets, world-class compute, and a team of top scientists and engineers building the future of e-commerce? If so, you'll be a great fit for our team at Amazon. We build large-scale ML solutions that deliver personalized, up-to-date recommendations to millions of customers. Our team is uniquely positioned to shape how customers think about their shopping journey. We're looking for scientists with deep LLM expertise to build our next generation of models. The team focuses on post-training-instruction tuning, reward modeling, reinforcement learning, and multi-modal alignment. You'll design and run large-scale experiments, analyze model behavior, and develop training recipes that improve core capabilities like reasoning, personalization, and other frontier paradigms. Key job responsibilities - Own the scientific roadmap for personalization initiatives, identifying high-impact research directions and translating ambiguous business problems into well-defined ML formulations - Design and lead end-to-end systems spanning recommendations, information retrieval, and LLM fine-tuning, from problem framing through offline experimentation to production A/B testing and launch - Drive technical decisions on model architecture, training methodology, and evaluation frameworks, balancing scientific rigor with business impact and operational constraints - Mentor and raise the bar for the science team through design reviews, paper discussions, and establishing best practices for experimentation and reproducibility - Influence cross-functional strategy by partnering with engineering, product, and leadership to define the product vision informed by what's technically feasible and

Senior Data Engineer Senior Data Engineer

Software Development Engineer - Big Data, Brand Metrics and Benchmarks New York, New York, USA Are you passionate about Big Data, Distributed systems, and large scale datasets can be integrated with new state-of-the-art AI agent-friendly data interfaces? Be part of a team of world-class engineers and industry leading experts operating one of the largest big data platforms for advertising at Amazon scale. Key job responsibilities Join our team and take ownership of significant initiatives that solve some of the toughest technical challenges in big data: - Design and build highly scalable, fault-tolerant data pipelines processing petabytes of advertising data - Architect new data paradigms that enable real-time analytics and machine learning applications - Collaborate with data scientists, product managers, and partner teams across Amazon Ads to deliver data solutions that drive business impact - Mentor junior engineers through code reviews, design discussions, and pair programming - Contribute to technical strategy and architectural decisions for our data platform - Build monitoring, alerting, and data quality frameworks to ensure reliability at scale Our systems are built for stability and reliability, which means our on-call shifts are manageable and focused on continuous improvement rather than firefighting. About the team We're a collaborative team of engineers who are passionate about solving complex data challenges while maintaining a healthy work-life balance. Our engineers regularly work with latest technologies-recent projects include migrating our data lake to Apache Iceberg, implementing real-time streaming pipelines with Spark, and building unified and flexible reporting solutions. We invest heavily in your

Data Integration Operations Manager MetroPark posted: Posted 25 Days Ago