Spark jobs

Lead Analyst- Accounting Bangalore, Karnataka posted: Posted 30+ Days Ago

Principal Technology Manager, Business Data Technologies Seattle, Washington, USA Amazon's eCommerce Foundation (eCF) organization is responsible for the core components that drive the Amazon website and customer experience. Serving millions of customer page views and orders per day, eCF builds for scale. As an organization within eCF, the Business Data Technologies (BDT) group is no exception. We collect exabytes of data from thousands of data sources inside Amazon including the Amazon catalog system, inventory system, customer order system, page views on the website and Alexa systems. We also support Amazon subsidiaries such as IMDB and Audible. We provide interfaces for our internal customers to access and query the data hundreds of thousands of times per day, using Amazon Web Service's (AWS) Redshift, Hive, Spark and Oracle. We build scalable solutions that grow with the Amazon business. Are you passionate about Program Management and technology? Are you passionate about eliminating redundant efforts that solve similar customer needs? Here's your chance to lead a highly visible initiative with significant business impact. The senior leaders need a light weight mechanism to help bring visibility, share best practices, and provide better communication and collaboration across teams. This role works closely with Tech Director, VPs and Principal Engineers to identify patterns, new trends, goals and areas for cross-org collaboration. This role is unique in that it does not support just one organization, instead focuses on best practices that can be applicable to several or all within Stores, Devices and Others. In this role, you will

Databricks AWS Engineer - Vice President PUNE, Mahārāshtra, India

Pyspark and Databricks Engineer In the assigned Job Role of Technology Consultant 2, your Area Of Responsibility will be as below: ⦁ Contribute to the requirements elicitation process by documenting assigned parts of business requirements, in line with guidance provided ⦁ Facilitate software application design discussions, and document design decisions to guide the technical team towards building software solutions ⦁ Participate in coding and integrate new features or updates into existing applications, with a focus on maintaining system stability ⦁ Conduct code reviews, do changes to the codebase and maintain code repositories ⦁ Implement test strategies, analyse results, and coordinate bug fixes to uphold the software quality standards ⦁ Develop user training programs, documentation, and support frameworks to ensure a smooth transition to new software applications ⦁ Actively participate in resolving production issues and recommend preventive strategies to enhance system reliability ⦁ Maintain detailed records of code, testing techniques, and support activities to enrich the knowledge base and assist other similar projects Your contribution to the team: ⦁ A collaborative spirit and excellent communication skills. ⦁ The ability to handle end to end SDLC phases from requirement gathering to implementation. ⦁ A knack for translating complex requirements into actionable development tasks. ⦁ A passion for design and hands-on coding experience ⦁ A proactive approach to testing, troubleshooting, and refining our applications. ⦁ The ability to work with cross-functional teams and do software integration.

Director, Engineering Governance- Mountain View, CA Mountain View, California P-1494 Databricks is the Lakehouse company. More than 7,000 organizations worldwide - including Comcast, Condé Nast, H&M and over 50% of the Fortune 500 - rely on the Databricks Lakehouse Platform to unify their data, analytics and AI. Databricks is headquartered in San Francisco, with offices around the globe. Founded by the original creators of Apache Spark™, Delta Lake and MLflow, Databricks is on a mission to help data teams solve the world's toughest problems. To learn more, follow Databricks on Twitter, LinkedIn, andFacebook. For more information, visit www.databricks.com . About the Team We are seeking a highly technical and influential Director of Engineering, Governance Foundation to define how security, privacy, and responsible data practices are built into the core of Databricks' products and infrastructure. This role is not simply about running programs-it is about designing and scaling the engineering platforms, guardrails, and operating models that make secure and compliant product development the default across a rapidly growing, multi-cloud ecosystem. This is a strategic, hands-on technical leadership role with significant cross-company impact. You will lead a team of senior security engineers, partner with product and infrastructure leaders, and shape the foundational governance systems that enable Databricks to innovate quickly while maintaining world-class security and customer trust. What You'll Own Security & Privacy Governance Architecture - Define the company-wide strategy for embedding security, privacy, and responsible data use across engineering workflows and platform components. - Build scalable governance frameworks that help engineering

Brazil - Rio de Janeiro (Hybrid): Full Stack Developer (Java/Cloud) Brazil - Rio de Janeiro (Hybrid): Full Stack Developer (Java/Cloud) We are looking for the right people - people who want to innovate, achieve, grow and lead. We attract and retain the best talent by investing in our employees and empowering them to develop themselves and their careers. Experience the challenges, rewards and opportunity of working for one of the world's largest providers of products and services to the global energy industry. Job Duties We are seeking a highly experienced Full Stack Java Cloud Developer to design, build, and maintain cloud-native applications that support scalable, secure, and high-performance systems. This role combines backend Java development, cloud architecture, distributed data processing with Apache Spark, DevOps automation, and frontend UI development. The successful candidate will work within an Agile engineering team to deliver resilient microservices, modern user interfaces, and automated deployment pipelines, while leveraging emerging technologies to improve code quality, productivity, and system observability. Qualifications Design, develop, and maintain cloud-native applications and backend services using Java/Jakarta EE, Spring Boot, and microservices architecture Implement distributed data processing pipelines using Apache Spark Build RESTful APIs and event-driven services (Kafka, RabbitMQ, SNS/SQS, Azure Service Bus) Apply cloud security best practices, including identity management, encryption, and secure communication Leverage AI-based tools for coding, testing, automation, monitoring, and p

Data Center Technician (Night Shift), AWS Opportunity for Military Community Sparks, Nevada, USA This opportunity is for U.S. citizens who have served in the U.S. Armed Forces including Reserves, National Guard, and service member spouses. Join our dynamic AWS team and become a critical guardian of global cloud infrastructure! You'll play a pivotal role in maintaining the heartbeat of the world's most innovative technology platform, ensuring seamless data center operations that power millions of businesses and services worldwide. Data Center Technician position involves in-depth hardware and network diagnostics followed by physical repair as well as participating in an on-call rotation. AWS Infrastructure Services owns the design, planning, delivery, and operation of all AWS global infrastructure. In other words, we're the people who keep the cloud running. We support all AWS data centers and all of the servers, storage, networking, power, and cooling equipment that ensure our customers have continual access to the innovation they rely on. We work on the most challenging problems, with thousands of variables impacting the supply chain - and we're looking for talented people who want to help. You'll join a diverse team of software, hardware, and network engineers, supply chain specialists, security experts, operations managers, and other vital roles. You'll collaborate with people across AWS to help us deliver the highest standards for safety and security while providing seemingly infinite capacity at the lowest possible cost for our customers. And you'll experience an inclusive culture that welcomes bold ideas and empowers you to own them

Data Scientist & Machine Learning Researcher 2 Locations posted: Posted 30+ Days Ago

Cloud Engineering, Advanced 2 Locations posted: Posted Today

Lead - Data Engineer 2 Locations posted: Posted 22 Days Ago

Senior Software Engineer - JVM Language Clients United States (remote) About ClickHouse Recognized on the 2025 Forbes Cloud 100 list, ClickHouse is one of the most innovative and fast-growing private cloud companies. With more than 3,000 customers and ARR that has grown over 250 percent year over year, ClickHouse leads the market in real-time analytics, data warehousing, observability, and AI workloads. The company's sustained, accelerating momentum was recently validated by a $400M Series D financing round. Over the past three months, customers including Capital One, Lovable, Decagon, Polymarket, and Airwallex have adopted the platform or expanded existing deployments. These customers join an established base of AI innovators and global brands such as Meta, Cursor, Sony, and Tesla. We're on a mission to transform how companies use data. Come be a part of our journey! About the team The Integrations Ecosystem org is the bridge between ClickHouse and the rest of the data world. We build and maintain the connections that make ClickHouse accessible to the developers, data engineers, and analysts who rely on it in production every day. Our scope spans low-level database drivers, ingestion and CDC pipelines, data framework integrations, and BI connectivity - ensuring ClickHouse feels native in whatever stack our users already have. We maintain the official language clients (Python, JavaScript, Java, Go, Rust, C++, C#) and ODBC driver; ship first-party connectors for Kafka Connect, Apache Spark, Apache Flink, dbt, and Airbyte; and own integrations with BI platforms. This role sits on the Language Clients team, which

Staff Machine Learning Engineer Hybrid - Denver, Denver, Colorado, United States Ibotta is seeking a Staff Machine Learning Engineer to join our Core Data & Analytics team and contribute to our mission to Make Every Purchase Rewarding. We're looking for someone who has a passion for building scalable machine learning solutions using technologies and tools such as Python, Spark, Databricks, AWS, Sagemaker, etc. As a Staff Machine Learning Engineer, you will act as a technical leader and delivery owner for complex, high-impact ML initiatives spanning foundation models, data systems, and large-scale ML platforms. This role will immediately impact ML strategy and drive impact through the support, guidance, and collaboration with multiple teams as part of our analytics organization. We're looking for a technical leader to further mature our current infrastructure and abstractions and identify new opportunities to add value through ML applications. This position is available in multiple locations: Denver, Colorado - if you are in or willing to relocate to the Denver area, this is a hybrid position requiring 3 days in office (Tuesday, Wednesday, and Thursday). Remote options are available for the following states - AZ, AR, CA, FL, GA, IL, IN, IA, KS, MD, MA, MI, MN, MO, MT, NV, NJ, NY, NC, OH, OK, PA, TN, TX, UT, VA, WA, WI. Not based in one of these areas? We will offer a relocation bonus to help make your move to the Mile High City a smooth one. What you will be doing: - Lead the design,

Senior Power Systems Engineer, Data Centers 2 Locations posted: Posted 25 Days Ago

Senior Platform Engineer - SAP Business Technology Platform Big Data Team Region Europe | Country Bulgaria | Internal Posting Location Sofia We help the world run better At SAP, we keep it simple: you bring your best to us, and we'll bring out the best in you. We're builders touching over 20 industries and 80% of global commerce, and we need your unique talents to help shape what's next. The work is challenging - but it matters. You'll find a place where you can be yourself, prioritize your wellbeing, and truly belong. What's in it for you? Constant learning, skill growth, great benefits, and a team that wants you to grow and succeed. Overview As a Senior Platform Engineer, you will design, implement, and operate the Big Data services (Kafka, Cassandra, Spark) that power SAP's enterprise‑scale cloud solutions. This includes distributed systems, multi‑cloud infrastructure, Kubernetes‑based runtimes, and automation frameworks that enable reliable, scalable, and secure platform operations. Your contributions will strengthen platform performance, developer experience, operational resilience, and the technical foundations of SAP Business Technology Platform (SAP BTP). We welcome individuals who bring curiosity, diverse experiences, and a commitment to reliable, high‑quality engineering. What you will build Build and automate provisioning, configuration, and lifecycle operations for Big Data components within SAP Business Technology Platform, ensuring scalable, repeatable, and reliable deployment workflows Design and implement cloud‑agnostic platform capabilities that run seamlessly across major hyperscale environments,

Project Engineer (Senior / Staff) – Power & Automation 3 Locations posted: Posted 3 Days Ago

Data Center Manager, , AWS Opportunity for Military Community Sparks, Nevada, USA This opportunity is for U.S. citizens who have served in the U.S. Armed Forces including Reserves, National Guard, and service member spouses. Manage a team that maintains cloud infrastructure for global data center operations! AWS is looking for technology managers with experience in people management, strong technical understanding and the drive and vision to take AWS Data Center Operations to the next level. The successful candidate will have experience creating/devising strategies, mentoring people at all levels, sponsoring projects, and proposing technical solutions. This position involves leading teams in hardware and network diagnostics followed by physical repair and includes participation in an on-call rotation. AWS Infrastructure Services owns the design, planning, delivery, and operation of all AWS global infrastructure. In other words, we're the people who keep the cloud running. We support all AWS data centers and all servers, storage, networking, power, and cooling equipment that ensure our customers have continual access to the innovation they rely on. We work on the most challenging problems, with thousands of variables impacting the supply chain - and we're looking for talented people who want to help. You'll join a diverse team of software, hardware, and network engineers, supply chain specialists, security experts, operations managers, and other vital roles. You'll collaborate with people across AWS to help us deliver the highest standards for safety and security while providing seemingly infinite capacity at the lowest possible cost for our customers. And you'll experience

Data Team Lead 2 Locations posted: Posted 17 Days Ago

Staff Backline Engineer - Data & AI Dallas, Texas; San Francisco, California; Vancouver, Canada P-1381 At Databricks, we are passionate about enabling Data & AI teams to solve the world's toughest problems - from making the next mode of transportation a reality to accelerating the development of medical breakthroughs. We do this by building and running the world's best data and AI infrastructure platform so our customers can use deep data insights to improve their business. Founded by engineers, we leap at every opportunity to tackle technical challenges, from designing next-gen UI/UX for data interaction to scaling our services and infrastructure across millions of virtual machines. And we're only getting started. About the Team: The Backline Engineering Team serves as the critical bridge between Frontline Support and Engineering. We handle complex technical issues and escalations across the Data and AI ecosystem. With a strong focus on customer success, we are committed to delivering exceptional customer satisfaction by providing deep technical expertise, proactive issue resolution, and continuous platform improvements. We emphasise automation and tooling to enhance troubleshooting efficiency, reduce manual efforts, and improve the overall supportability of the platform and the health of our products. By developing smart solutions and streamlining workflows, we drive operational excellence and ensure a delightful experience for both customers and internal teams. What your impact will be: - Deep Dive Troubleshooting : Conduct deep-dive forensics into Spark core internals and the broader Databricks Data and AI ecosystem to resolve high-priority architectural failures and complex system anomalies.

Cloud Services Engineer Cloud Services Engineer

Facility Manager (Night Shift) Sparks, Nevada, USA Join our dynamic team and become a critical technical leader who ensures the heartbeat of our global infrastructure remains strong and resilient. In this pivotal role, you'll be the guardian of our data center operations, driving reliability and performance while optimizing operational efficiency across complex technical environments. Data Center Facility Manger role acts as Amazon's front line when it comes to hands-on electrical and mechanical equipment troubleshooting. They will maintain, operate, and troubleshoot mission-critical data center facility equipment including electrical support equipment such as stand-by diesel generators and related fuel systems, 3 phase electrical systems that include but not limited to switchgear, UPS units, PDUs, and wet cell batteries and associated systems. Mechanical equipment includes CRAC units, centrifugal chillers, cooling towers/water chemical systems, air handlers and associated systems, pumps, and motors. Additional support equipment is included in the scope of the role which includes fire suppression systems, building automation systems, and general facilities equipment. AWS Infrastructure Services owns the design, planning, delivery, and operation of all AWS global infrastructure. In other words, we're the people who keep the cloud running. We support all AWS data centers and all the servers, storage, networking, power, and cooling equipment that ensure our customers have continual access to the innovation they rely on. We work on the most challenging problems, with thousands of variables impacting the supply chain - and we're looking for talented people who want to help. You'll join a diverse team of software, hardware,

Principal Applications Engineer - Remote Field Office - Texas posted: Posted 30+ Days Ago

Senior Applied Scientist, FinTelligence Bellevue, Washington, USA At Amazon's FinTech organization, we are building AI systems that process hundreds of millions of financial transactions, turn complex documents into actionable intelligence, and power autonomous agents that learn from every customer interaction. We are looking for a Senior Applied Scientist to lead the development of generative AI applications that change how finance teams work, tackling problems at the intersection of large language models, multi-agent systems, and real-world financial operations. Key job responsibilities What You'll Work On - Building AI systems that finance teams trust enough to rely on without manual review, where precision isn't a nice-to-have, it's a compliance requirement - Designing agents that learn from user corrections and get measurably better with every interaction, not just at the next model release - Solving inference at massive scale using tiered model architectures, intelligent routing, and small language models that deliver production-grade accuracy at a fraction of frontier model cost - Developing evaluation frameworks that catch quality regressions before customers do and gate every model change before it ships Who Thrives Here - You're someone who cares as much about shipping as about research. - You've built models that run in production, not just in notebooks. - You're comfortable working across the full stack, from model architecture to deployment to measuring whether the customer's workflow actually changed. - You operate well in cross-functional settings where science, engineering, and business teams inform each other continuously. - You'd rather solve a hard real-world problem than

Senior Forward Deployed Engineer - Databricks Location not specified

Talent Acquisition Manager New York Join Pigment: The AI Platform Redefining Business Planning Pigment is the AI-powered business planning and performance management platform built for agility and scale. We connect people, data, and processes in one intuitive, feature-rich solution, empowering every team-from Finance to HR-to build, adapt, and align strategic plans in real time. Founded in 2019, Pigment is one of the fastest-growing SaaS companies globally. Industry leaders like Unilever, Snowflake, Siemens, and DPD use Pigment daily to make more informed decisions and confidently navigate any scenario. With a team of 600+ across Paris, London, New York, Toronto, San Francisco and Austin, we've raised nearly $400M from top-tier investors and were named a Visionary in the 2024 Gartner® Magic Quadrant™ for Financial Planning Software. At Pigment, we take smart risks, celebrate bold ideas, and challenge the status quo-all while working as one team. If you're driven by innovation and ready to make an impact at scale, we'd love to hear from you. Join Pigment: The AI Platform Redefining Business Planning Pigment is the AI-powered business planning and performance management platform built for agility and scale. We connect people, data, and processes in one intuitive, feature-rich solution, empowering every team-from Finance to HR-to build, adapt, and align strategic plans in real time. Founded in 2019, Pigment is one of the fastest-growing SaaS companies globally. Industry leaders like Unilever, Snowflake, Siemens, and DPD use Pigment daily to make more informed decisions and confidently navigate any scenario. With a team of 600+ across