Redpanda Data jobs

Data Engineer II, Zappos Analytics New York, New York, USA As a Data Engineer at Zappos, you will play a crucial role in designing, developing, and maintaining our data infrastructure. You will work closely with cross-functional teams to ensure that data is collected, processed, and made available for analysis, reporting, and machine learning applications. Your expertise in data pipelines, ETL processes, and data warehousing will be instrumental in shaping our data ecosystem. The right candidate will be excited by the opportunity to redesign our company's data architecture to support our next generation of data initiatives. Key job responsibilities • Design, build, and maintain robust data pipelines to acquire, process, and store data from various sources such as databases, APIs, and external data providers. • Develop and optimize ETL (Extract, Transform, Load) processes to clean, enrich, and structure raw data into a usable format for analysis and reporting. • Implement and manage data warehousing solutions to ensure efficient data storage, retrieval, and query performance. • Establish data quality standards, perform data validation, and proactively identify and address data quality issues. • Optimize data pipelines and storage solutions to handle large volumes of data while maintaining high performance and reliability. • Ensure data privacy and security by implementing access controls, encryption, and compliance with data protection regulations. • Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and provide the necessary data infrastructure to support their needs. • Maintain comprehensive documentation for data pipelines, data models, and processes to

Senior Data Engineer - Ag & Trading State/Province1 | Job Posting Country IN India | State/Province Karnataka | Job Posting City Bangalore | City Zip Code 560087 Job Purpose and Impact The Senior Professional, Data Engineering job designs, builds and maintains complex data systems that enable data analysis and reporting. With minimal supervision, this job ensures that large sets of data are efficiently processed and made accessible for decision making. Key Accountabilities DATA INFRASTRUCTURE: Prepares data infrastructure to support the efficient storage and retrieval of data. DATA FORMATS: Examines and resolves appropriate data formats to improve data usability and accessibility across the organization. DATA & ANALYTICAL SOLUTIONS: Develops complex data products and solutions using advanced engineering and cloud based technologies, ensuring they are designed and built to be scalable, sustainable and robust. DATA PIPELINES: Develops and maintains streaming and batch data pipelines that facilitate the seamless ingestion of data from various data sources, transform the data into information and move to data stores like data lake, data warehouse and others. DATA SYSTEMS: Reviews existing data systems and architectures to identify areas for improvement and optimization. STAKEHOLDER MANAGEMENT: Collaborates with multi-functional data and advanced analytic teams as well as with business teams to gain requirements and ensure that data solutions meet the functional and non-functional needs of various partners. DATA FRAMEWORKS: Builds complex prototypes to test new concepts and implements data engineering frameworks and architectures that impr

Manager, Data Engineering State/Province1 | Job Posting Country IN India | State/Province Karnataka | Job Posting City Bangalore | City Zip Code 560087 Job Purpose and Impact The Supervisor II, Data Engineering job sets goals and objectives for the achievement of operational results for the team responsible for designing, building and maintaining robust data systems that enable data analysis and reporting. This job leads implementing the end to end process to ensure that large sets of data are efficiently processed and made accessible for decision making. Key Accountabilities DATA & ANALYTICAL SOLUTIONS: Oversees the development of data products and solutions using big data and cloud based technologies, ensuring they are designed and built to be scalable, sustainable and robust. DATA PIPELINES: Develops and monitors streaming and batch data pipelines that facilitate the seamless ingestion of data from various data sources, transform the data into information and move to data stores like data lake, data warehouse and others. DATA SYSTEMS: Reviews existing data systems and architectures to lead identification of areas for improvement and optimization. DATA INFRASTRUCTURE: Oversees the preparation of data infrastructure to drive the efficient storage and retrieval of data. DATA FORMATS: Reviews and resolves appropriate data formats to improve data usability and accessibility across the organization. STAKEHOLDER MANAGEMENT: Partners collaboratively with multi-functional data and advanced analytic teams to capture requirements and ensure that data solutions meet the functional and non-functional needs of various partners. DATA FR

Senior Data Engineer San Francisco About the Role Together AI is looking for a Senior Data Engineer to help define, build, and operate the data infrastructure that handles millions of events every day to power Together's mission-critical systems. As a Senior Data Engineer, you will work with our Data and Commerce engineering team to scale the data processing components of Together's usage-based billing system, real-time customer-facing analytics product, and internal business intelligence tools. You will work across both cloud-native services and globally distributed data centers. If you thrive in fast-paced environments and have a passion for defining and building early-stage data platforms for a rapidly scaling and data-intensive company, this is for you. Requirements - 5+ years of demonstrated experience in building large scale, fault tolerant, distributed data platforms, stream processing pipelines, ETLs, etc - Expert-level skills in designing, building, and operating stream processing pipelines with services like AWS Kinesis, Apache Kafka, or Redpanda - Expert-level knowledge of building real-time customer facing analytics systems using services like AWS TimeStream or Clickhouse - Proficiency in writing and maintaining Infrastructure as Code (IaC) using tools like Terraform, AWS CDK, or Pulumi - Proficiency in version control practices and integrating IaC with CI/CD pipelines. - Proficiency in implementing and managing GitOps workflows with tools such as ArgoCD, Github Actions, TeamCity, or similar - Proficiency in one or more of Golang, Rust, Python, Java, or TypeScript - Bachelor's or Master's degree in Computer Science, Computer Engineering, or a related technical field, or equivalent practical

Actuarial Data Engineer - Centralized Data Solutions Actuarial Data Engineer - Centralized Data Solutions

Portfolio Data Analyst - Portfolio Data Feeds Poland Portfolio Data Analyst - Portfolio Data Feeds

Consultant, Data Management & Governance - Bioindustrial State/Province1 | Job Posting Country IN India | State/Province Haryana|Karnataka | Job Posting City Gurgaon|Bangalore | City Zip Code 122002|560087 Job Purpose and Impact The Consultant, Data Management & Governance - Bioindustrial job ensures the accuracy, security and effective use of the Bioindustrial business's core data in Asia Pacific operations. With limited supervision, this role supports the development and implementation of global data governance processes, handles data quality in region, and ensures compliance with regulations. This job also helps design data architecture, cultivates data literacy and partners with technology business teams to support better decision making, operational efficiency and strategic advantage for the organization. Key Accountabilities STRATEGIC PLANNING: Implements and supports the development of data management and governance strategies aligned with organizational goals. DATA STORAGE, INTEGRATION & CONSUMPTION: Implements and maintains standards for data storage, data in transit and data use in both raw and insight formats. DATA STANDARDS: Reviews and documents data standards to ensure consistency and quality across all data domains, and that policies are followed. DATA QUALITY: Helps ensure the accuracy, consistency and reliability of data throughout its lifecycle by handling data quality standards, monitoring data quality metrics and implementing corrective processes while supporting data quality audits as needed to ensure data standards are met. . POLICY DEVELOPMENT & GOVERNANCE: Implements data governance policies that help maintain data integrity ensuring data practices align

Senior Data Engineer Job Location (US/Canada) MI-Southfield, Main Office | Job Location (UK) Career Site Header Job Summary The Senior Data Engineer is a key resource in building and maintaining a modern enterprise data lake and data warehouse in Sun's Azure data platform. They will facilitate modern data platform processes and procedures for Sun and will establish robust data governance and management best practices. The Sr. Data Engineer works under the leadership of the Manager, Data Architecture to ensure core enterprise data objectives are accounted for in the design and implementation of the data platform infrastructure. Further, they will liaison with key business leadership and stakeholders to ensure that the design, validation, and implementation of the new data platform infrastructure meets the needs of Sun's business. Job Duties Builds and maintains Azure data platform for Sun including enterprise data lake and data warehouse in alignment with Sun's strategic objectives and organizational goals (Essential) Incorporates Azure data platform best practices into Sun's Azure data platform (Essential) Develops policies and implements mechanisms for data ingestion into Azure data platform from various sources (Essential) Supports & mentors the data team on best practices & methodologies (Essential) Leads navigating challenging or technical situations to find best business outcomes (Essential) Works with Microsoft support to ensure an ongoing stable data system. Configures, validates, and implements various Azure tools such as but not limited to Databricks, Data F

Senior Data Engineer, Investments Technology (Enterprise Data Management) Boston, MA, US Description Note: This role has a required hybrid/Boston work arrangement. At Liberty Mutual Investments Technology, we are seeking a highly skilled Senior Data Engineer to join our Financial - Enterprise Data Management team. This role is focused on developing and supporting enterprise data pipelines and data management solutions for market, reference and financial data in the EDM (Enterprise Data Management) platform. Previously called Markit EDM, it is currently hosted externally in a SaaS model by Gresham Technologies and runs on Microsoft SQL Server. You will play a critical role in developing robust data pipelines, new features and enhancements in the enterprise data management system that allow our Data Integrity Team to reconcile market and reference data. This is an opportunity to utilize your investment knowledge and work closely with a business team on a critical data function. Key Responsibilities Design, develop, maintain, and support data management workflows, data models, and integration processes. Configure and support data ingestion, validation, enrichment, matching, and distribution processes within the enterprise data management system. Support reporting, analytics, and downstream data integration processes connected to the EDM platform. Build and maintain scalable ETL/ELT pipelines for market, reference, and transactional data. Integrate data from internal and external vendors, APIs, flat files, and enterprise applications. Implement data quality checks, r

Senior Data Engineer, WW Standardization & Automation Luxembourg, LUX Are you passionate about standardizing data platforms and automating data engineering to drive analytics and reporting? Do you excel in dynamic, fast-paced environments and find joy in converting data into actionable insights? Are you adept at implementing data governance practices and defining data access and quality standards? If you thrive in innovation and can deliver scalable Data Engineering Solutions, then the Worldwide Operations Finance Standardization & Automation (SnA) team has an exciting opportunity for you! We are seeking a customer-centric Senior Data Engineer to establish a reliable and accessible data platform, ensuring Operations Finance customers have complete trust in the data, technology, and tools to make data-driven business decisions. As a Senior Data Engineer in WWSNA, you will be working in one of the world's largest and most complex data warehouse environments. You will design, implement and support scalable data infrastructure solutions and implement complex data models for Amazon Inbound & Outbound Transportation business. As a Sr. DE you will create solutions to integrate with multi heterogeneous data sources, aggregate and retrieve data in a fast and safe mode, curate data that can be used in reporting, analysis, machine learning models and ad-hoc data requests. You should have excellent business and communication skills to be able to work with business owners, Product teams and Tech leaders to gather infrastructure requirements, design data infrastructure, build up data pipelines and data-sets to meet business needs. You will be responsible for designing, developing, and

Senior Software Engineer, Data Engineering Remote, USA Omada Health is on a mission to bend the curve of chronic disease. Job overview: We are dedicated to leveraging data to drive strategic decision-making and operational efficiency. Our team is passionate about harnessing the power of data to solve complex problems and deliver impactful insights. We are seeking a highly skilled and motivated Data Engineer to join our team. The ideal candidate will be responsible for designing, building, and maintaining robust data architectures and engineering data models and pipelines. This role will play a critical part in ensuring the integrity, scalability, and performance of our data processing and products. Key Responsibilities: - Data Architecture: Design, develop, and implement scalable, secure, and efficient data solutions that meet the needs of the organization. - Data Modeling: Create and maintain logical and physical data models to support business intelligence, analytics, and reporting requirements. - Pipeline Engineering: Design, build, and optimize ETL (Extract, Transform, Load) processes and data pipelines to ensure smooth and efficient data flow from various sources. - Data Integration: Integrate diverse data sources, including APIs, databases, and third-party data, into a unified data platform. - Performance Optimization: Monitor and optimize the performance of data systems and pipelines to ensure low latency and high throughput. - Data Quality and Governance: Implement data quality checks, validation processes, and governance frameworks to ensure the accuracy and reliability of data. - Collaboration: Partner closely with data scientists, analysts, and other stakeholders to understand data requirements and deliver

Sr. Data Engineer, WW Standardization & Automation Bellevue, Washington, USA Are you passionate about standardizing data platforms and automating data engineering to drive analytics and reporting? Do you excel in dynamic, fast-paced environments and find joy in converting data into actionable insights? Are you adept at implementing data governance practices and defining data access and quality standards? If you thrive in innovation and can deliver scalable Data Engineering Solutions, then the Worldwide Operations Finance Standardization & Automation (SnA) team has an exciting opportunity for you! We are seeking a customer-centric Senior Data Engineer to establish a reliable and accessible data platform, ensuring Operations Finance customers have complete trust in the data, technology, and tools to make data-driven business decisions. As a Senior Data Engineer in WWSNA, you will be working in one of the world's largest and most complex data warehouse environments. You will design, implement and support scalable data infrastructure solutions and implement complex data models for Amazon Inbound & Outbound Transportation business. As a Sr. DE you will create solutions to integrate with multi heterogeneous data sources, aggregate and retrieve data in a fast and safe mode, curate data that can be used in reporting, analysis, machine learning models and ad-hoc data requests. You should have excellent business and communication skills to be able to work with business owners, Product teams and Tech leaders to gather infrastructure requirements, design data infrastructure, build up data pipelines and data-sets to meet business needs. You will be responsible for designing, developing,

GDB/XD - Senior Specialist Data Governance (m/f/d) Country / Location India Senior Specialist Data Governance (m/f/d) Location: Hyderabad Unit: GDB/XD Contract: Permanent Job Grade: TBD Why this role matters Within the Data & AI Office, we implement BASF's Data & AI Governance Framework. Your role is focused on implementing the data strategy and enabling efficient, secure and compliant use of data across the company. Our goal is to make data easy to find, easy to use and safe - while supporting business value, innovation and AI use cases. You play a key part in federated data governance: governance is coordinated centrally but implemented close to the business. You will help embed data governance into daily work without slowing teams down. Your responsibilities You will facilitate and operationalize data governance across BASF, with a strong focus on collaboration and value creation: • Coordinate data governance activities across different stakeholders • Support the execution of BASF-wide data principles and governance guidelines • Define, implement and continuously improve data roles and responsibilities (e.g. Data Owners, Data Product Owners, Data Domain Owners) • Promote and support the nomination and onboarding of data roles in business • Develop and share processes, templates and best practices for data governance across the organization • Create transparency on data governance maturity, risks and improvement areas • Facilitate data usage with clear and secure agreements integrated into the Data Catalog • Identify, assess, and manage data-relate

AWS Data Engineer Hyderabad, Telangana, India This is a highly influential member of our data team. Working within our Data group, you will be responsible for leading creation of data architecture that produces our data assets to enable our data platform.

Data Automation Engineer EG-Banha, Egypt (EI Corniche Street) Position: Data Automation Engineer Job Description: As a Data Automation Engineer, you will play a crucial role in developing and maintaining automated data processing systems. You will work closely with data analysts, and other stakeholders to design, implement, and optimize data pipelines, ensuring the efficient and accurate flow of data throughout the organization. What You'll Do: Develop and implement automated data pipelines for ingesting, processing, and storing structured and unstructured data. Integrate data from various sources in both real-time and batch mechanisms and perform transformations to ensure consistency, accuracy, and reliability. Implement data quality checks and monitoring mechanisms to identify and address issues proactively. Write code and scripts for automation of data processes using languages such as Python, Java, or Scala. Leverage ETL (Extract, Transform, Load) tools and frameworks [Apache spark, AWS Glue] for efficient data processing. Collaborate with other intelligent data teams. Ensure seamless data flow for machine learning and analytics applications. Implement monitoring systems to track data pipeline performance and identify areas for optimization. Continuously optimize data processes for efficiency and scalability. Document data processes, system architecture, and codebase for knowledge sharing and future reference. Keep documentation up-to-date with any changes or enhancements to the data infrastructure. Implement and adhere to data security and compliance standards. Collaborate with security teams to ensure data protection and privacy. You're a Perfect Fit If You Have: 1+ years of experience in data engineering, automation, or a related field. Strong understanding of data pipelines

Senior Data Quality Analyst Rumst, Belgium Leads, develops, and implements project(s) of varying complexity and size in assigned functional area. Partners with business stakeholders to ensure a successful project completion and organizational improved capability to data quality standards. Leads the quality and integrity of the data within the assigned functional area. Serves as an expert on data quality tools. Conducts data profiling for data. Develops Key Performance Indicators (KPIs) to measure data quality. Creates and executes action plans to resolve data quality issues. Maintains existing rules and develops new rules to support a robust data quality process. Escalates issues and critical conflicts in a timely fashion. Identifies and mitigates data quality risks. Reports data quality risks and KPI measurement statistics to data governance council. Manages metadata and data standards to ensure adherence, working closely with IT. Coordinates activities of data stewards located within the business, working closely with the business process owner. Supports business data analysis and proposal creation as a member of a data governance council. Guides the business to prepare data. Leads initial data exploration steps (binning, pivoting, summarizing and finding correlations, for example). Catalogues business attributes to enable data discovery. Coaches business to establish and enforce guidelines for data collection, integration and processes. Leads data and systems analysis as required to support data governance processes. Understands data governance roles and processes, and ensures that they are followed. Prepares and presents communications to leaders and stakeholders. College, university, or equivalent degree in statistics, information systems or related field

Sr Data Engineer- Data Platform & AI Enablement United States Sr Data Engineer- Data Platform & AI Enablement

Senior External Data Specialist - FSP Warsaw, Poland Senior External Data Specialist - Clinical Data Management / Vendor Data. The Senior External Data Specialist role is a great opportunity to support Clinical Data Management activities focused on e...

Senior Data Engineer - Data Bricks CHENNAI, India This position participates in the support of batch and real-time data pipelines utilizing various data analytics processing frameworks in support of data science practices for Marketing and Finance bu...

Forward Deployed Data Engineer (Data Platform & Solutions) San Jose, California Data Integrations at FloQast FloQast is building the indispensable home base for the modern accountant: an AI-powered platform where accounting work is connected, automated, and intelligently executed. As we expand our Data Platform and Data Studio capabilities, we are making it faster and easier to connect source systems, model data, and publish trusted information across FloQast applications. As a Forward Deployed Data Engineer (Data Platform & Solutions), you will own data integrations end to end. You will use FloQast's Data Platform and Data Studio to connect to source systems, extract data, create mappings, transform it into FloQast-ready models, validate the results, and publish it to FloQast applications. You will work cross-functionally with customers, partners, Implementation, Customer Success, Product, and Engineering to deliver successful launches and durable data solutions. This is a hands-on role for someone who can turn source data, business needs, and platform capabilities into working data flows. What You'll Do: What You'll Bring: Nice to have: Why You'll Love Working With Us:

Executive Director, Enterprise Data Management Waltham, MA, United States The Executive Director, Enterprise Data Management is responsible for leading and expanding enterprise data management capabilities that support all functions-with an initial focus on R&D and Commercial. This role will drive the continued development and execution of the company's enterprise data strategy, strengthen data governance, oversee data acquisition and master data management (MDM), scale data operations, and accelerate the delivery of data solutions that meet key business needs and support AI / analytics / reporting solutions. This role will lead several teams of data professionals, including an existing Commercial Data Management team, a new R&D Data Management team, and a new Enterprise Data Governance specialist. Depending on start date, this position will lead the hiring of new positions for R&D Data Management and Enterprise Data Governance. This role will work a weekly hybrid schedule in our Waltham office. This role will report to the Vice President, Data, Analytics & Insights. Partner with leaders across R&D, Commercial, Enterprise AI, IT, Information Security, and Program Management to understand key business needs and existing data management capabilities. Deliver data solutions that enhance and augment existing capabilities to accelerate scientific, operational, and commercial decision-making and execution. R&D data capabilities vary by sub-function. For example, Clinical has their own data management capabilities. Commercial uses centralized capabilities provided by the existing Commercial Data Management team. Continue the development of enterprise data management capabilities: Data Strategy: Lead the continued development and execution of the enterprise data strategy, ensuring

Lead Cloud Data Engineer - Multi-Cloud Data Platforms Irving, TX, United States The ideal candidate brings deep experience in modern cloud data platforms, distributed data processing, and secure data integration. Experience with data architecture and platform design in large ente...

Senior Data Analyst (SNOWFLAKE, SQL) Los Angeles, California; Charlotte, North Carolina SENIOR DATA ANALYST WHAT IS THE OPPORTUNITY? The Senior Data Analyst works within the Insights, Data Engineering & Analytics organization to meet the data pipeline strategies and objectives across the bank. The Senior Data Analyst will guide and lead end-to-end process, standards, and guidelines to ensure accurate and efficient data delivery of tasks and projects. WHAT WILL YOU DO? Lead data analysis via data discovery, exploration, investigation, and impact assessments. Apply best practices and illustrate understanding via data flows of systems and processes. Conduct and leads data profiling and data preparation by identifying gaps and qualifying data. Validates data by verifying against requirements and success criteria. Confirms understanding of requirements by modeling data with Entity Relationship diagrams and data flow diagrams Leads understanding by researching business processes and documenting inconsistent and incomplete data, value misinterpretations and misclassified data. Lead development/quality measurement, benchmarking and goal setting, performance measurement analysis Identify the appropriate data sources and map the data from source to target Create and manage the catalog and metadata of data related objects Guide integration across data sources and targets Perform one or more of the following: Actively mentor other colleagues by contributing experience and insight to the team Manage large programs & projects of significant complexity and risk. Independently (with minimal oversight) develop and maintain trusted advisor relationships, including at the senior leadership level and with external partners, that include guidance, as appropriate, for optimizing use of analytic capabilities

Senior Data Engineer Hyderabad - India Data Engineering at Blackbaud is responsible for ingestion, transformation and processing of data that powers Blackbaud Data Intelligence and Advanced analytics products . The data managed by the team supports data science, data enrichment, research and data analysis as well as making data operationally able to be consumed by Blackbaud's products and services. As a Senior Data Engineer, you will take responsibility for technical design , implement complex features and provide mentorship in the Data Engineering discipline of Infrastructure Engineering & Cloud Operations (IECO) . You build data intensive solutions that are highly available , scalable, reliable, secure, and cost-effective. You contribute to our common code repositories; help establish and enforce coding standards for data engineering teams. What will You be doing: Design, develop and operate high performance, large volume data structures for data-powered products and data science. Implement efficient, distributed and scalable pipelines and integrate data from multiple sources to create data products Implement design patterns that support data ingestion, data movement, transformation, aggregation, and much more. Collaborate with Product managers, Software engineers, and Data Scientists, and work towards achieving key results. Build high quality production level code, test and deploy data pipelines Undertake complex tasks that are large and diverse in scope and/or critical in nature Design and develop breakthrough products, services or technological advancements that expand our business Participate in code reviews, brainstorm design approaches with peers and mentor junior engineers Work in fast paced agile environment , p articipate in