Spark jobs

Battery Electrical Power Distribution Engineer India - Bangalore posted: Posted 30+ Days Ago

Americas Business Process Re-Engineering Data Engineer Austin, United States of America Apple is where extraordinary people do their best work. If making a real impact excites you, a career here might be your dream - just be prepared to dream big. Apple's growing supply chain complexity demands innovative approaches beyond traditional data engineering. You'll join a team designing and building modern, scalable data infrastructure that powers analytics, machine learning, and AI-driven decision-making across Operations. You're passionate about building reliable data systems, staying ahead of technology trends, and thrive navigating ambiguity in a fast-paced environment. If this sounds like you, we'd love to talk. Engage with business and analytics teams to deeply understand data needs and translate requirements into robust, scalable engineering solutions that directly impact Operations decisions Design and implement end-to-end data pipelines and architectures from ingestion and transformation to delivery across batch and real-time streaming workloads Build and maintain high-quality data models (dimensional, relational, or knowledge graph-based) using modern transformation frameworks such as dbt, powering analytics and AIML use cases at scale Architect and operate data workflows using orchestration tools (e.g., Apache Airflow, etc) with built-in monitoring, alerting, and SLA management Implement data observability, lineage tracking, and validation frameworks to uphold data integrity and trustworthiness across the platform Collaborate with Data Scientists, ML Engineers, Software Engineers and Analysts to operationalize models and ensure data infrastructure supports production AIML workflows Partner with infrastructure and platform teams to manage cloud-native data environments (Snowflake, Spark, Delta Lake / Apache Iceberg) with a

Principal Software Engineer, AI Platform Engineering El Segundo, CA ABOUT SAVIYNT Saviynt is a leader in identity security, delivering an AI-powered platform that governs and secures access to applications, data, and business processes for global enterprises and government institutions. Built for the AI era, Saviynt helps organizations move faster - securely and compliantly. ABOUT THE ROLE You set the architectural direction for how training data flows, evolves, and is governed across the AI Platform. You define the standards ML engineers and scientists build on, and ensure every training signal is tenant-isolated, PII-free, and traceable from source to model. WHAT YOU'LL OWN AI Data Lake on GCS: bucket layout, raw → silver → gold tier separation, CMEK encryption, lifecycle rules Batch pipelines: Spark on Dataproc for TB-scale feature backfills, Iceberg compaction, and daily S3→GCS incremental sync Streaming pipelines: Apache Beam on Dataflow for sub-5-min CDC ingestion with exactly-once semantics and PII assertion gates Schema registry: Avro / Protobuf schema versioning, compatibility modes, and migration playbooks for safe schema evolution Orchestration: Flyte as primary DAG layer - task authoring standards, domain isolation, retry policies, DataCatalog memoization; evaluate Kubeflow Pipelines where relevant Multi-tenancy: strict per-tenant GCS prefix isolation, quota policies, and cross-tenant contamination validation Data Anonymizer and Data Labeler microservices: strip PII and attach ML labels before signals leave each customer environment Feature store: Feast offline (GCS Parquet) and online (Redis) with point-in-time correctness and < 0.1% consistency SLA Vector database: operate Pgvector (Cloud SQL) for POC and Qdrant on GKE for production-scale

Electrical Engineer - Solar EPC

Senior Applied Scientist , Buyer Risk Prevention (BRP) Bengaluru, Karnataka, IND Do you want to lead the development of advanced machine learning systems that protect millions of customers and power a trusted global eCommerce experience? Are you passionate about modeling terabytes of data, solving highly ambiguous fraud and risk challenges, and driving step-change improvements through scientific innovation? If so, the Amazon Buyer Risk Prevention (BRP) Machine Learning team may be the right place for you. We are seeking a Senior Applied Scientist to define and drive the scientific direction of large-scale risk management systems that safeguard millions of transactions every day. In this role, you will lead the design and deployment of advanced machine learning solutions, influence cross-team technical strategy, and leverage emerging technologies-including Generative AI and LLMs-to build next-generation risk prevention platforms. Key job responsibilities Lead the end-to-end scientific strategy for large-scale fraud and risk modeling initiatives Define problem statements, success metrics, and long-term modeling roadmaps in partnership with business and engineering leaders Design, develop, and deploy highly scalable machine learning systems in real-time production environments Drive innovation using advanced ML, deep learning, and GenAI/LLM technologies to automate and transform risk evaluation Influence system architecture and partner with engineering teams to ensure robust, scalable implementations Establish best practices for experimentation, model validation, monitoring, and lifecycle management Mentor and raise the technical bar for junior scientists through reviews, technical guidance, and thought leadership Communicate complex scientific insights clearly to senior leadership and cross-functional stakeholders Identify emerging scientific trends and translate

Lead Systems Services Engineer GB-SO-NAILSEA-2 HIGH STREET posted: Posted 30+ Days Ago

Data Architect Atlanta, Georgia Overview The Data Architect designs, develops and maintains integration solutions using the MuleSoft Anypoint Platform. They create and manage APIs, ensure seamless data flow between systems, and provide technical support for integration solutions. The Data Architect will work closely with the IT team and business stakeholders to develop integration solutions that support the organization's goals and objectives, develop scalable data and API solutions, ensure seamless data flow across systems, and support the Data Analytics platform to drive data-driven decision-making. Primary Duties & Responsibilities Maintain a deep understanding of Microsoft data solutions, data modeling, and database design, ensuring data systems are scalable, secure, and aligned with business goals. Develop and implement cohesive Data Fabric solutions that integrate disparate data sources across cloud and on-premises environments. Design and manage data transformation and processing workflows using tools like Fabric Spark, PySpark, and SQL. Ensure seamless data flow across systems and platforms, supporting real-time analytics, business intelligence, and operational reporting. Design, develop, and enhance integration solutions, APIs, and flows using the MuleSoft Anypoint Platform, ensuring reliability, scalability, and performance. Create, test, and manage APIs and integration flows that enable seamless connectivity between internal systems, third-party applications, and cloud-based services. Perform unit testing, quality assurance, debugging, and ongoing troubleshooting of integrations for all databases and applications across all environments. Develop solutions and proof of concepts (PoCs) based on business and project needs to validate integration approaches. Implement and enforce best practices, reusable patterns, and standards in integration design; manage API lifecycle

BI Engineer, Prime Data Engineering and Anaytics Bengaluru, Karnataka, IND Join Amazon's Prime Data Engineering & Analytics (PDEA) team as a Business Intelligence Engineer (BIE) to revolutionize how we understand, measure and serve our global Prime customer base. You'll architect and build enterprise-level data analytics solutions, including scalable ETL pipelines, automated AI powered reporting systems, and self-service analytics platforms that process petabytes of data daily across multiple dimensions. Working with technologies like AWS, Apache Kafka, Spark, and modern AI/ML tools, you'll create data analysis products that power critical decision-making across Amazon. As a BIE, you'll be instrumental in defining and implementing key business metrics, building efficient dimensional data models, and leveraging advanced AI technologies like QuickSuite and ML-powered analytics. You'll design intuitive self-service platforms that enable business users to independently explore data, discover insights, and create automated reports. Beyond technical implementation, you'll lead end-to-end project execution, manage worldwide Prime core services data infrastructure, and establish key relationships across Amazon's business units. This role offers the opportunity to provide technical mentorship, drive architectural decisions, and shape the future of Amazon Prime's data ecosystem, directly impacting millions of customers globally. Key job responsibilities Responsibilities 1. Own design and execution of end to end projects 2. Own managing WW Prime data infrastructure 3. Establish key relationships which span Amazon business units and Business Intelligence teams 4. Implement standardized, automated operational and quality control processes to deliver accurate and timely data and reporting to meet or exceed SLAs Basic Qualifications: - 3+ years of

Electrical Engineer (Operations) 2 Locations posted: Posted 30+ Days Ago

Data Architecture – Advisor I Berkeley Heights, New Jersey posted: Posted Today

Lead Data Scientist Bengaluru, Karnataka, India The Company CommerceIQ is building the AI platform that runs commerce for the world's largest brands. We are not selling AI demos. We are shipping AI agents for content, media, and sales into the workflows of the Fortune 100 every week. 2,200+ Customers 10 of Top 12 CPG Companies 900+ Retailers Connected $200M+ Raised Customers include Coca-Cola, Nestlé, Colgate-Palmolive, Mondelez, Samsung, and Kellogg's. Backed by SoftBank, Insight Partners, and Madrona. Headquartered in Mountain View with teams across the US, India, Canada, and the UK. Pre-IPO. Technical Expertise - Strong background in machine learning, deep learning, and NLP, with proven experience in training and fine-tuning large-scale models (LLMs, transformers, diffusion models, etc.). - Hands-on expertise with Parameter-Efficient Fine-Tuning (PEFT) approaches such as LoRA, prefix tuning, adapters, and quantization-aware training. - Proficiency in PyTorch, TensorFlow, Hugging Face ecosystem and good to have distributed training frameworks (e.g., DeepSpeed, PyTorch Lightning, Ray). - Basic understanding of MLOps best practices, including experiment tracking, model versioning, CI/CD for ML pipelines, and deployment in production environments. - Experience working with large datasets, feature engineering, and data pipelines, leveraging tools such as Spark, Databricks, or cloud-native ML services (AWS Sagemaker, GCP Vertex AI or Azure ML). - Knowledge of GPU/TPU optimization, mixed precision training, and scaling ML workloads on cloud or HPC environments. - Applied Problem-Solving Mandatory skill - - Demonstrated success in adapting foundation models to domain-specific applications through fine-tuning or transfer learning.Mandatory skill - - Strong ability to design, evaluate,

Staff Software Engineer, ISPM - India Bangalore, India - Remote All roles at JumpCloud® are Remote unless otherwise specified in the Job Description. About JumpCloud® JumpCloud® is the AI-powered unified IT management platform designed to secure the modern workforce. By consolidating identity, device, and access management, JumpCloud provides intelligent, secure IT that scales from human users to autonomous AI agents. We help organizations around the globe eliminate complexity and turn AI risk into an optimized advantage, ensuring the right people and agents have secure access to the right resources at all times. JumpCloud is Intelligent, Secure IT. The Problem Statement: The Invisible Identity Perimeter The Opportunity: Founding the ISPM Platform We're looking for: Bonus Points:

Senior Data Center Engineer II Seattle Senior Data Center Engineer II

Senior Software Development Engineer, AI Data Platform United States, US Oracle's Forward Deployed Engineer (FDE) team is hiring a Senior Software Development Engineer - AI Data Platform to help global customers unlock the full potential of their data. You will provide expert architectural guidance focused on designing, optimizing, and scaling modern AI/ML-centric data platforms. As a key member of Oracle's Analytics and AI Service Excellence organization, you will work closely with enterprise customers, product management, and engineering to ensure seamless adoption of Oracle AI Data Platform, Gen AI services, and associated cloud technologies. Key Responsibilities: Strong data engineering, HPC, and data science experience. Spark, PySpark, Delta Lake, Parquet, Feature Extraction, MLOps, Flink, with a deep understanding of the distributed systems techniques that make these services work. Full stack development experience. This includes web app development, RESTful APIs, SSE, all the way to deploying your solution. Design, implement, and maintain scalable software components and services that support AI/ML workloads. Build APIs, SDKs, and automation frameworks to streamline the adoption of Oracle AI Data Platform and Gen AI services. Optimize performance, scalability, and reliability of distributed data/AI systems. Collaborate with cross-functional teams (engineering, product, and field) to solve complex technical challenges. Participate in code reviews, testing, and CI/CD to ensure high-quality deliverables. Document technical designs and contribute to knowledge-sharing (e.g., blogs, internal docs, demos). Continuously explore new tools, frameworks, and best practices in AI, cloud, and data engineering. Primary Skills: Experience with LLMs and agentic frameworks (e.g., MCP, LangChain, CrewAI, Semantic Kernel).

Customer Support Specialist

Sr. QA Engineer Denver, CO (Greenwood Village) Job Description The STARZ Data Products & Engineering team is seeking a Senior QA Data Engineer who is passionate about ensuring data quality, integrity, and trust at scale. In this role, you will design and build data testing frameworks-primarily using Snowflake and supporting scripting pipelines-to ensure our ETL/ELT processes deliver timely, accurate, and reliable data products across the organization. You will work across a diverse data ecosystem, validating data ingestion and transformation from database systems, vendor-supplied files, API endpoints, and AWS S3. Partnering closely with Data Engineering and Business Intelligence teams, you will embed data quality checks early in the development lifecycle and help establish consistent QA standards that scale across teams. Our data pipelines support the entire organization, making data integrity a critical and highly visible responsibility. Strong experience with Snowflake or equivalent cloud data platforms is preferred. Responsibilities Design, develop, test, and maintain automation-first, scalable, and reliable data QA and validation processes Implement testing practices, integrating data quality checks early in pipeline design and development Validate data accuracy, completeness, and consistency across complex, high-volume datasets from multiple internal and external sources Analyze and interpret entity-relationship diagrams, relational models, and dimensional (star/snowflake) schemas Write, review, and optimize complex SQL queries in Snowflake (or equivalent platforms) Create and execute detailed test cases based on business and technical requirements, including interpretation of data mapping documents Define and document test strategies, test plans, and test summary reports Partner closely with Data Engineers and BI teams