We are looking for an AI & Data Engineer who has strong foundational expertise in data engineering and has extended that into building intelligent AI systems.

Company: Quest Global

Role: Senior Engineer - Data & AI

Location: Bengaluru, Karnataka, India

Experience:

Strong proficiency in Python for data pipeline development, ETL/ELT automation, and data processing
Advanced SQL — window functions, CTEs, query optimisation, and performance tuning on large-scale datasets
Hands-on experience with dbt for data transformation, modelling, incremental strategies, testing, and documentation
Experience with Apache Airflow for orchestrating multi-step workflows — DAG design, retry logic, dependency management, and SLA monitoring
Proficiency with PySpark and Databricks for distributed data processing, including Delta Lake and batch/streaming workloads
Experience with cloud data warehouses — Snowflake, BigQuery, or Redshift — including schema design, partitioning, clustering, RBAC, and query optimisation
Solid understanding of data modelling paradigms: dimensional modelling (star/snowflake schema), SCDs, and medallion/lakehouse architecture
Familiarity with streaming ingestion using Apache Kafka or equivalent event-driven platforms
Experience with data quality frameworks, schema validation, and pipeline observability
Experience building production-grade RAG (Retrieval-Augmented Generation) pipelines with hybrid retrieval — dense vector search combined with structured or graph-based retrieval
Proficiency with LangChain, LangGraph, or equivalent frameworks for LLM orchestration and agentic pipeline design
Working knowledge of vector databases — Pinecone, Qdrant, Weaviate, or equivalent — including embedding model selection and indexing strategies
Experience integrating LLM APIs (OpenAI, Azure OpenAI, Anthropic) into production data and AI pipelines
Understanding of agentic system patterns: MCP, A2A frameworks, tool use, memory, and multi-step reasoning
Hands-on experience deploying data and AI systems on Azure or AWS — including AKS, Azure Data Factory, S3, Glue, or equivalent managed services
Experience with Docker, Kubernetes, and CI/CD pipelines for reliable, reproducible deployments
Familiarity with Redis for caching and FastAPI for building lightweight API service layers

Key Skills:

Python
PySpark
Databricks
dbt
Apache Airflow
Snowflake
BigQuery
Redshift
Apache Kafka
LangChain
LangGraph
Pinecone
Qdrant
Weaviate
OpenAI
Azure OpenAI
Anthropic
Docker
Kubernetes
Redis
FastAPI

Qualification:

Bachelor's or Master's degree in Computer Science, Data Engineering, Information Systems, or a related technical discipline
Demonstrable experience in a data engineering or AI engineering role with progressively increasing scope and ownership
Portfolio of delivered work — production pipelines, warehouse implementations, RAG systems, or AI applications — is strongly preferred over credentials alone
dbt Analytics Engineering Certification is a strong plus
Cloud certifications — Google Professional Data Engineer, Azure DP-203, or AWS Data Analytics Specialty — are valued but not mandatory

Role Focus:

Design, build, and maintain end-to-end batch and real-time data pipelines
Develop and manage ELT/ETL workflows
Build and maintain cloud data warehouse and lakehouse architectures
Implement dimensional models, SCD strategies, and incremental load patterns
Integrate data from diverse enterprise sources
Manage data quality, lineage tracking, and metadata documentation
Optimise query performance, partitioning strategies, and materialization approaches
Architect and build production-grade RAG pipelines
Build distributed agent-based systems
Design LLM evaluation and validation layers
Develop LLM-based automation workflows, recommendation systems, and document intelligence solutions
Build explainability layers
Deploy data and AI systems on Kubernetes with robust CI/CD pipelines
Design FastAPI microservices
Apply observability and monitoring best practices

Nice to have:

Knowledge of semantic web standards: RDF, SPARQL, OWL/TTL, JSON-LD
Experience with knowledge graph platforms — Neo4j, Apache Jena — including Cypher querying and ontology design
Exposure to ML pipelines, object detection, NLP models, or structured data extraction from documents
Familiarity with data cataloguing and lineage tools: Apache Atlas, DataHub, OpenMetadata, or similar
Open-source contributions, technical blog posts, or documented project case studies

Other:

We wholeheartedly believe in the diversity of thought that comes with fostering a culture rooted in respect, where everyone belongs, is valued, and feels inspired to share their ideas
We know embracing our unique differences makes us better, and that solving the worlds hardest engineering problems requires diverse ideas, perspectives, and backgrounds
Questians are driven by hunger, humility, and aspiration
Strong communication skills: able to write clear technical documentation and participate in architecture reviews and client-facing discussions

Senior Engineer - Data & AI

Skills Required

Description