Hi, I'm Sathiyamurthy.
I am a Staff Data Engineer and systems architect. I specialize in designing and scaling distributed data systems (using Spark and Kafka), engineering robust pipeline orchestration frameworks (Airflow), running self-hosted local infrastructure (Homelabs & Home Automation), and building next-gen generative AI applications (RAG & AI Agents).
Knowledge Hubs
Data Engineering
Distributed data systems, optimization strategies, and architecture patterns. Hands-on guides for Apache Spark, Kafka streaming, and Apache Airflow dynamic pipelines.
Explore Hub →Generative AI
Deep dives into production LLM implementations. Developing autonomous AI Agents, implementing advanced RAG systems, and designing vector search retrieval loops.
Explore Hub →Homelab & Automation
Self-hosting and automation guides. Setting up highly available K3s clusters on Proxmox VE, smart home automation with Home Assistant, and writing DevOps automations.
Explore Hub →Recent Articles
A guide to building agentic Retrieval-Augmented Generation (RAG) pipelines that can autonomously execute queries, retrieve context, and reason over databases.
How to avoid writing duplicate pipeline code by auto-generating Airflow DAGs from YAML configurations and database metadata.
Mastering real-time streaming architectures with Apache Kafka, from partition design to managing schemas using Avro and Confluent Schema Registry.