Principal Data Engineer at Vantive
Cloud Data Platforms · Distributed Processing · Real-Time Analytics
LinkedIn · Email · Selected projects
I work across cloud data engineering, distributed processing, and analytics. My technical foundation spans Python, SQL, Apache Spark, and Azure, with public projects that connect event ingestion, lakehouse processing, and operational reporting.
My engineering focus is the complete path from source data to useful decisions: clear data models, reliable ingestion, validated transformations, and analytics that reflect the business problem.
- Cloud data platforms — connecting ingestion, processing, storage, and reporting into a coherent architecture.
- Streaming and lakehouse pipelines — event-driven ingestion, PySpark Structured Streaming, and layered Delta Lake datasets.
- Data modeling and quality — explicit schemas, validation rules, and transformations that make downstream data easier to trust.
- Operational analytics — translating retail and healthcare events into measures of sales, patient flow, and resource utilization.
| Area | Technologies & practices |
|---|---|
| Languages & processing | Python, SQL, Apache Spark, PySpark, Structured Streaming |
| Cloud & lakehouse | Microsoft Azure, Azure Event Hubs, Databricks, Delta Lake |
| Data architecture | Batch and streaming pipelines, bronze/silver layers, schema enforcement, data validation |
| Databases & warehousing | PostgreSQL, MySQL, Teradata |
| Analytics | Power BI, operational dashboards, business KPIs |
| Engineering workflow | Git, GitHub, Bitbucket |
An end-to-end streaming demonstration that turns simulated order events into sales analytics. The pipeline ingests events through Azure Event Hubs, processes them with Databricks PySpark Structured Streaming, and stores raw and validated data in Delta Lake for Power BI reporting.
Architecture
Python event generator → Azure Event Hubs → Databricks Structured Streaming → Bronze / Silver Delta tables → Power BI
Engineering scope
- Event ingestion through the Kafka-compatible Event Hubs interface.
- Separation of raw ingestion from cleaned, validated, and enriched datasets.
- Reporting on sales, orders by state, product performance, and time-based KPIs.
An Azure data engineering project designed around the operational needs of a multi-branch hospital network. It combines streaming and batch pipelines to support visibility into admissions, discharges, transfers, bed occupancy, and department load.
Engineering scope
- Centralized ingestion for patient movement and hospital utilization analytics.
- Near-real-time operational reporting alongside historical trend analysis.
- Architecture documentation, sample events, and Power BI report examples.
Extending my work in streaming data pipelines and exploring retrieval-augmented generation (RAG) and agentic AI. I am also working through Snowflake data modeling and SQL constraint behavior.
Connect
For conversations about cloud data platforms, streaming architecture, or data engineering, connect on LinkedIn or reach me at g.singh.bora@gmail.com.