Summary
Overview
Work History
Education
Skills
Accomplishments
Awards
Timeline
Generic

Shivam Gahoi

Senior Data Engineer
Abu Dhabi

Summary

Experienced Data Engineer with expertise in AWS, Azure, SQL, Python, PySpark, and Redshift. Specializes in ETL pipeline development, data modeling, and big data technologies, optimizing performance for large-scale data processing. Skilled in automating workflows, integrating diverse data sources, and enhancing dashboard efficiency for e-commerce and retail. Proven ability to architect scalable data solutions and streamline pipeline orchestration to drive business insights and operational efficiency. Based in Abu Dhabi, UAE, holding a UAE Golden Visa, with a focus on building high-impact, scalable data solutions that enable business transformation.

Overview

6
6
years of professional experience
4
4
years of post-secondary education

Work History

Data Engineer

Mensa Brands
06.2022 - Current
  • Designed and implemented an AWS CDK-based data pipeline, improving development efficiency.
  • Built sales views and SKU-level financial metrics using Redshift and SQL, enabling data-driven decisions.
  • Optimized SQL queries, reducing dashboard refresh time from 2 minutes to 5 seconds.
  • Developed SKU-level inventory reports to enhance working capital management.
  • Streamlined ETL pipelines using AWS Step Functions, S3, and Redshift, ensuring data integrity.
  • Managed data architecture for 25+ brands across multiple channels (Amazon, Flipkart, Ajio, Myntra, etc.) in India, UAE, Germany, USA, ensuring seamless data integration and reporting.
  • Tech Stack: AWS CDK, Python, SQL, Redshift, S3, Step Functions, ETL, Data Modeling, Data Warehousing

Data Engineer I

Flipkart India Private Limited
06.2021 - 05.2022
  • Unified Flipkart Wholesale (online) & Best Price (offline) data, enabling faster decision-making.
  • Automated data pipelines, integrating multiple sources into a common platform.
  • Enabled event-triggered processing for manual file uploads, reducing manual effort.
  • Migrated key dashboards (leadership, supply chain, pricing, finance) to a unified platform.
  • Optimized dashboard refresh time from 2 hours to 10 minutes.
  • Tech Stack: Azure Synapse, ADLS, SQL Server, PySpark, Python, SQL, ETL

Senior Data Engineer

Tredence Analytics Solutions Pvt. Ltd.
07.2019 - 06.2021
  • Developed a competitive pricing dashboard for a leading restaurant chain, analyzing price variations across stores, demographics, and competitors to optimize daily pricing and coupon strategies.
  • Designed and implemented data models to ensure efficient data organization and reporting.
  • Optimized data ingestion by implementing stored procedures to load data into Snowflake from Amazon S3, improving processing efficiency.
  • Enhanced Tableau performance, reducing dashboard refresh time from 5 minutes to 5 seconds for datasets exceeding 1 billion records.
  • Implemented orchestration for Rate-Volume-Mix (RVM) analytics for a leading chemical chain, automating data integration, reporting, and email-based result distribution for cost analysis and strategy formulation.
  • Automated custom report generation, enabling multi-user parallel execution via a web-based UI.
  • Tech Stack: Azure, SQL, Python, Snowflake, Amazon S3, Tableau, ETL, Data Modeling, Orchestration, Apache Airflow

Education

B.Tech - Electrical Engineering

IIT TIRUPATI
India
08.2015 - 06.2019

Skills

  • SQL

  • Python

  • PySpark

  • Pandas

  • Redshift

  • SQL Server

  • Snowflake

  • Hive

  • Hadoop

  • AWS

  • AWS Step Functions

  • AWS CDK

  • AWS Glue

  • Azure Synapse

  • Azure Data Factory

  • Apache Airflow

  • E-commerce

  • Retail

  • Data Warehousing

  • Data Modelling

  • MS Excel

Accomplishments

  • AIR 6110 (Top 5%ile) of 1.1+ lac candidates in JEE Adv-2015, 06/2015
  • Top 1%ile of 12+ lac candidates in JEE Mains-2015, 06/2015

Awards

  • Pat on the back: for leading and setting up the processes for orchestrating data pipelines
  • Pursue Excellence: for making tableau dashboard run faster, showing great sense of ownership and intelligence.

Timeline

Data Engineer

Mensa Brands
06.2022 - Current

Data Engineer I

Flipkart India Private Limited
06.2021 - 05.2022

Senior Data Engineer

Tredence Analytics Solutions Pvt. Ltd.
07.2019 - 06.2021

B.Tech - Electrical Engineering

IIT TIRUPATI
08.2015 - 06.2019
Shivam GahoiSenior Data Engineer