Shivam Gahoi
Senior Data Engineer
Abu Dhabi
Summary
Experienced Data Engineer with expertise in AWS, Azure, SQL, Python, PySpark, and Redshift. Specializes in ETL pipeline development, data modeling, and big data technologies, optimizing performance for large-scale data processing. Skilled in automating workflows, integrating diverse data sources, and enhancing dashboard efficiency for e-commerce and retail. Proven ability to architect scalable data solutions and streamline pipeline orchestration to drive business insights and operational efficiency. Based in Abu Dhabi, UAE, holding a UAE Golden Visa, with a focus on building high-impact, scalable data solutions that enable business transformation.
Overview
6
6
years of professional experience
4
4
years of post-secondary education
Work History
Data Engineer
Mensa Brands
06.2022 - Current
- Designed and implemented an AWS CDK-based data pipeline, improving development efficiency.
- Built sales views and SKU-level financial metrics using Redshift and SQL, enabling data-driven decisions.
- Optimized SQL queries, reducing dashboard refresh time from 2 minutes to 5 seconds.
- Developed SKU-level inventory reports to enhance working capital management.
- Streamlined ETL pipelines using AWS Step Functions, S3, and Redshift, ensuring data integrity.
- Managed data architecture for 25+ brands across multiple channels (Amazon, Flipkart, Ajio, Myntra, etc.) in India, UAE, Germany, USA, ensuring seamless data integration and reporting.
- Tech Stack: AWS CDK, Python, SQL, Redshift, S3, Step Functions, ETL, Data Modeling, Data Warehousing
Data Engineer I
Flipkart India Private Limited
06.2021 - 05.2022
- Unified Flipkart Wholesale (online) & Best Price (offline) data, enabling faster decision-making.
- Automated data pipelines, integrating multiple sources into a common platform.
- Enabled event-triggered processing for manual file uploads, reducing manual effort.
- Migrated key dashboards (leadership, supply chain, pricing, finance) to a unified platform.
- Optimized dashboard refresh time from 2 hours to 10 minutes.
- Tech Stack: Azure Synapse, ADLS, SQL Server, PySpark, Python, SQL, ETL
Senior Data Engineer
Tredence Analytics Solutions Pvt. Ltd.
07.2019 - 06.2021
- Developed a competitive pricing dashboard for a leading restaurant chain, analyzing price variations across stores, demographics, and competitors to optimize daily pricing and coupon strategies.
- Designed and implemented data models to ensure efficient data organization and reporting.
- Optimized data ingestion by implementing stored procedures to load data into Snowflake from Amazon S3, improving processing efficiency.
- Enhanced Tableau performance, reducing dashboard refresh time from 5 minutes to 5 seconds for datasets exceeding 1 billion records.
- Implemented orchestration for Rate-Volume-Mix (RVM) analytics for a leading chemical chain, automating data integration, reporting, and email-based result distribution for cost analysis and strategy formulation.
- Automated custom report generation, enabling multi-user parallel execution via a web-based UI.
- Tech Stack: Azure, SQL, Python, Snowflake, Amazon S3, Tableau, ETL, Data Modeling, Orchestration, Apache Airflow
Education
B.Tech - Electrical Engineering
IIT TIRUPATI
India 08.2015 - 06.2019
Skills
- SQL
- Python
- PySpark
- Pandas
- Redshift
- SQL Server
- Snowflake
- Hive
- Hadoop
- AWS
- AWS Step Functions
- AWS CDK
- AWS Glue
- Azure Synapse
- Azure Data Factory
- Apache Airflow
- E-commerce
- Retail
- Data Warehousing
- Data Modelling
- MS Excel
Accomplishments
- AIR 6110 (Top 5%ile) of 1.1+ lac candidates in JEE Adv-2015, 06/2015
- Top 1%ile of 12+ lac candidates in JEE Mains-2015, 06/2015
Awards
- Pat on the back: for leading and setting up the processes for orchestrating data pipelines
- Pursue Excellence: for making tableau dashboard run faster, showing great sense of ownership and intelligence.
Timeline
Data Engineer
Mensa Brands
06.2022 - Current
Data Engineer I
Flipkart India Private Limited
06.2021 - 05.2022
Senior Data Engineer
Tredence Analytics Solutions Pvt. Ltd.
07.2019 - 06.2021
B.Tech - Electrical Engineering
IIT TIRUPATI
08.2015 - 06.2019
Similar Profiles
Deepak RaghavDeepak Raghav
International Operations Manager at Mensa BrandsInternational Operations Manager at Mensa Brands
Arun Kumar GArun Kumar G
Team Member at Mensa Brands Technologies Private LimitedTeam Member at Mensa Brands Technologies Private Limited
Manish GusainManish Gusain
Senior Video Editor at Mensa Brands Team HYPPSenior Video Editor at Mensa Brands Team HYPP