Karthik Karavadi
Summary
Experienced Data Engineer with 10+ years of expertise in ETL development, data integration, and pipeline orchestration. Skilled in data modeling, data quality assurance, database performance optimization, and workflow automation for actionable insights through analytics and visualization. Proficient in leveraging technical skills to streamline processes and improve efficiency.
Overview
8
8
years of professional experience
Work History
Senior Data Engineer
EMAAR
02.2025 - Current
- Leading the migration of data workflows from Informatica PC to IIDC, ensuring seamless transition and
optimized performance.
• Troubleshot and resolved bugs in existing Informatica workflows and monitors, enhancing reliability
and reducing downtime.
• Supported and maintained Airflow orchestration for data pipelines, ensuring efficient scheduling and
execution.
• Provided ongoing support for Snowflake Data Warehouse, ensuring high availability, performance
tuning, and efficient data management.
Data Analytics Engineer
Pepsico
01.2023 - Current
- Established a seamless data pipeline integrating diverse cloud storage sources for unified data ingestion
- Improved data quality and consistency by 100%, leveraging DBT for robust data transformations
- Boosted efficiency by 70% through streamlining task orchestration and scheduling with Airflow, minimizing manual intervention
- Utilized Snowflake and GCP Data Warehousing for scalable, cost-effective advanced analytics and reporting
- Designed and automated end-to-end data pipelines with Cloud Composer, orchestrating PySpark jobs on Google Cloud Dataproc for real-time analytics in BigQuery
Senior Data Lead
Ernst & Young
01.2021 - 01.2023
- Built a data warehouse on Azure Synapse with Slowly Changing Dimension (SCD) transformations, improving customer data accuracy by 95%
- Automated data ingestion from multiple sources using Azure Data Factory (ADF) pipelines, reducing manual data entry by 70%
- Optimized Azure Synapse performance by 60% through indexing, query optimization, and partitioning
- Enhanced PySpark job performance by optimizing using broadcast joins, data partitioning, caching, and filter pushdown, improving processing speed by 40%
- Implemented a hybrid data warehouse model combining Inmon's top-down architecture with Kimball's dimensional modeling, optimizing both enterprise-wide integration and subject-specific reporting
Technical Data Engineer
State Street Corp
01.2020 - 01.2021
- Leveraged Alteryx for efficient extraction, transformation, and loading (ETL) of client data, significantly boosting processing efficiency and reducing data handling time
- Developed and optimized complex SQL queries to ensure robust data quality, supporting detailed, in-depth analysis and accurate decision-making
- Designed data models and documented data flows, strengthening system architecture and promoting transparency, leading to improved data integrity and streamlined processes
Big Data Engineer
Infosys
01.2017 - 01.2020
- Migrated data from Oracle to HDFS using Sqoop/Talend for efficient extraction and loading
- Optimized Hive and Impala queries to enhance performance and shorten data processing times
- Developed and maintained ETL Talend/Sqoop pipelines, integrating Oracle data with Hive and Impala for analytics
- Executed data transformation and cleansing using HiveQL and Python, preparing data for reporting
Education
Bachelor of Science - Electronics And Communications Engineering
K L University
Vijayawada, AP, India04-2013
Skills
- Airflow
- DBT
- SQL
- Informatics IICS
- Pyspark
- Azure Synapse
- Databricks
- ADF
- Data Modelling
- Snowflake
- GCP
- CloudComposer
- Data Proc
- Big query
- Github
- Hive
- Impala
- Sqoop
- PowerBI
Personal Information
Nationality: Indian
Languages
English
Hindi
Telugu
Timeline
Senior Data Engineer
EMAAR
02.2025 - Current
Data Analytics Engineer
Pepsico
01.2023 - Current
Senior Data Lead
Ernst & Young
01.2021 - 01.2023
Technical Data Engineer
State Street Corp
01.2020 - 01.2021
Big Data Engineer
Infosys
01.2017 - 01.2020
Bachelor of Science - Electronics And Communications Engineering
K L University
Similar Profiles
JANITHA FERNANDOJANITHA FERNANDO
RECIEVER CUM STORE KEEPER AND ADMINISTRATION at EMAARRECIEVER CUM STORE KEEPER AND ADMINISTRATION at EMAAR
Rahul BhagatRahul Bhagat
Residence Facilities Associate at EMAARResidence Facilities Associate at EMAAR
BASIYA AKTHARBASIYA AKTHAR
Head of QA at EmaarHead of QA at Emaar