Barah Kar
Sr. Data Engineer
Dubai
Summary
Dedicated and results-oriented Cloud Data Engineer with 7 years of experience specializing in On premise(Cloudera), Azure and Google Cloud environments. Proven track record of designing, implementing, and optimizing data solutions to drive business objectives. Adept at leveraging cutting-edge cloud technologies and data engineering best practices to deliver scalable, high-performance, and cost-effective solutions.
Overview
6
6
years of professional experience
4
4
years of post-secondary education
Work History
Cloud Data Engineer
Damac
Dubai
02.2023 - Current
- · Responsible for data driven platform in DAMAC across all Azure and Google cloud platform.
- · Implement Databricks for revamp to datalakehouse of existing Azure platform.
- · Developed pySpark based data pipeline using databricks for digital marketing use cases.
- · Handel various digital market raw data and perform ELT as per business need.
- · Developed lambda functions using google cloud to integrate third party API’s.
- · Developed dashboards for various KPI’s for digital marketing team.
- · Responsible for all social media data integration with google cloud platform.
- Developed database architectural strategies at modeling, design and implementation stages to address business or industry requirements
- Designed data models for complex analysis needs
Data Engineer
Walamrt
Bangalore
04.2022 - 02.2023
- · Working with one of largest dataset in Walmart.
- · Deals with daily volume of 25-30 TB data generate from various devices user impressions across Walmart online web pages.
- · Developed spark ETL pipeline using Scala to handle large data volume in batch.
- · Stores source raw impression Data in Google GS bucket and preform ETL operation using spark Scala using GCP data proc clusters.
- · Data Governance practice for best data quality after communicating with stake.
- · Automate data quality issues using python and data governance tools.
- Collaborated on ETL (Extract, Transform, Load) tasks, maintaining data integrity and verifying pipeline stability
Azure Data Engineer
Wipro
Bangalore
01.2021 - 04.2022
- · Involved in Azure cloud cost optimization.
- · Involved in Azure CI/CD pipeline creation for seamless azure component deployment.
- · Worked on end-to-end Azure Data Engineering project implementation for new use case.
- · Migrate data from various source (Service Now and Oracle) to Azure platform.
- · Performed ETL/ELT to re-structured data based on business need.
- · Developed incremental load using Azure data flow.
- · Migrate spark CPU based application to GPU processing engine with Databricks.
- · Data Governance integration with Databricks for profiling and data quality check.
- · Developed dashboard using Power BI to track various demographic data asset.
- · Developed dynamic schema creation based on user data types using data bricks notebook.
Data Engineer
GAP
Hyderabad
09.2018 - 01.2021
- · Developed pySpark script in Databricks.
- · Wrapped Hive and Spark SQL with Scala for ELT operation.
- · Worked with data event and cloud migration data lift and shift projects.
- · Extensively handled multiple file format data like XML, JSON, CSV, AVRO and Parquet.
- · Bulk data migration from Oracle to Azure.
- · Consuming streaming data from kafka topic and transforming using spark.
- · Load aggregated sales, pricing data from spark to Azure data lake.
- · Configured all streaming and batch data into Azure Blob storage.
- · Migrated all HQL script to Spark script and scheduled job using yaml script.
- · Tuning experience with Oracle Database.
Software Engineer
Sears & Kmart Holdings
Bangalore
06.2017 - 09.2018
- · Implementation for inventory accelerate applications.
- · Developed and scheduled HIVE, SQOOP and PIG script using Control-M, Oozie and UNIX jobs.
- · Involved in ETL batch design and implementation using penthao.
- · Optimization of existing queries on production database as per business requirements.
- · Migrated Teradata records into Hive and HDFS.
- · Used HBASE to load online allocation transactions.
- · Daily standup with product team regarding abends and issues during Support.
- · Daily Scrum Meeting for New Jira’s Development in Agile method.
Education
Bachelor of Engineering Technology - Engineering Technology
Biju Patnaik University of Technology
Bhubaneswar, India 09.2009 - 12.2013
Skills
Azure Cloud
Python Programming
Scala Programming
SQL
Adapt ETL Tools
Shell Scripting
Google Cloud
Databricks
Managing Databases
Data Quality Engineering
Reading Technical Blogs, Gymnastics, Boxing and stock chat analysis.
My hobbies include reading technical blogs, practicing gymnastics, boxing for fitness and self-defense, and analyzing stock charts to make informed investment decisions.
Work Availability
monday
tuesday
wednesday
thursday
friday
saturday
sunday
morning
afternoon
evening
swipe to browse
Timeline
Cloud Data Engineer
Damac
02.2023 - Current
Data Engineer
Walamrt
04.2022 - 02.2023
Azure Data Engineer
Wipro
01.2021 - 04.2022
Data Engineer
GAP
09.2018 - 01.2021
Software Engineer
Sears & Kmart Holdings
06.2017 - 09.2018
Bachelor of Engineering Technology - Engineering Technology
Biju Patnaik University of Technology
09.2009 - 12.2013