Barah Kar
Sr. Data Engineer
Dubai
Summary
Dedicated and results-oriented Cloud Data Engineer with 7 years of experience specializing in On premise(Cloudera), Azure and Google Cloud environments. Proven track record of designing, implementing, and optimizing data solutions to drive business objectives. Adept at leveraging cutting-edge cloud technologies and data engineering best practices to deliver scalable, high-performance, and cost-effective solutions.
Overview
6
6
years of professional experience
4
4
years of post-secondary education
Work History
Cloud Data Engineer
Damac
Dubai
2023.02 - Current
- · Responsible for data driven platform in DAMAC across all Azure and Google cloud platform.
- · Implement Databricks for revamp to datalakehouse of existing Azure platform.
- · Developed pySpark based data pipeline using databricks for digital marketing use cases.
- · Handel various digital market raw data and perform ELT as per business need.
- · Developed lambda functions using google cloud to integrate third party API’s.
- · Developed dashboards for various KPI’s for digital marketing team.
- · Responsible for all social media data integration with google cloud platform.
- Developed database architectural strategies at modeling, design and implementation stages to address business or industry requirements
- Designed data models for complex analysis needs
Data Engineer
Walamrt
Bangalore
2022.04 - 2023.02
- · Working with one of largest dataset in Walmart.
- · Deals with daily volume of 25-30 TB data generate from various devices user impressions across Walmart online web pages.
- · Developed spark ETL pipeline using Scala to handle large data volume in batch.
- · Stores source raw impression Data in Google GS bucket and preform ETL operation using spark Scala using GCP data proc clusters.
- · Data Governance practice for best data quality after communicating with stake.
- · Automate data quality issues using python and data governance tools.
- Collaborated on ETL (Extract, Transform, Load) tasks, maintaining data integrity and verifying pipeline stability
Azure Data Engineer
Wipro
Bangalore
2021.01 - 2022.04
- · Involved in Azure cloud cost optimization.
- · Involved in Azure CI/CD pipeline creation for seamless azure component deployment.
- · Worked on end-to-end Azure Data Engineering project implementation for new use case.
- · Migrate data from various source (Service Now and Oracle) to Azure platform.
- · Performed ETL/ELT to re-structured data based on business need.
- · Developed incremental load using Azure data flow.
- · Migrate spark CPU based application to GPU processing engine with Databricks.
- · Data Governance integration with Databricks for profiling and data quality check.
- · Developed dashboard using Power BI to track various demographic data asset.
- · Developed dynamic schema creation based on user data types using data bricks notebook.
Data Engineer
GAP
Hyderabad
2018.09 - 2021.01
- · Developed pySpark script in Databricks.
- · Wrapped Hive and Spark SQL with Scala for ELT operation.
- · Worked with data event and cloud migration data lift and shift projects.
- · Extensively handled multiple file format data like XML, JSON, CSV, AVRO and Parquet.
- · Bulk data migration from Oracle to Azure.
- · Consuming streaming data from kafka topic and transforming using spark.
- · Load aggregated sales, pricing data from spark to Azure data lake.
- · Configured all streaming and batch data into Azure Blob storage.
- · Migrated all HQL script to Spark script and scheduled job using yaml script.
- · Tuning experience with Oracle Database.
Software Engineer
Sears & Kmart Holdings
Bangalore
2017.06 - 2018.09
- · Implementation for inventory accelerate applications.
- · Developed and scheduled HIVE, SQOOP and PIG script using Control-M, Oozie and UNIX jobs.
- · Involved in ETL batch design and implementation using penthao.
- · Optimization of existing queries on production database as per business requirements.
- · Migrated Teradata records into Hive and HDFS.
- · Used HBASE to load online allocation transactions.
- · Daily standup with product team regarding abends and issues during Support.
- · Daily Scrum Meeting for New Jira’s Development in Agile method.
Education
Bachelor of Engineering Technology - Engineering Technology
Biju Patnaik University of Technology
Bhubaneswar, India 2009.09 - 2013.12
Skills
Azure Cloud
Reading Technical Blogs, Gymnastics, Boxing and stock chat analysis.
My hobbies include reading technical blogs, practicing gymnastics, boxing for fitness and self-defense, and analyzing stock charts to make informed investment decisions.
Work Availability
monday
tuesday
wednesday
thursday
friday
saturday
sunday
morning
afternoon
evening
swipe to browse
Timeline
Cloud Data Engineer
Damac
2023.02 - Current
Data Engineer
Walamrt
2022.04 - 2023.02
Azure Data Engineer
Wipro
2021.01 - 2022.04
Data Engineer
GAP
2018.09 - 2021.01
Software Engineer
Sears & Kmart Holdings
2017.06 - 2018.09
Bachelor of Engineering Technology - Engineering Technology
Biju Patnaik University of Technology
2009.09 - 2013.12
Similar Profiles
Dhananjay ShembekarDhananjay Shembekar
Team Lead - Analytics, Sales Performance & MIS at DAMACTeam Lead - Analytics, Sales Performance & MIS at DAMAC
Karan NairKaran Nair
Client Relationship Manager at DAMACClient Relationship Manager at DAMAC
Mohammed EssaMohammed Essa
Real Estate Agent at DAMACReal Estate Agent at DAMAC