Experienced Lead Data Engineer excelling in transforming complex datasets into actionable insights. Successfully led multiple projects to optimise data pipelines, resulting in significant time and cost savings.
Driven professional with keen interest in data engineering, bringing strong analytical skills and proficiency in SQL and Python. Demonstrated ability in problem-solving and team collaboration, ensuring seamless data integration and management. Poised to drive impactful data solutions that enhance business intelligence and operational efficiency.
Overview
16
16
years of professional experience
1
1
Certification
Work history
Lead Data Engineer
Kagool Ltd
Dubai, United Arab Emirates
03.2023 - Current
Worked in the capacity of Lead Data Engineer for the Data Modernization Project for Al-Futtaim Group and analyzing and rebuilding their BW systems (Auto and Retail) into Azure using Meta data driven pipeline development.
The platform modernization program will activate data across the enterprise by extracting data from SAP ERP systems (SAP ECC , S4 HANA), SAP BW, NLS , Oracle Rpro into Azure.
Modernize the group's data analytics capabilities by re-platforming existing analytics solution into an azure based Intelligent data platform with advanced capabilities for analytics and reporting.
Data ingested from source to raw until Gold(CDMs)/Gold Serve(Business KPIs - equivalent to BW Objects (DSO's, Info-cubes, Info-objects) controlled through metadata driven development.
Ensured Data security and compliance by implementing Azure role based access control(RBAC) and Data classification and masking and encryption for sensitive Data (PII.PHI,PCI) using hash and MD5 techniques.
Entire IDP Data build on Unity Catalogue structure and cross lob access driven using unity catalogue workspace binding and data sharing capabilities.
Data ingested being brought forward into the Common azure container and then being bifurcated to cross LOB's containers (Retail/Auto/Healthcare/HR/Blue/Engg etc). using the filter of either company code or sales organization.
Access control driven using Centralized access management including Row level security driven on each object to show data to only the users who have the relevant access.
Data Cleansing and Validation rules build using Python and implemented using metadata driven development including feature like currency decimal shift.
Job triggers created using scheduler's for weekly, daily monthly based on the needs of customer along dependency based schedule(Hard and soft dependency).
Different Delta propagation functionalities build as part of Intelligent Data Platform ( Delta on Full , Delta on Base Table , Custom Delta)
Collaborated with stakeholders for clear understanding of data requirements.
Spearheaded a cross-functional team for successful deployment of large-scale data projects.
Improved data security with advanced encryption and access control methods.
Senior data engineer
Kagool Pvt Ltd
Coventry, UK
06.2020 - 03.2023
Worked as a Senior Data Engineer to collaborate with stakeholders - Smiths Group and Atlas Copco in UK for clear understanding of data requirements.
Build, Unit tested and deployed to production different modules using Azure Data Factory , Function Apps , Logic Apps , Service Bus , SFTP, File Share , Azure Data-bricks.
Designed and Delivered Common Data Models for a number of business functions to expedite building KPI's on top of it.
Created ER Diagrams,Visio Diagrams for architecture flow for end to end system.
Enhanced user experience by automating repetitive tasks using Python scripts.
Improved data quality with rigorous validation techniques and error handling protocols including email alerts to be send to stakeholders for any pipeline failures.
Suggested actionable improvements to address operational bottlenecks with new and emerging technology.
Troubleshot complex system issues, ensuring minimal disruption to business operations.
Managed end-to-end data pipeline construction for seamless integration of disparate data sources.
Information Technology Specialist
NIIT Technologies Limited
Milton Keynes, UK
09.2018 - 06.2020
Worked as a Technical Business Analyst for development and maintenance of all the applications that fall under Customer Communication Platform for Santander Bank located at Milton Keynes
I was involved in end to end process start from interacting with source applications till we finally send the communications from Bank to customers through the channel of Letter, Email, SMS and Push
Developed Project Plans, Functional Specifications, project charter for Business Process Refinement Re-Engineering, Business Intelligence and Automation
Delivered two major programmes going in the bank: Migrating the process flows/source-code/databases from legacy platforms (Mainframe/Banksphere) to Big data/Hadoop/Inspire
Paper-free programme: To turn-off paper communications for the customers opting to be paper-free via Online Portals
Involved in creating Use cases, Test cases, Test Plans, Wireframe Diagrams, Sequence diagrams, Class and Context Diagrams, Data Flow Diagrams (DFD)
Conducted Pre-UAT and User Accepting Testing (UAT) cycles with users and stakeholders, prepared Standard Operating Procedures (SOPs)
Identified and mitigated team risks and impediments (Blocking issues, Critical Defects) on a daily basis to allow the team to deliver the sprint goals and deliverables
Ability to analyze functional requirements, generate a requirement traceability matrix to map the requirements from start of analysis, design, test and uptil deployment of code specs
Assisting and conducting Unit testing, System Integration testing and Regression testing phases and coordinating with different teams to accomplish them as well as supporting the pre and post deployment process of the enhancements
Experience of working alongside the Product Owner to prioritize product backlogs, creating user stories, performing daily stand-ups and coordinating for sprint planning and retrospectives
Well-versed in conducting daily scrum meetings and in identifying and mitigating team risks and impediments to enable the team to deliver the sprint goals and deliverables
Lead Consultant
Genpact Headstrong Capital Markets
, India
07.2013 - 02.2018
Worked in the capacity of Project Lead/Analyst under Capital Market – Fixed Income Application for Morgan Stanley
Analyzing Fixed Income applications to design, modify and implement using COBOL and CICS programming languages whilst interacting with different external applications using Web-services and MQ's as interface
I was handling the below Fixed Income online/back-end applications for Morgan Stanley: IMS (Inventory Management System) is the backend for Fixed Income Trading Platform
Different front-end applications (FI Offerings, FI Ticket, Global Currency, Bond Calculator, Manager Approval and Order Status) interact with IMS via MQ or Webservices on the backend side for Trade processing (Buy, Sell, Cross-Sell etc), approval and various other trade inquiries
MTS (Muni Trading System) deals with Municipal securities under fixed income umbrella
The application provides all the Muni Primary & Secondary Offerings through an automated deal setup process which are traded through CMAT and from VB based MTS application
Understanding of the following Fixed Income Products: Treasury bills, Treasury notes, Municipal bonds, Corporate Bonds, Fixed income mutual funds and certificates of deposit (CDs)
Team management and liaising with cross-functional teams like Practice, Presales, Delivery and Solution Design
Managing and handling team of 9 members at offshore and conducting code/test plan reviews
Resource Management for new or existing client requirement
Client relationship management - Monthly meeting with the clients to provide status update and understanding any underlying issues and opportunities
Handling Escalation for existing accounts and liaising with team towards the fix
Monitoring/running and supporting batch jobs and schedulers to assist the business application in order to provide feed to different interacting applications(upstream/downstream) and also feeding data to online CICS screens to keep them updated as per business hours
Ability to analyze functional requirements, translates them into test cases, and prepares a Detailed Test Plan Specification utilizing knowledge of the functional application, and creation of Test Case Design, Test estimates, Test Data setup and Test Case Execution
Assisting and conducting Unit testing, System Integration testing and Regression testing phases and co-ordinating with different teams to accomplish them
Involved in bug fixes and tracked/logged/monitored defects using QC and JIRA
Senior Analyst Programmer
Syntel Private Limited
, India
06.2009 - 06.2013
Global New Accounts system core function is to process new applications for American Express Credit cards and other applications which were related to Transaction Based Automated Survey System and ACR (Balancing and Control Tool).
The main processing involves verifying applications for Fraud, Duplicate and other checks, real time/batch Credit Bureau review scoring, decision, card assignment and creating different downstream files for further processing of application
Worked on Development Requests provided by the Client including doing impact analysis and guiding the team members in providing the timely updates
Concerned with analysis, design, code, component and unit testing of all application code using mainframe components COBOL, CICS, DB2 , JCL , VSAM
Conversion of business requirement into technical split modules, Delivery of code modules and formulating production verification plans and back out plans
Formulated design specification and unit test plans, UTR's and testing of enhancements documents
Conducting Code walkthroughs & Code reviews with Technical & Business teams both at onshore & offshore
Supported SIT, UAT and Implementation of the enhancements in the real time environment
Education
B-Tech - Computer Science
IEC CET Technologies
India
PG Diploma - Operations Management
Symbiosis International University
India
Skills
Azure Technologies (Azure Data Factory, Azure Databricks, Azure Functions, Azure DevOps ,Azure Function Apps, Service Bus, Python , SQL, SFTP , Fileshare)
Well versed with SDLC and Agile development methodologies
Received Global Star Award from Kagool in July 2024.
Received Most Valuable Player Award from Kagool in December 2022
Received Customer Choice Promotion in Category of Think Customer issued by Client (Santander Bank) in April 2020 associated with NIIT (Co-forge) Technologies.
Received Bronze Award issued by Client(Morgan Stanley Bank) in August 2017 associated with Genpact Headstrong.
Received Cause for Applause Award from Genpact Headstrong in Jan 2014.
Centurion Certification of Appreciation issued by Client(American Express Bank) in August 2011 associated with Syntel.
Certification
Certified Azure Fundamentals (AZ 900) from Microsoft on 4 Sep 2020
Certified Scrum master from Scrum Alliance in Feb 2018