Vishwanath Kadapa
Dubai
Summary
A highly experienced Data Engineer with 15 years of industry expertise, specializing in Microservices (8 years), Data Engineering (5 years), and Java/J2EE Development (13 years). Strong background in developing real-time data transformation and ingestion solutions, integrating systems with Apache Kafka, Apache Flink, and Apache Beam. Proven track record of working with distributed systems and cloud platforms, optimizing data pipelines for performance, scalability, and fault tolerance. Adept in collaborating with cross-functional teams, including architects, IT teams, and data scientists, to deliver scalable, efficient solutions.
Overview
15
15
years of professional experience
Work History
Senior Data Engineer
Emirates NBD
01.2020 - Current
- Developed real-time data pipelines to send financial and non-financial transactions to SAS for fraud detection using Apache Kafka, Apache Flink, and Apache Beam
- Optimized Flink jobs for scalability, fault tolerance, and high performance, ensuring real-time data is processed without loss
- Designed real-time detection pipelines for fraud detection by transforming, enriching, de-duplicating, and sorting financial data
- Collaborated with cross-functional teams to design custom analytics applications for fraud alerts
- Developed real-time enterprise event pipelines for Apache Ignite updates using Apache Kafka and Apache Flink
- Configured Kafka connectors for seamless integration with Ignite caches for low-latency data access
- Deployed Flink clusters with Helm 3 and managed CI/CD pipelines for efficient deployments
- Developed real-time streaming solutions to display financial and credit card transactions in the Emirates NBD mobile app
- Worked on optimizing performance for real-time data updates with Kafka Streams and Flink
- Worked on enriching customer transaction data in real-time using the Personetics platform for personalized insights
- Integrated Personetics with existing Kafka and Flink pipelines to provide automated transaction insights for improved customer experience
- Optimized the pipeline for low-latency processing of enriched data to be displayed in customer-facing applications
- Led the global expansion of the Global Transactions project by extending it to new markets, including London, Egypt, and KSA
- Developed and optimized real-time data pipelines to process financial and credit card transactions in these new regions, ensuring consistency and performance at scale
- Implemented Kafka connectors for seamless integration with regional data sources, supporting transactions across multiple countries and currencies
- Spearheaded the migration of data from Apache Ignite to Redis to improve data processing speed and scalability
- Designed and developed migration strategies, ensuring minimal downtime and data consistency during the transition
- Created real-time data pipelines using Flink and Kafka for smooth data synchronization between Ignite and Redis
- Worked on optimizing data access patterns in Redis, significantly reducing latency for real-time applications
Senior Technical Engineer
Emirates Airlines
01.2018 - 12.2019
- Led full-stack development for the NEO transformation project to provide dynamic pricing and recommendations for flights and services
- Developed responsive UIs using Angular and React and integrated backend APIs in Node.js and Java Spring Boot
- Migrated legacy .NET pages to React and Node.js, improving performance, efficiency, and speed to market
Senior Technology Specialist
Wells Fargo Enterprise Global Services
10.2011 - 01.2018
- Developed microservices to support online and mobile banking services for customer accounts and products
- Deployed services using Pivotal Cloud Foundry and implemented Jenkins pipelines for continuous integration and deployment
- Developed web services for credit and debit card operations, including eligibility checks and credit line updates, using Apache CXF and SOAP web services
Software Engineer
Nisum Technologies
07.2009 - 10.2011
- Worked on Java-based applications and provided L3 support for critical ecommerce systems during peak retail events for Gap and Macy’s
Education
B.Tech - Electronics and Communication Engineering
VNR Vignana Jyothi Institute of Engineering & Technology
01.2009
Skills
- Git version control
- ETL development
- Big data processing
- Python programming
- NoSQL databases
- Kafka streaming
- Data pipeline design
- API development
- Performance tuning
- Hadoop ecosystem
- Spark development
- Java development
- SQL and databases
Key Resposibilities
- Real-Time Data Pipelines: Developed real-time data processing pipelines for fraud detection, financial transactions, and real-time system updates using Apache Flink, Kafka, and Beam.
- Optimization: Focused on optimizing job performance, fault tolerance, and memory usage, ensuring robust and scalable data processing systems.
- Deployment & CI/CD: Led the creation and deployment of Flink clusters using Helm and integrated CI/CD pipelines for automated deployment and management of data jobs and systems.
- Team Collaboration: Worked in close collaboration with architects, data scientists, and IT teams to implement solutions aligned with business needs and project goals.
- Monitoring & Troubleshooting: Developed and maintained monitoring systems using Prometheus, Grafana, and Kibana for tracking system health and resolving issues proactively.
Work Preference
Work Type
Full TimeWork Location
On-SiteHybridImportant To Me
Career advancementWork-life balanceCompany CultureFlexible work hoursHealthcare benefitsWork from home optionStock Options / Equity / Profit Sharing4-day work weekTeam Building / Company RetreatsPersonal development programsPaid sick leavePaid time offLanguages
English
Advanced (C1)
Quote
The way to get started is to quit talking and begin doing.
Walt Disney
Timeline
Senior Data Engineer
Emirates NBD
01.2020 - Current
Senior Technical Engineer
Emirates Airlines
01.2018 - 12.2019
Senior Technology Specialist
Wells Fargo Enterprise Global Services
10.2011 - 01.2018
Software Engineer
Nisum Technologies
07.2009 - 10.2011
B.Tech - Electronics and Communication Engineering
VNR Vignana Jyothi Institute of Engineering & Technology
Similar Profiles
Najiba MohammadNajiba Mohammad
Account Service/SigCap Specialist/Supervisor at Tanfeeth, Emirates NBD/Emirates Islamic BanksAccount Service/SigCap Specialist/Supervisor at Tanfeeth, Emirates NBD/Emirates Islamic Banks
AHMED FARIDAHMED FARID
Regional Sales Manager at Emirates NBD/Emirates Islamic BankRegional Sales Manager at Emirates NBD/Emirates Islamic Bank
Fatiha AboulkhairFatiha Aboulkhair
Human Resources Specialist at Emirates National Bank of Dubai (Emirates NBD)Human Resources Specialist at Emirates National Bank of Dubai (Emirates NBD)
Christopher RajeshChristopher Rajesh
Senior Solution Architect at Emirates NBDSenior Solution Architect at Emirates NBD
ERICK SAIMON MMBAGAERICK SAIMON MMBAGA
Ramp Operations Team Leader at Dnata, The Emirates GroupRamp Operations Team Leader at Dnata, The Emirates Group