ABOUT ME
I am a highly motivated and progress-focused student currently pursuing Master of Science in Information Systems at Northeastern University and have a strong theoretical basis for computer science and data analytics.
I have a industry experience of 4 years as a Data Engineer at State Street Corporation and have worked on many big data technologies (Hadoop, Spark, HDFS, M/R, Hive, Pig, and Hue) with cross-platform proficiency (Windows, Unix) and strong programming skills (Python, Java, Scala, and Shell scripting) having profound knowledge on data infrastructure and frameworks. My job responsibilities primarily included the building of data pipelines for data cleansing and data migration process from different State Street data sources to the Hadoop environment and analyzing the data to generate the Hive reports according to the business requirements by maintaining quality assurance. Within a span of just 9 months, I have worked on 3 projects simultaneously, but my greatest achievement was to design and develop an automated solution that migrates data from various servers (2000 servers approx.) of different database platforms like Oracle, MS SQL, and Sybase to Hadoop environment and apply transformation techniques to generate reports according to the client’s requirement thereby eliminating the manual intervention.
SKILLS
PROGRAMMING LANGUAGES
Java , pYTHON , C , SQL
Machine LEARNING
NumPy, Pandas, Matplotlib, Scikit-learn, NLTK, TensorFlow
BIG DATA TOOLS
Apache Hadoop, StreamSets, HDFS, Hive, Hue, Sqoop, Spark, Linux, Shell Scripting, PowerBI , TABLEAU
DATABASES
MySQL, Oracle, Teradata, MS SQL, Cassandra, PostgreSQL, MongoDB
WEB TECHNOLOGIES
·HTML, CSS, Bootstrap, JavaScript, Node JS, React
Others
Data structures, object-oriented programming, Agile Methodologies, Lean Practitioner
EXPERIENCE
Data Engineer Co-op , Aura Sub LLC, Boston, JUNE 2023 - DECEMBER 2023
· Implemented a key independent project using Python in Databricks, to automatically identify and remove over 200 redundant Snowflake databases that had been migrated to Databricks which resulted in significant cost savings and improved data management
· Engineered data ingestion for Aura protection plans, extracting data from marketing APIs, and seamlessly processing it with Python in Databricks. The data was then ingested into AWS RDS and S3, improving accessibility and analytics across multiple sources
· Successfully migrated 70 DAGs from Airflow 1 to Airflow 2, ensuring a smooth transition and minimal disruption to vital workflows
Data Engineer - Senior Associate, State Street Corporation , JULY 2018 - APRIL 2022
• Designed and developed an automated solution that migrates data from various servers (1200 servers approx.) of different database platforms like Oracle, MSSQL, and Sybase to the Hadoop environment.
• Carried out the data analysis and data metrics generation process on the resultant report for the project which can be used by the business user to check the metric change periodically.
• Trained a team of 5 members on technical and functional knowledge of the data migration projectS.
• Worked on highly scalable data migration applications to transfer structured data from multiple sources to the Hadoop environment using python & HDFS and generated weekly/monthly reports using Hive.
• Built ETL workflow with pre-processing and data transformation techniques to enhance data quality and reliability by using Apache Spark for faster batch processing.
• Built the automated and scheduled data pipelines using StreamSets ETL tool to extract data from different data platforms like MSSQL, Oracle DB, SQL Server, and Excel to the Hadoop environment.
PROJECTS
CHARGE UP
- Designed a responsive event management website which acts as an online ticketing tool where users can create events among 10 different categories like music, theater, etc. with automatic generation of event pages and register for events as per their interests
- Incorporated MongoDB to store the user data and event information to achieve data persistence and search capabilities for event.
Intensive Pet Care
The project aim is to create a java swing application to enhance and provide better health care services to animals by implementing an online system for pet owners where they can enable to register the pets under hospitals and health camps according to their preference and convenience.
BANK CHURN PREDICTION
- Customer churn has become a major problem in banking industry and banks have always tried to track customer interaction with the company to detect early warning signs in customer’s behavior.
- Performed Data Cleaning, Exploratory data analysis on the dataset and carried out AutoML and model interpretability on the bank churn data to perform classification on whether a customer would be exit or not based on some factors and patterns.
Sentimental Analysis using Neural NETWORK
- Devised a platform to predict the emotion behind the user responses contributed through comments among 10 different emotional categories by applying artificial intelligence techniques, NLTK libraries and scikit-learn to model factors that predict user input
- Performed data cleaning and data preprocessing of the user input to create a corpus of meaningful words and built the neural network using Keras and TensorFlow backend for training and making predictions
Escapade - VR PROJECT
- Developed 3D, 2D, and virtual reality games developed using Unity and deployed it into an a gaming android/iOS application used by end-user
- Implemented gameplay mechanics and interaction logic using C# programming and created 3D models, textures, and animation that supports cross-functional platforms like PC and mobile using Unity Assets
CAREER SKETCH
Developed website which enables the students to identify one's interest in a career best suited for them. The common theories concerning individual career planning are briefly outlined. The website has been developed giving importance to even minute details of Webpages by using Bootstrap & CSS and incorporated few careers planning games like spin the wheel to enhance the project.
AWARDS
Award Name | Year | Description |
Well Done | May-22 | Awarded for automating the process of data migration and report generation by 20% by pulling in data from different platforms into hadoop environment and delivering it with integrity and speed. |
Applause - Break Through Silos | Nov-21 | Awarded for efficiently aligning the deliverables with functional aspects and coordinating with different teams globally to get the deliverables achieved in time as per the Business expectation during the tight schedules. |
Well Done - Choose to Own it! | Apr-21 | Awarded for driving automation initiatives By working with cloakware and unix teams to resolve technical issues which resulted in reducing manual processing involved. |
Spot Award | Nov-18 | Awarded for successfully organizing Global CXO's meet at State Street on visit of the then CEO Ronald O Hanley to India. |
Feel free to contact me for any queries.
