Working with antiquated ETL systems without documentation is like trying to navigate a maze while wearing a blindfold in today's data-driven world. However, Sushil Kumar Tiwari saw this as a challenge that needed to be overcome rather than a barrier. Sushil has a background in data transformation and has gained recognition for his ability to effectively rebuild intricate data infrastructures from the ground up, frequently without the benefit of legacy knowledge or documentation.
Sushil has been gradually moving upward in the domain of data engineering, contributing to the large-scale data modernization projects of famous companies. His vast knowledge of ETL frameworks, data warehousing, and pipeline orchestration has contributed to him being recognized as a key player in enterprise data transformation projects. He managed the transition to a modern, cloud-based data facility at his present company by being a principal factor in the upgrade of the old systems, which had turned weak, undocumented, and inefficient. His work has not only received internal recognition but also strengthened the company’s adaptability and decision-making ability.
Leading the conversion of more than 200 undocumented ETL jobs into a modular, scalable framework using cutting-edge technologies like Apache Airflow, Spark, and Snowflake was one of his most significant accomplishments. By reducing job failures by 70% and batch processing times by 60%, this effort resulted in significant annual infrastructure maintenance savings. In addition to a technical improvement, the outcome was a cultural change that moved the company away from reliance on tribal knowledge and toward a more open, transparent data environment. Additionally, by reducing the learning curve from weeks to days, this work facilitated the onboarding of new engineers.
Sushil's pathway was very bright, indeed, as he was involved in and implemented several powerful projects. His list of achievements includes the transfer of business-critical data workloads from outdated on-premises systems to the cloud and the development of tools for the mapping of data lineage in undocumented environments. Sushil's work has always been practical, scalable, and fit for the future. In one project, for instance, he created frameworks for automated data validation and testing that kept an eye on more than 5,000 data dependencies, thus securing data integrity in business reporting systems. He also performed redesign with 15 Tb of data and got 99.98% job success rate after migration.
Sushil's achievements are significant but they would not have been the same if he hadn't faced and conquered the challenges associated with them. A lot of the obsolete ETL processes he took over from his predecessor were constructed using outdated tools that had no active owners, no documentation, and, in some cases, no understanding of their original purpose. These processes had executive dashboards as the end-users and therefore one could not afford to make even the slightest mistake. By combining data profiling, log parsing, iterative validations, and a strong intuition for how data flows through systems, Sushil managed to completely reconstruct, optimize, and future-proof each and every component without disrupting the business.
Beyond his technical contributions, Sushil has authored several internal whitepapers and blogs that reflect his practical philosophy toward data engineering. His articles such as “Scaling ETL in the Absence of Documentation” and “Metadata-First Design: How to Rebuild Without Breaking” have been widely read by peers and have sparked important conversations within the data community. He also regularly mentors junior engineers, advocating for modular design, code reusability, and the importance of storytelling in data engineering.
Sushil believes that intelligent orchestration and AI-assisted metadata management will be key components of ETL in the future. According to him, self-healing capabilities, real-time observability, and increased transparency are essential for the next generation of data infrastructure. "ETL shouldn't be a black box anymore," he said. Like any contemporary engineering system, it ought to be collaborative, explainable, and maintainable.
Sushil Kumar Tiwari is a prime example of the data infrastructure of the future, where legacy does not imply outdated and where rebuilding entails rethinking what is feasible. He has a history of resolving issues that seemed insurmountable and has a vision for the future.