Scalable Big Data Processing Using Cloud Computing and Distributed Machine Learning

Abstract

The volume, velocity, and variety of data produced by digital systems have outgrown what single machines can store, process, or learn from. Cloud computing provides elastic, on-demand infrastructure, and distributed machine learning (ML) spreads model training across many machines. Together they make large-scale data analytics practical. This review synthesizes fifteen studies published between 2016 and 2025, organized around four themes: (1) cloud data platforms, including container orchestration and lakehouse storage; (2) distributed data processing for batch and streaming workloads; (3) distributed ML, from data-parallel training to large-model and federated approaches; and (4) resource scheduling, workflow orchestration, and cost efficiency, including the cost of running large language models (LLMs) at scale. The review finds that unified processing engines, data-parallel training frameworks, and intelligent scheduling have greatly improved scalability. Practical performance, however, depends on communication overhead, efficient resource allocation, governed pipeline workflows, and cost control. The review proposes a layered framework and outlines directions for future research.