This book presents a thorough discussion of the role of entropy in thermodynamic systems, with particular emphasis on its influence on the stability of matter. Challenging conventional energy-centric paradigms, the book explores how entropy leads to stable phases in systems, especially systems characterized by frustration. Through detailed explanations of key concepts of entropy, and concrete examples, such as entropic forces, adiabatic demagnetization, and entropy-driven phase transitions, the book elucidates the multifaceted nature of entropy. Special attention is given to the phenomenon of "order by disorder" in frustrated systems, where entropy plays a crucial role in determining thermodynamic stability. Both discrete and continuous spin systems are analyzed to illustrate the mechanisms by which entropy governs ordering processes. The treatment also extends to quantum fluctuations and their contribution to phase transitions, offering a comprehensive perspective across classical and quantum regimes. Extension of entropy to non-equilibrium states and also to non-extensive states is also explained. Intended for researchers and graduate and advanced undergraduate students in physics and related disciplines, this volume offers a detailed explanation of the concept of entropy, giving theoretical insights with illustrative case studies. It serves as a valuable resource for those who would like to more deeply understand entropy and its foundational role in the thermodynamics of complex systems.