This paper aims to present the fundamental mathematical framework necessary for understanding reinforcement learning (RL) and provides an overview of RL algorithms, focusing on their application in Heating, Ventilation, and Air Conditioning (HVAC) systems. Specifically, the paper addresses the use of RL in Building Energy Management Systems (BEMS) to tackle the issue of high CO2 emissions resulting from HVAC operation, with RL proposed as a potential solution for reducing emissions by enhancing energy efficiency while maintaining occupant comfort. Additionally, the paper highlights the key advantages and limitations of RL when applied in intelligent buildings. The review bridges theoretical concepts and findings from the literature to identify appropriate algorithms for various problems and highlight research gaps. Furthermore, the future research direction of meta-RL is discussed, which trains agents on diverse tasks, offering strong generalization capabilities, making RL algorithms more adaptable to real-world conditions.