As one of the latest emerging HVAC technologies, the Variable Refrigerant Flow (VRF) system with heat recovery (HR) configurations has obtained extensive attention from both the academia and industry. Compared with the conventional VRF systems with heat pump (HP) configurations, VRF-HR is capable of recovering heat from cooling zones to heating zones and providing simultaneous cooling and heating operations. This can further lead to substantial energy saving potential and more flexible zonal control. In this paper, a novel model is developed to simulate the energy performance of VRF-HR systems. It adheres to a more physics-based development with the ability to simulate the refrigerant loop performance and consider the dynamics of more operational parameters, which is essential for representing more advanced control logics. Another key feature of the model is the introduction of component-level curves for indoor units and outdoor units instead of overall performance curves for the entire system, and thus it requires much fewer user-specified performance curves as model inputs. The validation study shows good agreements between the simulated energy use from the new VRF-HR model and the laboratory measurement data across all operational modes at sub-hourly time steps. The model has been adopted in the official release of the EnergyPlus simulation program since Version 8.6, which enables more accurate and robust assessments of VRF-HR systems to support their applications in energy retrofit of existing buildings or design of zero-net-energy buildings.