Over the last few years, Deep learning (DL) approaches have been shown to outperform state-of-the-art machine learning (ML) techniques in many applications such as vegetation forecasting, sales forecast, weather conditions, crop yield prediction, landslides detection and even COVID-19 spread predictions. Several DL algorithms have been employed to facilitate vegetation forecasting research using Remotely Sensed (RS) data. Vegetation is an extremely important component of our global ecosystem and a necessary indicator of land cover dynamics and productivity. Vegetation phenology is influenced by lifecycle patterns, seasonality and weather conditions, leading to changes in their spectral reflectance. Various relevant information, such as vegetation indices (VIs), can be extracted from RS data for vegetation forecasting. Therefore, the Normalized Difference Vegetation Index (NDVI) is known as one of the most widely recognized indices for vegetation related studies. This paper reviews the related works on DL-based spatio-temporal vegetation forecasting using RS data over the period between 2015 and 2021. In this review, we present several DL-based studies and discuss DL algorithms and various sources of data that have been used in these studies. The purpose of this work is to highlight the open challenges such as spatio-temporal prediction issues, spatial and temporal non-stationarity, fusion data, hybrid approaches, deep transfer learning and large parameter requirements. We also attempt to figure out the future directions and limits of DL for vegetation forecasting.