Distinctive regional characteristics in different natural gas markets have increased the difficulty in accurately forecasting natural gas prices. Moreover, the natural gas markets have experienced great structural instability due to advancement in technology and rapid financialization over the past few decades. We employ three classes of flexible time-varying parameters models to evaluate the effects of the regional characteristics and structural instability on natural gas prices forecasts. Using the data from the US, EU and Japanese markets from 1992 to 2019, we find that allowing different time-varying dynamics of the model parameters is crucial in forecasting natural gas prices. For Japan and the EU, models allowing gradual changes in coefficients and drastic changes in volatility have the best forecasting performance, while most of forecasting gains appear to have come from allowing gradual changes in volatility for the US. In addition, embedding t-distributed errors can further improve the forecast accuracy.