Hydrogen produced using fossil fuel feedstocks causes greenhouse gas (GHG) emissions, even when carbon capture and storage (CCS) is used. By contrast, hydrogen produced using electrolysis and zero-emissions electricity does not create GHG emissions. Several countries advocating the use of 'clean' hydrogen put both technologies in the same category. Recent studies and strategies have compared these technologies, typically assuming high carbon capture rates, but have not assessed the impact of fugitive emissions and lower capture rates on total emissions and costs. We find that emissions from gas or coal based hydrogen production systems could be substantial even with CCS, and the cost of CCS is higher than often assumed. Carbon avoidance costs for high capture rates are notable. Carbon prices of $22/tCO2e would be required to make hydrogen from fossil fuels with CCS competitive with hydrogen produced from fossil fuels without CCS. At the same time there are indications that electrolysis with renewable energy could become cheaper than fossil fuel with CCS options, possibly in the near-term future. Establishing hydrogen supply chains on the basis of fossil fuels, as many national strategies foresee, may be incompatible with decarbonisation objectives and raise the risk of stranded assets.