As machine learning systems are increasingly employed in high-stakes tasks, algorithmic fairness has become an essential requirement for deep learning models. In this paper, we study how to transfer fairness under distribution shifts, a crucial issue in real-world applications. We first derive a sufficient condition for transferring group fairness. Guided by it, we propose a practical algorithm with a fair consistency regularization as the key component. Experiments on synthetic and real datasets demonstrate that our approach can effectively transfer fairness as well as accuracy under distribution shifts, especially under domain shift which is a more challenging but practical scenario.