In this paper, we propose a new skeleton detection method that is geometry-aware and can be learned in an end-to-end fashion. Recent approaches in this area are based primarily on the holistically-nested edge detector (HED) that is learned in a fundamentally bottom-up fashion by minimizing a pixel-wise cross-entropy loss. Here, we introduce a new objective function inspired by the Hausdorff distance that carries both global and local shape information and is made differentiable through an end-to-end neural network framework. When compared with the existing approaches on several widely adopted skeleton benchmarks, our method achieves state-of-the-art results under the standard F-measure. This sheds some light towards directly incorporating shape and geometric constraints in an end-to-end fashion for image segmentation and detection problems --- a viewpoint that has been mostly neglected in the past.