A locally metric connection on a smooth manifold M is a torsion-free connection D on TM with compact restricted holonomy group . If the holonomy representation of such a connection is irreducible, then D preserves a conformal structure on M. Under some natural geometric assumption on the life-time of incomplete geodesics, we prove that conversely, a locally metric connection D preserving a conformal structure on a compact manifold M has irreducible holonomy representation, unless or D is the Levi-Civita connection of a Riemannian metric on M. This result generalizes Gallot's theorem on the irreducibility of Riemannian cones to a much wider class of connections. As an application, we give the geometric description of compact conformal manifolds carrying a tame closed Weyl connection with non-generic holonomy.