Osh/Orp proteins transport sterols between organelles and are involved in phosphoinositide metabolism. The link between these two aspects remains elusive. Using novel assays, we address the influence of membrane composition on the ability of Osh4p/Kes1p to extract, deliver, or transport dehydroergosterol (DHE). Surprisingly, phosphatidylinositol 4-phosphate (PI(4)P) specifically inhibited DHE extraction because PI(4)P was itself efficiently extracted by Osh4p. We solve the structure of the Osh4p-PI(4)P complex and reveal how Osh4p selectively substitutes PI(4)P for sterol. Last, we show that Osh4p quickly exchanges DHE for PI(4)P and, thereby, can transport these two lipids between membranes along opposite routes. These results suggest a model in which Osh4p transports sterol from the ER to late compartments pinpointed by PI(4)P and, in turn, transports PI(4)P backward. Coupled to PI(4)P metabolism, this transport cycle would create sterol gradients. Because the residues that recognize PI(4)P are conserved in Osh4p homologues, other Osh/Orp are potential sterol/phosphoinositol phosphate exchangers.