The inert Higgs doublet model contains a stable neutral boson as a candidate of dark matter. We calculate cross section for spin-independent scattering of the dark matter on nucleon. We take into account electroweak and scalar quartic interactions, and evaluate effects of scattering with quarks at one-loop level and with gluon at two-loop level. These contributions give an important effect for the dark matter mass to be around m h/2, because a coupling with the standard model Higgs boson which gives the leading order contribution should be suppressed to reproduce the correct amount of the thermal relic abundance in this mass region. In particular, we show that the dark matter self coupling changes the value of the spin-independent cross section significantly.