We present spectra taken with the Infrared Spectrograph on Spitzer covering the 5-38 μm region of the 10 ultraluminous infrared galaxies (ULIRGs) found in the IRAS Bright Galaxy Sample (BGS). There is a factor of 50 spread in the rest-frame 5.5-60 μm spectral slopes, and the 9.7 μm silicate optical depths range from at least τ 9.7≤ 0.4 (A V~ 8) to τ 9.7≥ 4.2 (A V≥ 78). There is evidence for water ice and hydrocarbon absorption and C 2 H 2 and HCN absorption features in 4 and possibly 6 of the 10 BGS ULIRGs, indicating shielded molecular clouds and a warm, dense ISM. We have detected [Ne V] emission in 3 of the 10 BGS ULIRGs, at flux levels of 5-18× 10-14 ergs cm-2 s-1 and [Ne V] 14.3/[Ne II] 12.8 line flux ratios of 0.12-0.85. The remaining BGS ULIRGs have limits on their [Ne V]/[Ne II] line flux ratios, which range from≤ 0.15 to≤ 0.01. Among the BGS ULIRGs, the AGN fractions implied by either the [Ne V]/[Ne II] or [O IV]/[Ne II] line flux ratios (or their upper limits) are significantly lower than implied by the MIR slope or strength of the 6.2 μm PAH EQW feature. There is evidence for hot (T> 300 K) dust in five of the BGS ULIRGs, with the fraction of hot dust to total dust luminosity ranging from~ 1% to 23%, before correcting for extinction. When integrated over the IRAC-8, IRS blue peak-up, and MIPS-24 filter bandpasses, the IRS spectra imply very blue colors for some ULIRGs at z~ 1.3. The large range in diagnostic parameters among the nearest ULIRGs suggests that matching survey results to a small number of templates may lead to biased results about the fraction of luminous dusty starbursts and AGNs at high z.