Formation control of autonomous underwater vehicles is regarded as a promising way in situ sensing and monitoring of marine activities. However, due to the harsh marine environment, the full autonomy is still unreachable to fulfill complex marine tasks. This letter develops a teleoperation formation control system toward human-on-the-loop for AUVs. A board learning (BL) based estimator is first designed to estimate the real-time states of master operator and slave AUVs, through which the BL-based formation controller is developed to steer AUVs to keep specific formation shape. Compared with the previous works, the BL-based estimator can capture the real-time states even with time delays, and meanwhile the BL-based formation controller can achieve bilateral teleoperation without model parameters. Simulation results are conducted to verify our solution.