In this paper, the artificial intelligence genetic optimization algorithm is introduced into the signal demodulation process of acoustic telemetry system while drilling for the first time. And the modulation and demodulation processes of binary amplitude keying and binary phase keying, which are common in the field of telemetry while drilling, were numerically simulated. The experimental results show that the artificial intelligence genetic optimization algorithm has a very strong adaptability to the demodulation of acoustic telemetry signal while drilling. Especially for the binary phase keying modulation mode, it has the advantages of strong anti-noise interference ability and high demodulation accuracy. Obviously, artificial intelligence genetic optimization algorithm is a very promising demodulation method for acoustic telemetry signal while drilling.