Colorectal cancer (CRC) is one of the most common cancers worldwide, and its diagnosis and classification remain challenging for pathologists and imaging specialists. The use of artificial intelligence (AI) technology, specifically deep learning, has emerged as a potential solution to improve the accuracy and speed of classification while maintaining the quality of care. In this scoping review, we aimed to explore the utilization of deep learning for the classification of different types of colorectal cancer. We searched five databases and selected 45 studies that met our inclusion criteria. Our results show that deep learning models have been used to classify colorectal cancer using various types of data, with histopathology and endoscopy images being the most common. The majority of studies used CNN as their classification model. Our findings provide an overview of the current state of research on deep learning in the classification of colorectal cancer.