Semantic segmentation is a fundamental task in computer vision and image processing. Although existing methods based on the fully convolutional network (FCN) have greatly improved the accuracy, it still does not show satisfactory results on tiny objects and boundary regions. One of the problems is that the current FCN-based methods ignore details such as the image’s contours and edges because of over downsampling operations in the CNN encoder backbone. In signal processing, excessive down-sampling will incur spectrum aliasing, thus losing high-frequency details. This work presents a high-frequency graph convolution operation to solve the above problems. Traditional image processing generally uses the high-pass filter to extract image contours. We accordingly suppose that the high-frequency information is vital for the extractions of boundary clues and details. We implement our strategy and conduct experiments on the Cityscapes dataset, which demonstrate the effectiveness of our high-frequency graph convolution block on semantic segmentation. The proposed method achieves comparable performance and dramatically improves the performance of small objects like the rider, traffic signs, etc.