Abstract: Hetian Yu is a polycrystalline aggregate. After polishing, it can show greasy luster, waxy luster, or glassy luster, and a small proportion may show porcelain luster, but the predomioant is the greasy luster, which is referred to the market as oiliness in the market. The commercial value of Hetian Yu is directly proportional to the intensity of greasy luster. When it is quite similar to mutton fat, the jade is called Yangzhi Yu, which is of high value. At present, the evaluation of the intensity of Hetian Yu's greasy luster relies primarily on the naked-eye judgment based on the experience of gemmological professionals, which inevitably involves subjectivity and errors. Polarizing microscope test showed that the luster of Hetian Yu relates to the thickness and uniformity of the structure, the more delicate and uniform the structure, the stronger the greasy luster. Therefore, in this study, the authors labeled and divided the image dataset of Hetian Yu, and comprehensively compared the transfer learning of different models. The results showed that the transfer learning on the ImageNet pre-trained model can achieve a 92.06% classification accuracy by using the MobileNetV3_large_x1_0 deep learning network. This model demonstrates excellent classification performance and effectively addressed the long-standing challenge in Chinese gemmological community regarding the quality assessment of greasy luster of Hetian Yu. Moreover, the algorithmic model that classified the degree of greasy luster of Hetian Yu—into strong, medium, and weak categories—exhibits strong generalization capability.