Minerals, with their intricate chemical compositions and crystalline structures, play a pivotal role in diverse chemical processes, applications, and research. Traditionally, their classification was achieved through observational and chemical techniques. However, with increasing sample sizes, these methods often proved time-consuming. Recent advances in Artificial Intelligence (AI) and Deep Learning (DL) promise transformative improvements in the speed and accuracy of mineral classification. However, DL models, for all their precision, often operate as “black boxes”, making their decision-making opaque. To address this, our study introduces an innovative AI-powered framework for mineral classification, integrating state-of-the-art models with Explainable AI (XAI) and generative AI large language models (LLMs) like GPT-4. This framework not only categorizes a wide-ranging number of minerals but also elucidates the reasoning behind each classification. Through a combination of Swin Transformer V2 models for mineral identification, GradCAM for model transparency, and GPT-4 for detailed mineral information retrieval, the framework offers a balanced blend of performance, interpretability, and user-centric information. Available for public access, this system underscores the potential of AI to revolutionize mineral classification while staying attuned to the demands of clarity, transparency, and user education. The framework can be publicly accessed via https://huggingface.co/spaces/minatosnow/mineral_framework.