The work presents the characterization of clays from sterile mining of the concentrated phosphatic materials of Araxá in Minas Gerais as pozzolanic materials. Three clays of distinct tones, namely, yellow (YC), red (RC), and intermediate (IC) clays, were used at different levels of excavation depth. The clays were calcined at three temperatures (680, 760, and 840 °C) in a muffle-type electric oven. The pozzolanic activity levels of the calcined clays was measured through the conductivity change in saturated Ca(OH)2 solutions and also from the result of the compressive failure load achieved by mortars with 35% of the Portland cement replaced with the calcined material. The results indicated that the calcined clays showed a high level of pozzolanic activity and can be used as a partial substitute for Portland cement, thus suggesting the possibility of recovering this sterile material.