Abstract Radiotherapy for malignancies in the head and neck can cause common complications that can result in tooth damage that are also known as radiation caries. The aim of this study was to examine damage to the surface topography and calculate changes in friction behavior and the nano-mechanical properties (elastic modulus, nanohardness and friction coefficient) of enamel and dentine from extracted human third molars caused by exposure to radiation. Enamel and dentine samples from 50 human third molars were randomly assigned to four test groups or a control group. The test groups were exposed to high energy X-rays at 2 Gy/day, 5 days/week for 5 days (10 Gy group), 15 days (30 Gy group), 25 days (50 Gy group), 35 days (70 Gy group); the control group was not exposed. The nanohardness, elastic modulus, and friction coefficient were analyzed using a Hysitron Triboindenter. The nano-mechanical properties of both enamel and dentine showed significant dose-response relationships. The nanohardness and elastic modulus were most variable between 30-50 Gy, while the friction coefficient was most variable between 0-10 Gy for dentine and 30-50 Gy for enamel. After exposure to X-rays, the fracture resistance of the teeth clearly decreased (rapidly increasing friction coefficient with increasing doses under the same load), and they were more fragile. These nano-mechanical changes in dental hard tissue may increase the susceptibility to caries. Radiotherapy caused nano-mechanical changes in dentine and enamel that were dose related. The key doses were 30-50 Gy and the key time points occurred during the 15th-25th days of treatment, which is when application of measures to prevent radiation caries should be considered.