A new source of natural dyes, which possessed antibacterial and UV protection properties, was proposed. Natural dyes were extracted from Fuji apple peel. The extract contained two anthocyanins and five quercetin glycosides. Silk and cotton fabrics were dyed using the extract with and without six types of mordants: Mg2+, Ca2+, Al3+, Ti4+, Fe3+, and Cu2+. The properties of the dyed fabrics were investigated by various means. The K/S values were used to show relative color strength of the dyed fabrics and the CIELAB color system was used to show the color hue of the dyed fabrics. The crystallinity of the silk was lower than that of the cotton. The silk was dyed deeper than the cotton regardless of the dyeing conditions, including dyeing temperature, dyeing period, and solution pH. This result comes from the existence of carboxyl and amino groups and low crystallinity of silk fibers. Fabrics with various colors, including pink, green, brown, and gray, were obtained by mordant dyeing. The dyed fabrics had splendid performance; that is, antibacterial and UV shielding properties. Color fastness to washing (color change) was comparably low but was enhanced with Ti4+ and Fe3+ mordants.