This paper investigates the influence of adding vegetal fibers on thermal and acoustic performance based on natural hydraulic lime. Mortar samples with 10% weight of vegetal fibers were fabricated adding water to obtain easily workable mortars with good consistency; their performance was compared to mortar samples without vegetal fibers. The fibers were of different types (rice husk, spelt bran, and Khorasan (turanicum) wheat chaff) and size (as-found and ground form). Thermal performance was measured with the Small Hot Box experimental apparatus. Thermal conductivity was reduced in the 1-11% range (with Khorasan wheat chaff and rice husk); no significant reduction was found with spelled bran in the mixture. When ground, fibers were characterized by both good thermal and acoustic absorption performance; a reduction of 6-22% in thermal conductivity λ was achieved with spelled bran (λ = 0.64 W/mK) and rice husks (λ = 0.53 W/mK), whereas the Khorasan wheat chaff had the highest sound absorption average index (0.38). However, the addition of fibers reduced sound insulation properties due to their low weight densities. This reduction was limited for rice husks (transmission loss value was only 2 dB lower than the reference).