The patron saint of Oliveri is Saint Joseph, celebrated on the second Sunday of October; the church dedicated to him stands in the town.
The first to record the town's name, then Labiri, was the Arab geographer Edrisi, commissioned by the Grand Count Roger, who described it as «a beautiful and graceful village, with a great castle by the sea, houses, good farming land, perennial streams on whose banks some mills were set up, and a fine port where abundant tuna fishing was carried out». The town's current name instead derives from the condottiero Carlo Oliveris, whose courtesy the inhabitants wished to commemorate. The castle, of Arab origin and dating from the 11th century, was in 1360 detached from the grant made to the monks by King Ferdinand of Aragon, who gave it, together with the fief and the tuna fishery, to his second son. Among the town's other religious buildings is the church of the Nativity.