The word wear is it only used to talk about clothes and to refer to a weary life (for example) or?

Yes. You can wear a smile or jewelery. Using the same word you can wear down your rival and mountains can wear away.