Preheat oven to 375° F. Place chopped walnuts on a baking sheet in an even layer. Toast in the preheated oven for 7-10 minutes until lightly golden and fragrant. Be careful not to over toast. Remove from oven, toss with melted butter, rosemary, brown sugar, salt, and pepper. Place back in the oven for about a minute so everything gets nice and toasty, melty, and combined. Remove and set aside.