Wash and peel the potatoes, then cut them into ~3/4 inch cubes. Place chopped potatoes in a large pot. Add water to cover potatoes, plus 1 extra inch on top. Cover pot with a lid, and heat on high until boiling. Remove lid, turn down heat to medium low, and cook an additional ~5 minutes, or until potatoes are fork tender. Cover most of the pot with the lid, and very carefully pour out excess water into a sink. Let potatoes sit in pot, uncovered for 5 minutes to let any additional water evaporate.

While potatoes are cooking, finely chop the fresh rosemary, and set aside until ready to use.

After cooked potatoes have rested for 5 minutes, add the butter, olive oil, goat cheese, and milk to the pot. Mash, and mix ingredients together with a potato masher, or hand mixer until desired consistency is achieved. Stir in rosemary and salt to combine well. Taste, and add additional seasoning if desired before serving.