Preheat oven to 350°F. Grease 9-inch round cake pan and line bottom with parchment paper. Arrange peach slices in spiral pattern covering bottom of pan.
Melt 1/4 cup butter in saucepan over medium heat. Stir in brown sugar until dissolved and bubbling. Pour caramel evenly over peaches.
Cream remaining 1/4 cup butter with granulated sugar until light and fluffy. Beat in eggs one at a time, then add vanilla extract.
Combine flour, baking powder, and salt in separate bowl. Add half dry ingredients to butter mixture, followed by milk, then remaining dry ingredients. Mix until just combined.
Spread batter carefully over peaches, starting from edges and working toward center. Create even layer covering all peaches.
Bake for 35-40 minutes until top springs back and toothpick inserted in center comes out clean. Cool 10 minutes in pan, then invert onto serving plate.
