Heat oven to 375ºF. Line two baking sheets with parchment; set aside.
In a bowl sift together the flour, cocoa powder, baking soda, and salt; set aside.
In the bowl of an electric mixer fitted with the paddle attachment combine the butter and sugar on medium-high speed until pale and fluffy, about 2 minutes. Reduce speed and mix in egg, milk and vanilla. Mix in flour mixture, ½ cup at a time, until combined; scraping down the sides and bottom of the bowl as needed.
Use a 1¾ inch cookie scoop(#40 size) to drop 1½ tablespoons dough onto prepared baking sheets, spacing about 2 inches apart. Bake, rotating sheets halfway through, until cookies are just set, 8 minutes.
Remove and immediately press a marshmallow half on top of each cookie. Bake again until marshmallows begin to melt, 2 minutes more. Let cool completely on baking sheets on wire racks.
For the frosting:
Make frosting by placing confectioners’ sugar in a mixing bowl. Melt butter with cocoa powder in a saucepan on the stove or in the microwave. Add the melted butter mixture to the confectioners’ sugar and add the milk and vanilla. Whisk until smooth.
Spread about 1 tablespoon frosting on top of each cookie to cover marshmallow. Let stand until set, about 10 minutes. Cookies can be stored in single layers in airtight containers at room temperature for up to 2 days.