Bring ¾ cup water to a simmer in a medium saucepan over medium-high heat. Whisk in 3 Tbsp. cocoa powder until no lumps remain, then add milk and return to a simmer. Whisk in chocolate and sugar and cook, whisking frequently, until mixture is smooth and creamy and chocolate is melted, about 5 minutes.
Divide hot chocolate among mugs. Top with a marshmallow and dust with cocoa powder.
