Preheat the oven to 400°F (200°C). Roll out the puff pastry on a lightly floured surface and cut into even rectangles.
Place the pastry rectangles on a baking sheet lined with parchment paper. Prick them with a fork and place another baking sheet on top to keep them flat.
Bake for 15-20 minutes or until golden brown. Let them cool completely.
In a saucepan, heat the milk and 2 tablespoons of sugar. In another bowl, mix the cornstarch and remaining sugar.
Gradually add the cornstarch mixture to the hot milk, stirring constantly until thickened. Remove from heat and stir in butter and vanilla extract. Let cool.
Fold the whipped cream into the cooled pastry cream.
Layer the puff pastry with the pastry cream in between. Use 3 layers of pastry with 2 layers of cream.
Mix powdered sugar, water, and lemon juice to make the icing. Spread on top of the mille-feuille.
Refrigerate the assembled mille-feuille for at least 1 hour before serving.