Peel the bananas and cut them in half crosswise. Insert a popsicle stick into the flat end of each banana half.
Place the bananas on a baking sheet lined with parchment paper and freeze for at least one hour, or until firm.
In a microwave-safe bowl, combine the chocolate chips and coconut oil. Microwave in 30-second increments, stirring after each, until completely melted and smooth.
Dip each frozen banana into the melted chocolate, turning to coat evenly. Allow the excess chocolate to drip off.
If using, immediately sprinkle colorful sprinkles over the chocolate before it sets.
Place the coated bananas back on the parchment-lined baking sheet and return to the freezer until the chocolate is set, about 15 minutes.