You will need to buy some Popsicle sticks, most stores with a craft or cake decorating department have then, and they are fairly cheap.

1 box instant chocolate pudding (you can use the sugar free if you like)
1/4 cup sugar or sugar substitute
3 1/2 cups milk (you can use whole milk, or a low fat milk)

Mix ingredients and pour into molds or just use plastic cups, place in freezer until partly frozen, add the stick and let freeze until solid.

If you run a little warm water over the outside of the cup, they will drop right out easily for you.