Prepare a loaf pan by wiping with a little softened coconut butter (or spray with oil), and line with a strip of parchment paper (letting excess come up the sides to assist removing the fudge to slice). Place all ingredients except banana in a food processor. Process until smooth (no pieces of coconut butter should remain, should smooth out fully into the sweet potato). Use a spatula to transfer the mixture to loaf pan. If using the banana, add about half of the mixture, lightly smooth out, layer on the banana, and then add remaining fudge mixture. Smooth over, and sprinkle with coconut sugar. Cover with foil and transfer to the freezer to set (for several hours). Once firm, remove fudge by lifting up the parchment (and using a spatula to stabilize underneath). Use a sharp knife to cut into squares. Transfer pieces of fudge to an airtight container and return to the freezer. Enjoy fudge frozen!