Preheat your oven to 350°F (175°C). Grease and flour a 9x13 inch baking pan.
In a large mixing bowl, cream together the softened butter and granulated sugar until light and fluffy.
Add the mashed bananas, eggs, and vanilla extract to the butter mixture, mixing until well combined.
In a separate bowl, whisk together the flour, baking soda, salt, cinnamon, and nutmeg. Gradually add the dry ingredients to the banana mixture, stirring until just combined. If using, fold in the chopped nuts.
Pour the batter into the prepared baking pan and smooth the top with a spatula.
Bake in the preheated oven for 25-30 minutes, or until a toothpick inserted in the center comes out clean.
Allow the cake to cool in the pan for about 10 minutes, then transfer to a wire rack to cool completely.
While the cake is cooling, prepare the cream cheese frosting. In a medium bowl, beat together the softened cream cheese and butter until smooth. Gradually add the powdered sugar and vanilla extract, mixing until well combined. If the frosting is too thick, add milk one tablespoon at a time until the desired consistency is reached.
Once the cake is completely cool, spread the cream cheese frosting evenly over the top.