"and that to retain it would be like keeping new wine in old bottles..."
See in text (Economy)
Thoreau is referring to the biblical book of Matthew 9:17. In biblical times, "bottles" were fashioned from animal skins. Over time, the skin could become brittle, so if an old skin were filled with new wine, the wine would ferment, expand, and destroy the old skin, thereby not retaining any of the wine.
See in text (Where I Lived, and What I Lived For)
In Greek mythology, the Titan Atlas was condemned by Zeus to hold up the sky and the heavens on his shoulders for eternity. Thoreau invokes this myth to illustrate his willingness to enjoy these advantages even if they require enormous amounts of strength and endurance.