Just four years ago, only a few high redshift galaxies were known within the first billion years. However, since then, sufficient HST ACS and NICMOS data have been taken that we can construct very large samples of galaxies at early times and study the evolution of the rest-frame UV LF all the way out to redshifts of 8 -- just 700 million years after the Big Band. The larger, more statistically significant samples come from the deep, wide-area ACS data (including more than 4800, 1500, and 600 galaxies at redshifts of 4, 5, and 6, respectively), allowing us to come to firm conclusions about how the luminosity function of galaxies evolves at very early times. Our own analyses demonstrate that the faint-end slopes of these luminosity functions are remarkably uniform and steep (alpha = -1.7), indicating little evolution from z~6 => z~4. On the other hand, the characteristic luminosity L* of galaxies brightens considerably over this period, demonstrating that the galaxy population actually builds up quite dramatically at early times -- exactly as we might have expected from popular hierarchical models of galaxy formation. Pushing back even further in cosmic history to 400-700 million years after the Big Bang, we also have detected galaxies at z~7-8 in deep NICMOS observations and set limits at z~10 directly. Spitzer observations of our z~7-8 sample have been used to estimate masses and ages, suggesting substantial formation as early as z~9-10. Our high-redshift findings strongly suggest that the evolution we see at z~4-6 continues into the reionization epoch.