We discuss the use of gravitational wave sources as probes of cosmology. The inspiral and merger of a binary system, such as a pair of black holes or neutron stars, is extraordinarily bright in gravitational waves. By observing such systems it is possible to directly measure an absolute distance to these sources out to very high redshift. When coupled with independent measures of the redshift, these "standard sirens" enable precision estimates of cosmological parameters. We review proposed GW standard sirens for the LIGO and LISA gravitational wave observatories. Percent-level measurements of the Hubble constant and the dark energy equation-of-state may be feasible with these instruments.