We present Keck/HIRES observations of six metal-poor stars in two of the ultra-faint dwarf galaxies orbiting the Milky Way, Ursa Major II and Coma Berenices. These observations include the first high-resolution spectroscopic observations of extremely metal-poor stars ([Fe/H]<-3.0) stars not belonging to the Milky Way (MW) halo field star population. We obtain abundance measurements and upper limits for 26 elements between carbon and europium. The entire sample of stars spans a range of -3.2<[Fe/H]<-2.3, and we confirm that each galaxy contains a large intrinsic spread of Fe abundances. A comparison with MW halo stars of similar metallicities reveals substantial agreement between the abundance patterns of the ultra-faint dwarf galaxies and the MW halo for the light, alpha and iron-peak elements (C to Zn). This agreement contrasts with the results of earlier studies of more metal-rich stars (-2.5<[Fe/H]<-1.0) in more luminous dwarf spheroidal galaxies (dSphs), which found significant abundance discrepancies with respect to the MW halo data. The abundances of neutron-capture elements (Sr to Eu) in the ultra-faint dwarf galaxies are extremely low, consistent with the most metal-poor halo stars, but not with the typical halo abundance pattern at [Fe/H]>-3.0. Our results are broadly consistent with a galaxy formation model that predicts that massive dwarf galaxies are the source of the metal-rich component ([Fe/H]>-2.5) of the MW halo, but we also suggest that the faintest known dwarfs may be the primary contributors to the metal-poor end of the MW halo metallicity distribution.