It is commonly believed that the heavenly messenger took the Urim and Thummim and the plates from Joseph following the loss of the 116 pages by Martin Harris. Actually the plates and interpreters, as they were called, were taken earlier and not as a result of the loss of the translated pages but as Joseph later said, "in consequence of my having wearied the Lord in asking for the privilege of letting Martin Harris take the writings."  Shortly after the loss of the pages, the angel returned the interpreters long enough for Joseph to receive a revelation (D&C 3) on the reason for the loss of the privileges, before taking them again. Shortly after sincere repentance both the plates and interpreters were returned for the work to continue.

HC 1:21-23