Using advanced in situ transmission electron microscopy, the lithiation and delithiation processes into graphene sheets are studied and significant differences are detected in the structural evolution of the system. Thin fcc lithium crystals with faceted shapes are formed between graphene sheets during lithiation, but are transformed into irregular patches during delithiation. It is found that defects such as vacancies in graphene and impurity atoms play the key role in these processes. Specifically, during intercalation the lithium crystals nucleate at vacancies in graphene, while upon delithiation the impurity oxygen atoms initially embedded at octahedral interstitial positions inside the lithium crystals agglomerate at the edges of the crystals, thus giving rise to the formation of amorphous lithium oxide patches, where lithium ions are trapped.