The temple was build in 967 by king Yajnavaraha,a Brahmin priest and counsellor of king Rajendravarman II and his brother. Before the foundation stele was discorered in 1936 it was assumed that the temple was from a much later date(almost 1300) because of the refinement of its ornamentations. The stele praises King JayavarmanV,who was king when the Banteay Srey was build and Yajnaveraha and his brothers, the builders of the temple.

The pink sandstone structure is small compared to other Angkor monuments. This is probably because the Banteay Srey was not build by a king, and it would be considered disrespectful to build a monument larger and grander than a Royal sponsored temple. To prevent further deterioration of this magnificent and much visited temple ,the inner sanctuary has been closed to visitors.