The reopening is met with resistance from Muslims who are refusing to enter through the new metal detectors.
The site is known to Jews as Temple Mount and to Muslims as Noble Sanctuary and is an area of huge religious importance to both faiths.
The compound was closed for the first time in decades after three Arab citizens of Israel fled to the site after killing two police officers.
The gunmen fled to the holy site where they were shot dead by security forces.
The reopening was met with resistance from Muslims who refused to enter due to the installation of metal detectors outside.
Sheikh Omar Kiswani, director of the Al-Aqsa mosque that sits within the compound, said: “We reject the changes imposed by the Israeli government – we will not enter through these metal detectors.”
Police said some worshippers knelt to pray outside after refusing to go through them.
Israeli police said on Sunday morning that security forces had shot dead a Palestinian assailant behind another shooting attack on Saturday.
Police spokesman Micky Rosenfeld said police tracked down the 34-year-old suspect in a joint operation with the military and shot him dead after he opened fire with an automatic weapon.
The Jerusalem shrine has been the scene of repeated confrontations, including during the current wave of violence.
It is managed by the Jordanian authorities and is adjacent to the Western Wall – the holiest prayer site in Judaism.