It is considered that the origin of this temple is that a statue of Buddha was constructed on the grounds in order to put down an internal insurrection at the end of the 7th century. Starting in the Heian Period (710~793), this temple was the Tendai School of Buddhism. This school lasted up until the Kamakura Period (began in 1192), but at the beginning of the Kamakura Period, this temple was changed into a Zen school.

Ashikaga Takauji, the first shogun of the Ashikaga Shougnate, deeply respected this temple. Thanks to him, this temple was considered as a very important temple and was given government support. The support from the government made the temple very prosperous.

However, during the Sengoku Period (Age of the Civil War), the grounds of the temple were ruined because the temple was located on a strategy point and was considered an essential point of defense.

The highest priest, Ohara, who has been credited as the brains behind Imagawa Yoshimoto’s actions (the leading daimyo/feudal lord), re-constructed this temple and became the chief. When Tokugawa Ieyasu (a child at the time) was sent to the Imagawa Clan as a hostage, Ohara educated him and greatly influenced his life.

After Ieyasu became a shogun during the Edo period, he invited the diplomatic mission from Korea to Sunpu Castle in order to restore relations with the dynasty in Korea. It is recorded that there were 467 people invited. Tokugawa Ieyasu used the Seikenji Temple as a residence for them.