The island represents part of an ancient volcano that erupted about 40 million years ago. The large, jagged rocks (Waverly Heights basalt) found throughout the island were formed by lava flows, and may be the oldest exposed rock in the Portland area. The island contains seven distinct habitats, including wetlands, forests, and grasslands. Across the river, there are high cliffs which are Elk Rock proper. A Native American legend holds that this was a good spot to stampede a herd of elk over the cliff. The island gets its name from being near Elk Rock.

Elk Rock Island was part of the original donation land claim of Milwaukie pioneer Lot Whitcomb and was known as Lot Whitcomb Island during the 1860s. It went through six owners before Scottish grain exporter and Portland businessman Peter Kerr (1862-1957) acquired the property in 1910 from the Rock Island Club, which operated a dance hall on the island. He gave the island to the City of Portland in 1940 with the requirement that it be preserved in its natural state. As Kerr put it, "Preserve it as a pretty place for all to enjoy." On October 29, 1954, the Kerr formally dedicated Peter Kerr Park with a bronze plaque.