This center was built in the early 1940s as part of an enormous federal housing project for ship workers mobilized for the World War II effort. At the time of its construction, it included recreational facilities as well as a restaurant, a fire department with three engines, and administrative offices for the housing project. The center was acquired by PP&R in 1953. With funding from the 1994 bond measure, a major renovation resulted in the expansion of the east and west wings. A portion of the 2002 parks levy was used to build a new active recreation wing in 2007, nearly doubling the center's size with PP&R’s largest gymnasium, a fitness room, youth game room, dance studio, and computer lab.

A wind-driven kinetic sculpture by Jerry Mayer was installed in front of the center on August 31, 1999. Mayer worked closely with the North Portland community to develop Whirlymajig, an altered water pump windmill with a 5-ft diameter fan wheel atop a 30-ft steel flag pool. Driven by the wind through a system of gears, drive shafts, axels, and drive chains, the sculpture's tail section consists of variously moving aluminum cutouts of arms and legs performing physically and mentally challenging tasks.

As part of the center's extensive 2007 renovation, a wall sculpture by Laura Bender and John Early was installed in the lobby. From Here to There is a metaphor for the venture of exploration, a story about movement, coordination, and change that references things historical, nautical, and navigational. Movement refers to people relocating here, ships sailing rivers and seas, driftwood swept up on the shores of the Columbia River, and people meeting and forming relationships as they dance, exercise, and move around one another.

In 2012, the center was renamed in honor of Charles Jordan, Portland’s first African-American City Commissioner and Portland Parks & Recreation Director. He was the force behind Portland landmarks such as Pioneer Courthouse Square, the Interstate Firehouse Cultural Center, Delta Park, and Southwest Community Center. On the national stage, Charles Jordan broadened the agenda of the environmental movement and land conservation to make it more inclusive. He is known for his groundbreaking approach to putting people - particularly people of color - at the heart of the American Conservation Movement.