We studied metal pollution in the sediments of Rijeka harbor, including anthropogenic influence during recent decades and at the present time. Sediment profiles were collected at ten sampling points. The concentrations of 63 elements in bulk sediment were obtained using ICP–MS, and the concentrations of selected elements were evaluated by statistical factor analyses. We also calculated metal-enrichment factors and geoaccumulation indices and constructed spatial-distribution maps. Mercury (Hg) was the heaviest pollutant, with concentrations exceeding 4 mg/kg. Silver (Ag) was the second most important pollutant, with constantly increasing values. The average concentrations of the most toxic elements were comparable to those found in sediments of other ports throughout the world, and their toxicity ranged from threshold values [chromium (Cr), arsenic (As)] and midrange-effect values [cadmium (Cd), lead (Pb), copper (Cu), zinc (Zn), nickel (Ni)] to extreme-effect values (Hg). Metal pollution has decreased during recent decades, except for Ag and barium (Ba).