President Donald Trump said on Tuesday that US threats from Pyongyang would be met with "fire and fury", which prompted North Korea to say it was considering plans for a missile strike on the USA pacific territory of Guam.
Disappointing earnings also helped pull the market lower Wednesday.
Mattis said North Korea must stop isolating itself and "stand down" in its pursuit of nuclear weapons, and Pyongyang "should cease any consideration of actions that would lead to the end of its regime and the destruction of its people".
While markets shrugged off news earlier in the day that North Korea had developed nuclear warheads, Trump's threatening reaction later in the day prompted a more nervous reaction from investors.
The influential financial stocks were among the biggest drags on the index, with Royal Bank of Canada down 1.0 percent to C$93.36, and Manulife Financial Corp falling 2.8 percent to C$24.93.
The Dow Jones industrial average lost 71 points, or 0.3 percent, to 22,013.
Nine of the 11 major S&P sectors were lower, with the consumer discretionary index's 10.2 percent fall leading the decliners.
The pan-European STOXX 600 was down 0.7% at closing, France's CAC 40 fell 1.4% after a vehicle hit a group of soldiers in Paris in what is thought to have been a deliberate act, and a fall in bond yields saw Germany's DAX down 1.1%.
Markets were also lower in Europe and Asia. Earlier in the week, Trump said the United States would unleash "fire and fury" on North Korea if it continued to threaten the United States.
Brian Jacobsen, chief portfolio strategist at Wells Fargo Funds Management, said: "While the tough talk about the potential for war (between the USA and North Korea) is scary, investors have heard it many times before". Germany's DAX Index fell 0.6 percent, while the UK's FTSE 100 Index retreated 1.2 percent, and France's CAC 40 Index slid 1.4 percent.
The CBOE Volatility Index, the most widely followed barometer of expected near-term stock market volatility, was up 1.17 points at 12.12 points, its highest level in a month. The company posted better-than-expected quarterly earnings but its sales missed expectations.
Bond prices rose. The yield on the 10-year Treasury note slipped to 2.19 percent from 2.20 percent late Thursday.
Avis Budget Group slumped 9.9% after the auto rental company cut its guidance following a weak second quarter.
ENERGY: Benchmark U.S. crude lost 23 cents to $48.94 per barrel in electronic trading on the New York Mercantile Exchange. Brent crude, the worldwide standard, lost 23 cents to 52.14 dollars a barrel in London. The euro slid to 1.1752 dollars from 1.1793.