Worries about higher prices from tariffs, a slowing economy and falling stock prices have taken a toll on consumers.