Sounds like that's your issue. I suspect a fairly heavy spring(maybe 1" in diameter)?
They'll be one idler that moves back and forth to keep tension on the belt. Connected
to that, likely a moving (back and forth) with a hole in it for that spring.
There should be another hole(stationary), nearby where the idler pulley would go to keep spring tension on the belt.
Here's the parts diagram, but it's hard to make out what goes where:
https://www.jackssmallengines.com/jacks-parts-lookup/manufacturer/husqvarna/tractors-ride-mowers/yth23v48-96043017500-2013-09/drive .