Idler Stand off question

Changed belt type and had to change the idler stand-off to line everything up. Idler is the large 3” with the stacked bearings offset to one side. The old stand off went inside the idler bearings and was almost flush with the outside of the outer bearing (slight gap between the washer and the end), this standoff goes through the rear bearing and just into the second bearing and the gap between where the washer will sit and the end of the standoff is quite a bit larger. Wanting to make sure it’s not going to cause a problem, if so can I turn a spacer down to sit inside the bearing and take up that space ?