Make sure that your hub, hub lip, spacer and all surfaces, as well as your wheel bore are all clean and free of any dirt/corrosion/etc.
You really shouldn’t be getting any vibration from hub centric spacers if all of those surfaces are clean. If you are, it might be something else like your wheels.