Swatches revert to default when items are moved in Build mode

I found that several items were resetting to their default swatch when I was moving them. I was using the swatch tool to change it, so it could be that.