I delete the Accessory and Groups caches almost every time I finish playing, or before I start up the next time. I occasionally delete thumbnails from the Thumbnails folder within Legacy and the Thumbnails within each neighborhood. Within the Logs folder you can also delete all of those. Any of these files can hold onto outdated or unnecessary information or become corrupted, or just take up space as they grow.
The game will generate new files as needed for all of these when you start up again.
What absolutely cannot be deleted without some work in SimsPe, and with batbox and hoodchecker, is the character files. The ones like N004_user00000.package and the lot files like N004_lot00.package. Those are Sims and Lots in your game. Deleting character files would result in Sims without any information, causing corruption. Some of the lots might be okay if you deleted them, others are tied to Sims that live or lived there and you'd be leaving behind references to something that didn't exist anymore, again causing corruption.
If you really want to have fewer of those types of files you either need to download and install cleaned neighborhoods (there are some out there available, like those from MeetMetotheRiver) or aquaint yourself with the needed programs and do the appropriate steps yourself. The easiest thing to do obviously, is leave them and deal with it.