Bluebell knows more about computer tech than I do, so take her advice first. But I figured I'd throw my two cents in here just because I too have a 2017 iMac and recently looked into possible upgrades. My specs are a bit different than yours, but here they are just for reference. I'm able to run my game currently at highest graphic settings, with around 13 GB in my mods folder and with all DLC installed.
iMac, Retina 4K, 21.5-inch, 2017
3.4 GHz Quad-Core Intel Core i5
8GB 2400 MHz DDR4 Memory
Radeon Pro 560 Graphics with 4GB VRAM
I understand the feeling of disappointment from spending a lot of money on a Mac desktop only to run into problems with your gaming. My guess here would be the same as Bluebell's - it probably comes down to your Intel graphics card just having a hard time handling that quantity of CC. Which sucks, because the graphics cards on Macs are not upgradeable. The only option is using an external graphics card to give your computer more VRAM, but I'm not sure if that even works with TS4 - it's just something I stumbled across when I thought my video card was the problem with a graphics issue I've been having (turns out, it's not). So before considering an external graphics card (which from what I've seen is pretty expensive) I'd research it a good deal more and find out if it would even be compatible with or helpful for a game like TS4.
Also, from what I understand, the memory (RAM) in a 2017 iMac is service-center-upgradeable (not sure if this is only for the Retina one, I'll put instructions for how to check below). When I looked into it for my own computer, because I'd like to be able to run TS4 simultaneously with a few other programs, like I used to before I added so much CC/DLC to my game, I found out that with my specific model, the memory is not user-upgradeable, but it is upgradeable through an Apple Authorized Service Provider. Depending on where you're located, these providers could be either all over the place and very easy to work with, or next-to-impossible to find/afford. I live in a major metropolitan area so I've got Apple Authorized Service Providers everywhere and the labor cost is somewhat reasonable. However the parts cost is still expensive of course... and I personally don't think that upgrading your memory cards would totally fix your problems. For a few reasons.. I don't think any iMac, except maybe the brand new Pros which are around $5000, are going to run TS4 on highest graphic settings with lots of CC+mods with absolutely no lag. The graphics cards in "normal" (non-$5000) Macs just aren't suited for intensive gaming.. at least when compared with PC's that have NVIDIA graphics cards. Additionally, the graphics drivers in Macs are not updated super often, because they come with Apple's software updates, instead of being offered by the card-maker itself.
All of this to say, try out the things that Bluebell mentioned to reduce lag. Also, you can also launch Activity Monitor to monitor how much of your memory/graphics capabilities TS4 is using. You'll want to look at GPU for graphics and CPU for memory, I believe.
You could also try doing a SMC reset to reset the graphics-related functions.. sometimes that helps. But it's not a miracle-fix. https://support.apple.com/en-us/HT201295 Make sure to look for the instructions for your specific model. For my desktop it's just a matter of shutting down, and actually unplugging the power cord, then waiting a bit and plugging it back in. It's different for different models though.
And to find more info about upgrading memory for your specific model, click the Apple icon at the top left, click About This Mac, click the Memory tab, and click Memory Upgrade Instructions. This will take you to an Apple Support page that will tell you if your specific model of iMac is user-upgradeable (it's not), service-center upgradeable (maybe), or not upgradeable at all.
Before sinking any money/time into upgrading your Mac, I would definitely compare the cost/specs of a gaming PC versus those Mac upgrades. Odds are you could find a PC that would run TS4 much better at a price unrivaled by Apple. That said, I personally will stick with Apple forever, because they have so many benefits over PC outside of gaming.
Lastly, call Apple and ask for help. They actually can be super helpful. If your computer is laggy in any way outside of TS4, fixing those issues with the help of an Apple rep could very well improve your TS4 performance. Doing a reinstall of the Catalina operating system fixed a lot of general Finder lag for me, and that helped lag in TS4 as well. There are lots of options to try before thinking you have to upgrade! And just know I'm right here with you struggling lol 🙂