Ideas
the fact you can get it to drop back down without a restart, but just by changing the graphics settings makes me think its probably just caching shaders or something like that. I have not experienced this problem at all in beta or release so might help to bump your settings down so the cache will not fill up maybe. Still something that should be addressed though if its caching more than a high end card can hold. But as a general statement for any game with all these new age fancy lighting settings you cant really tell what settings you should be using until after about 15 minutes (BL4 for example that came out around the same time). In BL4 you have to play for 15 minutes to let the shaders cache before you know if your system can actually handle the settings you chose.