Basically your second and first floor rooms need to match. So, you have to remove both the floor of the second floor and the ceiling of the first floor. If they don't match, it won't be removable unless you make an even smaller room within that where you want there to still be a floor/ceiling.
First make your rooms, one on top of the other:
https://i.imgur.com/yX3eLoZ.jpg
https://i.imgur.com/IY65ktI.png
Then remove the floor of the top story:
https://i.imgur.com/CSpkl5d.jpg
Then you need to click the yellow portion to remove the ceiling of the bottom story while on the top story after the floor has been removed (you may have to "remove the ceiling" then "build" it, then remove it again for it to actually go away):
https://i.imgur.com/2RQsmHW.jpg
You're left with an open floor where you can add rooms and doors and such as you wish:
https://i.imgur.com/gYL3Pwa.jpg
https://i.imgur.com/0IcIg90.jpg
https://i.imgur.com/vaJ25o0.jpg