Is it possible to configure Generals and/or Zero Hour to play in a window?

Yes, that should be possible. You can pass multiple command line arguments to Zero Hour. For example, to force Zero Hour to run in windowed mode with an 800x600 resolution, you would use the following shortcut:

"C:\Program Files (x86)\Origin Games\Command & Conquer Generals Zero Hour\generals.exe" -win -xres 800 -yres 600

 

You can either create a new shortcut on your desktop that points to the .exe file with these additional arguments or edit the properties of your generals.exe inside your game folder. Additionally, Origin now allows us to set command line arguments for games directly within the Origin client, to do so, right-click the game and select Game Properties, then add the command line arguments and confirm by clicking Ok.

 

Hope that can help.