Can't load or continue any save: "content that has not completed installation"

The problem seems obvious now, but it turns out the game had not finished downloading and it was just allowing me to play the downloaded part of the game. It does not show up anywhere in the PS4 UI, you have to go to "Library", select the game, and then select "Information".