You just have to make sure your sim is very inspired when they finish a book. Some things you can do to ensure they are very inspired when they complete a book, which may be kind of cheaty, but eh, whatever are:
Make sure you have lot traits home studio and chefs kitchen and own a shower.
When you've completed 3/4 of the book get your sim to stop writing and take a thoughtful shower, stargaze/cloudgaze just until the moodlet appears, click on the fridge and select cook then just close the food selection menu - they should be very inspired after all that then you can finish the last 1/4 of the book and will have a higher chance of a best seller.