Life on a farm, how do you manage it, and still leave time for other fun stuff?

Get a butler, maid and gardening service.
Farmers usually don't have other day jobs, they are just that, farmers.
Knitting and cross stitching can be done in winter when there should be no gardening.
On the same note, take Sunday off and make it a day of rest.
Automation helps, upgrade everything that can be upgraded and get bots to help.
Splice plants so that you have different fruits on one plant, so there will be less gardening.