Why exercise at all? Is it not simply easier to modify my diet?

The simple answer is that yes you will get results from only dieting alone but the benefits you get will be far less.
 
The benefits you will get from physical exercise will contribute greatly to your overall results with things such as:
 
– Improved muscle tone and lean muscle mass
– Improved heart function
– Improved lung function
– Improved immune system
– Stress relief
– Improved skin tone and health
 
The other important aspect of improving your fitness for weight loss is that the fitter you are, the harder and longer you can work out for which means you will be able to burn more calories per session.