In spring an ill-timed frost can do significant damage to an apple crop. If the weather turns warm then cold the apple trees may have already begun budding and a frost then kills off the buds thus preventing any apples from growing. The most recent severe frost damage we had was in 2012 when we lost a majority of the crop due to high temperatures in March and frost in April.
In the fall a frost is not as damaging to the apple crop. A fall frost can alter the taste of some apples by causing sugars to concentrate. It may also cause certain varieties to have a change in their peel texture, but the fruit is still fine for eating and baking.