I wouldn't worry about the ammo cooking off but the heat does shorten the "shelf life" so replace it maybe once a month in the summer.
An insulated storage container is a great idea too.
An insulated storage container is a great idea too.