You will have two problems if you wait to long to kill off the bermuda. One is timing.
Bermuda is a beautiful grass and makes great turf grass but it is basically a weed because of its aggressive nature. What makes it so aggressive is that it spreads through runners above the ground and ones below the ground. So eradicating it in a lawn typically takes several treatments. Now about "time" I mentioned earlier.
To plant fescue you must start either mid July or no later than the first day after labor day. Here is why. Fescue needs soil temperatures around 65 to 75 degrees to germinate quickly. It takes about 7 to 15 days to germinate. Once the grass is established it needs another 3 weeks to develop a good rooting system to survive the winter. So if you started on labor day by mid September you should have grass sprouting. Which would give you about 3 or for weeks for the grass to develop a root system.
If you wait to late to plant, depending on when the first winter frost, your grass may not be mature enough to handle the cold weather and die. Also waiting to late will cause slow seed germination which is not a good thing.
Let's say you wait until the fall to kill the bermuda, you would have to wait at least two weeks but 3 weeks to be on the safe side to seed any new grass and that would probably put you to close to that first winter seed of lower temperatures that would cause slow germination
The guy has a wonderful youtube channel The lawn care nut. He moved to Florida this past December so he has warm season grass now. But he has tons of videos when he had cools season grasses which was Fescue. I learned so much from him. Check out his older fescue videos and he will have your fescue looking right. Watch his maintenance videos also.
He also got vids showing him seeding a fescue yard from scratch and what to do