If you add the new evaluation template to a course definition you also have the button Update Evaluations if you click on it the evaluations will also be added to the existing courses.
Please note that the status of the course has to be startable. Otherwise the courses will not be updated.
If you set the old evaluation to inactive, they will not be inactive in the existing courses, you can only manually remove them from the current courses.
