Configure template without republishing


I know that when I’m defining a publish location I can use “configure template”. That’s great, but what if I want to configure a template without setting up a new location? I want to configure the template for an existing location but I can’t see any way to do that; the “file publication” dialog doesn’t allow it.