Use ControllerBuilder.SetDefaultControllerFactory(typeof(MyControllerFactory)).  Then you can implement the base class resolution logic in your own factory and you do not need to call SetControllerFactory for all of your controllers.