Ok, I think I figured out the semi-generic approach to this problem. I was able to use GET, DELETE, etc. the trick was to not specify the action. I should have tried this approach sooner, I had wondered if I needed separate controllers, and it turns out for what I wanted that was necessary.

1. Create an ApiController for the 'parent' resource, and allow the default route to handle it
2. Create an ApiController for each 'child' resource.
3. Map HTTP routes for each child resource like this:

routes.MapHttpRoute(
   name:           "MyChildResourceRoute",
   routeTemplate:  "api/contacts/{contactId}/adresses/{id}",
   defaults:       new { controller = "contactaddresses", id = RouteParameter.Optional }
);