.NET custom validators in Framework 2.0 have two components, Server side validation, and client side validation. I'm by no means a guru when it comes to client side validation - because me and JavaScript dont get along, but it's pretty easy once you get to the server...and you could incorporate a few different methods.
1.) Because you know the textboxes are in direct relation to the radio buttons. Check to see which radio button is selected and use a CASE statment to perform validation based on which textboxes are showing. You'd do this inside a custom validator.
2.) You can easily - inside a custom validator check the visible state of the checkbox, to determine if you should validate it. If textbox1.visible then -> Valdate -> Else -> Move on
3.) Add the controls and validators dynamically, so that the textboxes you dont want to validate are truly not even on the page when the validation fires. This is the hardest method, but also the most flexible because you havent hard coded the amount, nor the id of the textboxes anywhere for validation purposes.