I arrived here trying to find out how to close a modal with the escape key as well, i found a solution so I am posting it to help others in future.

Add this javascript to the head section of the aspx page you want to perform the event on.

document.onkeydown = function(e){    
                    if (e == null) { // ie
                        keycode = event.keyCode;
                    } else { // mozilla
                        keycode = e.which;
                    }
                    if(keycode == Sys.UI.Key.esc){
                        $find('<%= commentsModal.ClientID %>').hide();
                    }
                };

where commentsModal is your server side ID of the extender.