A known issue in InDesign Creative Suite (CS) 5 has to do with the new Live Corners feature. You may find that this feature does not present itself quite the way it is supposed to when working with rectangular interactive buttons.
Live Corners lets you apply corner effects to rectangular frames and edit the radius of each corner together or individually using the small, yellow diamond shapes that appear on a frame when it is selected.
When working with rectangular interactive buttons, however, the initial small, yellow box you need to click in order to access the corner diamonds is nowhere to be seen.
As a workaround, simply double click the button to activate the small, yellow box and then click the small, yellow box to activate the four yellow diamonds in the corners of the button.
Use these yellow diamonds to edit the corners of your button. For example: