Users can retract/remove their vote from an idea if community administrator has enabled the setting for it.
How to retract vote
To retract a vote, simply click on the vote button which you had give eg. if you have down voted, click on down vote and if up voted, click on up vote.
Though if you want to retract the vote and give the opposite vote eg. you have down-voted and idea and want to up vote it, click on the opposite vote (up vote button in this use case). This will cause the retraction of down vote and also addition of an up vote.
Setup vote retraction by administration
Administrator of a community can enable vote retraction feature from the following path Customization >> Customize Site Behavior >> General >> Allow to Retract Votes.
Simply enable the setting and click on 'Save changes' button at the end of the list.