In the list of web forms a Delete-link shows for all forms that are closed.
If the form is closed, you can also delete it. However, you must actively set the form to be closed under the Settings tab to display the delete link in the list of forms. It is not enough for a form to be closed by limitation of response date or maximum number of responses.
Keep in mind that deleting a form will also delete all response data! These cannot be restored! Download all answers before you delete forms.