To remove a widget from the dashboard, click on More Settings on the top right of the widget you want to remove and select "Remove from Dashboard" option from the dropdown list.
To permanently delete a widget, click on the Widgets from the left, hover your cursor on the widget that you want to delete, and click on the widget settings.
A new modal window will appear providing the widget setting options. Click on delete from the top right corner of the interface and this will delete the widget and the dataset (if the dataset is not linked to other widgets).