You can delete categories by clicking on the cross icon that appears when hovering over the category on the Categories page.