The category only show up if it have the posts!
1- Go to Appearance –> Widgets –> Add “List Categories with images” widget to sidebar.
At the “Filter by Category“, you should keep it as “All categories”.
2- Go to Posts –> Categories
3- Click to Category you want add featured image. Then, at bottom of page, click Upload/Add image.