If you don't feel like manually deleting empty folders, use this PowerShell script to query and delete all empty folders and subfolders. Make sure you change the $WebURL to your SharePoint site or subsite, and the $listName to your library.
Sometimes your users don't need to see every single SharePoint library when they're in file explorer. Here is a bit of PowerShell that will hide necessary libraries from file explorer. Make sure you change the URL to your SharePoint site's URL and the folder name to the library you want to hide.