To remove fields from your import mapping, go to "Setup > Project Fields". Select the field you want to remove and click on "Delete selected fields", as shown in the image below.
It is important to know that if the field is still used somewhere in your import, it will be re-added when importing. The only way you can use this feature is when the field does not exist in your import (anymore) and is not used anywhere in the rules, exports, APIs, or Ads. If the field is still being used somewhere, you will see a warning message appearing showing you where the field is still being used.
By clicking on each line in the warning you can clean the rules, exports, APIs, etc. from the used field and after that you can remove the field from the import mapping.