Drupal allows its core path.inc file that controls the way path aliases are fetched/stored/handled to be overridden, through the 'path_inc' variable. Some modules make use of this to add their own functionality - for example, the language_hierarchy_paths submodule in https://www.drupal.org/project/language_hierarchy (which leverages https://www.drupal.org/project/pathinc exists to make that easier), which allows fallbacks for languages. For example, so that a British English language can exist, but aliases can just be set in a 'parent' language, regular English, without needing to be duplicated into the 'child' language.
The https://www.drupal.org/project/redis project is another example of one that overrides Drupal's path alias handling.
Anyway, the option to prefetch path aliases may not be compatible with some of these path handling implementations - it's certainly not with the language hierarchy example explained above. If the option is enabled, then paths end up unaliased in the sitemap, when they ought to be.
Patch to follow...