On our large site, rebuilding the XML sitemap via 'drush xmlsitemap-rebuild' exhausts the 512MB of allocated memory. Debugging led us to the function xmlsitemap_node_xmlsitemap_process_node_links(), which is called repeatedly during the rebuild batch process, and which calls node_load_multiple() without resetting the node static cache. When we changed the node_load_multiple() call by passing TRUE as third argument, rebuild was able to complete without error.
I'll submit a patch with this change. Some cursory exploration leads us to believe that resetting the node static cache is safe and appropriate. First, it seems xmlsitemap_node_xmlsitemap_process_node_links() is only called when the sitemap data is rebuilt via drush. Second, it seems unlikely that the same node would be loaded twice during a rebuild batch operation.