Crawling wrong urls and generating unnesecary folders #95
Description
Suppose we are embedding multiple markdown files with links using -- $!{readmeContent}.
These readme files which have been taken from github generally contain links in the form of /something instead of https://github.com/something so, the crawler thinks that /something is a route of my website and generates a folder like /something/index.html which is the main issue. There should be a way to control this, like an option in vite.config.js to for example add only the urls that I want to generate or disable the crawler altogether for specific routes.