Uses the supplied rules for this path. URLs are parsed locally.
A few things to know.
Checks 500 KiB of UTF-8 text. Uses exact case-insensitive product-token matching and the shared path matcher. Compares paths and queries up to 2,047 characters, strips fragments and applies Allow on equal normalized lengths. Does not fetch a deployed file, check server responses or implement provider-specific aliases.
Paste robots.txt or open a file. Enter the crawler product token and a target path or HTTP URL.
Select Explain rules. Review exact groups, wildcard fallback and the normalized comparison path, then compare matched rule lengths.
Change the path or agent to see which group or rule wins. Download a text trace for review.