Texting Robots: Taming robots.txt with Rust and 34M tests

6 points by Smerity 3 years ago | 0 comments