Siunam's Website

My personal website

Home About Me Blog CTF Writeups Projects

where are the robots | Mar 3, 2023


Welcome to my another writeup! In this picoGym challenge, you’ll learn: Reading web crawler file (robots.txt)! Without further ado, let’s dive in.


Author: zaratec/Danny


Can you find the robots? (link) or


Home page:

Pretty empty.

In the challenge’s title and the home page, it’s referring to a file called robots.txt, which is a file for web crawler (an Internet bot that systematically browses the World Wide Web).

Let’s try to read that file:

└> curl                           
User-agent: *
Disallow: /8028f.html

As you can see, it’s disallowing web crawler (not us) to view /8028f.html.

Let’s go there!

└> curl -s | html2text

Guess you found the robots

We found the flag!

What we’ve learned:

  1. where are the robots