Content Discovery Room on TryHackMe

Overview

• Objective: Discover hidden or private content on a web server
• Tasks:
  • Manual Discovery
  • Ascent
  • Automated Discovery
• Tools & Scripts: Various tools and scripts will be used (details to follow)

Task 1: Content Discovery Methods

• Question: What is the content discovery method that begins with 'm'?
  • Answer: Manually
• Question: What is the content discovery method that begins with 'a'?
  • Answer: Automated

Task 2: Robots.txt

• Question: What is the directory in the robot's text that is not allowed to be viewed by web crawlers?
  • Concept: Robots.txt lists pages allowed or not allowed for search engine results
  • Steps: Go to the web server and check robots.txt
  • Example: unknown stuff/path (needs to be checked)

Task 3: Favicon Identification

• Concept: The favicon is the icon found on the tab of a browser
• Practical Exercise: Determine which framework a favicon belongs to
  • Steps: Use MD5 sum of the favicon
  • Tool: Use OWASP database to identify the type
  • Example: CGI IRC

Task 4: Manual Discovery - Sitemap.xml

• URL: sitemap.xml
• Question: Find the secret area in the sitemap
  • Example Paths: article id 3, contact, customers, login, secret area
  • Answer: Secret area

Task 5: HTTP Headers

• Question: Identify a tryhackme flag by examining HTTP headers
  • Steps: Send request to the web server
  • Example: Use options and extract flags

Task 6: Framework Identification

• Concept: Find the framework used by the web server
• Steps: View page source, find comments and links
  • Example Path: /documentation, /login with default username and password (admin/admin)
  • Outcome: Obtain a flag

Task 7: Google Dorking

• Question: What Google Dork operator can be used to only show results from a particular site?
  • Answer: site:

Task 8: Online Tool - Wappalyzer

• Purpose: Identify technologies a website is running
• Example Technologies: JavaScript frameworks (Node, anime.css, jQuery)
• Tool: Wappalyzer

Task 9: Wayback Machine

• Question: What is the website address for the Wayback Machine?
  • Answer: http://archive.org/web

Task 10: Git and S3 Buckets

• Concept: URL format for Amazon S3 buckets
• Example Format: bucket-name.s3.amazonaws.com

Task 11: Automated Discovery

• Tools: GoBuster
  • Steps: Use GoBuster with a URL and path
  • Example: Use word lists located in /usr/share/wordlists
• Practical Questions
  • Name of the directory beginning with /mod -> monthly
  • Name of the log file discovered -> log
  • Other directories and files to discover

Conclusion

• Summary: Completed tasks focusing on different aspects of content discovery on web servers
• Tools Used: Various manual techniques, Google dorking, Wappalyzer, GoBuster, etc.