Content Audit Process with Screaming Frog & Sitebulb

Jul 7, 2024

Content Audit Process with Screaming Frog & Sitebulb

Overview

  • Aim: Determine the list of pages on a website.
  • Tools:
    • Screaming Frog (less than 500 pages)
    • Sitebulb (more than 500 pages)
  • Screaming Frog (200 pounds/year)
  • Sitebulb ($50/month)

Steps to Create the List of Pages

  1. Crawl with Screaming Frog: Filter out blog, taxonomy pagination pages, author pages.
  2. Using Sitemap: Load a page of URLs instead of crawling all URLs.
  3. Filtering: Only include required pages such as blog posts.

Free Version Demonstration

  • Screaming Frog Modes: Switch from spider mode to list mode.
  • Download from XML Sitemap: Paste post XML sitemap.
  • Filter Options: Filter out HTML for blog posts.
  • Exporting Data: Export HTML data for use.

Data Analysis

  • Extracting Data: Export the data and analyze using tools like Google Sheets.
  • Columns to Consider: Populating data like H1 titles, meta descriptions, etc.
  • Using Unique Spreadsheets: Simple Excel or Google Sheets formulas can help de-duplicate URLs.

Using Other Tools

  • Google Search Console (GSC): Useful for pulling impressions and clicks data.
    • Performance Report: Extracts last 12 months data.
    • Filtering: Uses regex to filter all blogs except the root.
  • Google Analytics (GA): Helps track conversions and page views.
    • Add Goals: Filter conversion data for blog landing pages.
    • Reporting: Use the all pages report and page views data.
  • Ahrefs/Sitebulb: For backlink data and top keyword analysis.
    • Best Pages by Backlinks: Export the URL rating and referring domains data.

Data Considerations

  • Date Information: Ensure content has had enough time to perform before making decisions.
  • Traffic and Visibility: Check for click and impression levels, especially with seasonal content.
  • Conversions: Evaluate using GA goal completions or CRM data.
  • Page Views: Helps identify pages part of user journeys that may not get tracked by other metrics.
  • Authority and Top Keywords: Use data from Ahrefs or similar to identify high authority pages and their top keywords.

Content Decision-Making

  • URL Rating and Referring Domains: Identify pages with authority.
  • Top Keyword Analysis: Determine whether pages rank within striking distance to be improved.
  • Filtering for Content to Remove: Use GSC and GA data to spot content with poor metrics for removal.
    • Consider redirecting URLs that are removed.
  • Secondary Data Analysis: Use a combination of GA, GSC, and Ahrefs to refine content strategies.

General Recommendations

  • Audit Frequency: Perform annual audits, ideally in December.
  • Informed Decision-Making: Use data to guide not only removals but improvements and refining opportunities.
  • Cross-functional Work: Ensure that changes consider marketing, SEO goals, and user journeys.

Practical Takeaways

  • Simplified Processes: Start with free versions where possible, then scale up based on traffic needs.
  • Iterative Refinement: Continuous updates and monitoring can provide opportunities for improvement.
  • Collaboration: Communication with stakeholders, including content creators, can enhance the overall auditing process.