Facebook Group Crawler

MVP crawler to extract posts and comments from Facebook Groups you've joined.

Features

Cookie-based session persistence (login once, reuse session)
Anti-detection measures (random delays, user-agent rotation)
Extracts: posts, authors, timestamps, media URLs, reactions, comments
Export to JSON (full data) and CSV (flat structure)

Requirements

Python 3.11+
Facebook account with access to target groups

Quick Start

1. Setup Environment

# Create virtual environment
python -m venv venv

# Activate (Windows)
venv\Scripts\activate

# Activate (Linux/Mac)
source venv/bin/activate

# Install dependencies
pip install -r requirements.txt

# Install Playwright browser
playwright install chromium

2. Configure

# Copy example config
cp .env.example .env

# Edit .env with your credentials

.env contents:

FB_EMAIL=your_email@example.com
FB_PASSWORD=your_password
TARGET_GROUP_URL=https://www.facebook.com/groups/your-group-id
HEADLESS=false
TARGET_POSTS=20
MAX_SCROLL_ATTEMPTS=50
REQUEST_DELAY_MIN=2
REQUEST_DELAY_MAX=5
MAX_COMMENT_EXPANDS=20
MAX_COMMENTS_EXTRACT=100

3. Run

python main.py

On first run:

Browser opens Facebook login page
Credentials auto-filled, click login
Complete 2FA if prompted (120s timeout)
Cookies saved for future runs

4. Output

Files saved to data/output/:

posts_YYYYMMDD_HHMMSS.json - Full data with comments
posts_YYYYMMDD_HHMMSS.csv - Flat post data
comments_YYYYMMDD_HHMMSS.csv - Comments with post_id reference

Project Structure

fbbot/
├── src/
│   ├── config.py       # Settings, logging
│   ├── browser.py      # Playwright management
│   ├── auth.py         # FB login, cookies
│   ├── crawler.py      # Group navigation, scrolling
│   ├── extractor.py    # HTML parsing
│   ├── exporter.py     # JSON/CSV export
│   └── models.py       # Post, Comment dataclasses
├── data/
│   ├── cookies/        # Session cookies (gitignored)
│   └── output/         # Crawled data (gitignored)
├── main.py
├── requirements.txt
└── .env.example

Configuration Options

Variable	Default	Description
`FB_EMAIL`	-	Facebook login email
`FB_PASSWORD`	-	Facebook password
`TARGET_GROUP_URL`	-	Group URL to crawl
`HEADLESS`	false	Run browser headless
`TARGET_POSTS`	20	Number of posts to collect (scroll until reached)
`MAX_SCROLL_ATTEMPTS`	50	Maximum scroll attempts to prevent infinite loop
`REQUEST_DELAY_MIN`	2	Min delay between actions (sec)
`REQUEST_DELAY_MAX`	5	Max delay between actions (sec)
`MAX_COMMENT_EXPANDS`	2	Comment expansion clicks (0=top only, -1=all)
`MAX_COMMENTS_EXTRACT`	50	Max comments to extract per post

Troubleshooting

Login fails:

Check credentials in .env
Try HEADLESS=false to see browser
Complete 2FA manually if prompted

No posts extracted:

Facebook may have changed DOM selectors
Check data/crawler.log for errors
Try increasing TARGET_POSTS or MAX_SCROLL_ATTEMPTS

Account blocked:

Use longer delays (REQUEST_DELAY_MIN=5)
Don't run too frequently
Use your real account (not fake)

Legal Notice

This tool is for personal/research use only. Crawling Facebook may violate their Terms of Service. Use at your own risk. The authors are not responsible for any consequences.

License

MIT

Name	claude-code
Description	Use when users ask about Claude Code features, setup, configuration, troubleshooting, slash commands, MCP servers, Agent Skills, hooks, plugins, CI/CD integration, or enterprise deployment. Activate for questions like 'How do I use Claude Code?', 'What slash commands are available?', 'How to set up MCP?', 'Create a skill', 'Fix Claude Code issues', or 'Deploy Claude Code in enterprise'.

claude-code

SKILL.md