ChatGPT Data Processor

A streamlined toolkit for analyzing ChatGPT conversations with a focus on neurodivergent patterns and traits.

Features

• Pattern Analysis: Detect patterns related to:

  • Hyperfocus
  • Executive Function
  • Special Interests
  • Sensory Processing
  • Social Interaction

• Data Processing:

  • Process raw ChatGPT conversation data
  • Extract meaningful patterns and insights
  • Generate recommendations based on detected patterns

• Analysis Tools:

  • NLP-based conversation analysis
  • Pattern sequence detection
  • Temporal pattern analysis

• Export Options:

  • JSON format for further processing
  • CSV format for spreadsheet analysis
  • HTML reports with visualizations

Installation

1. Clone the repository:

git clone https://github.com/yourusername/chatgpt-data-processor.git
cd chatgpt-data-processor

2. Create and activate a virtual environment:

python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

3. Install dependencies:

pip install -r requirements.txt

Usage

The processor provides three main commands: process, analyze, and export.

Process Conversations

Process a single conversation file or directory of files:

# Process a single file
python main.py process path/to/conversation.json

# Process a directory recursively
python main.py process path/to/directory --recursive

Analyze Patterns

Analyze processed conversations for specific patterns:

# Analyze all patterns
python main.py analyze

# Analyze specific pattern type
python main.py analyze --pattern hyperfocus

# Analyze within date range
python main.py analyze --from-date 2025-01-01 --to-date 2025-03-19

Export Results

Export analysis results in various formats:

# Export as JSON
python main.py export results.json

# Export as CSV
python main.py export results.csv --format csv

# Export as HTML report
python main.py export report.html --format html

Project Structure

chatgpt-data-processor/
├── core/                     # Core functionality
│   ├── pipeline/             # Processing pipeline
│   ├── analyzer/             # Pattern analysis
│   └── utils/                # Helper utilities
├── explorer_html/            # Sample HTML output
├── templates/                # HTML templates
├── main.py                   # Main entry point
└── requirements.txt          # Dependencies

Configuration

Create a config.ini file to customize settings:

[Database]
path = chatgpt_data.db
backup_path = backups/

[Logging]
level = INFO
file = chatgpt_processor.log

[Analysis]
embedding_model = all-MiniLM-L6-v2
min_similarity_score = 0.8
max_tokens = 512

Development

1. Set up development environment:

pip install -r requirements.txt

2. Run tests:

pytest

3. Format code:

black .

4. Run linter:

flake8

Contributing

Contributions are welcome! Please see our Contributing Guidelines for details.

License

This project is licensed under the MIT License - see the LICENSE file for details.