Forge Data Branch - Git Integration

Overview

Forge workspace data is stored in an orphan branch called forge-data, separate from the main codebase. This approach provides significant performance and organizational benefits.

Why Orphan Branch?

Benefits

1. Faster Clones: Developers can clone only the code without downloading large forge data
2. Smaller Repository: Code and data histories are separate
3. Better Performance: Git operations on main branch are faster
4. Independent Versioning: Track code and data changes separately
5. Cleaner History: git log on main shows only code changes
6. Selective Access: Team members can opt-in to forge data
7. Improved CI/CD: Pipelines can skip forge data when not needed

Architecture

Main Repository
├── main branch              # Application code
│   ├── src/
│   ├── tools/
│   ├── docs/
│   │   ├── architecture/
│   │   └── forge-home/     # Worktree → forge-data branch
│   └── ...
│
└── forge-data branch        # Orphan (no shared history)
    ├── forge/              # All forge resources
    │   ├── project/
    │   ├── task/
    │   └── ...
    ├── .forge/
    │   └── config.json
    └── README.md

Setup

Initial Setup (One-Time)

Run the setup script:

./scripts/forge/setup-forge-data-branch.sh

This script will:

1. Create the forge-data orphan branch
2. Move existing forge-home contents to the new branch
3. Setup a Git worktree at docs/forge-home
4. Update .gitignore
5. Create forge configuration
6. Optionally push to remote

Manual Setup

If you prefer manual setup:

# 1. Create orphan branch
git checkout --orphan forge-data
git rm -rf .

# 2. Move forge data
git checkout main -- docs/forge-home
mv docs/forge-home/* .
rm -rf docs

# 3. Create .gitignore
cat > .gitignore << 'EOF'
.forge/graph.db
.forge/graph.db-shm
.forge/graph.db-wal
.DS_Store
EOF

# 4. Commit
git add .
git commit -m "Initialize forge data branch"

# 5. Return to main
git checkout main

# 6. Setup worktree
rm -rf docs/forge-home
git worktree add docs/forge-home forge-data

# 7. Push (optional)
git push -u origin forge-data

Daily Workflow

Clone Repository (First Time)

Option 1: Code Only (Recommended for most developers)

git clone --branch main --single-branch git@gitlab.savvifi.com:platform/savvi-studio.git
cd savvi-studio
# Skip forge data, work on code only

Option 2: Code + Forge Data

git clone git@gitlab.savvifi.com:platform/savvi-studio.git
cd savvi-studio
git worktree add docs/forge-home forge-data

Work on Code (Main Branch)

# Normal development workflow
git checkout main
# Edit code files
git add src/
git commit -m "Add feature"
git push origin main

Work on Forge Data

# Navigate to forge-home (automatically on forge-data branch)
cd docs/forge-home

# Check branch
git branch  # Shows: * forge-data

# Make changes
echo '{"id": "new-project"}' > forge/project/v1/new-project

# Commit changes
git add forge/project/v1/new-project
git commit -m "Add new project"

# Push forge data
git push origin forge-data

# Return to main
cd ../..
git branch  # Shows: * main

Pull Updates

Update Code:

git checkout main
git pull origin main

Update Forge Data:

cd docs/forge-home
git pull origin forge-data
cd ../..

Update Both:

# Update main
git pull origin main

# Update forge data
cd docs/forge-home && git pull && cd ../..

Git Commands Reference

Branch Management

# List all branches (including forge-data)
git branch -a

# Switch between branches
git checkout main
git checkout forge-data  # Usually not needed (use worktree)

# View worktrees
git worktree list

Worktree Management

# Add worktree (if not present)
git worktree add docs/forge-home forge-data

# Remove worktree
git worktree remove docs/forge-home

# List worktrees
git worktree list

View History

# View forge data history
cd docs/forge-home
git log
git log --oneline
git log --follow forge/project/v1/my-project

# View main branch history
cd ../..
git log

Check Status

# Check main branch status
git status

# Check forge data status
cd docs/forge-home
git status
cd ../..

CI/CD Integration

GitLab CI Example

.gitlab-ci.yml:

# Code pipeline (fast, no forge data)
test:code:
  only:
    - main
  script:
    - pnpm install
    - pnpm test
    - pnpm build

# Forge data pipeline (separate)
test:forge:
  only:
    - forge-data
  before_script:
    - git worktree add forge-home forge-data
  script:
    - cd forge-home
    - pnpm run forge:validate
    - pnpm run forge:test

# Combined pipeline (when both needed)
test:integration:
  only:
    - main
  before_script:
    - git worktree add docs/forge-home forge-data
  script:
    - pnpm install
    - pnpm test:integration

Advanced Usage

Get File History

# View history of a specific resource
cd docs/forge-home
git log --follow --patch forge/project/v1/my-project

Compare Versions

# Compare current with previous version
cd docs/forge-home
git diff HEAD~1 forge/project/v1/my-project

# Compare two commits
git diff abc123..def456 forge/project/v1/my-project

Restore Previous Version

cd docs/forge-home

# View file at specific commit
git show abc123:forge/project/v1/my-project

# Restore file to previous version
git checkout abc123 -- forge/project/v1/my-project
git commit -m "Restore project to previous version"

Create Data Snapshot (Tag)

cd docs/forge-home

# Create annotated tag
git tag -a v1.0 -m "Production data snapshot"

# Push tag
git push origin v1.0

# List tags
git tag -l

Work with Branches (Experiments)

cd docs/forge-home

# Create experimental branch
git checkout -b experiment-new-structure

# Make experimental changes
# ... edit files ...

# Commit
git commit -am "Try new structure"

# Switch back to main forge-data
git checkout forge-data

# Merge if successful
git merge experiment-new-structure

Troubleshooting

Worktree Not Setup

Symptom: docs/forge-home is a regular directory

Solution:

# Remove existing directory
rm -rf docs/forge-home

# Add worktree
git worktree add docs/forge-home forge-data

Can't Push Forge Data

Symptom: error: src refspec forge-data does not match any

Solution:

cd docs/forge-home
git push -u origin forge-data

Worktree is Locked

Symptom: fatal: 'docs/forge-home' is already checked out

Solution:

git worktree remove docs/forge-home --force
git worktree add docs/forge-home forge-data

Large Repository Size

Symptom: Clone taking too long

Solution: Clone only main branch

git clone --branch main --single-branch <repo-url>

FAQ

Q: Do I need forge data for development?
A: No! Most developers can work without forge data. Clone only main branch.

Q: How do I share my forge changes?
A: Commit and push from docs/forge-home: cd docs/forge-home && git push

Q: Can I use normal git commands in forge-home?
A: Yes! docs/forge-home is a normal git working directory on the forge-data branch.

Q: What if I accidentally commit forge data to main?
A: Remove it from main and commit to forge-data instead. Keep branches separate.

Q: How do I backup forge data?
A: Push to remote: cd docs/forge-home && git push origin forge-data

Q: Can I have multiple forge environments?
A: Yes! Create separate branches: forge-dev, forge-staging, forge-prod

Best Practices

1. Commit Frequently: Small, focused commits in forge data
2. Descriptive Messages: Clear commit messages for data changes
3. Use Tags: Tag important data snapshots
4. Review Changes: Use git diff before committing forge data
5. Sync Regularly: Pull forge data updates frequently
6. Separate Concerns: Keep code in main, data in forge-data
7. Document Changes: Add README files for major data changes

See Also

• Forge Backend README
• Graph Backend Documentation
• Git Worktree Documentation