# Job Taxonomy Implementation Plan

> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.

**Goal:** Replace tag-based tab matching with explicit group/subgroup fields in jobs.yaml, collapsible sub-groups in the Finance tab, and group-level pause/run operations.

**Architecture:** Add `groups:` registry and per-job `group:`/`subgroup:` fields to jobs.yaml. Parse in `Job` dataclass. Dashboard reads groups from YAML instead of hardcoded `JOB_GROUPS`. AG Grid row grouping for sub-groups. CLI gets `--group`/`--tag` flags.

**Tech Stack:** Python, NiceGUI AG Grid, ruamel.yaml (for comment-preserving YAML), Click CLI

**Design doc:** `docs/plans/2026-03-09-job-taxonomy-design.md`

---

### Task 1: Add groups registry to jobs.yaml

**Files:**
- Modify: `jobs/jobs.yaml:1-5` (add groups section before `runner:`)

**Step 1: Add groups registry**

Insert at the very top of `jobs/jobs.yaml`, before the `runner:` section:

```yaml
# --- Group taxonomy (dashboard tabs + CLI operations) ---
groups:
  finance:
    name: Finance
    emoji: "📊"
    subgroups:
      market-monitoring: Market Monitoring
      company-research: Company Research
      pltr: PLTR
      supply-chain: Supply Chain
      value-investing: Value Investing
  research-feeds:
    name: Research Feeds
    emoji: "🎓"
  people:
    name: People
    emoji: "👤"
  system:
    name: System
    emoji: "🔧"
  projects:
    name: Projects
    emoji: "📋"

runner:
  discovery_interval: 60
  ...
```

**Step 2: Verify YAML parses**

Run: `python -c "import yaml; d = yaml.safe_load(open('jobs/jobs.yaml')); print(list(d['groups'].keys()))"`
Expected: `['finance', 'research-feeds', 'people', 'system', 'projects']`

**Step 3: Commit**

```bash
git add jobs/jobs.yaml
git commit -m "feat(jobs): add groups registry to jobs.yaml"
```

---

### Task 2: Add group/subgroup to each job definition

**Files:**
- Modify: `jobs/jobs.yaml` (add `group:` and `subgroup:` to every job)

**Step 1: Add group/subgroup fields**

For each job, add `group:` right after `kind:`, and `subgroup:` where applicable. Remove the grouping-only tags (`finance`, `longform`, `vic`) — keep operational tags (`yt-channel`, `co-research`, `podcast`, `newsflow`, etc.).

Assignment table:

| Job | group | subgroup | Remove tag |
|-----|-------|----------|------------|
| earnings_backfill_largecap | finance | company-research | `finance` |
| dumb_money_live | finance | market-monitoring | `finance` |
| a16z | research-feeds | — | `longform` |
| healthy_gamer_all | research-feeds | — | `longform` |
| dwarkesh_podcast | research-feeds | — | `longform` |
| lex_fridman | research-feeds | — | `longform` |
| pltr_discovery | finance | pltr | `finance` |
| pltr_content_processing | finance | pltr | `finance` |
| newsflow_monitor | finance | market-monitoring | `finance` |
| newsflow_yt_investors | finance | market-monitoring | `finance` |
| company_analysis | finance | company-research | `finance` |
| semi_assessment | finance | company-research | `finance` |
| quantpedia | finance | value-investing | `finance` |
| supplychain_anchors | finance | supply-chain | — |
| supplychain_expand | finance | supply-chain | — |
| supplychain_news | finance | supply-chain | — |
| vic_ideas | finance | value-investing | `vic` |
| vic_text_extract | finance | value-investing | `vic` |
| vic_wayback | finance | value-investing | `vic` |
| vic_returns | finance | value-investing | — |
| person_intel | people | — | — |
| vc_data_bulk | people | — | — |
| vc_data_openvc | people | — | — |
| vc_data_signal_nfx | people | — | — |
| vc_firm_analysis | people | — | — |
| moltbook_monitor | projects | — | — |
| nonprofit_990s | projects | — | — |
| _autodo_template (and all autodo_*) | system | — | — |

Example for one job:

```yaml
  pltr_discovery:
    emoji: "🔎"
    kind: monitor
    group: finance
    subgroup: pltr
    tags: [pltr]
    # ... rest unchanged
```

**Step 2: Verify**

Run: `python -c "import yaml; d = yaml.safe_load(open('jobs/jobs.yaml')); jobs = d['jobs']; missing = [j for j in jobs if 'group' not in jobs[j]]; print(f'{len(missing)} jobs missing group:', missing[:5])"`
Expected: `0 jobs missing group: []`

**Step 3: Commit**

```bash
git add jobs/jobs.yaml
git commit -m "feat(jobs): add group/subgroup to all job definitions"
```

---

### Task 3: Parse group/subgroup in Job dataclass

**Files:**
- Modify: `jobs/lib/job.py:146-170` (add fields to Job)
- Modify: `jobs/lib/job.py:277-314` (parse in load_jobs)

**Step 1: Add fields to Job dataclass**

In `jobs/lib/job.py`, add after `kind`:

```python
    group: str = ""           # taxonomy group key (e.g. "finance", "research-feeds")
    subgroup: str = ""        # sub-group within group (e.g. "pltr", "supply-chain")
```

**Step 2: Parse in load_jobs**

In the `Job(...)` constructor call, add:

```python
            group=cfg.get("group", ""),
            subgroup=cfg.get("subgroup", ""),
```

**Step 3: Add load_groups helper**

Add a function to load the groups registry:

```python
def load_groups(yaml_path: Path) -> dict[str, dict]:
    """Load group taxonomy from YAML. Returns {group_key: {name, emoji, subgroups}}."""
    with open(yaml_path) as f:
        data = yaml.safe_load(f)
    raw = data.get("groups", {})
    groups = {}
    for key, cfg in raw.items():
        groups[key] = {
            "name": cfg.get("name", key.replace("-", " ").title()),
            "emoji": cfg.get("emoji", ""),
            "subgroups": cfg.get("subgroups", {}),
        }
    return groups
```

**Step 4: Verify**

Run: `python -c "import sys; sys.path.insert(0,'.'); from jobs.lib.job import load_jobs, load_groups; from pathlib import Path; j=load_jobs(Path('jobs/jobs.yaml')); print(j['pltr_discovery'].group, j['pltr_discovery'].subgroup); g=load_groups(Path('jobs/jobs.yaml')); print(list(g.keys()))"`
Expected: `finance pltr` and `['finance', 'research-feeds', 'people', 'system', 'projects']`

**Step 5: Commit**

```bash
git add jobs/lib/job.py
git commit -m "feat(jobs): parse group/subgroup fields in Job dataclass"
```

---

### Task 4: Replace JOB_GROUPS with YAML-driven grouping in dashboard

**Files:**
- Modify: `jobs/app_ng.py:415-425` (replace JOB_GROUPS)
- Modify: `jobs/app_ng.py:428-449` (_load_job_meta)
- Modify: `jobs/app_ng.py:452-515` (get_jobs_overview)

**Step 1: Load groups from YAML**

Replace the hardcoded `JOB_GROUPS` list with:

```python
def _load_groups() -> list[tuple[str, dict]]:
    """Load group taxonomy from YAML. Returns [(display_name, group_config), ...]."""
    from jobs.lib.job import load_groups
    from jobs.ui.styles import JOBS_YAML
    groups = load_groups(JOBS_YAML)
    return [(cfg["name"], {"key": key, **cfg}) for key, cfg in groups.items()]

# Loaded once at module level
JOB_GROUPS = _load_groups()
```

**Step 2: Update _load_job_meta to include group/subgroup**

Add to the meta dict:

```python
            "group": job.group,
            "subgroup": job.subgroup,
```

**Step 3: Replace tag-matching in get_jobs_overview**

Replace the tag-matching logic with direct group field lookup:

```python
    # Group by explicit group field
    group_name_map = {cfg["key"]: label for label, cfg in JOB_GROUPS_WITH_CONFIG}

    for jid, row in sorted(jobs.items()):
        group_key = row.get("group", "")
        label = group_name_map.get(group_key, "Projects")  # fallback
        if label in grouped:
            grouped[label].append(row)
        else:
            grouped["Projects"].append(row)
```

Remove the old autodo prefix matching and tag-matching blocks.

**Step 4: Propagate subgroup to overview rows**

In the job row dict, include subgroup for AG Grid grouping:

```python
            "subgroup": m.get("subgroup", ""),
```

**Step 5: Verify**

Run: `python -c "import sys; sys.path.insert(0,'.'); from jobs.app_ng import get_jobs_overview; o = get_jobs_overview(); [print(f'{t:20} ({len(r):2})') for t, r in o.items()]"`
Expected: Finance (~18), Research Feeds (4), People (5), System (9), Projects (2+)

**Step 6: Commit**

```bash
git add jobs/app_ng.py
git commit -m "feat(jobs): YAML-driven group taxonomy replaces hardcoded JOB_GROUPS"
```

---

### Task 5: AG Grid row grouping for sub-groups in Finance

**Files:**
- Modify: `jobs/app_ng.py` (_build_job_group_tab, overview grid setup)

**Step 1: Add subgroup column to JOB_OVERVIEW_COLS**

Add a hidden subgroup column used for row grouping:

```python
    {"field": "subgroup", "headerName": "Sub-Group", "rowGroup": True, "hide": True},
```

**Step 2: Configure AG Grid for row grouping**

In `_build_job_group_tab`, when the tab has sub-groups (Finance), enable grouping:

```python
        has_subgroups = any(r.get("subgroup") for r in jobs)
        grid_options = {
            "columnDefs": JOB_OVERVIEW_COLS,
            "rowData": jobs,
            "rowSelection": "single",
            "headerHeight": 32,
            "rowHeight": 36,
            "defaultColDef": {"sortable": True, "resizable": True},
            "suppressColumnVirtualisation": True,
        }
        if has_subgroups:
            grid_options["groupDefaultExpanded"] = -1  # expand all
            grid_options["autoGroupColumnDef"] = {
                "headerName": "",
                "width": 30,
                "cellRendererParams": {"suppressCount": True},
            }
```

**Step 3: Map subgroup keys to display names**

Before passing to AG Grid, replace subgroup keys with display names from the groups registry. In the row prep loop:

```python
    # Look up subgroup display names from groups registry
    group_config = next((cfg for _, cfg in JOB_GROUPS_WITH_CONFIG if cfg.get("key") == group_key), {})
    sg_names = group_config.get("subgroups", {})
    for row in jobs:
        sg_key = row.get("subgroup", "")
        row["subgroup"] = sg_names.get(sg_key, sg_key.replace("-", " ").title()) if sg_key else ""
```

**Step 4: Verify visually**

Start server: `python jobs/app_ng.py` and check Finance tab shows collapsible sub-groups.

**Step 5: Commit**

```bash
git add jobs/app_ng.py
git commit -m "feat(jobs): collapsible sub-groups in Finance overview via AG Grid row grouping"
```

---

### Task 6: Group-level pause/run buttons

**Files:**
- Modify: `jobs/app_ng.py` (_build_job_group_tab, overview header area)

**Step 1: Add pause/run group buttons to overview header**

In `_build_job_group_tab`, after the summary label and time toggle:

```python
                pause_grp_btn = ui.button("⏸ Pause All", on_click=lambda: None).props(
                    "flat dense size=sm").style("color: #d29922; font-size: 0.8em;")
                run_grp_btn = ui.button("▶ Run All", on_click=lambda: None).props(
                    "flat dense size=sm").style("color: #2ea043; font-size: 0.8em;")
```

**Step 2: Wire pause/run group handlers**

```python
    async def on_pause_group():
        for row in overview_grid.options["rowData"]:
            if not row.get("paused"):
                pause_job(row["job"], reason="group pause")
        _refresh_overview()
        ui.notify(f"Paused all {label} jobs", type="info")

    async def on_run_group():
        for row in overview_grid.options["rowData"]:
            if row.get("paused"):
                unpause_job(row["job"])
        _refresh_overview()
        ui.notify(f"Resumed all {label} jobs", type="positive")

    pause_grp_btn.on_click(on_pause_group)
    run_grp_btn.on_click(on_run_group)
```

**Step 3: Verify**

Click "Pause All" on Finance tab → all finance jobs paused. Click "Run All" → all resumed.

**Step 4: Commit**

```bash
git add jobs/app_ng.py
git commit -m "feat(jobs): group-level pause/run buttons on overview"
```

---

### Task 7: CLI --group and --tag flags

**Files:**
- Modify: `jobs/ctl.py:37-43` (list command)
- Modify: `jobs/ctl.py:59` (pause command)
- Modify: `jobs/ctl.py:167` (run command)
- Modify: `jobs/lib/job.py` (helper to resolve group → job_ids)

**Step 1: Add group resolver helper in job.py**

```python
def resolve_group(yaml_path: Path, group_spec: str) -> list[str]:
    """Resolve group or group/subgroup spec to job IDs.

    Examples: 'finance', 'finance/pltr', 'research-feeds'
    """
    jobs = load_jobs(yaml_path)
    parts = group_spec.split("/", 1)
    group_key = parts[0]
    subgroup_key = parts[1] if len(parts) > 1 else None

    result = []
    for jid, job in jobs.items():
        if job.group != group_key:
            continue
        if subgroup_key and job.subgroup != subgroup_key:
            continue
        result.append(jid)
    return result


def resolve_tag(yaml_path: Path, tag: str) -> list[str]:
    """Resolve tag to job IDs (cross-group)."""
    jobs = load_jobs(yaml_path)
    return [jid for jid, job in jobs.items() if tag in job.tags]
```

**Step 2: Add --group and --tag to list command**

```python
@cli.command("list")
@click.option("--group", "-g", default=None, help="Filter by group (e.g. finance, finance/pltr)")
@click.option("--tag", "-t", default=None, help="Filter by tag (e.g. yt-channel)")
def list_jobs(group, tag):
```

Filter the job list by group/tag before display.

**Step 3: Add --group and --tag to pause command**

```python
@cli.command()
@click.argument("job_id", required=False)
@click.option("--group", "-g", default=None, help="Pause all jobs in group")
@click.option("--tag", "-t", default=None, help="Pause all jobs with tag")
```

When `--group` or `--tag` is given instead of `job_id`, resolve to job list and pause each.

**Step 4: Same for run command**

**Step 5: Verify**

Run: `jobctl list --group finance/pltr`
Expected: Shows pltr_discovery and pltr_content_processing

Run: `jobctl list --tag yt-channel`
Expected: Shows all YouTube channel jobs across groups

**Step 6: Commit**

```bash
git add jobs/ctl.py jobs/lib/job.py
git commit -m "feat(jobs): --group and --tag flags for jobctl list/pause/run"
```

---

### Task 8: Clean up old tag-matching artifacts

**Files:**
- Modify: `jobs/app_ng.py` (remove old is_autodo checks, tag-matching code)
- Modify: `jobs/jobs.yaml` (remove grouping-only tags if not already done)

**Step 1: Remove is_autodo special-casing**

Replace `is_autodo = label == "Code Maintenance"` with `is_system = group_key == "system"`. Update all `is_autodo` references to use `is_system`. The System tab uses scanner check columns (AUTODO_ITEMS_COLS), not dynamic job columns.

**Step 2: Remove stale comments**

Remove references to "Autodo", old tab names, tag-matching comments.

**Step 3: Verify full dashboard**

Start server, click through all 5 tabs, verify:
- Finance shows sub-groups with collapse/expand
- Research Feeds shows 4 channels flat
- People shows 5 jobs flat
- System shows scanner checks with existing scanner UI
- Projects shows moltbook + nonprofit

**Step 4: Commit**

```bash
git add jobs/app_ng.py jobs/jobs.yaml
git commit -m "refactor(jobs): clean up old tag-matching and autodo references"
```

---

### Task 9: Update documentation

**Files:**
- Modify: `jobs/CLAUDE.md` (update job grouping section)

**Step 1: Update CLAUDE.md**

Add a "Job Groups" section documenting the taxonomy:

```markdown
## Job Groups

Jobs are organized into 5 groups (configured in `groups:` section of jobs.yaml):

| Group | Key | Sub-Groups | Description |
|-------|-----|------------|-------------|
| Finance | finance | market-monitoring, company-research, pltr, supply-chain, value-investing | Markets, companies, investing |
| Research Feeds | research-feeds | — | Cross-domain knowledge intake |
| People | people | — | VC/investor intelligence |
| System | system | — | Codebase health (autodo) |
| Projects | projects | — | Standalone efforts |

Group-level operations: `jobctl pause --group finance/pltr`, `jobctl list --tag yt-channel`
```

**Step 2: Commit**

```bash
git add jobs/CLAUDE.md
git commit -m "docs(jobs): update CLAUDE.md with group taxonomy"
```