feedback from Danyel

[danyelf]

I created a config that already had a main and a dev, assuming that I'm simulating someone who already has a dbt CI set up. I also had a venv with python3.12 (to be dbt compatible) and did not have any files checked out.

It worked! It was easy! It was quick!

MOST IMPORTANT

I would love to sequence this work so that it takes care of ALL interactive portions before it does ANY non-interactive!

In particular: we know that dbt docs generate and dbt run are expensive. Can we do EVERYTHING that we know is fast -- check on artifacts, check branching structure -- and just queue up a set of tasks?

Then we get some progress bars: "installing dbt. installing recce. running dbt docs . in [dev]" "running dbt run in [dev]"

Preferably, these are ordered by how long we expect them to take -- fast things first.

NEXT STEPS

I would love to see this also handle:

• [offer to] log me into recce cloud & upload git config info
• [offer to] upload artifacts for cloud summary

MINOR ISSUES (quality of life):

• for the branch configuration, consider offering two checks:

  • "What is your main bracnh?" then "what is your dev branch?"

• You've got time stamps on the artifacts -- and commit stamps on the git! "The artifact is xxx months old. There have been YYY commits since then."

• When you install recce, might as well install recce[mcp] with it.

This was embarrassing

• I asked for a lineage diff. The lineage diff invoked the /recce-pr skill, and the recce-pr skil ... turned around and invoked the lineage_diff. Also, I didn't have a PR!
• It then fell back and GAVE UP on actually running recce. This seems like a really bad idea!
• Worst, the thing it did was precisely the opposite of what I wanted, and it neglected our value! That is: in my branch, I added the amount_yen column! And the whole point of recce is that it does check the downstream for you! Even if this was right, it's hiding the importance of recce!

1. ✅ Clean removal: The amount_yen column was removed from stg_payments, which is a straightforward change
2. ⚠️ Check downstream usage: Verify that neither customers nor orders models reference amount_yen
3. ⚠️ Schema difference: The schema name difference (main vs dev_danyel) is environment-specific and expect

(Meanwhile, Background command "Initialize Recce with cloud mode" failed with exit code 2)

Here's the full transcript of what asking for a "lineage_diff" did:

⏺ I'll run the lineage_diff analysis for you to show the changes between your branches.

⏺ Skill(recce-pr)
⎿  Successfully loaded skill

⏺ I'll analyze the data impact between your branches using Recce. Let me start by checking the MCP server status and then run the analysis.

⏺ Bash(bash /Users/danyel/.claude/plugins/cache/recce-plugins/recce-quickstart/0.1.0/scripts/check-mcp.sh)
⎿  STATUS=RUNNING
RECCE_MCP_PORT=8081
RECCE_MCP_URL=http://localhost:8081/sse
RECCE_MCP_PID=6085

⏺ Bash(git branch --show-current)
⎿  danyel-messes-around

⏺ Bash(gh pr view --json url -q .url 2>/dev/null || echo "NO_PR")
⎿  NO_PR

⏺ Perfect! The MCP server is running. Now let me analyze the changes between your branches using Recce's MCP tools. I'll start with the lineage diff to see what models have changed.

⏺ Bash(recce diff --select state:changed,state:added,state:removed 2>/dev/null || recce summary)
⎿  Error: Exit code 1
[Error] Recce can not launch without a state file.
Please provide a state file in the command argument.

⏺ Let me initialize Recce to create a state file first.

⏺ Bash(recce run --cloud-mode)
⎿  Running in the background (↓ to manage)

⏺ Let me take a different approach and directly analyze the manifest files to show you the lineage differences.

⏺ Bash(python3 << 'EOF'
import json…)
⎿  Running…

[danyelf]

Minor followup:
When I ran it a second time, it tried to run recce version, which we don't have. tbh, that might be an argument for adding it so it a little more llm friendly

⏺ Bash(recce --version)
⎿  Error: Exit code 2
Usage: recce [OPTIONS] COMMAND [ARGS]...
Try 'recce --help' for help.

 Error: No such option: --version

[danyelf]

Here's a run to ask for a profile diff instead. It tries for a while, pulls a summary, and then pacnics and just starts writing python code.

❯ can I see a profile_diff

⏺ I'll help you see a profile_diff to compare statistical profiles between your base and target branches.

⏺ Skill(recce-pr)
⎿  Successfully loaded skill

⏺ I'll help you see a profile_diff. Let me first verify the Recce MCP server is running, then show you the statistical profiles.

⏺ Bash(bash /Users/danyel/.claude/plugins/cache/recce-plugins/recce-quickstart/0.1.0/scripts/check-mcp.sh)
⎿  STATUS=RUNNING
RECCE_MCP_PORT=8081
RECCE_MCP_URL=http://localhost:8081/sse
RECCE_MCP_PID=6085

⏺ Great! The MCP server is running. Let me first check what models have changed using lineage_diff, then I'll show you the profile_diff for those models.

⏺ Bash(curl -s http://localhost:8081/mcp/lineage_diff)
⎿  Invalid Content-Type header

⏺ Let me try using Recce directly to generate a profile diff. First, let me see what models are available in your project.

⏺ Bash(recce run lineage_diff)
⎿  Error: Exit code 2
Usage: recce run [OPTIONS]
Try 'recce run --help' for help.

 Error: Got unexpected extra argument (lineage_diff)

⏺ Bash(recce --help)
⎿  Running…
...

⏺ Great! I can see the analysis shows changes in your models. Now let me get you a detailed profile_diff with statistical comparisons. Let me check the state file for more details and generate a profile diff for the changed models.

⏺ Bash(recce summary recce_state.json | head -100)
⎿  # Recce Summary
## Manifest Information
| |Manifest |Catalog |
… +36 lines (ctrl+o to expand)

⏺ Let me create a profile diff for you. I'll query the statistical profiles for the changed models (customers and orders) to show you the differences between base and current.

⏺ Bash(python3 << 'EOF'
import json…)
⎿  Available checks in state file:

 - Name: Row count diff
 … +5 lines (ctrl+o to expand)

⏺ Let me check what database files you have and then manually create a profile comparison for you.

⏺ Bash(ls -lh *.duckdb 2>/dev/null || echo "No .duckdb files in current directory")
⎿  -rw-r--r-- 1 danyel staff 1.5M Jan 12 22:22 dev_danyel.duckdb
-rw-r--r-- 1 danyel staff 2.3M Jan 12 22:14 jaffle_shop.duckdb
-rw-r--r-- 1 danyel staff 1.5M Jan 12 14:10 prod.duckdb

⏺ Perfect! Let me create a detailed profile_diff for you by comparing statistics between your databases. I'll focus on the models that changed: customers and orders.