This post was originally published on LinkedIn.

But unfortunately, it probably will. Here's why:

AI should write code to perform data analysis. If it doesn’t use code it will struggle as LLMs struggle a bit with numbers.

But, even if it does, due to LLMs being non-deterministic (i.e. "a computational process that can exhibit different execution paths on different runs, even when given the exact same input") it may write code differently each time you ask it the same question

e.g. “give me the conversion rate over time from meeting>pipeline”

>> 1st time asked: AI writes code to use your CRM meeting data and Opp data

>> 2nd timeasked: AI writes code to use your Opp data only as you have Stage 0 Opps for Meeting Booked stage

Issue: Conversion rate is different each run.

HERE IS WHAT'S WORKING FOR US @CS2:

• We used Claude Code to write python scripts that perform the analysis. The scripts are fixed and do not change each run.
• We make sure to use clean unified data.
• We created a Skill to walk through each step: config/stage mapping (for each client one time) > data quality > analysis run > human review > HTML page generated with analysis and recommendation.

The key parts are:

• The analysis scripts do not change each run (you get the expected output each time)
• AI can be more creative in the pattern finding, summarization, insight finding, etc
• AI can be creative in building a great looking HTML dashboard for review

This is how you don’t end up like this person on Reddit.