Back-and-forth: The life of a PM and Data Analyst

Playing ping-pong with data analysts again?

At any given moment, deep within the scaleup universe, a Product Manager’s soul dies just a tad. Here's how it goes down…

(Inspired by a post by the witty Elena Verna (LinkedIn, Newsletter))

Scene: A PM clicks the “Create ticket” CTA, thus closing the loop on the preceding meeting spent with the data analyst, where data requirements were raised and “defined”.

While the meeting was set for 60 minutes, after 20 minutes, it seemed there wasn’t much more to say, and everyone was happy to finish early.

Looks promising.

The ticket reads - “Just so we have a documented ticket - please create a dashboard for the feature A/B test.”

A week later, the ticket is closed by the data analyst.

It takes a few more days before the PM can review the dashboard when they realize, “This isn’t what I needed…”

1st follow-up

PM reopens the ticket - “Could we please exclude users who haven’t done any meaningful actions during their session?”

As straightforward as it seems to the PM, the data analyst didn’t think to do this.

Busy with other tickets, they squeeze some time in and update the query.

Done.

Ticket closed.

Again.

2nd follow-up

Peeved, the PM reopens the ticket - “Why can’t I break the data down into activated vs. non-activated and free vs. paying users?”

The analyst has bigger fish to fry.

Another week passes.

The analyst adds two additional charts to the dashboard.

Done.

Ticket closed.

Again.

“Hope that’s it, I’ve got other tickets to tend to”, grumbles the analyst rightfully impatient.

3rd follow-up

Following 3 hours of analysis, the PM realizes - “Wait a sec… why did the analyst decide on a 24-hour window for Activation? That completely skews everything!”

Looks like it’s time to regroup and have that 60-min meeting 🤦

How to avoid this?

Define dashboards just like product requirements

A dashboard is an internal product. You need to think of its functionality, UI, and UX.

You need to think about how you'll be using it and what insights you want to uncover. Ahead of time.

Anything you leave undefined will be autonomously defined by the analyst. That’s just how life works.

You don’t half-bake product requirements, so don’t half-bake analysis requirements either.

Define dashboard requirements using questions

Ask proper questions, and you’ll get the answers you seek.

Ask partial questions, and you’ll get partial/irrelevant answers.

Dashboard requirements should effectively be a set of questions you need to answer.

Be very specific.

Include any filters, breakdowns, chart types, events, and properties you may need.

Worst case, the analyst will correct you, but at least they’ll understand what you're after.

Examples:

❌ Partial requirement: How many users clicked on feature X?

✅ Well-defined requirement: How many users (broken down by free vs. paid) clicked on feature X (after at least 3 hours of signup)? I would like to see this data as a pie chart.

❌ Partial requirement: Did the experiment increase the activation rate?

✅ Well-defined requirement: Did the experiment increase the activation rate of users who engaged with feature X more than 3 times AND received a successful output? I would like to see this data as a table that I can filter based on feature usage.

Empower your analysts

Are your analyst colleagues business oriented?

Involve them as much as possible, pull them in, and discuss the reasoning and goals.

This won’t only decrease overhead for you, but it will empower analysts to focus on the right things before you’ve had a chance to even ask.

Self-serve product analytics (e.g. Heap, PostHog, Mixpanel)

Why go back and forth with analysts when you can control your own queries and dashboards?

Products like Heap don’t just let you self-serve data, but they also let you self-serve the event and property definitions, thus not having to go back and forth with developers too.

What!?

Yes.

Using products like this gives you independence and, perhaps most importantly, frees up analysts' time to sophisticated queries you wouldn't be able to do on your own.

---------------

Have you faced a similar situation?

What do you do to decrease the ping pong?