We launched Jitsu on Product Hunt on June 2nd (Wednesday). While Product Hunt is not a typical medium for launching products such as Jitsu (open-source platform for data engineers), we decided to do it anyway.
Our objective was pretty simple: to attract users. We didn't care much about getting a badge (becoming Product of the day). In our opinion, it’s a vanity metric rather than an objective measurement.
We didn't expect much from this experiment, and the results were quite surprising.
Traffic#
Excess Uniques is the number of unique users above our daily average.
It's a pretty accurate estimate of the effect of the launch. Product Uniques is the number of users who can be explicitly linked with Product Hunt (through the referer or landing page URL which ends with ?ref=producthunt.
As you may see, the difference is pretty substantial (from 70% to 47%, see green line on a plot). Some users clearly came to jitsu.com as a result of the launch, but not directly. We identified two major sources: github, Twitter. They account for 30% of excess traffic.
It’s worth mentioning that, on a launch day, I’ve been approached by a few people on LinkedIn and have have been offered the service of becoming Product of the Day at ProductHunt. It would assume that, if such services existed, people would tend to buy them from time to time. I can only guess how those service providers get the upvotes.
"If you were beaten by another product, don’t worry! Somebody probably wasn't playing fair!"
What about conversions?#
Of course, website traffic wasn't the main objective of the launch. What we cared about, is conversions. Since Jitsu has both self-hosted and cloud, version conversion can happen in two ways:
- User signed up for cloud.jitsu.com (signups)
- User installed Jitsu on a server (installs)
While signups' tracking is pretty straightforward, tracking installs and especially, attributing them to an acquisition channel is not an easy task. Installs statistics will be covered later, let's take a look at signups.
The same as with traffic, not all signups can be directly linked to clicks from Product Hunt. We added Excess Signups (signups above our daily average).
Surprisingly, the conversion rate went down from 5% to 2% after day one. We don't have a good explanation for that.
Open-Source Tracking (aka Telemetry)#
In the open-source world, user tracking is called Telemetry. It's way less accurate than any SaaS tracking. Unlike SaaS, we:
- Collect only anonymized data (no emails). We can't identify users accurately
- Allow users to opt out from tracking easily. If Jitsu is deployed with UI through Docker compose we ask users if they are OK with anonymized usage collection. Opting out from Server Only deploy is less obvious (it's done through config variable).
If the user deploys UI and switches off Telemetry, we send a last event about that fact. Surprisingly, more than half the users decided to opt out from telemetry collection
However, it's not possible to link the user to the traffic source even if the telemetry is on. The only way to measure the result of the launch is to calculate Excess Installs (installs above our averages)
If we compare Installs with Signups, the latter will prevail. It's not surprising, since the Product Hunt audience is less technical and more product-oriented.
Time#
Another interesting topic is distribution of traffic over time. Product Hunt features products in daily batches where the day starts at 12:00 am PST. American users are usually asleep at that time, while European users are just starting their day. As you may see, we got way more traffic at the beginning of the PST day. It's been coming mostly from the EU. It can be explained by two factors:
- Historically, our current user base was mostly coming from the EU
- It's been a long holiday for us before our launch (Wed, Jun 2nd), Monday was a Memorial Day
Tools#
We used those amazing tools to build the data above:
- Jitsu 🙂 We're big fans of dog-fooding! Retroactive User Recognition helped a lot with attributing signups to the traffic source
- DBT is another open-source tool. I would characterize them as "SQL Views on Steroids". We used DBT to post-process raw data collected by Jitsu
- Usually we do visualization in Metabase (as you may already guess, they are open-source too). However, this time, Google Sheets worked better, because we needed to match the plot colors with the website theme.
