Orviqa

How to measure open-source adoption with a record you keep every week

By Bright Oparaji10 min read

Pick four or five signals that only move when someone uses the project, write them down on the same day every week, and judge the trend against your own past. The hard part is not choosing the metrics. It is that most of the platforms that hold them forget within days or months.

Most advice on measuring adoption stops at a list of metrics. The list is the easy part. If you want the comparison of what each number means, the post on GitHub stars vs active users goes through stars, forks, contributors, downloads, dependents and issues one by one, and the guide to GitHub stars covers stars on their own. You do not need either to follow this one. It is about the practical job: where each useful number lives, how long it stays there, and how to turn it into a record you can still read in six months.

Every source has a short memory

The reason a weekly habit matters is that almost none of the places holding your numbers keep them for long.

  • GitHub traffic (Insights, then Traffic) shows views, unique visitors, clones, referring sites and popular content for the past 14 days only, and only to people with push access. GitHub's documentation also notes that the data is in UTC and that referrers and popular content update daily rather than hourly.
  • The traffic API has the same 14-day window, and its referrers and popular paths endpoints return the top 10 of each. The REST reference is short and worth reading before you script anything.
  • npm keeps daily download counts for up to 18 months through its download counts API, but the per-version breakdown covers only the previous 7 days.
  • pypistats keeps its daily time series for 180 days, according to its API page. The full history lives in Google BigQuery, which the Python Packaging User Guide explains how to query.
  • Read the Docs keeps traffic analytics for 90 days on its community site, per its analytics reference.

Fourteen days is the binding constraint. If you look at GitHub traffic once a month, half of every month is gone before you see it. A weekly record is the shortest habit that loses nothing, and a small scheduled script can keep it for you.

The signals that mean someone used it

A useful adoption signal is one that is hard to move without real use. Here are the ones worth recording, with what to watch for in each.

Issues and discussions from people you do not know

This is the most honest signal a small project has. Someone who opens an issue describing their own setup has installed the project, run it, and cared enough about the result to write to you. Count them separately from issues opened by you or your regular contributors.

GitHub's API marks every issue with an author_association field, which distinguishes owners, members and collaborators from everyone else, so the count can be scripted. By hand, a glance at the week's new issues is usually enough. Record the number and keep a one-line note of anything that tells you who the person is or what they were building. Over a few months those notes say more about your users than any count.

Dependents and "Used by"

For a library, a public repository that declares your package as a dependency has done the integration work. GitHub's dependency graph documentation says dependents are listed for public repositories, counting other public repositories that depend on yours or on the packages it publishes. The "Used by" box in the sidebar appears only once more than 100 repositories depend on your package, and only for supported ecosystems, so most young projects will not see it yet. The dependents list itself is still there.

Two cautions. Private code is invisible, so the number is a floor. And it moves slowly, so record it weekly but only expect to read it monthly.

Package downloads, read carefully

Downloads are the number closest to installs, and the easiest to over-read. The Python Packaging User Guide is blunt about it: PyPI does not display download counts because they are inaccurate, with pip's cache lowering them and mirrors and inflation scripts distorting them in both directions. pypistats excludes known mirrors by default and offers a "with mirrors" series if you want the raw figure.

Release days are the biggest trap. In the record from Orviqa's six-week digline case study, 91 percent of downloads in the 40 days after promotion began landed on release days. The median release day had 343 downloads and the day after had 71. A new version is fetched by bots, scanners and CI jobs, and the count cannot tell you how many of those were people.

So record downloads, but record two things alongside them: whether there was a release that week, and the weekly total excluding mirrors. Read the trend across weeks without releases, and treat any single spike as unexplained until something else confirms it.

Referrers and popular content in GitHub traffic

Referring sites tell you where visitors came from, which is the closest you get to knowing which of your efforts sent anyone. GitHub excludes search engines and GitHub itself from the list. Popular content tells you what visitors opened once they arrived: the README, the docs folder, the releases page, a specific example.

Record the top referrers by name with their unique visitor counts, not just the total. A new referrer you did not expect is often the most interesting line in the week. Bear in mind that links shared in chat, direct messages and email often arrive without a referrer at all. In the digline record, 39 percent of site visits in one week landed as "unknown".

Documentation analytics

If your project has a docs site, its analytics show something the repository cannot: whether people get past the front page. Visits to the installation page, the quickstart and the API reference come from people trying to use the thing. A rise in reference pages over several weeks is a better sign than a rise in homepage visits. Read the Docs also shows the top pages and the pages returning 404, and the 404 list is a free list of broken links your users are hitting.

The signals that mislead

Some numbers look like adoption and mostly are not.

  • Clones. GitHub counts full clones and unique cloners, and neither tells you whether a person or a script did the cloning. Record clones if you like, but do not read a rise as new users without something else moving with it.
  • Total page views. A single link on a busy site can multiply views for a day, almost all from people who will never return. Unique visitors and referrers are more useful than raw views.
  • Downloads on release day. Covered above. Most of that number is machines.
  • Stars and forks. They measure attention and intent, which is worth knowing, but not use. The stars guide explains why.
  • Your own activity. Commits, releases and your own issues make a repository look busy. They describe you, not your users.

CHAOSS, the Linux Foundation project that defines open-source health metrics, files many of these under its Project Popularity metric, which lists stars, forks, clones, new issues, downstream dependencies and mentions as things to collect. The useful lesson from that framing is that popularity is a bundle of signals, each with its own bias, and none of them stands in for the others.

Setting up the weekly record

A spreadsheet is enough. One row per week, recorded on the same weekday, ideally the same one each time so the 14-day traffic window always overlaps the last entry.

Columns that work for most projects:

  1. Week starting (a date, in UTC to match GitHub and the registries)
  2. Release that week (yes or no, and the version)
  3. Downloads, excluding mirrors (weekly total from your registry)
  4. Unique visitors (from GitHub traffic)
  5. Top three referrers (name and unique visitors)
  6. New issues and discussions from outside the project (a count)
  7. Dependents (the count from the dependency graph, for libraries)
  8. Docs visits to installation or quickstart pages (if you have a docs site)
  9. What you did that week (posts, talks, README changes, anything that could explain a change)
  10. What you expect next week (one sentence, written before you look)

The last two columns are the ones most people skip, and they are what make the record usable later. Without a note of what you did, a change in the numbers has no candidate explanation. Without a written expectation, every result looks like confirmation. The digline case study turned on exactly this: releases and promotion fell on the same days so often that nothing could be attributed to either, which is why it now separates them deliberately.

If you want to automate it, the traffic endpoints, the npm range endpoint and the pypistats overall endpoint all return JSON and can be called from a scheduled job that appends a row each week. Start by hand for a month first. Doing it manually teaches you which columns you actually read.

Reading the record

A few rules keep the record honest.

Compare against yourself. Totals invite comparison with other projects, and the comparison is almost never fair, because category, ecosystem and age dominate. Week-on-week change against your own baseline is the measure that means something.

Wait for several weeks. A small project's numbers move by chance from one week to the next. Two or three quiet weeks at the start give you a sense of the normal range before you read anything into a change.

Look for agreement. One signal moving alone is noise until shown otherwise. Downloads rising in a week without a release, together with a new referrer and an issue from a stranger, is a story. Downloads rising on their own is a release or a bot.

Ask what the number is for. If visitors arrive and leave, the record is telling you about the first screen, not about reach. That is the point to check what a README has to do in its first ten seconds, or run yours through the free README quality checker.

A checklist for this week

  • Open a spreadsheet with the ten columns above
  • Copy this week's GitHub traffic, including the top referrers, before the 14-day window moves on
  • Pull downloads excluding mirrors, and note whether you released
  • Count issues and discussions opened by people outside the project
  • Record dependents if you maintain a library
  • Write one line about what you did and one about what you expect
  • Put a weekly reminder on the same day, and keep going for at least a month before you judge anything

← All posts