The Subscription Problem
At some point in the last two years, many knowledge workers found themselves accumulating AI tool subscriptions the way previous generations accumulated streaming services. Each new tool promised to save hours, automate tedium, and supercharge productivity. The monthly bills added up. And the honest question — whether any of this actually delivers value proportionate to its cost — remained mostly unasked.
Evaluating AI tools well requires a framework that most people don’t use. The default is to demo the tool, find something impressive it can do, and subscribe. What’s missing is the harder question: does this tool actually change how much useful work I produce, or does it just feel productive?
The Four Questions That Matter
First: what task am I trying to do, and how long does it currently take? If you can’t name a specific task and estimate its current cost in time or money, you don’t have enough specificity to evaluate whether a tool helps. “AI writing assistant” is not specific enough. “Drafting first versions of client reports, which currently takes me about three hours each” is specific enough.
Second: does this tool actually reduce that time or cost in practice, not in demos? The demo is always the best version. What matters is whether the tool works in your actual context, with your actual data, on your actual problems — including the edge cases and the weird ones. Free trials exist precisely to answer this question, and most people don’t use them rigorously enough.
Third: what does the output quality look like, and does it require significant correction? A tool that saves you an hour of first-draft writing but requires 45 minutes of editing may still be net positive — but by less than it appeared. Count the full workflow cost. Fourth: is this tool solving a real bottleneck, or a convenient one? The real test is whether the tool addresses something that was actually slowing you down.
Running a Proper Trial
A good trial involves three things: a defined set of real tasks you’ll use the tool for, a comparison baseline of how long those tasks took before, and an honest accounting of total workflow time including correction and verification. Run this for two to four weeks — long enough that the novelty effect fades and you’re using the tool on a typical mix of work.
The novelty effect is real and worth naming: new tools feel more useful in the first two weeks because we seek out their strengths and avoid their weaknesses. After four weeks, you have a more honest picture of how the tool handles the full range of what you actually do.
What Worth It Actually Means
Cost is not just the subscription fee. It’s the learning curve time, the prompt engineering investment, the cognitive overhead of managing another tool, and the time spent reviewing and correcting outputs. A $20/month tool that requires four hours of setup and learning may not break even for months. For organisations evaluating tools at scale, deployment, training, security review, and integration costs often dwarf the per-seat subscription.
The Portfolio Approach
Rather than subscribing to every AI tool with a compelling demo, the more effective approach is to identify your two or three highest-value use cases and invest in understanding one tool well for each. Depth beats breadth. A developer who deeply understands how to get maximum value from GitHub Copilot will outperform one who has five AI tools they use shallowly.
The AI tool landscape will continue to consolidate. Some of today’s specialised tools will be absorbed by larger platforms. Investing in deep familiarity with a small number of durable tools is a better bet than constantly chasing the newest release. The goal isn’t to have the most tools — it’s to have the right ones, used well.
Watch: Related Video
Sources
- McKinsey Global Institute. (2023). The Economic Potential of Generative AI. mckinsey.com.
- Mollick, E., and Mollick, L. (2023). Assigning AI: Seven Approaches for Students. SSRN.
- Nielsen, J. (2006). The 90-9-1 Rule for Participation Inequality. Nielsen Norman Group.