In Part 1, we completed a thorough reshaping of your thinking. You are no longer a "programmer" waiting for requirements, but a "super individual" actively seeking problems. You understand the essence of business and are equipped with the powerful strategic weapon of "gap thinking."
But thoughts and strategies must eventually touch the ground. You might ask: "I understand the principles, but where exactly is that specific 'gap' that can make me money?"
This is the eternal question for every indie developer. Fortunately, the process of finding answers does not rely entirely on luck and inspiration. It is more like a science -- a detective discipline that combines cold data with burning human nature.
In this part, we will transform into detectives of the digital world. We will learn to interpret the "crime scenes" scattered across every corner of the internet -- users' search histories, angry reviews, and late-night ramblings. They are all clues, pointing to unmet needs, those value gaps waiting for your product to fill.
Ready your magnifying glass and notebook. Our treasure hunt begins now.
On the path of finding demand, there are two main schools:
- The Data-Driven School: They believe data is the only objective truth. They obsess over keyword analysis, search trends, and market reports, trying to find the "optimal solution" representing the greatest common denominator of demand from vast amounts of information.
- The Intuition and Empathy School: They believe all great products come from deep insight into human nature. They immerse themselves in user communities, talk to users, and try to feel their pain, anxiety, and desire, starting from a specific, emotional pain point.
Which school is right?
The answer is: both are right, and both are one-sided.
Relying only on data, you might find a "pseudo-demand" that many people search for but has very weak emotional intensity, resulting in a "toy" nobody wants to pay for. Relying only on intuition, you might be misled by the voices of one or two extreme users, wasting your energy on an extremely niche "personal problem" with zero commercial value.
A true master of demand mining cultivates both data and human nature. They hold an Ahrefs analysis report in one hand while soaking in the rant sessions on Reddit with the other. They use data to validate human impulses, and human nature to interpret the coldness of data.
In this chapter, we will cultivate both inner skills simultaneously, making you a demand-mining expert capable of left-right coordination at an extraordinary level.
3.1 Data Archaeology: Using Ahrefs/Semrush to Mine Low-Competition, High-Value Long-Tail Keywords
Imagine the entire internet as a vast ancient ruin buried with countless treasures. Every user's Google search is a "footprint" left on this site. Some footprints lead to bustling markets, already overcrowded. Others wind their way toward some unknown cave hiding a gold mine.
Data archaeology is the use of professional SEO tools, like Ahrefs or Semrush, to systematically analyze these "footprints" and discover those "less-traveled" yet "priceless" treasure caves.
Why search keywords?
A search behavior, especially those long-tail keywords containing specific questions or needs, is the most honest and proactive "expression of demand" a user can make on the entire internet. When someone searches at midnight for "how to automatically remove filler words from podcast audio," they are not browsing -- they are calling for help. Behind this keyword is a real, urgent potential user who is almost ready to pay for a solution.
Our mission is to find these "distress signals."
The Archaeological Tools: Ahrefs/Semrush
These two tools are the Swiss Army knives of the SEO industry. Think of them as "Google's backend database." They tirelessly crawl the entire internet, analyzing the traffic of every website and the search data of every keyword. For us independent developers, their most important feature is "Keywords Explorer."
We will focus on three core metrics:
Search Volume (SV): How many people search for this term per month. Conventional wisdom says higher is better, but for us, medium-to-low search volume (say, 100-1,000 per month) often means lower competition and more precise users.
Keyword Difficulty (KD): A 0-100 value representing how hard it is to rank on Google's first page. Higher KD means more competition, with many large sites vying for this keyword. Our target is to find KD < 20, or even KD < 10 -- "no man's land."
Cost Per Click (CPC): How much advertisers are willing to pay for a single click on this keyword. This is an extremely important "commercial value" indicator. If a keyword has a high CPC (e.g., > $2), it means companies are already spending real money to acquire these search users. High CPC equals the market has already validated the commercial value of this demand.
The Core Mindset of Archaeology: Understanding "Keyword Intent"
Not all keywords have equal value. Someone searching "what is podcasting" just wants information. Someone searching "best podcast editing software for beginners" is already in the purchase decision phase.
Understanding the "intent" hidden behind keywords is the soul of data archaeology. We mainly focus on the following types of intent:
- Informational Intent: The user wants to learn something. Usually starts with "what is," "how to," "why." Lower value, but can be used for content marketing to attract potential users.
- Commercial Investigation Intent: The user is comparing and researching solutions. This is our gold mine! Usually contains words like "best," "top," "alternative," "review," "vs," "tool," "software," "platform," "template."
- Transactional Intent: The user is ready to buy. Usually contains words like "buy," "price," "discount," "trial," "for sale." This is the most direct purchase signal.
Our strategy: Find long-tail keywords with "Low KD + High CPC + Commercial Investigation Intent."
Hands-on Archaeology Process: Digging Out Your Gold Mine Step by Step
Let us simulate a complete "archaeology" process. Suppose you are somewhat interested in the "real estate" field.
Step 1: Sow the "Seed Keyword"
In Ahrefs' Keywords Explorer, enter a broad "seed word," e.g., real estate agent.
Step 2: Filter "Matching Terms," Set Filters Ahrefs will return thousands of related keywords. Now, use filters -- the "sieve" -- to sift out gold from the sand.
- Set KD Filter:
Keyword Difficulty<=20. We only look at blue oceans. - Set CPC Filter:
CPC>=$2. We only look at promising areas. - Include specific modifiers: In the "Include" box, enter words that indicate commercial intent, like
software, tool, platform, template, generator, calculator. Select "Any word."
Step 3: Analyze Filtered Results, Find "Demand Patterns" Click "Show results." Now you are looking at a preliminary "treasure map." It might look like this:
| Keyword | KD | Volume | CPC |
|---|---|---|---|
| real estate cma software free | 12 | 500 | $11.00 |
| best crm for real estate agents | 19 | 1200 | $15.00 |
| real estate listing presentation template | 8 | 300 | $5.00 |
| automatic real estate flyer generator | 5 | 150 | $7.00 |
| real estate commission calculator florida | 15 | 400 | $3.00 |
Now, do not just look at individual words. Look for "patterns" and "stories."
Pattern One: "CMA Software"
- Keyword:
real estate cma software free(CMA: Comparative Market Analysis) - Archaeological Interpretation: 500 agents search for "free" CMA software every month. This suggests: 1) CMA reports are a core need for agents; 2) existing CMA software may be expensive, hence the search for free alternatives; 3) The high CPC ($11) indicates that even with free-demand searchers, the paid market in this area is very mature -- companies are willing to spend heavily to acquire these customers.
- Product opportunity: Could you build a simpler, cheaper CMA software, or offer a feature-limited but good-enough free version? For example, a Freemium version that only allows 3 reports per month.
- Keyword:
Pattern Two: "Template"
- Keyword:
real estate listing presentation template - Archaeological Interpretation: 300 agents need to create polished presentations for homeowners each month to win listing agreements. They may not be good at design and need ready-made templates. Low KD (8) and moderate CPC ($5) indicate a valuable but low-competition gap.
- Product opportunity: This is not a complex SaaS, but an excellent digital product opportunity. You could design a set of 10 different professionally styled PowerPoint/Keynote/Canva templates, package them as a product, and sell for $49 on Gumroad or your own site. A classic "minimalist product."
- Keyword:
Pattern Three: "Generator"
- Keyword:
automatic real estate flyer generator - Archaeological Interpretation: Agents mass-produce promotional flyers -- a repetitive, time-consuming task. They crave "automation." Extremely low KD (5) and a substantial CPC ($7) are practically waving at you.
- Product opportunity: A perfect opportunity for an AI dimensionality reduction strike! Build a web tool where agents enter property details (address, price, bedrooms/bathrooms) and upload a few photos, and AI automatically generates 10 different layouts and copy for beautiful flyers to choose from and download. This tool could be offered as a monthly subscription, say $19/month.
- Keyword:
Through this "data archaeology," without relying on any seat-of-the-pants inspiration, we have discovered at least three concrete, data-backed product directions with clear user profiles and business models.
The greatest advantage of this method is its scalability and objectivity. You can replace the "seed word" with any field you are interested in -- musician, lawyer, scientist, youtube creator... and repeat the process. You are systematically scanning the demand landscape of the entire market.
But data can only tell you "what" (what people are searching for). It cannot tell you "why" (why they are suffering). To understand the deep struggles inside users' minds, we need to put down the data reports, pick up a stethoscope, and listen to the "heartbeat" of the market -- those complaints full of negative emotion.
3.2 Negative Emotion Is a Gold Mine: Lurking on Reddit and in App Store Negative Reviews, Finding the User's "Anger Level"
If data archaeology is the rational left brain, then "emotion mining" is the emotional right brain.
A simple and profound business truth: the intensity of pain determines the willingness to pay. A minor annoyance, users might tolerate. But a pain point that drives them crazy, infuriates them, ruins their entire day -- they will do anything to find a "painkiller."
Your product is that "painkiller." And the user's negative emotions are the strongest signal pointing to where the disease is. You need to become a "collector of negative emotions," an "analyst of user anger levels." We jokingly call this process "anger level farming."
Where do you collect these high-concentration "anger levels"?
Gold Mine One: Reddit -- The Internet's "Rant Central"
Reddit is the world's largest collection of forums. Its anonymity and granular community culture make it a natural, raw "user complaint database."
Infiltration Strategy:
Find vertical communities: Do not just stare at
/r/SaaSor/r/Entrepreneur. Go to the dedicated communities for the professions or hobbies you are interested in.- Interested in designers? Go to
/r/graphic_design,/r/UI_Design,/r/figmaapp. - Interested in writers? Go to
/r/writing,/r/freelanceWriters. - Interested in podcasters? Go to
/r/podcasting.
- Interested in designers? Go to
Learn to use "magic search terms": Search within these communities. Your keywords are not product names, but emotions and problems.
"I wish" [software name] had..."frustrated with" [a specific task]"is there an app/tool for...""[competitor name]" alternative"the worst part about" [a workflow]
Case Study: Discovering an Opportunity in /r/podcasting
You search /r/podcasting for "editing" "hate" and find a post titled: "I love everything about podcasting EXCEPT editing. It's soul-crushing."
You click in, and the OP describes in detail how they spend 4 hours editing a 1-hour recording. They complain:
"...I must have said 'like' and 'you know' a thousand times. Hunting them down one by one in Audacity makes me want to throw my computer out the window. And then there are the long pauses where I was thinking... God, I wish I could just get a text version and delete the words I don't want."
Underneath this post, there are dozens of empathetic replies:
- "Same here! Editing is 80% of the work."
- "I tried hiring a freelancer on Fiverr, but it's too expensive for my small show."
- "Someone please invent a magic button for this."
Opportunity Interpretation:
- High-frequency pain point: Removing filler words and pauses is a universal, deeply painful issue for podcasters.
- Strong emotion: "soul-crushing," "throw my computer out the window" -- these words indicate extremely high "anger levels."
- Existing solutions inadequate: Manual editing is too time-consuming; hiring is too expensive.
- User has already given you the solution: "I wish I could just get a text version and delete the words..." -- the user has told you the product's core feature!
This post is a high-value requirements document. Companies like Descript answer precisely this kind of pain point (calling Descript a "unicorn" reflects media valuation language that shifts over time; take it here to mean a "successful product"). And you, by lurking and listening, could discover the same opportunity before those companies even existed.
Gold Mine Two: Software Store "Negative Reviews"
App Store, Google Play, Chrome Web Store, Shopify App Store... The review sections of these places, especially 1-star and 2-star reviews, are another rich mine.
Infiltration Strategy:
- Find the "top players" in your field: Want to build a note-taking app? Look at reviews for Evernote and Notion. Want to build a project management tool? Look at reviews for Trello and Asana.
- Ignore "crashes" and "bugs": These are usually technical issues, not feature requests.
- Look for "love-hate" reviews: The most valuable negative reviews often start with "I love this app, BUT..." or "It's almost perfect, IF ONLY...." These users are power users of the product. They are not being unreasonable; they have genuinely hit a "last-mile" problem blocking their workflow.
Case Study: Discovering an Opportunity in a Project Management App's Negative Reviews
You see a 2-star review titled: "Great for my team, useless for me."
"I manage 5 different projects for 5 different clients in this app. The app is fantastic for collaborating within each project. But there is NO WAY for me to see a combined view of ALL my tasks across ALL projects due today. I have to click into each project one by one every morning to figure out what I need to do. It's driving me crazy! I'm this close to going back to a simple spreadsheet."
Opportunity Interpretation:
- Clear user profile: Freelancers or project managers managing multiple projects.
- Specific pain point: Lack of a "global task view" or "cross-project dashboard."
- Severe consequences: "driving me crazy," even considering abandoning this powerful tool for a raw spreadsheet.
- Product opportunity:
- Gap product: Build a dedicated "aggregator" tool that connects via API to mainstream tools like Trello and Asana, providing a powerful, customizable "super dashboard."
- Differentiation: If you want to build a project management tool yourself, a "powerful global view" could be your core differentiator against Trello and Asana.
By systematically collecting and analyzing these "negative emotions," you will get a demand list more authentic and vivid than any market report. These complaints are users telling you, in the simplest way, what features they are willing to pay for.
But human needs are not always about "efficiency" and "solving problems." Sometimes we buy a product not to "do more," but to "feel better." This brings us to the third and deepest level of demand mining.
3.3 Non-Functional Needs: Beyond Efficiency, There Is Anxiety and Loneliness (The Underlying Logic of Tarot, Companionship, and Healing Products)
So far, the demands we have discussed fall under "functional needs." They help users complete a specific task, solve a practical problem -- like "edit podcasts faster" or "manage tasks better." These needs are characterized by clear, quantifiable value.
But beneath the surface lies a vaster, more turbulent ocean of demand -- non-functional needs. These do not directly help you "do things," but satisfy some profound emotional or psychological state.
They include:
- Relieving anxiety, seeking certainty
- Fighting loneliness, craving connection
- Self-exploration, identity confirmation
- Finding comfort, healing trauma
- Experiencing a sense of control, feeling empowered
In the AI era, for the first time, we have tools that can satisfy these deep emotional needs at scale and low cost. Those seemingly "unreliable," "mystical" AI products that go viral and command high prices do so precisely because they accurately target these non-functional needs.
Let us deconstruct a few typical examples and see how they work.
Case 1: AI Tarot/Astrology -- Selling "Certainty" and "Narrative"
- Product form: Users input their question or birth date, and AI generates a tarot card reading or astrological analysis full of symbolism.
- Surface function: Fortune-telling.
- Underlying need: Relieving anxiety about the future, giving life a "narrative framework."
- User psychology: A user feeling uncertain about their career draws the "Knight of Swords" card. The AI interprets: "This card represents the need for decisive, swift action, but it may also come with impulsiveness and lack of forethought. It reminds you to keep a clear head and not be swayed by emotions when pursuing your goals."
- Does this user truly believe AI can predict the future? Not necessarily.
- But they have gained a thinking tool. This interpretation gives them a "script," a fresh perspective to examine their situation. It organizes messy anxiety into a story about "courage and caution." What the user pays for is not "prediction," but this sense of "order" and "control."
- Your opportunity: The technical barrier for this type of product is extremely low. The core is GPT + a well-crafted Prompt. Your prompt needs to incorporate rich tarot knowledge, Jungian psychological archetypes, and scripts full of wisdom and comfort. The product's differentiation lies in the UI's mystical design, the interaction's ritualistic feel, and the depth of the prompt engineering. One boundary: the product may state plainly that it is an entertainment and self-reflection tool, but it must not claim to actually predict the future — the former is a narrative service, the latter exploits the anxious and brushes against false-advertising law in most jurisdictions.
Case 2: AI Virtual Companion -- Selling "Companionship" and "Unconditional Acceptance"
- Product form: An AI character available 24/7 to chat with you, with its own persona, memory, and emotional responses.
- Surface function: Chatbot.
- Underlying need: Fighting profound loneliness, gaining a "listener" who is always online, always patient, and never judges you.
- User psychology: A user feeling suppressed and misunderstood in real life can confide any secret to their AI companion without fear of ridicule or betrayal. The AI gives positive feedback ("You are so brave," "I understand how you feel"). This "unconditional positive regard" is extremely scarce in real human relationships. What users pay for is this safe, private emotional connection.
- Your opportunity: This is also a GPT/LLM + Prompt + database (for storing memory) combination. The key lies in:
- Persona building: Is your AI a gentle girl-next-door or a learned philosophy mentor? The charm of the persona determines user stickiness.
- Memory system: Remembering the user's name, hobbies, and past conversation topics is key to transforming AI from a "tool" into a "companion." This can be achieved with vector databases.
- Ethical boundaries: This is a controversial but hugely promising field. You need to carefully design safety and ethical boundaries to avoid creating harmful dependencies. Especially for minors and users in psychological crisis, AI companion products can cause real harm — build identification and exit mechanisms for high-risk users into the first priority of your product design.
Case 3: AI Personal Coach/Healing App -- Selling "Hope" and "Self-Empowerment"
- Product form: An AI application incorporating psychological theories like CBT (Cognitive Behavioral Therapy), using guided conversations to help users identify negative thought patterns and build positive habits.
- Surface function: Habit tracker, journal app.
- Underlying need: On the path of self-improvement, gaining a "tireless cheerleader" and "personal coach" to fight procrastination and self-doubt.
- User psychology: A user sets a goal of "exercise three times a week." When they complete it, the AI says: "Amazing! You have done it again. I can see your perseverance growing." When they fail to complete it, the AI does not scold but asks: "That is okay. Let us see what got in the way. Should we adjust the goal to something smaller?" This non-judgmental support and positive psychological reinforcement greatly boosts the user's self-efficacy.
- Your opportunity: You need to learn some basic psychological frameworks (like CBT, mindfulness) and integrate them into your prompt design. Your product's value lies not in recording but in "conversation" and "guidance." The UI/UX should convey a sense of calm, safety, and inspiration. One red line you must not cross: such a product is not psychotherapy. Do not promise clinical outcomes or substitute for professional help; if a user signals self-harm or severe distress, the product should guide them toward professional crisis-intervention resources rather than continuing to "soothe" via AI conversation.
How to Discover Non-Functional Needs?
These kinds of needs cannot be found through simple keyword searches. It requires you to be more of a social observer:
- Pay attention to social trends and cultural phenomena: Why is MBTI so popular? It satisfies people's needs for "self-exploration" and "finding belonging." Why are meditation apps popular? They cater to modern people's need to "relieve stress and fight information overload."
- Read psychology and sociology books: Understanding concepts like Maslow's hierarchy of needs, attachment theory, and cognitive dissonance opens up a whole new perspective.
- Analyze popular "non-tool" products: Analyze games, social apps, and content communities. What emotional needs do they satisfy? Is it a sense of achievement? Belonging? Showing off?
Summary of Chapter 3:
We have established a three-dimensional, three-layer deep demand mining model:
- Data Archaeology (Surface layer): By analyzing search data, find "functional needs" that have been initially validated by the market and have commercial value. This is your starting point.
- Emotion Mining (Middle layer): By listening to user complaints, understand the "pain level" behind the function, and judge whether a need is "worth doing." This is your filter.
- Non-Functional Needs (Deep layer): By understanding human nature, discover deep emotional needs related to anxiety, loneliness, and hope, and find opportunities to create high-premium, high-stickiness products. This is your path to advancement.
A mature independent developer should be able to switch freely among these three levels. Perhaps your first product is a tool solving a small pain point found through "data archaeology," but your ultimate ideal might be to build an "emotional product" that comforts people.
Now you have a "demand candidate list" full of potential. But they are still hypotheses. Before investing precious time and energy in development, we must use the fastest, lowest-cost method to verify one ultimate question:
Will anyone really pay for it?
This is the core problem we will solve in the next chapter: Validation and Filtering.