TL;DR
Painted doors and fake doors let us test product ideas before we build them. Painted doors check for basic interest, while fake doors dig for stronger signals.
Before launching anything, we set up a clear hypothesis, define success metrics, and decide when to stop. We track the whole user journey from start to finish.
We respect our users. That means we explain what’s happening, never take payments, keep data minimal, let people opt out, and make tests accessible.
We choose tests based on risk and fidelity. Higher risk ideas need more detailed tests.
We use simple decision rules like “click rate must be 5% AND waitlist quality must be 70%.” We always document what we learn.
What these tests are (one-line definitions)
A painted door test puts a button or menu option in front of users that looks real. If they click, we let them know the feature isn’t ready, then ask if they want updates or to take another action.
A fake door test lets users go deeper—maybe they fill out forms or make choices—before we reveal the feature doesn’t exist. We collect more detailed info this way.
Both tests measure interest before we invest in building. Painted doors catch quick curiosity. Fake doors measure stronger commitment by letting users take more steps.
When to use which
We use painted door tests for quick feedback with low friction. They’re great for checking if people notice a feature, where to put it, or if a pricing tier grabs attention.
Fake door tests help us see who really commits. We use these when testing configuration choices, trial willingness, or changes to workflows.
Smoke tests give us market-fit hints before we build any UI. We use them for new product lines or customer segments.
Step-by-step playbook
1) Frame the bet
Start with a clear hypothesis: “If we offer [capability], [segment] will show [behavior] at ≥ [threshold].”
Pick your main metric. It might be click-through rate, progression to a certain step, or qualified waitlist rate.
Set up guardrails. Watch for changes in NPS/CSAT, support tickets, complaints, refunds, and accessibility issues.
Write your decision rule before you start. Decide what counts as Pass, Fail, or Follow-up. This keeps things honest.
2) Choose the test type
Go with a painted door test if you want early signals or you’re testing something low-risk. You’ll get basic user interest data.
Pick a fake door test if you need higher confidence. Add more ethics controls, since this can feel more misleading.
Try a smoke test if you don’t have space in your product yet.
3) Target the right audience
Segment users by plan, role, region, or device. Don’t target people in critical workflows like financial close or medical tasks.
Keep exposure low—maybe 5-10% of eligible traffic. Avoid VIP and enterprise accounts unless you have a pilot agreement.
4) Design the experience
Write copy that focuses on value. Skip clickbait or tricks.
Make the button or link look real, but don’t mislead beyond the first click.
Reveal honestly. Explain what’s going on, give a timeline, and offer a next step. Let users join a waitlist, book a call, or find a workaround.
Check accessibility. Buttons should work with keyboards and screen readers. The reveal screen should be readable and polite.
5) Instrumentation
Track these events:
| Event Type | Details |
|---|---|
| Exposures | Who saw the test |
| Clicks | Who clicked the fake feature |
| Progression | Movement through each step |
| Waitlist submissions | Sign-ups for future access |
| Complaints | User feedback issues |
| Support tickets | Help requests related to test |
Log context like user segment, entry point, copy version, and time spent at each step.
6) Sample size & thresholds (quick math)
For click-through rates, use this for margin of error: ME ≈ 1.96 × √(p(1−p)/n)
Example: You expect 3% CTR and want ±0.5 percentage point precision. 0.005 ≈ 1.96 × √(0.03×0.97/n) So you need about 45,000 exposures.
If you can only get 10,000 exposures, your margin of error is ±1.07 percentage points. Decide if that’s good enough for your go/no-go call.
7) Ethics gate (before launch)
Don’t take payment for features that don’t exist.
Show disclosure right away when users click. Apologize sincerely—don’t bury it in fine print.
Collect only the data you’ll actually use. Make opting out and deleting data easy.
Offer users real value. Suggest workarounds, highlight existing features, or invite them to research sessions.
Let your support and sales teams know ahead of time. Give them FAQ answers and templates.
8) Run and monitor
Run the test until you reach your sample size or 2 weeks, whichever comes first. Stop early if guardrails trip.
Check daily metrics—support volume, complaint rate per exposure, and accessibility bugs.
9) Decide and log
Apply your decision rule. For example: Pass if CTR ≥ 2% and ≥ 30% of waitlist users fit your ideal profile. Fail otherwise. If CTR is high but customer quality is low, follow up.
Write down your assumptions, any odd results, and next steps. That could mean building the feature, doing more research, or dropping the idea.
Trade-offs at a glance
Each method has its upsides and risks. Painted door tests are quick and cheap. They validate ideas with little effort.
Fake door tests give stronger intent signals and richer data, but risk disappointing more users and take more work.
Smoke tests let us check demand before building anything, but they don’t show how people behave in the real product.
| Test type | Speed | Build effort | Signal strength | Risk level |
|---|---|---|---|---|
| Painted door | Fast | Minimal | Moderate | Low |
| Fake door | Medium | Higher | Strong | Medium |
| Smoke test | Fast | Low | Moderate | Medium |
Pick based on what you need to learn and what risks you can live with.
Realistic examples
A SaaS analytics company adds an “Export audit trail (CSV)” option in their settings. If users click, they see a message inviting them to join early access for secure exports.
The company collects info about user roles and export needs. They move forward if the click rate hits 1.5% and at least 50 companies from regulated industries show interest.
A fitness app adds a “Coach chat” bubble to workout screens. When users tap it, they answer questions about their goals and schedules.
Then they see a message saying coach chat is in pilot phase. The app watches support tickets. If more than 0.3% of users complain about missing chat, they pause and tweak the copy.
Worked decision example (RICE to prioritize which idea to test)
Product managers need to pick between Team Dashboards and Slack Alerts. We use RICE scores to decide.
Team Dashboards will reach 8,000 monthly active users this quarter. Slack Alerts hit 5,000. Both tests take a week.
We expect dashboards to have medium retention impact at 0.5. Alerts look higher at 0.7. We’re 60% confident about dashboards and 50% for alerts.
| Feature | Reach | Impact | Confidence | Effort | RICE Score |
|---|---|---|---|---|---|
| Team Dashboards | 8,000 | 0.5 | 0.6 | 1 week | 2,400 |
| Slack Alerts | 5,000 | 0.7 | 0.5 | 1 week | 1,750 |
With (R × I × C) / E, dashboards score 2,400 and alerts score 1,750.
We run the painted door test on dashboards first. If it passes, we queue up alerts next. This helps us focus on the highest-scoring bet.
Implementation specifics (so you don’t annoy users)
Copy patterns that work
Your landing page copy should feel natural and honest. Start with something like “New: Team dashboards (beta)” before users click.
When they land, use a headline like “We’re building this—here’s how to get early access.” Set expectations right away.
Keep the body text short. Try “Thanks for your interest. We’re testing demand before we launch. Join the list and help shape it.”
Your buttons should offer choices. Use “Join early access” as the main action, and “Show me alternatives” as a secondary.
Data you actually need
Keep your forms simple. Just ask for an email or use their existing login if they’re already signed in.
Ask about their role and company size. Find out how often they’d use this feature.
These questions help you get a sense of real demand. You can ask about pricing, but make sure it’s clearly marked as research.
Let users know this is for planning, not a real purchase.
| Required Data | Optional Data |
|---|---|
| Email/ID | Pricing preferences |
| Role | Feature trade-offs |
| Company size | |
| Usage frequency |
Where to place the door
Put fake doors where users naturally look for new features. Settings pages usually work well.
Empty states and screens that show up after tasks get finished are also good spots.
High-intent pages like your pricing page, upgrade screens, or feature cards in related sections of your user interface often make sense.
Never add fake doors in critical workflows. Skip checkout pages, emergency tools, or anything where interruptions could cause real problems.
Your user experience should feel helpful, not disruptive.
Guardrails & ethics (non-negotiables)
We explain tests in plain English on reveal pages. Clear consent comes first.
Dark patterns are forbidden. Don’t hide explanations or auto-opt people into tests.
We collect only the minimal data and link to our privacy policy. If users ask to delete data, we do it right away.
We follow WCAG accessibility guidelines. That means proper labels, focus states, and screen-reader text for beta features.
Fairness matters. We don’t target vulnerable users or test critical operations without a backup plan.
We close the feedback loop by emailing test outcomes to participants.
Pitfalls & better alternatives
Fake door tests often just measure curiosity, not true intent. For important features, we add a short setup step after the click.
This helps us see who actually wants to use the feature.
Testing the wrong people gives bad results. We filter for our ideal customer profile and place tests where users already do the task.
We also limit how many people see the test.
Over-promising in our copy creates headaches later. We label tests as beta or early access.
We use time ranges like “coming this quarter” instead of specific dates we might miss.
We need both numbers and user feedback. Adding an optional text box helps us understand why people clicked.
We check support tickets every day for complaints or confusion.
Stop rules protect us from bad outcomes. Before we start, we set limits for complaints, support tickets, and bugs.
If we hit those limits, we stop the test.
Writing our decision rule before the test starts keeps us honest. For unclear results, we run follow-up tests to confirm.
Mini FAQ
Is a painted door enough to approve a new feature? Nope. A painted door shows interest, but you need more data. Combine it with user details, willingness-to-switch tests, or smoke tests to get a better sense of market size.
Can we test pricing through fake doors? Yes, but never take payments for features that aren’t real. Use plan choices or price ranges to collect signals. Always tell users right away what happened.
How do we prevent angry users? We stay honest when we reveal the test. We offer other solutions and limit how many people see it.
Our support and sales teams need to know about the test so they can help users.
What click rate should we expect? It really depends on your product. Compare results to similar features or upgrade buttons you already have.
Most teams want 1-3% or higher click rates and good user quality. Set your own targets based on your traffic and customer value.
Should big enterprise customers see these tests? Only if they agreed to be part of pilots or early programs. Otherwise, leave out large accounts and run proper research pilots instead.
Closing notes
Let’s make hypothesis-driven, instrumented, and honest painted doors actually work. Pick the simplest test that still respects user time and privacy.
Clear decision rules? They help shape better roadmaps and keep our reputation intact. It’s not always easy, but it’s worth it.
