FeedbackLens
Turning scattered customer feedback into decisions you can defend
FeedbackLens
Product Manager & Technical Lead
End-to-End Product Build (0 to 1)
August 2026 (12-week roadmap planned)
Product teams don't need a smarter feedback dashboard; they need less manual synthesis work between "customers said X" and "we're building Y, and here's why." FeedbackLens closes exactly that gap.
Full PRD, working prototype, and source code available
The Problem
Understanding who we're building for and why it matters
FeedbackLens users
Mid-sized B2B SaaS teams (50-500 employees) receive 200+ feedback items weekly across support tickets, reviews, interviews, and surveys. The evidence exists, but synthesizing it into defensible priorities takes 4-6 hours per week of manual work.
Teams default to gut-feel prioritization not because evidence doesn't exist, but because gathering it is slower than trusting instinct. That friction compounds: slower decisions, inconsistent prioritization, undocumented rationale, and zero feedback loop on whether shipped work actually helped.
Export from 3-4 tools → paste into spreadsheet → manually spot themes → estimate impact by gut feel → prioritize → hope you remember why in 6 months
Product question: How do you turn 200+ weekly feedback items into defensible priorities without manual synthesis?
Product Direction & Solution
The product approach chosen and how it addresses the problem
FeedbackLens becomes the system of record for "why we built what we built", where the connection between customer evidence and product decisions is never more than one click away.
Key Features
Centralized feedback repository with AI-assisted tagging (sentiment, topic, complaint/request detection)
Automatic clustering of related feedback into named opportunities
Transparent, formula-based opportunity scoring (no black-box ranking)
Structured decision records permanently linked to evidence
Roadmap planning with every item traceable to its originating decision
Release validation: before/after sentiment comparison to check if shipped work helped
Product health dashboard showing sentiment trends, open opportunities, and release outcomes
The decision: Product teams don't need a smarter feedback dashboard; they need less manual synthesis work between "customers said X" and "we're building Y, and here's why." FeedbackLens closes exactly that gap.
Key Product Decisions
Trade-offs and rationale for major design choices
Built a transparent, formula-based scoring model instead of pure ML ranking
Rationale: Score = 0.35×Frequency + 0.30×Complaint Rate + 0.20×Sentiment + 0.15×Confidence. Simple enough to compute on a whiteboard, which means PMs can defend it in stakeholder meetings. A learned model would be marginally more accurate but impossible to explain.
Sacrificed some ranking precision to gain complete explainability
Capped frequency input at 90th percentile before normalizing
Rationale: Without a cap, one opportunity with 1,000 mentions would compress every other score toward zero, even if those opportunities affect real customer segments. Same principle RICE uses (Impact capped at 5).
Outliers still score highest (they should), but they don't silently erase the rest of the board
Made PII redaction automatic, not optional
Rationale: Privacy is a default, not a feature request. Every feedback item gets scrubbed before reaching the AI tagging API. Regex + NER for names, emails, phone numbers.
Adds processing overhead, but non-negotiable for trust
Paired speed metric with quality counterweight
Rationale: North Star is "time-to-decision," but tracking speed alone would reward fast bad decisions. Added Decision Reversal Rate (% of decisions overridden within 90 days) to prevent gaming.
More complex measurement, but prevents optimizing for the wrong thing
Batch import, not real-time sync (for MVP)
Rationale: Scheduled pull every 6 hours proves the core workflow without the engineering cost of webhooks and sync conflict handling. Real-time can come later if validated.
Not instant, but sufficient to test the hypothesis
Manual edits survive re-clustering
Rationale: When PM renames or merges opportunities, those edits live in a separate override layer. New feedback matches against existing (renamed) opportunities first before creating new clusters.
More complex data model, but required so manual cleanup isn't lost on next import
Cut full collaboration (comments, @mentions) from MVP
Rationale: Added lightweight read-only share link instead. Solves actual failure mode (showing a decision to someone without login) without the build cost of notifications and threading.
No async discussion in-tool, but share link covers 80% of the need
Technical Implementation
How engineering background informed product decisions
AI/ML Integration
- •
LLM API integration for sentiment analysis and topic extraction
- •
Confidence scoring on every AI-generated tag (flags items <50% for review)
- •
Custom clustering algorithm for grouping feedback into opportunities
- •
PII detection pipeline (regex + NER) before external API calls
Data Architecture
- •
Separate override layer for manual edits (renames, merges, splits)
- •
Append-only decision records (full history preserved)
- •
Reversible token map for PII redaction (sanitized for API, real values shown internally)
- •
Centroid-based matching for incoming feedback against existing opportunities
Scoring Engine
- •
Composite scoring formula with outlier protection
- •
Real-time score recalculation when new evidence added
- •
Visible calculation breakdown (each input's contribution)
- •
Manual rank override with required rationale
Product Features
- •
CSV/API ingestion with schema mapping
- •
Searchable, paginated evidence views
- •
Drag-and-drop roadmap board (Backlog → Planned → In Progress → Released)
- •
Before/after sentiment comparison with minimum volume thresholds
- •
Read-only share links with revocation support
PRD Process & Documentation
How product requirements were structured and validated
External Review Process
- →
PRD reviewed by product mentor; every piece of feedback traced to specific changes in v3.0
- →
Caught scope contradiction between sections, missing PII handling, speed-without-quality metric trap
- →
Added documentation of how manual edits survive re-ingestion (wasn't originally specified)
Success Metrics
- →
North Star: Median time from feedback received to decision logged (baseline 12 days → target <3 days)
- →
Paired with Decision Reversal Rate to prevent optimizing for speed alone
- →
All baselines labeled as estimates with validation plan, not fake precision
Functional Requirements
- →
23 requirements across 8 epics, MoSCoW prioritized
- →
Acceptance criteria written for engineering (specific, testable)
- →
Three requirements added/revised in v3.0 from external review
Competitive Analysis
- →
Benchmarked against Productboard, Canny, Zendesk, BI tools, spreadsheets
- →
Identified gap: no tool connects evidence → scoring → decision → validation in one chain
- →
Honest risk assessment: incumbent could add decision log feature and close gap
Product Principles
- →
9 principles used to break ties on scope/design decisions
- →
Examples: "Evidence before opinion," "AI assists, humans decide," "Prioritization should be transparent"
- →
Privacy principle added in v3.0 after review caught missing PII handling
Goals & Success Metrics
How we define success for users, the product, and the business
Automate feedback synthesis and make prioritization auditable
Spend less time gathering evidence, more time deciding what to do with it
Cut manual triage hours, build institutional memory that survives turnover
Decision Reversal Rate <10% (quality counterweight to speed)
≥90% of roadmap items with linked evidence
<1.5 hours/week manual triage time per PM (down from 4-6 hours)
≥75% of releases get before/after validation
User-reported confidence in prioritization ≥4.0/5.0
Learnings
What I learned, what surprised me, and what I'd do differently
What I Learned
- •
Transparent beats accurate: PMs need to defend scores in meetings, so explainability matters more than marginal ranking precision
- •
Speed without quality is a trap: pairing time-to-decision with reversal rate prevents gaming the metric
- •
External review is invaluable: mentor caught scope contradictions, missing PII handling, and outlier math gaps I wouldn't have seen
- •
Users don't need full collaboration if share links solve the actual pain point (showing decisions to stakeholders)
- •
Manual work should compound, not restart: edit survival across re-ingestion was critical but not in initial spec
- •
PII redaction must run before third-party APIs, not as an optional feature
- •
Outlier protection in composite scores prevents one data point from dominating the entire distribution
- •
Append-only data models preserve history better than update-in-place for audit trails
- •
Off-the-shelf LLMs are good enough for MVP; custom training is a scaling problem, not a day-one problem
- •
Confidence scores on AI tags turn uncertainty into a UX feature instead of hiding it
What Surprised Me
That external review caught scope contradictions and missing requirements I completely missed after weeks of writing
What I Would Change
Would have run a two-week shadow period with real PMs before finalizing the scoring model weights. The current weights (0.35/0.30/0.20/0.15) are principled but not validated against actual PM judgment. A shadow period would give real baseline data instead of estimated baselines, and surface edge cases in workflow the spec might be missing.
Run a two-week shadow period with real PMs to validate scoring model weights against actual judgment
This project taught me that building a product and documenting it as a PM are different skills. Writing the PRD after building the prototype revealed gaps in my original thinking (like PII redaction, which wasn't in the initial spec) that a PM-first approach would have caught earlier. The external review process made that lesson concrete: every piece of feedback traced directly to a change in the document, which is exactly how product decisions should work.