OpportunIQ

Gained

• +Full control over what data gets extracted
• +No per-request costs at scale
• +Can customize parsing for specific sites
• +No dependency on third-party service availability

Gave Up

• −Had to handle rate limiting, retries, and site-specific edge cases
• −More maintenance when sites change their structure
• −Initial development time was significant

Why Worth It

Needed deep control over how product information, forum posts, and pricing were extracted. Generic APIs would not capture the specific data needed for accurate recommendations.

Gained

• +No training data needed - works out of the box
• +Handles diverse image types (cracks, mold, mechanical issues)
• +Natural language output easy to integrate
• +Continuous improvements from OpenAI

Gave Up

• −Per-request API costs
• −Less control over model behavior
• −Dependent on OpenAI availability and pricing
• −Privacy concerns with sending images to third party

Why Worth It

Training a custom model would require thousands of labeled images. OpenAI Vision works well enough for the diagnostic use case and allowed faster iteration.

Gained

• +Works with OpenAI Vision which only accepts images
• +Cheaper than video-specific AI services
• +More control over which frames to analyze
• +Can deduplicate similar frames to reduce costs

Gave Up

• −Loses temporal information (motion, sound)
• −Processing overhead to extract frames
• −May miss issues only visible during motion

Why Worth It

For diagnosing static issues like cracks or damage, individual frames capture what is needed. Motion analysis would be overkill and more expensive.

Gained

• +Users trust the platform with sensitive home data
• +Reduced liability for data breaches
• +Differentiator from competitors
• +Simpler compliance story

Gave Up

• −Cannot use user data to improve ML models
• −Cannot do analytics on diagnostic patterns
• −More complex implementation

Why Worth It

Users are sharing photos and videos of their homes. Trust is essential. The privacy-first approach builds that trust even if it limits data collection.

Gained

• +Recommendations are actionable, not just technically correct
• +Can prioritize based on user circumstances
• +Differentiates from generic DIY advice sites

Gave Up

• −Users must share financial information
• −More complex recommendation engine
• −Privacy sensitivity around income data

Why Worth It

A $500 repair means different things to different people. Financial context makes the "fix now vs. defer" recommendation actually useful.

Gained

• +No expensive fine-tuning process needed
• +Recommendations always use latest scraped data
• +Can cite sources - users see where advice comes from
• +Easy to add new products/content without retraining
• +Cheaper than maintaining fine-tuned models

Gave Up

• −Additional latency for retrieval step (~50-200ms)
• −Quality depends on chunking and embedding strategy
• −Context window limits how much info can be included
• −More infrastructure (vector database, embeddings pipeline)

Why Worth It

Home improvement advice changes constantly - new products, updated prices, seasonal recommendations. RAG ensures recommendations are always current without costly model retraining.

Gained

• +Single database for all data - simpler architecture
• +No sync between relational and vector data
• +Full Postgres features available (joins, transactions, RLS)
• +Lower infrastructure costs - no separate vector DB service
• +Can query vectors alongside relational data in one query

Gave Up

• −Less specialized performance at very large scale (10M+ vectors)
• −Fewer advanced vector features (hybrid search less mature)
• −Postgres tuning needed for vector workloads
• −May need to migrate to dedicated vector DB if scaling significantly

Why Worth It

For thousands of product embeddings, pgvector is fast enough. Keeping everything in Postgres eliminates sync complexity and reduces costs. If I hit millions of vectors, I can migrate - but that is a problem for later.

Gained

• +Cuts OpenAI API costs by 60-70% per video
• +Faster processing - fewer API calls
• +Reduces redundant analysis of nearly identical frames
• +Runs locally with no external dependencies

Gave Up

• −May miss subtle differences between similar frames
• −Additional processing step before AI analysis
• −Threshold tuning required (how similar is too similar)
• −Does not understand semantic similarity, only visual

Why Worth It

Users often hold camera still or pan slowly - 30 frames may contain only 8-10 unique views. Deduplication is essential for cost-effective video analysis at scale.

Gained

• +User does not need to keep browser open during processing
• +Can handle longer videos without timeout issues
• +Better failure recovery - can retry individual frames
• +Scales better under load - queue-based processing

Gave Up

• −User waits longer for results
• −Need to handle notification delivery (email, push)
• −More complex UX - not immediate feedback
• −Must store processing state somewhere

Why Worth It

A 30-second video takes ~45 seconds to fully analyze. Blocking the UI that long is a terrible experience. Async processing with notification means users can continue their day and get results when ready.

Gained

• +Postgres, Auth, Storage, Edge Functions in one platform
• +Generous free tier for MVP validation
• +No DevOps overhead - managed infrastructure
• +Built-in Row Level Security for authorization
• +Real-time subscriptions if needed later

Gave Up

• −Less flexibility than raw AWS
• −Vendor lock-in to Supabase ecosystem
• −Scaling ceiling compared to custom AWS architecture
• −Some advanced Postgres features require workarounds

Why Worth It

For a 0-to-1 product, I need to validate the idea fast, not manage infrastructure. Supabase lets me focus on the product. If I hit scale limits, that means the product is working and I can afford to migrate.

Gained

• +Server Components for SEO and performance
• +Server Actions for mutations - no separate API layer
• +Single deployment unit - simpler ops
• +Vercel handles scaling automatically
• +Streaming and Suspense for progressive loading

Gave Up

• −Less separation between frontend and backend
• −Server Actions are newer - less ecosystem support
• −Harder to share backend with mobile app
• −Vercel-specific optimizations may limit portability

Why Worth It

For a web-only product with no mobile app, the simplicity of a unified Next.js app outweighs the flexibility of separate frontend/backend. Ship faster, refactor later if needed.

Gained

• +5x cheaper than text-embedding-3-large
• +Lower latency - faster embedding generation
• +Smaller vectors (1536 dim) - faster similarity search
• +Good enough quality for product recommendation use case

Gave Up

• −Slightly lower quality embeddings
• −May miss subtle semantic differences
• −Less suitable for highly nuanced retrieval

Why Worth It

For home improvement product recommendations, the small model captures enough semantic meaning. The cost savings compound - every scraped page and every user query needs embeddings.

Gained

• +Cheerio is 10x faster for static pages
• +Lower compute costs - no browser overhead
• +Puppeteer reserved for JavaScript-heavy pages only
• +Parallel scraping is easier without browser instances

Gave Up

• −Need to detect which pages need Puppeteer
• −Two code paths to maintain
• −Some pages might fail detection and return empty content

Why Worth It

At scale, running Puppeteer for every page would be extremely expensive. 70% of pages are static HTML (forums, articles) and can use the fast path.

Gained

• +Can scale workers horizontally
• +Automatic retry with exponential backoff
• +Priority queue for different content types
• +Progress tracking and dead letter queue
• +Graceful handling of rate limits

Gave Up

• −More complex infrastructure
• −Need to manage queue persistence
• −Harder to debug than simple sequential scraping
• −Overkill for small-scale scraping

Why Worth It

Scraping 10,000+ pages reliably requires proper job queue infrastructure. A cron job would fail silently, retry incorrectly, or overwhelm target sites. The queue adds reliability at the cost of complexity.