Try to do everything in NodeJS, if possible. This will help get more traction in the JAMStack community
Phase 1: Fundamental infrastructure
Backend
- Scrapers to scrape data from various sites
- PostgreSQL database to store data which can later be aggregated
- ElasticSearch cluster to store text data
- Some way of storing images efficiently to store image data
- BigQuery —> PostgreSQL for analytics data
- Redis for in-memory stuff
- Code to perform operations on data in databases
Frontend
- Frontend with auth
- Components for different kinds of visualizations
- Components for exporting
- Drag and drop interface for dashboards
- Sample investment dashboard
Phase 2: Rich data science libraries in JS
JS libraries to do computation in the browser, and get really cool correlations/findings on the fly.
⇒ Basically limit backend to just storing data, and periodically running cronjobs. Keep all other processing on the front-end. This will lead to lightning fast queries!