All projects

live

Global Financial Pulse

Prices, headlines, and a sentiment score, on a schedule.

  • Python
  • Dagster
  • dbt
  • PostgreSQL
  • Docker
  • Streamlit

Global Financial Pulse is an end-to-end ELT pipeline that tracks global market sentiment.

Pipeline

  • Ingests Yahoo Finance prices and News API headlines with async Python, using 10 concurrent workers.
  • Scores each headline with TextBlob, on a scale from -1 to +1.
  • Dagster runs the pipeline every 30 minutes during US market hours, plus a daily retention cleanup.
  • dbt transforms the data into a Bronze / Silver / Gold medallion architecture in PostgreSQL.
  • The stack is containerized with Docker and self-hosted on a Synology NAS.
  • A Streamlit dashboard reads the curated data.