We currently generate individual tables. This could in fact be improved with using a SQL library. We could perhaps use SQLModel to support pydantic and pandas.

• Add SQLModel models for clone, traffic and referring
• Develop pydantic models for each data view/stats
• Add db module
  • configure function for setting up SQLAlchemy engine
  • query function to be able to query SQLite by repository_name and date
  • migrate_csv that will work with merged and individual run CSV files (difference is with skip_rows settings).
• Data Migration:
  • Script that can be executed within the Docker container to pull in merged data and parse that over to sqlite3.db
• Will need to load the sqlite3 database at some point when the gts_run_all_repos script is executed
• Be able to transform SQL data to pandas DataFrame to construct figures/charts
• Update documentation
• Test migration with a separate repo: astrochun/sqltest-github-stats