Create a roadmap

[patcon]

Going to start scaffolding out a loose roadmap. For now, it will be a single issue list of tasks, just as things take shape.

This is a work-in-progress body of text.

• admin
  • add Agora Citizen Network sponsorship details to README Add Agora sponsorship section to README #5
  • transfer repo to https://github.com/polis-community
  • initiate community call format
  • add tasks from completed backlog to this list
  • migrate this task list to project board
  • documentation
    • get auto-generation and deployment of doc site working
    • finish API references in doc site
    • migrate to material theme for mkdocs
      • add LaTeX support, and document math from polis paper
    • add documentation for models/types: https://github.com/pyapp-kit/griffe-fieldz
• data loading
  • from conversation_id
  • from report_id
  • from local files
  • from remote directory path
  • infer moderation type when importing from API
  • infer moderation type from API when loading local or remote CSVs
    • allow override, in case API mismatches state when archived
• dimensional reduction steps
  • add support for minimal basic PCA for dimensional reduction algorithm
  • properly account for meta-tids in PCA step
  • add support for power iteration method PCA
  • research and/or add support for more advanced dimensional reduction algorithms (see below)
• clustering steps
  • implement basic kmeans (with random initialization or kmeans++)
  • find best k via silhouette scores
  • initialize kmeans centers from existing centers output from polismath (these should almost always match, barring k-smoothing effects)
  • figure out how often math ticks occur (number of votes? passing of time? every vote capped to minimum time?)
  • implement k-smoothing functionality
  • reproduce last 4 math_tick pca results from raw votes, and confirm clusters match with k-smoothing
• implement presentation helpers to simplify common analysis from jupyter notebooks (see below)
  • reproduce all the analysis notebooks using this library
• add support for bucketizing participants with kmeans (default k=100)
• implement comment projection on graph
• Add function to calculate representativeness of statement for groups #22
• presentation
  • render participants on plot
  • render projected comments on plot
  • render concave hulls around clusters
• Add fundtion to calculate group aware consensus (gca) scores on statements #23
• calculate comment priority on statements
• migrate presentation helpers from matplotlib to plot.ly
• migrate from pandas to polars for dataframes (faster rust implementation)
• bonus
  • create a react component to render a polis-like graph from a known minimal data structure
  • investigate imputation of missing data via TabPFN (which would allow more dimensional reduction methods that can't operate on sparse matrices)
    • https://news.ycombinator.com/item?id=42647343
    • Reference: https://news.ycombinator.com/item?id=42653722

Meta

• Paper: Polis: Scaling Deliberation by Mapping High Dimensional Opinion Spaces
• types of conversations to align with Polis math output
  • < 100 participants, NO meta comments
  • < 100 participants, WITH meta comments
  • > 100 participants, NO meta comments
  • > 100 participants, WITH meta comments
  • why?
    • above 100 participants, a first round of kmeans clustering is run, which is harder to reason about and compare results.
    • meta tids are treated in a way that I'm not clear on yet, and I worry there might be some bugs or edge-cases in Polis code around this, which are harder to reproruce (since meta statement functionality was added much later)
• datasets used to test dimensional reduction algos (global and local structure)
  • mammoth. 3-dimensional scan of 3D mammoth with labelled parts, and specific shape in global.
  • MNIST. handwriting. very high dimensional (pixels).
  • S-curve with hole. global structure important here.
• extended dimensional reduction algos of interest
  • comparison spreadsheet
  • see wiki page for details: https://github.com/patcon/red-dwarf/wiki/Dimensional-Reduction-Algorithms
• related Jupyter Notebooks for polis data analysis
  • https://github.com/compdemocracy/analysis/tree/master/notebooks/jupyter
  • https://github.com/ilivieris/pol.is
  • https://github.com/aadityabhatia/polis-argmap/tree/main/docs
  • https://github.com/collect-intel/osccai/blob/main/notebooks/osccai_simulation.py
  • https://github.com/saffronh/ccai/blob/main/ccai_data_processing.ipynb
  • https://scrapbox.io/nishio-en/Analyze_Polis_data
    • https://github.com/mielka/yoronchizu2024-data/blob/main/README.ipynb
    • https://github.com/mielka/yoronchizu2024-data/blob/main/experimental.ipynb
    • https://github.com/nishio/UTAS-UMAP/blob/main/UTAS-Polis.ipynb
    • https://github.com/nishio/UTAS-UMAP/blob/main/UTAS-UMAP.ipynb
  • https://github.com/Oliz888/Voting_Consensus_Detection/blob/main/00_voting_and_consensus.ipynb
  • https://github.com/Oliz888/Voting_Consensus_Detection/blob/main/voting_and_consensus_1.ipynb
  • https://github.com/Oliz888/Voting_Consensus_Detection/blob/main/voting_and_consensus.ipynb
  • https://github.com/AlgoTrading101/Polis-Algotrading101/blob/main/Polis.ipynb

[patcon]

Got a doc generation site auto-deploying: https://polis-community.github.io/red-dwarf/

[nicobao]

It's nice you wrote all this. It would be nice to prioritize now.
In general, I think priority should be about making a simple library, a la polislite (Eric's work). So, no UI, no internal state, no complex metadata / math_tick, etc.

Some notes:

properly account for meta-tids in PCA step

=> I think metadata are orthogonal to the clustering. At least for a first version.

figure out how often math ticks occur (number of votes? passing of time? every vote capped to minimum time?)

=> I don't think we should care, since it's an advanced feature that requires holding state. I don't think the lib function should hold any state. The user of the library can remember when they last updated the conversation polis data by using the lib.

What's interesting is incremental re-calculation for performance purpose, but this should come only later, and only if really needed, for performance reason. We can simply make a specific API for incremental recalculation, and it's up to the library user to store the state of the incremental results for each conversation.

I will open an issue to describe the functions API I have in mind for the basic functionalities of the library.