Co-occurrence matrix

[nealhaddaway]

We need to produce a co-occurrence matrix with sources as columns/rows and the number of shared records across source pairs.

Not sure if co-occurrence or frequency matrix is the right word - can't find anything that shows frequencies rather than correlations or p-values, but maybe because it's very basic...

But something that looks like this:

[LukasWallrich]

I wonder if the naniar::gg_miss_upset visualisation would be a better / alternative style for this? That would allow one to see which combinations of databases sources were found in - and I'd find it helpful to see the size of each set included in the chart?

[DrMattG]

Something like this? But only one side?
`

Sources as columns/rows and the number of shared records across source pairs.

library(tidyverse)
library(hrbrthemes)
library(plotly)

Dummy data

SourceX <- paste0("Source", seq(1,20))
SourceY <- paste0("Source", seq(1,20))
data <- expand.grid(SourceX=SourceX, SourceY=SourceY)
data$shared_records <- round(runif(400, 0, 25))
data=data %>%
mutate(shared_records=ifelse(data$SourceX==data$SourceY, NA, shared_records))

text for tooltip:

data <- data %>%
mutate(text = paste0("Shared_records: ",round(shared_records,2), "\n"))

ggplot

p <- ggplot(data, aes(SourceX, SourceY, fill= shared_records, text=text)) +
geom_tile() +
labs(x="", y="")+
theme_ipsum()+
theme(axis.text.x = element_text(angle=45))

plotly

ggplotly(p, tooltip="text")
`

[DrMattG]

I wonder if the naniar::gg_miss_upset visualisation would be a better / alternative style for this? That would allow one to see which combinations of databases sources were found in - and I'd find it helpful to see the size of each set included in the chart?

We could design a bespoke plot to include these elements (gg_miss will fail where there are no missing data... which might happen I guess?)

[LukasWallrich]

We could design a bespoke plot to include these elements (gg_miss will fail where there are no missing data... which might happen I guess?)

For sure - I meant to suggest showing the number of articles per source, rather than missing values in this plot, just based on this visualisation. Present/absent in a source is very similar to missing/not-missing in variable, so we might be able to derive some of the code from naniar - licence permitting ....

[LukasWallrich]

In fact, the UpSetR::upset function that the naniar vis is based on directly gives the desired result, without worrying about missing data. This visualisation is obviously only helpful when the number of sources is quite small - but that seems to be a realistic situation, maybe with a long tail grouped as "other"?

if (!require(pacman)) install.packages("pacman")
#> Loading required package: pacman
pacman::p_load(tidyverse, UpSetR)

data <- data.frame(
  article = 1:100,
  source1 = rbinom(100, 1, .5),
  source2 = rbinom(100, 1, .2),
  source3 = rbinom(100, 1, .1),
  source4 = rbinom(100, 1, .6),
  source5 = rbinom(100, 1, .7),
  source6 = rbinom(100, 1, .3)
)

data[-1] %>% upset(order.by = "freq")

^{Created on 2022-02-18 by the reprex package (v2.0.1)}

[TNRiley]

Here is an example with the databases being listed both on the x and y axis, which can show overlaping citations between databases. (this is a snapshot of only part of a sheet so the numbers do not add up)

[LukasWallrich]

Here is an example with the databases being listed both on the x and y axis, which can show overlapping citations between databases. (this is a snapshot of only part of a sheet so the numbers do not add up)

That's helpful - though I probably wouldn't show the information twice (i.e. above and below the diagonal). Something that might be interesting (though probably not essential enough for a standard visualisation) is the overlap, which would not be symmetrical but might help with prioritizing databases (i.e. if 90% of results of DB1 are in DB2, but only 10% of results of DB2 are in DB1, then it might be worth focusing on DB2 over DB1 in future work / updates?)