Skip to content

Conversation

@mclegrand
Copy link

Note: I did not time "qlever index" on the server I ran it on, would probably not give anything comparable to other repo files

@mclegrand mclegrand marked this pull request as ready for review October 24, 2025 19:46
@mclegrand
Copy link
Author

index timings on a Ryzen 9900X:

2025-10-24 22:11:19.505 - INFO: QLever IndexBuilder, compiled on Fri Oct 24 12:48:38 UTC 2025 using git hash d6cae9
2025-10-24 22:11:19.505 - INFO: You specified "locale = en_US" and "ignore-punctuation = 1"
2025-10-24 22:11:19.505 - INFO: You specified "num-triples-per-batch = 5,000,000", choose a lower value if the index builder runs out of memory
2025-10-24 22:11:19.505 - INFO: By default, integers that cannot be represented by QLever will throw an exception
2025-10-24 22:11:19.505 - WARN: Implicitly using the parallel parser for a single input file for reasons of backward compatibility; this is deprecated, please use the command-line option --parse-parallel or -p
2025-10-24 22:11:19.505 - INFO: Processing triples from single input stream /dev/stdin (parallel = true) ...
2025-10-24 22:11:19.506 - INFO: Parsing input triples and creating partial vocabularies, one per batch ...
2025-10-24 22:20:01.131 - INFO: Triples parsed: 1,784,222,285 [average speed 3.4 M/s, last batch 3.6 M/s, fastest 4.1 M/s, slowest 1.4 M/s] 
2025-10-24 22:20:01.896 - INFO: Number of triples created (including QLever-internal ones): 4,769,834,199 [may contain duplicates]
2025-10-24 22:20:01.898 - INFO: Number of partial vocabularies created: 353
2025-10-24 22:20:01.898 - INFO: Merging partial vocabularies ...
2025-10-24 22:24:21.984 - INFO: Triples converted: 4,769,834,199 [average speed 34.8 M/s, last batch 34.4 M/s, fastest 102.7 M/s, slowest 5.9 M/s] 
2025-10-24 22:24:25.111 - INFO: Creating permutations SPO and SOP ...
2025-10-24 22:26:04.289 - INFO: Number of inputs to `uniqueView`: 1,784,222,285.1 M/s, last batch 59.3 M/s, fastest 91.6 M/s, slowest 5.8 M/s] 
2025-10-24 22:26:04.291 - INFO: Number of unique elements: 1,305,431,408
2025-10-24 22:26:04.291 - INFO: Triples sorted: 1,305,431,408 [average speed 13.2 M/s, last batch 59.3 M/s, fastest 91.6 M/s, slowest 5.8 M/s] 
2025-10-24 22:26:04.989 - INFO: Statistics for SPO: #relations = 49,688,174, #blocks = 27,865, #triples = 1,305,431,408
2025-10-24 22:26:04.993 - INFO: Statistics for SOP: #relations = 49,688,174, #blocks = 27,865, #triples = 1,305,431,408
2025-10-24 22:26:05.036 - INFO: Number of distinct patterns: 145,037
2025-10-24 22:26:05.036 - INFO: Number of subjects with pattern: 49,688,174 [all]
2025-10-24 22:26:05.036 - INFO: Total number of distinct subject-predicate pairs: 334,158,519
2025-10-24 22:26:05.036 - INFO: Average number of predicates per subject: 6.7
2025-10-24 22:26:05.038 - INFO: Average number of subjects per predicate: 2,716,736
2025-10-24 22:26:06.299 - INFO: Creating permutations OSP and OPS ...
2025-10-24 22:27:58.678 - INFO: Triples sorted: 1,305,431,408 [average speed 11.6 M/s, last batch 9.4 M/s, fastest 24.5 M/s, slowest 5.3 M/s]  
2025-10-24 22:27:58.707 - INFO: Statistics for OSP: #relations = 645,362,722, #blocks = 34,971, #triples = 1,305,431,408
2025-10-24 22:27:58.707 - INFO: Statistics for OPS: #relations = 645,362,722, #blocks = 34,971, #triples = 1,305,431,408
2025-10-24 22:27:59.060 - INFO: Adding 49,688,174 triples to the POS and PSO permutation for the internal `ql:has-pattern` ...
2025-10-24 22:28:02.084 - INFO: Creating permutations PSO and POS ...
2025-10-24 22:31:15.521 - INFO: Number of inputs to `uniqueView`: 3,035,300,0881 M/s, last batch 20.1 M/s, fastest 66.1 M/s, slowest 0.4 M/s] 
2025-10-24 22:31:15.521 - INFO: Number of unique elements: 1,570,979,896
2025-10-24 22:31:15.521 - INFO: Triples sorted: 1,570,979,896 [average speed 8.1 M/s, last batch 20.1 M/s, fastest 66.1 M/s, slowest 0.4 M/s] 
2025-10-24 22:31:15.550 - INFO: Statistics for PSO: #relations = 1,528, #blocks = 50,710, #triples = 1,570,979,896
2025-10-24 22:31:15.550 - INFO: Statistics for POS: #relations = 1,528, #blocks = 50,710, #triples = 1,570,979,896
2025-10-24 22:31:15.674 - INFO: Creating permutations PSO and POS ...
2025-10-24 22:33:27.310 - INFO: Triples sorted: 1,305,431,408 [average speed 9.9 M/s, last batch 14.1 M/s, fastest 51.7 M/s, slowest 0.4 M/s]  
2025-10-24 22:33:27.338 - INFO: Statistics for PSO: #relations = 123, #blocks = 41,828, #triples = 1,305,431,408
2025-10-24 22:33:27.338 - INFO: Statistics for POS: #relations = 123, #blocks = 41,828, #triples = 1,305,431,408
2025-10-24 22:33:27.552 - INFO: Index build completed


real    22m23.893s
user    0m0.012s
sys     0m0.050s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant