Accuracy issues

[seefun]

I conducted inference on USPTO, but I couldn't obtain the metrics in the paper. Only about 64% accuracy can be obtained (91.5% in papar).

Can you provide MolGrapher's inference results (SMILES) from USPTO dataset for comparison？

[lucas-morin]

Hello @seefun,

To evaluate MolGrapher, did you use the evaluation script?
The method used to compute accuracy is defined here. The evaluation is done without stereo-chemistry, maybe that is why you have different scores? (By default, stereo-chemistry is removed from MolGrapher's predictions.)

To get MolGrapher's SMILES on USPTO, you can run:

python3 molgrapher/scripts/evaluate/evaluate_molecular_recognition.py

Before running it:

1. Edit the config file to:

{   
    "experiment_name": "exp-default", 
    "batch_size": 4,
    "nb_workers": 4,
    "num_processes_mp": 2,
    "num_threads_pytorch": 10,
    "image_size": [3, 1024, 1024],
    "nb_atoms_classes": 182,
    "nb_bonds_classes": 6,
    "nb_sample": 1000,
    "nb_duplicates": 1,
    "nb_sample_fine-tuning": 10000,
    "nb_sample_benchmarks": [10000],
    "training_dataset_fine-tuning": "uspto-10k",
    "training_dataset": "molgrapher-synthetic",
    "benchmarks": ["uspto"],
    "on_fly": false
}

2. Add your code to convert the predicted molecules to SMILES and save them. For example, you can do it here.