File not found error / examples

[SBlair2020]

Hello,

Thank you for sharing your great work on this project!

I am trying to infer specific structures on the HNMR or CNMR spectrum, and need to troubleshoot an 'error: file not found' message given for the command 'none' in either hnmr_shifts or cnmr_shifts. I receive this error message even when the other (hnmr or cnmr, accordingly) is filled in, despite the directions to enter 'none' for this command.

I would love to troubleshoot this and further issues by studying an example code. Will example files of the code for the inference section be shared?

Thank you again for sharing.

[FranklinHu1]

Hi there,

Could you provide the files/commands that you are using that lead to the error? I can definitely also clarify the README, but it would also be helpful to see a reproducible example.

Thanks!

[SBlair2020]

Hello,

Here is the command that I am using:

--config NMR2Struct/myconfigs/training_spectrum_to_struct.yaml
--hnmr_file NMR2Struct/myconfigs/HNMR_file.tsv
--cnmr_file None
--hnmr_shifts NMR2Struct/example_configs/HNMR_shifts.p
--cnmr_shifts NMR2Struct/example_configs/CNMR_shifts.p
--ckpt CKPT
--normalize

And receiving the following error: File "/Users/skylerblair/Documents/GitHub/NMR2Struct/NMR2Struct/nmr/data/dataset_base.py", line 184, in init
assert (hnmr_shifts is not None) and (cnmr_shifts is not None)
AssertionError
zsh: command not found: --cnmr_file

I am not sure why I am receiving the cnmr error since I should be able to set it to none.

I also tried setting both the cnmr file and the cnmr shifts to none (assuming the shifts is unnecessary information), and received the same error.

Thank you for your help!

[FranklinHu1]

I think one issue could be that you did not include the backslashes required to break the command into multiple lines. In the example code snippet in the README, there are backslashes after each line to ensure that the CLI arguments are not misinterpreted. Can you try the following:

nmr_infer_single_spectrum \
--config NMR2Struct/myconfigs/training_spectrum_to_struct.yaml \
--hnmr_file NMR2Struct/myconfigs/HNMR_file.tsv \
--cnmr_file None \
--hnmr_shifts NMR2Struct/example_configs/HNMR_shifts.p \
--cnmr_shifts NMR2Struct/example_configs/CNMR_shifts.p \
--ckpt CKPT \
--normalize

Let me know if that variant works.

Thanks!

[SBlair2020]

Thank you for your response! I tried your command in my terminal. I am getting a TypeError message now, and I think the issue might be with the HMNR_file.tsv. Is there an example of a proper tsv file that I can study?

[FranklinHu1]

Hi there,

Attached to this post is an example 1H NMR spectrum that came from MestReNova. This format will work, but in theory any format that has the two columns separated by spaces should be fine.

Let me know if you run into any issues!

m0000010-1H.txt

[SBlair2020]

Thank you for sharing the file! I entered it in my HNMR tsv file. Here is the error message I received:

  File "/opt/anaconda3/bin/nmr_infer_single_spectrum", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/Users/skylerblair/Documents/GitHub/NMR2Struct/NMR2Struct/nmr/scripts/run_nmr_inference_single_spectrum.py", line 64, in main
    model, _ = create_model(model_args, dtype, device)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/skylerblair/Documents/GitHub/NMR2Struct/NMR2Struct/nmr/models/build_model.py", line 17, in create_model
    model = model_base(dtype=dtype,
            ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/skylerblair/Documents/GitHub/NMR2Struct/NMR2Struct/nmr/models/multitask_model.py", line 26, in __init__
    self.structure_model = getattr(nmr.models, structure_model)(
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/skylerblair/Documents/GitHub/NMR2Struct/NMR2Struct/nmr/models/transformer_model.py", line 74, in __init__
    tgt_embed_layer = nn.Embedding(target_size, d_model, padding_idx = tgt_pad_token)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/anaconda3/lib/python3.12/site-packages/torch/nn/modules/sparse.py", line 143, in __init__
    self.weight = Parameter(torch.empty((num_embeddings, embedding_dim), **factory_kwargs),
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: empty() received an invalid combination of arguments - got (tuple, dtype=NoneType, device=NoneType), but expected one of:
 * (tuple of ints size, *, tuple of names names, torch.memory_format memory_format, torch.dtype dtype, torch.layout layout, torch.device device, bool pin_memory, bool requires_grad)
 * (tuple of ints size, *, torch.memory_format memory_format, Tensor out, torch.dtype dtype, torch.layout layout, torch.device device, bool pin_memory, bool requires_grad)```

[FranklinHu1]

Hi, and sorry for the late response!

If possible, could you share your spectra files, config files, and your run script with me so I can try to reproduce the error? It's a little difficult to guess what the problem might be, and it might come down to the formatting of your spectra.

Thanks!

[SBlair2020]

Hello,

No problem, thank you for your continued help!

Config file:

training_spectrum_to_struct.txt

Spectra file:

HNMR_file.txt

Command file:

Run.spectrum.txt

Thank you!

[FranklinHu1]

Thanks for the files! I was able to reproduce the error on my end.

It seems that the documentation for the single inference entry point needs to be improved because there were a lot of changes I had to make that I omitted in the README. I had originally intended the entry point to be used after training a model which would provide you with a yaml configuration file that has a lot of the fields already populated, thus removing the necessity for manually editing the yaml file. I did not account for the fact that it would be much more useful to be able to use the entry point with a provided pretrained model, e.g. the ones that are provided as checkpoints in this repository.

Here is an updated version of yaml configuration file, albeit with my paths. I have marked with #CHANGED all the fields I had to edit. I also simplified the file by removing unnecessary sections such as data and analysis:

global_args:
  ngpus: 1
  dtype: float32
  savedir: "checkpoints"
  seed: 42

model:
  model_type: "MultiTaskModel"
  load_model: null
  model_args:
    src_embed: ConvolutionalEmbedding
    src_embed_options:
      d_model: 128
      n_hnmr_features: 28000
      n_cnmr_features: 80
      use_hnmr: True
      use_cnmr: True
    forward_fxn: src_fwd_fxn_conv_embedding
    substructure_model: "EncoderModel"
    substructure_model_args:
      d_model: 128
      d_out: 957
      dim_feedforward: 1024
      nhead: 4
      num_layers: 4
      output_head: SingleLinear
      output_head_opts:
        d_model: 128
        d_out: 957
      pooler: SeqPool
      pooler_opts:
        d_model: 128
      source_size: 28045 #CHANGED
      src_embed: null
      src_embed_options: {}
      src_forward_function: src_fwd_fxn_packed_tensor
      src_pad_token: null
    structure_model: "TransformerModel"
    structure_model_args: 
      src_embed: null
      src_embed_options: {}
      tgt_embed: "nn.embed"
      tgt_embed_options: {}
      src_forward_function: "src_fwd_fxn_packed_tensor"
      tgt_forward_function: "tgt_fwd_fxn_basic"
      source_size: 28045 #CHANGED
      target_size: 24 #CHANGED
      d_model: 128
      dim_feedforward: 1024
      src_pad_token: null 
      tgt_pad_token: 21 #CHANGED
    substructure_model_ckpt: null
    structure_model_ckpt: /home/frankhu/NMR2Struct/checkpoints/transformer_checkpoint.ckpt  #CHANGED
  
inference:
  model_selection: lowest
  splits: 
    - null
  train_size: 0.8
  val_size: 0.1
  test_size: 0.1
  dloader_args:
    shuffle: false
    batch_size: 1024
  sets_to_run: ['test']
  run_inference_args: 
    pred_gen_fn: 'infer_transformer_model'
    pred_gen_opts:
      num_pred_per_tgt: 15
      sample_val: 5
      tgt_start_token: 22 #CHANGED
      tgt_stop_token: 23 #CHANGED
      track_gradients: true
      alphabet: /home/frankhu/NMR2Struct/example_configs/alphabet.npy #CHANGED
      decode: true
      infer_fwd_fxn: multitask
    write_freq: 100

The specific values for the tokens can be computed based on the alphabet and the table in the README. I will also provide in a PR the token values for all the pretrained models in the repository as well as an updated yaml file for the single inference. I also changed the run script to the following:

#!/bin/bash
nmr_infer_single_spectrum \
    --config data/training_spectrum_to_struct.yaml \
    --hnmr_file data/HNMR_file.txt \
    --cnmr_file  None\
    --hnmr_shifts /home/frankhu/NMR2Struct/example_configs/HNMR_shifts.p \
    --cnmr_shifts /home/frankhu/NMR2Struct/example_configs/CNMR_shifts.p \
    --ckpt /home/frankhu/NMR2Struct/checkpoints/multitask_checkpoint.pt \
    --normalize

This setup worked for me, and should work for you as well once you switch all the file paths over correctly. Your HNMR data is formatted fine, so that's not a problem.

Feel free to follow up with any additional problems, and thanks for using the code!

[nes0zx]

Thanks for the files! I was able to reproduce the error on my end.

It seems that the documentation for the single inference entry point needs to be improved because there were a lot of changes I had to make that I omitted in the README. I had originally intended the entry point to be used after training a model which would provide you with a yaml configuration file that has a lot of the fields already populated, thus removing the necessity for manually editing the yaml file. I did not account for the fact that it would be much more useful to be able to use the entry point with a provided pretrained model, e.g. the ones that are provided as checkpoints in this repository.

Here is an updated version of yaml configuration file, albeit with my paths. I have marked with #CHANGED all the fields I had to edit. I also simplified the file by removing unnecessary sections such as data and analysis:

global_args:
ngpus: 1
dtype: float32
savedir: "checkpoints"
seed: 42

model:
model_type: "MultiTaskModel"
load_model: null
model_args:
src_embed: ConvolutionalEmbedding
src_embed_options:
d_model: 128
n_hnmr_features: 28000
n_cnmr_features: 80
use_hnmr: True
use_cnmr: True
forward_fxn: src_fwd_fxn_conv_embedding
substructure_model: "EncoderModel"
substructure_model_args:
d_model: 128
d_out: 957
dim_feedforward: 1024
nhead: 4
num_layers: 4
output_head: SingleLinear
output_head_opts:
d_model: 128
d_out: 957
pooler: SeqPool
pooler_opts:
d_model: 128
source_size: 28045 #CHANGED
src_embed: null
src_embed_options: {}
src_forward_function: src_fwd_fxn_packed_tensor
src_pad_token: null
structure_model: "TransformerModel"
structure_model_args:
src_embed: null
src_embed_options: {}
tgt_embed: "nn.embed"
tgt_embed_options: {}
src_forward_function: "src_fwd_fxn_packed_tensor"
tgt_forward_function: "tgt_fwd_fxn_basic"
source_size: 28045 #CHANGED
target_size: 24 #CHANGED
d_model: 128
dim_feedforward: 1024
src_pad_token: null
tgt_pad_token: 21 #CHANGED
substructure_model_ckpt: null
structure_model_ckpt: /home/frankhu/NMR2Struct/checkpoints/transformer_checkpoint.ckpt #CHANGED

inference:
model_selection: lowest
splits:
- null
train_size: 0.8
val_size: 0.1
test_size: 0.1
dloader_args:
shuffle: false
batch_size: 1024
sets_to_run: ['test']
run_inference_args:
pred_gen_fn: 'infer_transformer_model'
pred_gen_opts:
num_pred_per_tgt: 15
sample_val: 5
tgt_start_token: 22 #CHANGED
tgt_stop_token: 23 #CHANGED
track_gradients: true
alphabet: /home/frankhu/NMR2Struct/example_configs/alphabet.npy #CHANGED
decode: true
infer_fwd_fxn: multitask
write_freq: 100
The specific values for the tokens can be computed based on the alphabet and the table in the README. I will also provide in a PR the token values for all the pretrained models in the repository as well as an updated yaml file for the single inference. I also changed the run script to the following:

#!/bin/bash
nmr_infer_single_spectrum
--config data/training_spectrum_to_struct.yaml
--hnmr_file data/HNMR_file.txt
--cnmr_file None
--hnmr_shifts /home/frankhu/NMR2Struct/example_configs/HNMR_shifts.p
--cnmr_shifts /home/frankhu/NMR2Struct/example_configs/CNMR_shifts.p
--ckpt /home/frankhu/NMR2Struct/checkpoints/multitask_checkpoint.pt
--normalize
This setup worked for me, and should work for you as well once you switch all the file paths over correctly. Your HNMR data is formatted fine, so that's not a problem.

Feel free to follow up with any additional problems, and thanks for using the code!

Thank you very much for your project. I have successfully run it according to the current configuration and generated prediction results used the m0000010-1H.txt file you provided above.

The predictions_dataset_0_0.h5 is as follows:

import h5py
with h5py.File("./results/predictions_dataset_0_0.h5", "r") as f:
    print(list(f.keys()))
    if "test" in f: 
        test_group = f["test"]
        print(list(test_group.keys()))
        print(test_group["scores"][0])
        print(test_group["predictions"][0])
        print(test_group["targets"])

['predictions', 'scores', 'targets']

scores:  [-15.4992895  -1.4100332  -8.034058   -4.4576693 -13.100047   -1.4100332
  -6.8848996  -1.4100332  -1.4100332 -13.376823   -1.4100332  -4.428711
  -1.4100332  -5.9647965  -7.728926 ]

predictions:  [b'C1=NN(C23C4C5C4C2C5)CC3CC14' b'C1=NN2CCC1CC2'
 b'C1=NN(C2=CC3C(C2)C3)CC1' b'C1=NN2CCCN1CC2' b'C1=NN2CCCN1CN1C21'
 b'C1=NN2CCC1CC2' b'C1=NN2CCC3C4C(C1)C4C32' b'C1=NN2CCC1CC2'
 b'C1=NN2CCC1CC2' b'c1n[nH]n1C1CCN(N=C=O)CC1' b'C1=NN2CCC1CC2'
 b'C=NN1CCC1' b'C1=NN2CCC1CC2' b'c1cn2c(c1)CCC2' b'C1=NN(C2N3CC2CC3)CC1']

targets:  <HDF5 dataset "targets": shape (1,), type "|O">

When I input the SMILES in the predictions into PubChem to search, I can't find the corresponding molecular structures. Does this mean that the prediction results are incorrect?

I have tried another experimental file, and the results are still not very accurate.
C4H6O2.txt
C4H6O2.jdx

result:
['predictions', 'scores', 'targets']

[-5.2720385 -2.2016027 -5.3624716 -2.2016027 -1.3554496 -1.3554496
-4.991391 -1.3554496 -5.005814 -2.9482913 -1.3554496 -1.3554496
-1.3554496 -3.1533718 -6.1997194]

[b'C=CNCO' b'C=CC=NC' b'C=CCO' b'C=CC=NC' b'C=CN1CC1' b'C=CN1CC1'
b'C=CC(=C)CO' b'C=CN1CC1' b'C=CN(C)CO' b'C=CN(C)C=CC=C' b'C=CN1CC1'
b'C=CN1CC1' b'C=CN1CC1' b'C=CNC' b'C(C=CC=CN(C)CO)=CO']
<HDF5 dataset "targets": shape (1,), type "|O">

I hope to get your help. Thank you very much. ❤️

[FranklinHu1]

Hi there nes0zx,

Thanks for using the code! With regards to your issues on not finding the SMILES on PubChem, make sure that you first process the SMILES by checking they are valid SMILES strings. You can do this using the RDkit package as follows:

from rdkit import Chem
predictions = test_group["predictions"][0]
predictions = [x.decode('utf-8') for x in predictions]
valid_smiles = []
for smi in predictions:
    try:
        mol = Chem.MolFromSmiles(smi)
        assert mol is not None
        valid_smiles.append(smi)
    except:
        pass 

You can then check if the valid SMILES, which correspond to valid molecular graphs, appear in your PubChem search.

For the accuracy on the experimental spectra, note that this software was designed as a proof-of-concept and was trained and tested on simulated HNMR spectra. Some steps you can take to improve the performance on experimental spectra are to remove any impurity peaks and to do a baseline correction to try and remove as much noise as possible. Even then, note that accuracy on experimental spectra is not guaranteed.

Hope that answers your questions, and feel free to follow up with any additional thoughts.

Thanks!

[SBlair2020]

Thank you so much for the changes! I am past that initial error message now. I believe I redirected all the files correctly. Now, however, I wanted to ask about the following error message:

''Could not load network.src_embed.weight because it is not in the model state dictionary''

And also ''Missing key(s) in state_dict: "src_embed.c_embed.weight", "src_embed.post_conv_transform.weight", "src_embed.post_conv_transform.bias"'''.

It seems to have begun loading the transformer. I believe everything is downloaded correctly, but it seems the code cannot find these few details. Please let me know if it would be helpful to attach more of this feedbsck. Thank you!

[nes0zx]

Hi there nes0zx,

Thanks for using the code! With regards to your issues on not finding the SMILES on PubChem, make sure that you first process the SMILES by checking they are valid SMILES strings. You can do this using the RDkit package as follows:

from rdkit import Chem
predictions = test_group["predictions"][0]
predictions = [x.decode('utf-8') for x in predictions]
valid_smiles = []
for smi in predictions:
try:
mol = Chem.MolFromSmiles(smi)
assert mol is not None
valid_smiles.append(smi)
except:
pass
You can then check if the valid SMILES, which correspond to valid molecular graphs, appear in your PubChem search.

For the accuracy on the experimental spectra, note that this software was designed as a proof-of-concept and was trained and tested on simulated HNMR spectra. Some steps you can take to improve the performance on experimental spectra are to remove any impurity peaks and to do a baseline correction to try and remove as much noise as possible. Even then, note that accuracy on experimental spectra is not guaranteed.

Hope that answers your questions, and feel free to follow up with any additional thoughts.

Thanks!

Okay, thank you very much for your reply.