I have been trying to train some data using a model that utilizes src+img. When running the training script, I'm running into an error namely: RuntimeError: PytorchStreamReader failed locating file data.pkl: file not found
the .pkl file in this should have been the pickled .pt file.
First I figured that maybe I did not specify the preprocessed training.pt file correctly, but it is actually correctly specified.
data.pkl hasn't been dumped anywhere (or I failed to find it). I am guessing it has something do with pickle accordimg to docs:
"This save/load process uses the most intuitive syntax and involves the least amount of code. Saving a model in this way will save the entire module using Python’s pickle module. The disadvantage of this approach is that the serialized data is bound to the specific classes and the exact directory structure used when the model is saved. The reason for this is because pickle does not save the model class itself. Rather, it saves a path to the file containing the class, which is used during load time. Because of this, your code can break in various ways when used in other projects or after refactors."
I've tried a multitude of things, like changing data.pkl in the script in order to see if there was a generated corrupted file, but this seems to not be the case.
I would appreciate it if anyone is willing to help:
full error:
File "train_mm.py", line 448, in <module> main() File "train_mm.py", line 423, in main first_dataset = next(lazily_load_dataset("train")) File "train_mm.py", line 314, in lazily_load_dataset yield lazy_dataset_loader(pt, corpus_type) File "train_mm.py", line 305, in lazy_dataset_loader dataset = torch.load(pt_file) File "/usr/local/lib/python3.7/dist-packages/torch/serialization.py", line 607, in load return _load(opened_zipfile, map_location, pickle_module, **pickle_load_args) File "/usr/local/lib/python3.7/dist-packages/torch/serialization.py", line 878, in _load data_file = io.BytesIO(zip_file.get_record(pickle_file)) RuntimeError: PytorchStreamReader failed locating file data.pkl: file not found solved: I re-ran the preprocessing script and it generated new .pt files and the error was resolved
2 Answers
This issue has been resolved. the .pt file was heavily corrupted. After deleting the corrupt .pt file and re-running the preprocess script and consequently training script, I did not get the error anymore.
4This error can occur if you try to torch.load a model that has been saved after being traced, e.g.
# create model model = SomeModel() model.eval() # trace, for c++ or otherwise input_tensor = torch.rand(1, 2, 3) traced_script_module = torch.jit.trace(model, input_tensor) # save traced_script_module.save("path/to/model.pt") # load again weights = torch.load("path/to/model.pt") >>> RuntimeError: [enforce fail at inline_container.cc:208] . file not found: path/to/model/data.pkl In this case, you can torch.jit.load:
script_module = torch.jit.load("path/to/model.pt") This returns a ScriptModule object. See here: https://pytorch.org/docs/stable/generated/torch.jit.load.html
ncG1vNJzZmirpJawrLvVnqmfpJ%2Bse6S7zGiorp2jqbawutJobXJxZ259dICOq6ynrJmisqa%2B0aipZqipqbyzr8esq6udkaK%2Fpq3DnqlmnpGeuaawjKWmnJmknruoecWio55llJbBonnPpKNmnpmhsm66zq1kn6elo7E%3D