Deserialization from notebook#30
Conversation
|
Hello, thanks for this PR. Deserializing a notebook and restoring its kernel state is indeed a challenging task. Actually, I've tried several methods of restoring kernel state before, including the dill package you mentioned. I was satisfied with this package, but I haven't fully tested it. Additionally, using this package introduces the need for users to store two files (notebook and kernel state file) for recovering a conversion. Deserialization through code rerunning is a simple approach, but there are at least three problem:
As of now, all known methods have their limitations, and it might be necessary to find a suitable trade-off or discover a better approach. Given my current busy schedule, it may take me some time (maybe a week or longer) to thoroughly review and test the code. I will also carefully consider the implementation approach for kernel state deserialization. I'll reply as soon as I have completed this. |
edf9f7b to
252248d
Compare
4dfa445 to
75682bc
Compare
Hello, this is a follow up to my previous pull request.
The new code allows the user to pick up from a conversation saved into a notebook, or from any notebook even if it wasn't created by
Local-Code-Interpreter.The code cells get rerun to recreate the jupyter kernel state.
This behavior assumes that the output of the code is deterministic/reproducible which is usually the case for notebooks.
As of making this PR, if there happened to be a discrepancy between the old code output and the new one, the behavior is undefined.
Maybe we should ask the user what tho do when that happens, but it's not clear what options we would give him to remedy such discrepancy.
The code of
jupyter_backendandbot_backendmodules have been modified so that the working directory of the kernel is the same as the notebook this is to avoid output discrepancy due to file inputs.As of now, this branch is more of a place to discuss the implementation of jupyter kernel state deserialization than the actual final implementation.
The other possible solutions to this problem are:
This is effective but might add a layer of complexity though on every other feature.
It would also be more comber sum for sharing conversation compared to just sharing a notebook (and potentially some input files).
The main issues with this option is that dill cannot serialized every thing and that.
The serialized objects are kind of black boxes and might have compatibility issues between versions of
Local-Code-Interperter.Overall I believe that rerunning the code cell is the best option.
Let me know what you think, hopw this will be useful.