Subprocess debugging not supported #9886
Assignees
Labels
Milestone
Comments
|
Same trace when it fails, using a different Python kernel (from a different conda env): |
|
|
I created several conda environments in attempt to isolate the issue. My base environment currently has the following packages (output using |
|
Updating with one more thought I had: since it complained about the session already being started, I searched for all running processes with Python in their name (after closing Code), and killed them using |
|
Thanks for the issue. Your ipython and debugpy are rather out of date. Can you try creating a conda environment like so? As far as I can tell your two environments are identical so not sure how one could work but the other fails. But both IPython and debugpy have fixed issues in the last year related to debugging. If I try the conda environment listed above, I can continually debug a cell. |
rchiodo
added
bug
|
Thanks for the prompt reply, @rchiodo . It's odd that my versions are so out of date -- perhaps it's that I wasn't explicitly using the conda-forge channel? Back to the issue at hand. I managed to get the same behavior to replicate with my
I wondered if it might be an orphan process / port reuse issue -- after quitting Code, I still have a couple of debugpy process running in the background (found using I created an environment using the specification you provide, other than using python 3.9.12, since there's no official pytorch release for 3.10 as far as I know. I installed some other dependencies using pip, and let code install the dependencies for running notebooks using this environment. To test it, I:
Here's the output I get for Jupyer in code (in verbose mode). Could it be that something in my Code installation has gotten wonky, and that I should delete and reinstall? I can also try to disable most non-necessary extensions (the bisect didn't work because it disabled the Jupyter extension). |
|
That gives me some more questions to ask: Can you include the first 20 lines or so of your jupyter log? I want to see the version and whether or not ZMQ is working or not (should be the first thing we check). I think your error is coming from here: Which looks to me like we're attaching twice. Perhaps there's a timing issue in how we attach. Can you try this same thing with a simpler cell? Like maybe just a bunch of prints? Additionally are you running this directly on your machine or over remote SSH/WSL? |
|
To clarify, do you mean the first 20 lines after starting to debug this cell, or after restarting the notebook, or running the first cell in the notebook? I thought I had included the entire execution of the cell, but I might have missed the beginning. A timing issue would be consistent with one of the things I found at some point -- if I comment out enough of the cell, it doesn't happen. If I restart Code, run the previous cells, and then try to debug (with no breakpoints) a very minimal version of the cell (that terminates almost immediately), it succeeds. If I add a breakpoint, it fails. If I try to debug the same cell twice in a row (without adding breakpoints), it succeeds the first time and fails the second time. Running directly on my machine. |
|
First twenty lines of the log from when VS code opens. |
|
Looks like something goes awry: |
|
No that's expected on Mac M1. No ZMQ support. That's the message it generates. The new version of VS code is about to ship (like tomorrow), which may fix some things. If not we can try your original notebook here to see if we can repro in house. |
|
Thanks -- once I get the prompt to update, I will, and I'll report back what happens after. |
greazer
added
notebook-execution
|
I'm getting the same error "invalid message "session is already started" when attempting to debug cell number 5 in the lava notebook https://github.com/lava-nc/lava/blob/main/tutorials/in_depth/tutorial03_process_models.ipynb. I'm running on it on a Linux machine, with a conda env required to run lava, python version 3.10.4 ipykernel Version: 6.13.0 VsCode: The output when i try to debug the cell: Let me know if / which other logs can be useful to debug the problem. |
|
@waltergallegog the version of ipykernel, ipython, debugpy and python itself have an impact on debugging. Can you post those version numbers or try with the latest of all 4? |
|
Hi @rchiodo, here are the versions. I believe they are the latest available. I created the env just yesterday. |
|
Thanks, yeah those are the latest. Do you have a repro you can share too? I haven't gotten around to recreating @guydav's environment yet, was hoping something simpler would repro. |
|
The env I created was just to test the Lava framework tutorials. Then you can try to debug one of the notebooks in the |
17 tasks
15 tasks
|
Now the subprocess hangs because the I think you need to call Here's the code in ipykernel that starts debugging- it doesn't take arguments to configure debugpy- https://github.com/ipython/ipykernel/blob/aa53dcef1a282cba7dbd06d0715252ad10a9987c/ipykernel/debugger.py#L422 I also found this but I can't find which context it's used- https://github.com/microsoft/vscode-jupyter/blob/2023.02.100/src/interactive-window/debugger/interactiveWindowDebugger.node.ts#L47 |
16 tasks
|
Try to debug cell with multiprocess will make current notebook idle forever, really need a quick fix. |
|
I am also facing this issue. After trying to debug my code for several hours. I ended in this thread and realised the root cause is with the Jupyter debugging extension breaks when multiprocessing is involved. |
|
I get the same issue - is there a plan to get this fixed? This makes debugging something with e.g PyTorch data loaders impossible through VSCode Jupyter. Could we at least have a fix for the hanging debugger? |
|
Same issue... |
This is a pretty significant use case - and it is hard to find this issue as it doesn't include the common "couldn't find a debug adapter for debug type python kernel" message, which is a common stackoverflow question. |
|
This is so useful |
|
issue is still there |
|
I'm getting: Just by running import multiprocessing
def worker_fn(arg: list[int]) -> None:
pass
pool = multiprocessing.Pool(1).map(worker_fn, [[1, 2, 3], [4, 5, 6], [7, 8, 9], [10, 11, 12]])My debugger then becomes unresponsive for some time and as I hit disconnect & try to restart the kernel. PyTorch flavored: import torch.utils.data
dataloader = torch.utils.data.DataLoader(
dataset=[[1., 2., 3.], [4., 5., 6.], [7., 8., 9.], [10., 11., 12.]],
collate_fn=lambda batch: torch.tensor(batch),
batch_size=2,
num_workers=1,
)
next(iter(dataloader))Notably, these examples are not expected to raise errors. They just spawn child processes. |
|
I have the same issue with Tensorflow. |
|
@DonJayamanne any chance this is being investigated? After the July update of VS Code, I have not been able to use anymore debugging of Jupyter cells. |
|
I attempted the solution proposed in (microsoft/debugpy#1168 (comment), microsoft/debugpy#1166 (comment)) and it seems to work. |
UpdateMore details on on this issue can be found here microsoft/debugpy#1116 @ringohoffman @edervishaj This should work in the latest version of Jupyter, Python and Python Debubger extensions If this doesn't repro then I'm closing this issue, as for supporting debugging (i.e stepping into the sub process code is not yet supported and at the moment do not have any plans on supporting that feature in notebooks) |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
and others
Uh oh!
There was an error while loading. Please reload this page.
Does this issue occur when all extensions are disabled?: Yes/No (N/A, requires the Python and Jupyter extensions)
Steps to Reproduce:
I admit that I don't have any good intuition at this point if this is a bug with VS code, or if it's a coincidence with some wonkiness in my Python environments, or what exactly is happening, but I truly don't understand the behavior I'm observing.
My apologies for splitting the description over a few cells, but the stack traces are too long to fit in one post.
Below is a full trace of the Jupyter output in verbose, cleared before attempting to debug the relevant cell, when it succeeds:
The text was updated successfully, but these errors were encountered: