What are the exact semantics of DialPeer returning success ?

[hivenet-mathieu-lacage]

Hi,

I have two hosts A and B: A connects to B via a relay to request B to connect back to A via a direct connection (the network is configured so that A always has a public IP address so, I know this works all the time).

The stream handler on B essentially does this:

          // read request from A on stream
          ctx = network.WithForceDirectDial(ctx, "holepunch")
          ctx, cancel := context.WithTimeout(ctx, dialTimeout)
          _, err = server.host.Network().DialPeer(ctx, remotePeerID)
          // send response to B on stream

On A, the code essentially does this:

// open stream via relay to B
// write request to B on stream
// wait for response to come back from B on stream
// check if we now have a direct (inbound) connection to B ---> FAILS

So, what I was expecting on A, is that when I get the response, I have a reverse connection established by B that I can now use to send data to B.

Now, after a lot of testing, it looks like after I read the response from B on A, B thinks the connection back to A is established (DialPeer has returned success after all) but if I query host.Network().Conns(B) on A, there is no direct "inbound" connection in the list of connections for B. If I add an arbitrary sleep on A after I get the response from B over the relayed stream, I observe a direct connection from A to B 100% of the time.

i.e., it looks like as if the DialPeer to A on B returns before the connection is fully established between A and B. Is that expected ? If so, I am looking for hints on what might be triggering this behavior.

[hivenet-mathieu-lacage]

Hi,

After a bit of digging up, the answer is that yes, this is expected. More specifically, DialPeer returns once the connection has been secured and a muxer has been setup. On the remote side, once the muxer is setup on the connection, there is still quite a bit of work to do until a Conn appears on the host.Network().ConnsToPeer() so there is a relatively significant window of time during which the connection initiator has a valid Conn object returned by DialPeer and the connection receiver as an in-memory Conn object but this Conn object has not yet been fully added to the remote's data structures.

In my case, it was enough to add on B a single ping to A before sending the reply back to A to make all this work.