Symbol Store and Source Link support outside Vs/Devkit

[zickb]

Hi, motivated by seblyng/roslyn.nvim#116, I developed a simple symbol store and source link client that implements the ISourceLinkService interface (repo: https://github.com/zickb/RoslynExtension/tree/main). And for simple cases (no authentication, only msdn and nuget symbol store server support, ...) it's working great 🎉.
But there is currently one big problem, in order to use the client in the dotnet lsp server I have to fork roslyn and change the source code in two places, and I would love it if this would not be necessary.

The first issue that requires a code change is that the ISourceLinkService is internal. It would be nice to have a public interface. For example, the devkit implementation (https://github.com/dotnet/roslyn/blob/main/src/VisualStudio/DevKit/Impl/SourceLink/VSCodeSourceLinkService.cs) uses the Microsoft.VisualStudio.Debugger.Contracts.SourceLink and Microsoft.VisualStudio.Debugger.Contracts.SymbolLocator packages as a shared interface between the debugger and rosyln. While it is possible to reference these packages, I have not been able to use them because the license is too restrictive.
The second problem requiring a code change is that I could not register the client with the --extension option because extensions are loaded in a separate ALC. I tried to (mis)use the --devKitDependencyPath option (the devkit dependencies are loaded in the ALC of the lsp server), but then I have to implement all the contracts expected by the devkit extension (e.g. hot reload specific services) and I have the license problem again. So I changed the ExtensionAssemblyManager to add my extension to the ALC of the lsp server.

Now my question:
Have I missed a solution that does not require any changes to roslyn?
If not, is there a reason why there is a workaround for the devkit extension but no general solution?
If not, what would be needed to move this forward?

Thanks.

[dibarbet]

The first issue that requires a code change is that the ISourceLinkService is internal.

I'm not sure we would make ISourceLinkService public in its current form (it's mainly a private contract between us and the VS debugger). Maybe more appropriate would be

roslyn/src/Features/Core/Portable/MetadataAsSource/IMetadataAsSourceFileProvider.cs

Line 14 in 3782760

internal interface IMetadataAsSourceFileProvider

If you want to be able to plug into go to definition, please file an issue so we can get broader discussion, though it is fairly unlikely we would take changes here soon.

The second problem requiring a code change is that I could not register the client with the --extension option because extensions are loaded in a separate ALC.

Can you clarify the problem here? Why does the separate ALC cause issues for you?

[zickb]

If you want to be able to plug into go to definition, please file an issue so we can get broader discussion, though it is fairly unlikely we would take changes here soon.

I will Do that.

Can you clarify the problem here? Why does the separate ALC cause issues for you?

The issue is that only assemblies in the main ALC are considered by MEF.

[jasonmalinowski]

They should be included in MEF too:

roslyn/src/LanguageServer/Microsoft.CodeAnalysis.LanguageServer/ExportProviderBuilder.cs

Line 49 in b5d7ffb

assemblyPaths = assemblyPaths.Concat(extensionManager.ExtensionAssemblyPaths);

[zickb]

Hmm, that's strange.
I changed for better testability [Import(AllowDefault = true)] ISourceLinkService? sourceLinkService to [Import(AllowDefault = false)] ISourceLinkService? sourceLinkService in

roslyn/src/Features/Core/Portable/PdbSourceDocument/PdbFileLocatorService.cs

Lines 18 to 23 in a4f7dfd

	[Export(typeof(IPdbFileLocatorService)), Shared]
	[method: ImportingConstructor]
	[SuppressMessage("RoslynDiagnosticsReliability", "RS0033:Importing constructor should be [Obsolete]", Justification = "Used in test code")]
	internal sealed class PdbFileLocatorService(
	[Import(AllowDefault = true)] ISourceLinkService? sourceLinkService,
	[Import(AllowDefault = true)] IPdbSourceDocumentLogger? logger) : IPdbFileLocatorService

When I start the Lsp server with my extension registered in the default ALC and everything is working.

When I started the Lsp server with my extension registered in separate ALC and I got the following error:

fail: Microsoft.CodeAnalysis.LanguageServer.ExportProviderBuilder[0]
      Encountered errors in the MEF composition:
      ----- CompositionError level 1 ------
      Microsoft.CodeAnalysis.ExternalAccess.Pythia.PythiaSignatureHelpProvider.ctor(implementation): expected exactly 1 export matching constraints:
          Contract name: Microsoft.CodeAnalysis.ExternalAccess.Pythia.Api.IPythiaSignatureHelpProviderImplementation
          TypeIdentityName: Microsoft.CodeAnalysis.ExternalAccess.Pythia.Api.IPythiaSignatureHelpProviderImplementation
      but found 0.
         part definition Microsoft.CodeAnalysis.ExternalAccess.Pythia.PythiaSignatureHelpProvider
      
      Microsoft.CodeAnalysis.PdbSourceDocument.PdbFileLocatorService.ctor(sourceLinkService): expected exactly 1 export matching constraints:
          Contract name: Microsoft.CodeAnalysis.PdbSourceDocument.ISourceLinkService
          TypeIdentityName: Microsoft.CodeAnalysis.PdbSourceDocument.ISourceLinkService
      but found 0.
         part definition Microsoft.CodeAnalysis.PdbSourceDocument.PdbFileLocatorService
      
      ----- CompositionError level 2 ------
      Microsoft.CodeAnalysis.PdbSourceDocument.PdbSourceDocumentMetadataAsSourceFileProvider.ctor(pdbFileLocatorService): expected exactly 1 export matching constraints:
          Contract name: Microsoft.CodeAnalysis.PdbSourceDocument.IPdbFileLocatorService
          TypeIdentityName: Microsoft.CodeAnalysis.PdbSourceDocument.IPdbFileLocatorService
       but the matching part has been invalidated.
         part definition Microsoft.CodeAnalysis.PdbSourceDocument.PdbSourceDocumentMetadataAsSourceFileProvider

I have also validated that my extension is included in the assembly paths (2nd line from the bottom):

Composing MEF catalog using:
<folder with roslyn repo>\artifacts\bin\Microsoft.CodeAnalysis.LanguageServer\Debug\net9.0\Microsoft.CodeAnalysis.AnalyzerUtilities.dll    
<folder with roslyn repo>\artifacts\bin\Microsoft.CodeAnalysis.LanguageServer\Debug\net9.0\Microsoft.CodeAnalysis.CSharp.dll    
<folder with roslyn repo>\artifacts\bin\Microsoft.CodeAnalysis.LanguageServer\Debug\net9.0\Microsoft.CodeAnalysis.CSharp.Features.dll    
<folder with roslyn repo>\artifacts\bin\Microsoft.CodeAnalysis.LanguageServer\Debug\net9.0\Microsoft.CodeAnalysis.CSharp.Workspaces.dll    
<folder with roslyn repo>\artifacts\bin\Microsoft.CodeAnalysis.LanguageServer\Debug\net9.0\Microsoft.CodeAnalysis.dll    
<folder with roslyn repo>\artifacts\bin\Microsoft.CodeAnalysis.LanguageServer\Debug\net9.0\Microsoft.CodeAnalysis.Elfie.dll    
<folder with roslyn repo>\artifacts\bin\Microsoft.CodeAnalysis.LanguageServer\Debug\net9.0\Microsoft.CodeAnalysis.ExternalAccess.AspNetCore.dll    
<folder with roslyn repo>\artifacts\bin\Microsoft.CodeAnalysis.LanguageServer\Debug\net9.0\Microsoft.CodeAnalysis.ExternalAccess.CompilerDeveloperSDK.dll    
<folder with roslyn repo>\artifacts\bin\Microsoft.CodeAnalysis.LanguageServer\Debug\net9.0\Microsoft.CodeAnalysis.ExternalAccess.Razor.Features.dll    
<folder with roslyn repo>\artifacts\bin\Microsoft.CodeAnalysis.LanguageServer\Debug\net9.0\Microsoft.CodeAnalysis.ExternalAccess.RazorCompiler.dll    
<folder with roslyn repo>\artifacts\bin\Microsoft.CodeAnalysis.LanguageServer\Debug\net9.0\Microsoft.CodeAnalysis.ExternalAccess.VisualDiagnostics.dll    
<folder with roslyn repo>\artifacts\bin\Microsoft.CodeAnalysis.LanguageServer\Debug\net9.0\Microsoft.CodeAnalysis.ExternalAccess.Xaml.dll    
<folder with roslyn repo>\artifacts\bin\Microsoft.CodeAnalysis.LanguageServer\Debug\net9.0\Microsoft.CodeAnalysis.Features.dll    
<folder with roslyn repo>\artifacts\bin\Microsoft.CodeAnalysis.LanguageServer\Debug\net9.0\Microsoft.CodeAnalysis.LanguageServer.dll    
<folder with roslyn repo>\artifacts\bin\Microsoft.CodeAnalysis.LanguageServer\Debug\net9.0\Microsoft.CodeAnalysis.LanguageServer.ExternalAccess.Copilot.dll    
<folder with roslyn repo>\artifacts\bin\Microsoft.CodeAnalysis.LanguageServer\Debug\net9.0\Microsoft.CodeAnalysis.LanguageServer.Prowithcol.dll    
<folder with roslyn repo>\artifacts\bin\Microsoft.CodeAnalysis.LanguageServer\Debug\net9.0\Microsoft.CodeAnalysis.Razor.Compiler.dll    
<folder with roslyn repo>\artifacts\bin\Microsoft.CodeAnalysis.LanguageServer\Debug\net9.0\Microsoft.CodeAnalysis.Remote.Workspaces.dll    
<folder with roslyn repo>\artifacts\bin\Microsoft.CodeAnalysis.LanguageServer\Debug\net9.0\Microsoft.CodeAnalysis.Scripting.dll    
<folder with roslyn repo>\artifacts\bin\Microsoft.CodeAnalysis.LanguageServer\Debug\net9.0\Microsoft.CodeAnalysis.Workspaces.dll    
<folder with roslyn repo>\artifacts\bin\Microsoft.CodeAnalysis.LanguageServer\Debug\net9.0\Microsoft.CodeAnalysis.Workspaces.MSBuild.dll    
<folder with roslyn repo>\artifacts\bin\Microsoft.CodeAnalysis.LanguageServer\Debug\net9.0\Microsoft.ServiceHub.Client.dll    
<folder with roslyn repo>\artifacts\bin\Microsoft.CodeAnalysis.LanguageServer\Debug\net9.0\Microsoft.ServiceHub.Framework.dll    
<folder with roslyn repo>\artifacts\bin\Microsoft.CodeAnalysis.LanguageServer\Debug\net9.0\Microsoft.ServiceHub.Resources.dll    
<folder with my extension>\RoslynExtension\RoslynExtension\bin\Debug\net9.0\Microsoft.CodeAnalysis.RoslynExtension.dll    
<folder with vscode extension>\.vscode\extensions\ms-dotnettools.csharp-2.72.22-win32-x64\.razorExtension\Microsoft.VisualStudioCode.RazorExtension.dll.

Therefor i thought that there is (general) problem with the MEF and separate ALC, but maybe there is a problem with my extension. Is there more detailed documentation that I can use to investigate the problem?

[jasonmalinowski]

There's no documentation -- you'd just have to debug this.

[zickb]

After some debugging I found the problem(s). I had 2 configuration failures.

Just for reference if someone else find this discussion:
The first problem was that I have'd used a different System.Composition version that causes the described error 🤦‍♂️
But then I got a runtime CompositionFailedException that was thrown because the same type in different ALC's are treated as two different types by the runtime (https://learn.microsoft.com/en-us/dotnet/core/dependency-loading/understanding-assemblyloadcontext#type-conversion-issues). To solve this problem I followed https://learn.microsoft.com/en-us/dotnet/core/tutorials/creating-app-with-plugin-support#simple-plugin-with-no-dependencies and finally the extension is properly loaded. :)