NuGet strategy consolidation

Changelog.md

@@ -1,5 +1,9 @@
 # FOSSA CLI Changelog
 
+
+## Unreleased
+- NuGet: Consolidate `project.assets.json` and `PackageReference` strategies ([#1461](https://github.com/fossas/fossa-cli/pull/1461))
+
 ## 3.9.47
 - Licensing: Adds support for Zeebe Community License v1.1 and Camunda License v1.0
 

docs/features/manual-dependencies.md

@@ -58,7 +58,7 @@ Supported dependency types:
 
 The following dependency types are also supported but they require `arch`, `os`, and `osVersion` attributes:
 
-- `apk` - Alpine packages. 
+- `apk` - Alpine packages.
 - `deb` - Debian packages.
 - `rpm-generic` - Rpm packages.
 
@@ -180,7 +180,7 @@ If `version` is not provided, the system assumes the version is "latest", and ca
 
 For dependency types that require `arch`, `os`, and `osVersion` attributes, these fields are additionally considered for the cache.
 
-In the event caching is causing problems, FOSSA can be made to rebuild this kind of dependency: 
+In the event caching is causing problems, FOSSA can be made to rebuild this kind of dependency:
 Click the dependency in the UI and then click "Reanalyze".
 This button enqueues a background job to rebuild the dependency, which should resolve after a few minutes.
 
@@ -192,6 +192,6 @@ This button enqueues a background job to rebuild the dependency, which should re
 
 `remote-dependencies` are cached by their `(name, url, version)` fields, which are all required.
 
-In the event caching is causing problems, FOSSA can be made to rebuild this kind of dependency: 
+In the event caching is causing problems, FOSSA can be made to rebuild this kind of dependency:
 Click the dependency in the UI and then click "Reanalyze".
 This button enqueues a background job to rebuild the dependency, which should resolve after a few minutes.

docs/references/strategies/languages/dotnet/README.md

@@ -1,12 +1,12 @@
 # .NET Analysis
 
-There are several different methods of .NET analysis, that use both the `NuGet` (`nuspec`, `PackageReference`, `packages.config`, `project.json`, `project.assets.json`) and `Paket` package managers.
+There are several different methods of .NET analysis, that use both the [NuGet](./nuget.md) (`nuspec`, `PackageReference`, `packages.config`, `project.json`, `project.assets.json`) and [Paket](./paket.md) package managers.
 
-| Strategy                                 | Direct Deps | Transitive Deps | Edges |
-| ---------------------------------------- | ----------- | --------- | ----- |
-| [nuspec](nuspec.md)                         | ✅          | ❌        | ❌    |
-| [PackageReference](packagereference.md)     | ✅          | ❌        | ❌    |
-| [packages.config](packagesconfig.md)        | ✅          | ❌        | ❌    |
-| [paket](paket.md)                           | ✅          | ✅        | ✅    |
-| [project.assets.json](projectassetsjson.md) | ✅          | ✅        | ✅    |
-| [project.json](projectjson.md)              | ✅          | ❌        | ❌    |
+| Strategy                                    | Direct Deps | Transitive Deps | Edges |
+|---------------------------------------------|-------------|-----------------|-------|
+| [paket](paket.md)                           | ✅           | ✅               | ✅     |
+| [project.assets.json](projectassetsjson.md) | ✅           | ✅               | ✅     |
+| [PackageReference](packagereference.md)     | ✅           | ❌               | ❌     |
+| [project.json](projectjson.md)              | ✅           | ❌               | ❌     |
+| [packages.config](packagesconfig.md)        | ✅           | ❌               | ❌     |
+| [nuspec](nuspec.md)                         | ✅           | ❌               | ❌     |

docs/references/strategies/languages/dotnet/nuget.md

@@ -1,17 +1,22 @@
-# Quick reference: NuGet
+# NuGet Analysis
 
-## Requirements
+| Strategy                                    | Direct Deps | Transitive Deps | Edges |
+|---------------------------------------------|-------------|-----------------|-------|
+| [project.assets.json](projectassetsjson.md) | ✅           | ✅               | ✅     |
+| [PackageReference](packagereference.md)     | ✅           | ❌               | ❌     |
+| [project.json](projectjson.md)              | ✅           | ❌               | ❌     |
+| [packages.config](packagesconfig.md)        | ✅           | ❌               | ❌     |
+| [nuspec](nuspec.md)                         | ✅           | ❌               | ❌     |
 
-**Ideal/Minimum**
+NuGet analysis follows these strategies in sequence:
+1. `project.assets.json`
+2. `PackageReference`
 
-One more more of the following:
+`project.assets.json` files and their dependencies are generated from  `.csproj` files. `PackageReference` dependencies can be found in `.csproj`, `.xproj`, `.vbproj`, `.dbproj`, or `.fsproj` files. To consolidate findings from these two strategies, `project.assets.json` analysis is attempted first and falls back to `PackageReference` analysis. 
 
-- [`.nuspec`](https://docs.microsoft.com/en-us/nuget/reference/nuspec) formatted file in your directory.
-- [Package reference](https://docs.microsoft.com/en-us/nuget/consume-packages/package-references-in-project-files) file present in your project. Commonly with an ending such as `.csproj`, `xproj`, `vbproj` and others.
-- `project.assets.json`
-- `packages.config`
-- `project.json`
+The following strategies are executed independently::
+1. `project.json`
+2. `packages.config`
+3. `nuspec`
 
-## Project discovery
-
-Directories containing any of the files listed above are considered NuGet projects.
+`project.json` and `packages.config` files are deprecated in favor of `.csproj` and their usage of `PackageReference` format. `nuspec` serves as a manifest containing package metadata, used both for building the package and providing information to consumers. These strategies are isolated from the `project.assets.json` and `PackageReference` approaches and therefore run independently.

docs/references/strategies/languages/dotnet/paket.md

@@ -1,5 +1,9 @@
 # Paket
 
+| Strategy          | Direct Deps | Transitive Deps | Edges |
+|-------------------|-------------|-----------------|-------|
+| [paket](paket.md) | ✅           | ✅               | ✅     |
+
 Paket is a dependency manager for .NET projects. Paket enables precise and predictable control over your dependencies
 
 Paket manages your dependencies with three core file types:

integration-test/Analysis/NugetSpec.hs

@@ -6,8 +6,11 @@ module Analysis.NugetSpec (spec) where
 import Analysis.FixtureExpectationUtils
 import Analysis.FixtureUtils
 import App.Types (Mode (NonStrict))
+import Data.Set (member)
+import Data.Set qualified as Set
+import Discovery.Walk (fileName)
 import Path
-import Strategy.NuGet.PackageReference qualified as PackageReference
+import Strategy.NuGet qualified as NuGet
 import Strategy.NuGet.PackagesConfig qualified as PackagesConfig
 import Test.Hspec
 import Types
@@ -24,9 +27,21 @@ serviceStack discoveryFunc =
       [reldir|nuget/ServiceStack/|]
       [reldir|servicestack-5.13.2//|]
 
+dotnetCoreTwoExample :: (Path Abs Dir -> TestC IO [DiscoveredProject a]) -> AnalysisTestFixture a
+dotnetCoreTwoExample discoveryFunc =
+  AnalysisTestFixture
+    "dotnet-core-2.0-example"
+    discoveryFunc
+    LocalEnvironment
+    Nothing
+    $ FixtureArtifact
+      "https://github.com/james-fossa/dotnet-core-2.0-example/archive/refs/tags/0.0.1.tar.gz"
+      [reldir|nuget/dotnet-core-2.0-example-0.0.1/|]
+      [reldir|dotnet-core-2.0-example-0.0.1//|]
+
 testServiceStackForPkgReferences :: Spec
 testServiceStackForPkgReferences =
-  aroundAll (withAnalysisOf NonStrict $ serviceStack PackageReference.discover) $ do
+  aroundAll (withAnalysisOf NonStrict $ serviceStack NuGet.discover) $ do
     describe "ServiceStack" $ do
       it "should find targets" $ \(result, _) -> do
         length result `shouldBe` 64
@@ -38,7 +53,20 @@ testServiceStackForPkgConfig =
       it "should find targets" $ \(result, _) -> do
         length result `shouldBe` 4
 
+testDotnetCoreTwoExampleForProjectAssetsJson :: Spec
+testDotnetCoreTwoExampleForProjectAssetsJson =
+  aroundAll (withAnalysisOf NonStrict $ dotnetCoreTwoExample NuGet.discover) $ do
+    describe "dotnet-core-2.0-example" $ do
+      it "should find targets" $ \(result, _) -> do
+        length result `shouldBe` 2
+        let projectDataPaths = Set.fromList $ map (fileName . NuGet.nugetProjectFile . Types.projectData . fst) result
+        let doesProjectAssetsJsonTargetExist = member "project.assets.json" projectDataPaths
+        let doesCsprojTargetExist = member "example.csproj" projectDataPaths
+        doesProjectAssetsJsonTargetExist `shouldBe` True
+        doesCsprojTargetExist `shouldBe` True
+
 spec :: Spec
 spec = do
   testServiceStackForPkgReferences
   testServiceStackForPkgConfig
+  testDotnetCoreTwoExampleForProjectAssetsJson

spectrometer.cabal

@@ -486,6 +486,7 @@ library
     Strategy.Node.YarnV2.Lockfile
     Strategy.Node.YarnV2.Resolvers
     Strategy.Node.YarnV2.YarnLock
+    Strategy.NuGet
     Strategy.NuGet.Nuspec
     Strategy.NuGet.PackageReference
     Strategy.NuGet.PackagesConfig

src/App/Fossa/Analyze/Discover.hs

@@ -27,11 +27,10 @@ import Strategy.Maven qualified as Maven
 import Strategy.Mix qualified as Mix
 import Strategy.Nim qualified as Nim
 import Strategy.Node qualified as Node
+import Strategy.NuGet qualified as NuGet
 import Strategy.NuGet.Nuspec qualified as Nuspec
-import Strategy.NuGet.PackageReference qualified as PackageReference
 import Strategy.NuGet.PackagesConfig qualified as PackagesConfig
 import Strategy.NuGet.Paket qualified as Paket
-import Strategy.NuGet.ProjectAssetsJson qualified as ProjectAssetsJson
 import Strategy.NuGet.ProjectJson qualified as ProjectJson
 import Strategy.Perl qualified as Perl
 import Strategy.Pub qualified as Pub
@@ -65,15 +64,14 @@ discoverFuncs =
   , DiscoverFunc Mix.discover
   , DiscoverFunc Nim.discover
   , DiscoverFunc Node.discover
+  , DiscoverFunc NuGet.discover
   , DiscoverFunc Nuspec.discover
-  , DiscoverFunc PackageReference.discover
   , DiscoverFunc PackagesConfig.discover
   , DiscoverFunc Paket.discover
   , DiscoverFunc Pdm.discover
   , DiscoverFunc Perl.discover
   , DiscoverFunc Pipenv.discover
   , DiscoverFunc Poetry.discover
-  , DiscoverFunc ProjectAssetsJson.discover
   , DiscoverFunc ProjectJson.discover
   , DiscoverFunc Pub.discover
   , DiscoverFunc R.discover

src/App/Fossa/Container/Sources/Discovery.hs

@@ -36,11 +36,10 @@ import Strategy.Maven qualified as Maven
 import Strategy.NDB qualified as NDB
 import Strategy.Nim qualified as Nim
 import Strategy.Node qualified as Node
+import Strategy.NuGet qualified as NuGet
 import Strategy.NuGet.Nuspec qualified as Nuspec
-import Strategy.NuGet.PackageReference qualified as PackageReference
 import Strategy.NuGet.PackagesConfig qualified as PackagesConfig
 import Strategy.NuGet.Paket qualified as Paket
-import Strategy.NuGet.ProjectAssetsJson qualified as ProjectAssetsJson
 import Strategy.NuGet.ProjectJson qualified as ProjectJson
 import Strategy.Perl qualified as Perl
 import Strategy.Pub qualified as Pub
@@ -89,15 +88,14 @@ managedDepsDiscoveryF =
   , DiscoverFunc Maven.discover
   , DiscoverFunc Nim.discover
   , DiscoverFunc Node.discover
+  , DiscoverFunc NuGet.discover
   , DiscoverFunc Nuspec.discover
-  , DiscoverFunc PackageReference.discover
   , DiscoverFunc PackagesConfig.discover
   , DiscoverFunc Paket.discover
   , DiscoverFunc Pdm.discover
   , DiscoverFunc Perl.discover
   , DiscoverFunc Pipenv.discover
   , DiscoverFunc Poetry.discover
-  , DiscoverFunc ProjectAssetsJson.discover
   , DiscoverFunc ProjectJson.discover
   , DiscoverFunc Pub.discover
   , DiscoverFunc R.discover

src/Strategy/NuGet.hs

@@ -0,0 +1,94 @@
+module Strategy.NuGet (
+  discover,
+  findProjects,
+  getDeps,
+  mkProject,
+  NuGetProject (..),
+) where
+
+import App.Fossa.Analyze.Types (AnalyzeProject (analyzeProjectStaticOnly), analyzeProject)
+import Control.Effect.Diagnostics (
+  Diagnostics,
+  Has,
+  context,
+ )
+import Control.Effect.Reader (Reader)
+import Data.Aeson (
+  ToJSON,
+ )
+import Data.Foldable (find)
+import Data.List qualified as L
+import Discovery.Filters (AllFilters)
+import Discovery.Simple (simpleDiscover)
+import Discovery.Walk (
+  WalkStep (WalkContinue),
+  fileName,
+  findFileNamed,
+  walkWithFilters',
+ )
+import Effect.ReadFS (ReadFS)
+import GHC.Generics (Generic)
+import Path (Abs, Dir, File, Path, parent)
+import Strategy.NuGet.PackageReference qualified as PackageReference
+import Strategy.NuGet.ProjectAssetsJson qualified as ProjectAssetsJson
+import Types (
+  DependencyResults (..),
+  DiscoveredProject (..),
+  DiscoveredProjectType (NuGetProjectType),
+ )
+
+discover ::
+  ( Has ReadFS sig m
+  , Has Diagnostics sig m
+  , Has (Reader AllFilters) sig m
+  ) =>
+  Path Abs Dir ->
+  m [DiscoveredProject NuGetProject]
+discover = simpleDiscover findProjects mkProject NuGetProjectType
+
+findProjects :: (Has ReadFS sig m, Has Diagnostics sig m, Has (Reader AllFilters) sig m) => Path Abs Dir -> m [NuGetProject]
+findProjects = walkWithFilters' $ \_ _ files -> do
+  case findProjectAssetsJsonFile files of
+    Just file -> pure ([NuGetProject file], WalkContinue)
+    Nothing -> case find isPackageRefFile files of
+      Just file -> pure ([NuGetProject file], WalkContinue)
+      Nothing -> pure ([], WalkContinue)
+  where
+    findProjectAssetsJsonFile :: [Path Abs File] -> Maybe (Path Abs File)
+    findProjectAssetsJsonFile = findFileNamed "project.assets.json"
+
+    isPackageRefFile :: Path b File -> Bool
+    isPackageRefFile file = any (\x -> x `L.isSuffixOf` fileName file) [".csproj", ".xproj", ".vbproj", ".dbproj", ".fsproj"]
+
+mkProject :: NuGetProject -> DiscoveredProject NuGetProject
+mkProject project =
+  DiscoveredProject
+    { projectType = NuGetProjectType
+    , projectPath = parent $ nugetProjectFile project
+    , projectBuildTargets = mempty
+    , projectData = project
+    }
+
+newtype NuGetProject = NuGetProject
+  { nugetProjectFile :: Path Abs File
+  }
+  deriving (Eq, Ord, Show, Generic)
+
+instance ToJSON NuGetProject
+
+instance AnalyzeProject NuGetProject where
+  analyzeProject _ = getDeps
+  analyzeProjectStaticOnly _ = getDeps
+
+getDeps :: (Has ReadFS sig m, Has Diagnostics sig m) => NuGetProject -> m DependencyResults
+getDeps project =
+  context "NuGet" $
+    if "project.assets.json" == (fileName . nugetProjectFile) project
+      then getAssetsJsonDeps project
+      else getPackageReferenceDeps project
+
+getAssetsJsonDeps :: (Has ReadFS sig m, Has Diagnostics sig m) => NuGetProject -> m DependencyResults
+getAssetsJsonDeps = context "ProjectAssetsJson" . context "Static analysis" . ProjectAssetsJson.analyze' . nugetProjectFile
+
+getPackageReferenceDeps :: (Has ReadFS sig m, Has Diagnostics sig m) => NuGetProject -> m DependencyResults
+getPackageReferenceDeps = context "PackageReference" . context "Static analysis" . PackageReference.analyze' . nugetProjectFile