Sync with 2.43.4

* maint-2.43: (40 commits)
  Git 2.43.4
  Git 2.42.2
  Git 2.41.1
  Git 2.40.2
  Git 2.39.4
  fsck: warn about symlink pointing inside a gitdir
  core.hooksPath: add some protection while cloning
  init.templateDir: consider this config setting protected
  clone: prevent hooks from running during a clone
  Add a helper function to compare file contents
  init: refactor the template directory discovery into its own function
  find_hook(): refactor the `STRIP_EXTENSION` logic
  clone: when symbolic links collide with directories, keep the latter
  entry: report more colliding paths
  t5510: verify that D/F confusion cannot lead to an RCE
  submodule: require the submodule path to contain directories only
  clone_submodule: avoid using `access()` on directories
  submodules: submodule paths must not contain symlinks
  clone: prevent clashing git dirs when cloning submodule in parallel
  t7423: add tests for symlinked submodule directories
  ...

Author: dscho

Documentation/RelNotes/2.39.4.txt

@@ -0,0 +1,79 @@
+Git v2.39.4 Release Notes
+=========================
+
+This addresses the security issues CVE-2024-32002, CVE-2024-32004,
+CVE-2024-32020 and CVE-2024-32021.
+
+This release also backports fixes necessary to let the CI builds pass
+successfully.
+
+Fixes since v2.39.3
+-------------------
+
+ * CVE-2024-32002:
+
+   Recursive clones on case-insensitive filesystems that support symbolic
+   links are susceptible to case confusion that can be exploited to
+   execute just-cloned code during the clone operation.
+
+ * CVE-2024-32004:
+
+   Repositories can be configured to execute arbitrary code during local
+   clones. To address this, the ownership checks introduced in v2.30.3
+   are now extended to cover cloning local repositories.
+
+ * CVE-2024-32020:
+
+   Local clones may end up hardlinking files into the target repository's
+   object database when source and target repository reside on the same
+   disk. If the source repository is owned by a different user, then
+   those hardlinked files may be rewritten at any point in time by the
+   untrusted user.
+
+ * CVE-2024-32021:
+
+   When cloning a local source repository that contains symlinks via the
+   filesystem, Git may create hardlinks to arbitrary user-readable files
+   on the same filesystem as the target repository in the objects/
+   directory.
+
+ * CVE-2024-32465:
+
+   It is supposed to be safe to clone untrusted repositories, even those
+   unpacked from zip archives or tarballs originating from untrusted
+   sources, but Git can be tricked to run arbitrary code as part of the
+   clone.
+
+ * Defense-in-depth: submodule: require the submodule path to contain
+   directories only.
+
+ * Defense-in-depth: clone: when symbolic links collide with directories, keep
+   the latter.
+
+ * Defense-in-depth: clone: prevent hooks from running during a clone.
+
+ * Defense-in-depth: core.hooksPath: add some protection while cloning.
+
+ * Defense-in-depth: fsck: warn about symlink pointing inside a gitdir.
+
+ * Various fix-ups on HTTP tests.
+
+ * Test update.
+
+ * HTTP Header redaction code has been adjusted for a newer version of
+   cURL library that shows its traces differently from earlier
+   versions.
+
+ * Fix was added to work around a regression in libcURL 8.7.0 (which has
+   already been fixed in their tip of the tree).
+
+ * Replace macos-12 used at GitHub CI with macos-13.
+
+ * ci(linux-asan/linux-ubsan): let's save some time
+
+ * Tests with LSan from time to time seem to emit harmless message that makes
+   our tests unnecessarily flakey; we work it around by filtering the
+   uninteresting output.
+
+ * Update GitHub Actions jobs to avoid warnings against using deprecated
+   version of Node.js.

Documentation/RelNotes/2.40.2.txt

@@ -0,0 +1,7 @@
+Git v2.40.2 Release Notes
+=========================
+
+This release merges up the fix that appears in v2.39.4 to address
+the security issues CVE-2024-32002, CVE-2024-32004, CVE-2024-32020,
+CVE-2024-32021 and CVE-2024-32465; see the release notes for that
+version for details.

Documentation/RelNotes/2.41.1.txt

@@ -0,0 +1,7 @@
+Git v2.41.1 Release Notes
+=========================
+
+This release merges up the fix that appears in v2.39.4 and v2.40.2
+to address the security issues CVE-2024-32002, CVE-2024-32004,
+CVE-2024-32020, CVE-2024-32021 and CVE-2024-32465; see the release
+notes for these versions for details.

Documentation/RelNotes/2.42.2.txt

@@ -0,0 +1,7 @@
+Git v2.42.2 Release Notes
+=========================
+
+This release merges up the fix that appears in v2.39.4, v2.40.2
+and v2.41.1 to address the security issues CVE-2024-32002,
+CVE-2024-32004, CVE-2024-32020, CVE-2024-32021 and CVE-2024-32465;
+see the release notes for these versions for details.

Documentation/RelNotes/2.43.4.txt

@@ -0,0 +1,7 @@
+Git v2.43.4 Release Notes
+=========================
+
+This release merges up the fix that appears in v2.39.4, v2.40.2,
+v2.41.1 and v2.42.2 to address the security issues CVE-2024-32002,
+CVE-2024-32004, CVE-2024-32020, CVE-2024-32021 and CVE-2024-32465;
+see the release notes for these versions for details.

Documentation/fsck-msgids.txt

@@ -164,6 +164,18 @@
 `nullSha1`::
 	(WARN) Tree contains entries pointing to a null sha1.
 
+`symlinkPointsToGitDir`::
+	(WARN) Symbolic link points inside a gitdir.
+
+`symlinkTargetBlob`::
+	(ERROR) A non-blob found instead of a symbolic link's target.
+
+`symlinkTargetLength`::
+	(WARN) Symbolic link target longer than maximum path length.
+
+`symlinkTargetMissing`::
+	(ERROR) Unable to read symbolic link target's blob.
+
 `treeNotSorted`::
 	(ERROR) A tree is not properly sorted.
 

Documentation/git-upload-pack.txt

@@ -55,6 +55,37 @@ ENVIRONMENT
 	admins may need to configure some transports to allow this
 	variable to be passed. See the discussion in linkgit:git[1].
 
+`GIT_NO_LAZY_FETCH`::
+	When cloning or fetching from a partial repository (i.e., one
+	itself cloned with `--filter`), the server-side `upload-pack`
+	may need to fetch extra objects from its upstream in order to
+	complete the request. By default, `upload-pack` will refuse to
+	perform such a lazy fetch, because `git fetch` may run arbitrary
+	commands specified in configuration and hooks of the source
+	repository (and `upload-pack` tries to be safe to run even in
+	untrusted `.git` directories).
++
+This is implemented by having `upload-pack` internally set the
+`GIT_NO_LAZY_FETCH` variable to `1`. If you want to override it
+(because you are fetching from a partial clone, and you are sure
+you trust it), you can explicitly set `GIT_NO_LAZY_FETCH` to
+`0`.
+
+SECURITY
+--------
+
+Most Git commands should not be run in an untrusted `.git` directory
+(see the section `SECURITY` in linkgit:git[1]). `upload-pack` tries to
+avoid any dangerous configuration options or hooks from the repository
+it's serving, making it safe to clone an untrusted directory and run
+commands on the resulting clone.
+
+For an extra level of safety, you may be able to run `upload-pack` as an
+alternate user. The details will be platform dependent, but on many
+systems you can run:
+
+    git clone --no-local --upload-pack='sudo -u nobody git-upload-pack' ...
+
 SEE ALSO
 --------
 linkgit:gitnamespaces[7]

Documentation/git.txt

@@ -1049,6 +1049,37 @@ The index is also capable of storing multiple entries (called "stages")
 for a given pathname.  These stages are used to hold the various
 unmerged version of a file when a merge is in progress.
 
+SECURITY
+--------
+
+Some configuration options and hook files may cause Git to run arbitrary
+shell commands. Because configuration and hooks are not copied using
+`git clone`, it is generally safe to clone remote repositories with
+untrusted content, inspect them with `git log`, and so on.
+
+However, it is not safe to run Git commands in a `.git` directory (or
+the working tree that surrounds it) when that `.git` directory itself
+comes from an untrusted source. The commands in its config and hooks
+are executed in the usual way.
+
+By default, Git will refuse to run when the repository is owned by
+someone other than the user running the command. See the entry for
+`safe.directory` in linkgit:git-config[1]. While this can help protect
+you in a multi-user environment, note that you can also acquire
+untrusted repositories that are owned by you (for example, if you
+extract a zip file or tarball from an untrusted source). In such cases,
+you'd need to "sanitize" the untrusted repository first.
+
+If you have an untrusted `.git` directory, you should first clone it
+with `git clone --no-local` to obtain a clean copy. Git does restrict
+the set of options and hooks that will be run by `upload-pack`, which
+handles the server side of a clone or fetch, but beware that the
+surface area for attack against `upload-pack` is large, so this does
+carry some risk. The safest thing is to serve the repository as an
+unprivileged user (either via linkgit:git-daemon[1], ssh, or using
+other tools to change user ids). See the discussion in the `SECURITY`
+section of linkgit:git-upload-pack[1].
+
 FURTHER DOCUMENTATION
 ---------------------
 

INSTALL

@@ -139,7 +139,7 @@ Issues of note:
 	  not need that functionality, use NO_CURL to build without
 	  it.
 
-	  Git requires version "7.19.5" or later of "libcurl" to build
+	  Git requires version "7.21.3" or later of "libcurl" to build
 	  without NO_CURL. This version requirement may be bumped in
 	  the future.
 

builtin/clone.c

@@ -329,7 +329,20 @@ static void copy_or_link_directory(struct strbuf *src, struct strbuf *dest,
 	int src_len, dest_len;
 	struct dir_iterator *iter;
 	int iter_status;
-	struct strbuf realpath = STRBUF_INIT;
+
+	/*
+	 * Refuse copying directories by default which aren't owned by us. The
+	 * code that performs either the copying or hardlinking is not prepared
+	 * to handle various edge cases where an adversary may for example
+	 * racily swap out files for symlinks. This can cause us to
+	 * inadvertently use the wrong source file.
+	 *
+	 * Furthermore, even if we were prepared to handle such races safely,
+	 * creating hardlinks across user boundaries is an inherently unsafe
+	 * operation as the hardlinked files can be rewritten at will by the
+	 * potentially-untrusted user. We thus refuse to do so by default.
+	 */
+	die_upon_dubious_ownership(NULL, NULL, src_repo);
 
 	mkdir_if_missing(dest->buf, 0777);
 
@@ -377,9 +390,27 @@ static void copy_or_link_directory(struct strbuf *src, struct strbuf *dest,
 		if (unlink(dest->buf) && errno != ENOENT)
 			die_errno(_("failed to unlink '%s'"), dest->buf);
 		if (!option_no_hardlinks) {
-			strbuf_realpath(&realpath, src->buf, 1);
-			if (!link(realpath.buf, dest->buf))
+			if (!link(src->buf, dest->buf)) {
+				struct stat st;
+
+				/*
+				 * Sanity-check whether the created hardlink
+				 * actually links to the expected file now. This
+				 * catches time-of-check-time-of-use bugs in
+				 * case the source file was meanwhile swapped.
+				 */
+				if (lstat(dest->buf, &st))
+					die(_("hardlink cannot be checked at '%s'"), dest->buf);
+				if (st.st_mode != iter->st.st_mode ||
+				    st.st_ino != iter->st.st_ino ||
+				    st.st_dev != iter->st.st_dev ||
+				    st.st_size != iter->st.st_size ||
+				    st.st_uid != iter->st.st_uid ||
+				    st.st_gid != iter->st.st_gid)
+					die(_("hardlink different from source at '%s'"), dest->buf);
+
 				continue;
+			}
 			if (option_local > 0)
 				die_errno(_("failed to create link '%s'"), dest->buf);
 			option_no_hardlinks = 1;
@@ -392,8 +423,6 @@ static void copy_or_link_directory(struct strbuf *src, struct strbuf *dest,
 		strbuf_setlen(src, src_len);
 		die(_("failed to iterate over '%s'"), src->buf);
 	}
-
-	strbuf_release(&realpath);
 }
 
 static void clone_local(const char *src_repo, const char *dest_repo)
@@ -936,6 +965,8 @@ int cmd_clone(int argc, const char **argv, const char *prefix)
 	int hash_algo;
 	unsigned int ref_storage_format = REF_STORAGE_FORMAT_UNKNOWN;
 	const int do_not_override_repo_unix_permissions = -1;
+	const char *template_dir;
+	char *template_dir_dup = NULL;
 
 	struct transport_ls_refs_options transport_ls_refs_options =
 		TRANSPORT_LS_REFS_OPTIONS_INIT;
@@ -955,6 +986,13 @@ int cmd_clone(int argc, const char **argv, const char *prefix)
 		usage_msg_opt(_("You must specify a repository to clone."),
 			builtin_clone_usage, builtin_clone_options);
 
+	xsetenv("GIT_CLONE_PROTECTION_ACTIVE", "true", 0 /* allow user override */);
+	template_dir = get_template_dir(option_template);
+	if (*template_dir && !is_absolute_path(template_dir))
+		template_dir = template_dir_dup =
+			absolute_pathdup(template_dir);
+	xsetenv("GIT_CLONE_TEMPLATE_DIR", template_dir, 1);
+
 	if (option_depth || option_since || option_not.nr)
 		deepen = 1;
 	if (option_single_branch == -1)
@@ -1116,7 +1154,7 @@ int cmd_clone(int argc, const char **argv, const char *prefix)
 	 * repository, and reference backends may persist that information into
 	 * their on-disk data structures.
 	 */
-	init_db(git_dir, real_git_dir, option_template, GIT_HASH_UNKNOWN,
+	init_db(git_dir, real_git_dir, template_dir, GIT_HASH_UNKNOWN,
 		ref_storage_format, NULL,
 		do_not_override_repo_unix_permissions, INIT_DB_QUIET | INIT_DB_SKIP_REFDB);
 
@@ -1460,6 +1498,7 @@ int cmd_clone(int argc, const char **argv, const char *prefix)
 	free(dir);
 	free(path);
 	free(repo_to_free);
+	free(template_dir_dup);
 	junk_mode = JUNK_LEAVE_ALL;
 
 	transport_ls_refs_options_release(&transport_ls_refs_options);