Feb merge 25 (#134)

* [jak1] Fix wind (#3837)

I accidentally broke this with the camera matrix improvements, but it's
fixed now.


![image](https://github.com/user-attachments/assets/144ac770-d7bf-414a-8814-062bbe51c39e)

---------

Co-authored-by: water111 <awaterford1111445@gmail.com>

* [jak3] Improve overlord file size handling (#3828)

Try to avoid crashes/corruption if a file changes size after you've
started the game.

Co-authored-by: water111 <awaterford1111445@gmail.com>

* Fix camera-other (#3843)

There's a feature to draw levels with a different camera matrix. This
uses `camera-temp-other`, `camera-rot-other`, etc instead of
`camera-temp`, `camera-rot`... I assumed that `trans-other` was the
"other" version of `trans`, but it turns out this isn't true -
`trans-other` is combined with `quat-other` and the original camera
matrix.

This fixes the issue by using the translation part of
`inv-camera-rot-other` for othercam levels only. (it works in all cases,
but I didn't want to use it for normal levels since I think it will be
less accurate)

Co-authored-by: water111 <awaterford1111445@gmail.com>

* [decompiler] Cleanup for Jak 3 (#3845)

This PR does a few cleanups:
- improve method names/comments/flags for `enemy.gc` and a few other
files
- fix `new-stack-matrix0` not working for jak 3
- add `matrix-copy!` detection for jak 3
- add `vector-copy!` detection

---------

Co-authored-by: water111 <awaterford1111445@gmail.com>

* [decompiler] Detect vector*! (#3846)

Detect use of `vector*!`, which is inlined in jak 2 and jak 3.

---------

Co-authored-by: water111 <awaterford1111445@gmail.com>

* [decompiler] Update vector ops, reduce casts (#3849)

Update the decompiler to use the new vf macros.

Also, fix a bunch of silly casting issues where accessing inline fields
with an offset of 0 would be better than a cast:

![image](https://github.com/user-attachments/assets/885bbb07-634f-47b8-99f5-5a947941cdde)

---------

Co-authored-by: water111 <awaterford1111445@gmail.com>

* CI: Periodic Controller Database Update (#3847)

Updating Controller Database

Co-authored-by: OpenGOALBot <OpenGOALBot@users.noreply.github.com>

* custom levels: etie and build actor support for jak2/3 (#3851)

Custom levels for Jak 2/3 now support envmapped TIE geometry. The TIE
extract was also changed to ignore materials that have the specular flag
set, but are missing a roughness texture.

Jak 2/3 now also support the `build-actor` tool.

The `build-custom-level` and `build-actor` macros now have a few new
options:

- Both now have a `force-run` option (`#f` by default) that, when set to
`#t`, will always run level/art group generation even if the output
files are up to date.
- `build-custom-level` has a `gen-fr3` option (`#t` by default) that,
when set to `#f`, will skip generating the FR3 file for the custom level
and only generate the GOAL level file to skip the potentially slow
process of finding and adding art groups and textures. Useful for when
you want to temporarily edit only the GOAL side of the level (such as
entity placement, etc.).
- `build-actor` has a `texture-bucket` option (default 0) which will
determine what DMA sink group the model will be placed in, which is
useful to determine the draw order of the model. Previously, this was
omitted, resulting in shadows not drawing over custom actors because the
actors were put in a bucket that is drawn after shadows (this behavior
can be restored with `:texture-bucket #f`).

---------

Co-authored-by: water111 <48171810+water111@users.noreply.github.com>
Co-authored-by: water111 <awaterford1111445@gmail.com>
Co-authored-by: OpenGOAL Bot <99294829+OpenGOALBot@users.noreply.github.com>
Co-authored-by: OpenGOALBot <OpenGOALBot@users.noreply.github.com>
Co-authored-by: Hat Kid <6624576+Hat-Kid@users.noreply.github.com>

common/util/gltf_util.cpp

@@ -685,14 +685,15 @@ bool material_has_envmap(const tinygltf::Material& mat) {
 }
 
 bool envmap_is_valid(const tinygltf::Material& mat) {
-  if (material_has_envmap(mat) && mat.pbrMetallicRoughness.metallicRoughnessTexture.index < 0) {
+  auto envmap = material_has_envmap(mat);
+  if (envmap && mat.pbrMetallicRoughness.metallicRoughnessTexture.index < 0) {
     lg::warn(fmt::format(
         "Material \"{}\" has specular property set, but is missing a metallic roughness texture, "
         "ignoring envmap!",
         mat.name));
     return false;
   }
-  return true;
+  return envmap;
 }
 
 std::optional<int> find_single_skin(const tinygltf::Model& model,

custom_assets/jak2/levels/test-zone/test-zone.jsonc

@@ -56,6 +56,10 @@
   // Removed so that the release builds don't have to double-decompile the game
   // "art_groups": ["prsn-torture-ag"],
 
+  // If you have any custom models in the "custom_assets/jak2/models/custom_levels" folder that you want to use in your level, add them to this list.
+  // Note: Like with art groups, these should also be added to your level's .gd file.
+  "custom_models": ["test-actor"],
+
   // Any textures you want to include in your custom level.
   // This is mainly useful for textures which are not in the common level files and have no art group associated with them.
   // To get a list of all the textures, you can extract all of the game's textures
@@ -99,6 +103,17 @@
       "lump": {
         "name": "test-torture"
       }
+    },
+
+    {
+      "trans": [5.41, 3.5, 28.42], // translation
+      "etype": "test-actor", // actor type
+      "game_task": 0, // associated game task (for powercells, etc)
+      "quat": [0, 0, 0, 1], // quaternion
+      "bsphere": [5.41, 3.5, 28.42, 10], // bounding sphere
+      "lump": {
+        "name": "test-actor"
+      }
     }
   ]
 }

custom_assets/jak2/levels/test-zone/testzone.gd

@@ -5,5 +5,7 @@
 ("TSZ.DGO"
  (
   "prison-obs.o"
+  "test-zone-obs.o"
+  "test-actor-ag.go"
   "test-zone.go"
   ))

custom_assets/jak3/levels/test-zone/test-zone.jsonc

@@ -55,6 +55,10 @@
   // All art groups you want to use in your custom level. Will add their models and corresponding textures to the FR3 file.
   // "art_groups": [],
 
+  // If you have any custom models in the "custom_assets/jak3/models/custom_levels" folder that you want to use in your level, add them to this list.
+  // Note: Like with art groups, these should also be added to your level's .gd file.
+  "custom_models": ["test-actor"],
+
   // Any textures you want to include in your custom level.
   // This is mainly useful for textures which are not in the common level files and have no art group associated with them.
   // To get a list of all the textures, you can extract all of the game's textures
@@ -87,6 +91,17 @@
       "lump": {
         "name": "test-eco"
       }
+    },
+
+    {
+      "trans": [5.41, 3.5, 28.42], // translation
+      "etype": "test-actor", // actor type
+      "game_task": 0, // associated game task (for powercells, etc)
+      "quat": [0, 0, 0, 1], // quaternion
+      "bsphere": [5.41, 3.5, 28.42, 10], // bounding sphere
+      "lump": {
+        "name": "test-actor"
+      }
     }
   ]
 }

custom_assets/jak3/levels/test-zone/testzone.gd

@@ -3,6 +3,8 @@
 
 ;; the actual file name still needs to be 8.3
 ("TSZ.DGO"
- ("test-zone.go"
-  )
- )
+ (
+  "test-zone-obs.o"
+  "test-actor-ag.go"
+  "test-zone.go"
+  ))

decompiler/IR2/AtomicOp.cpp

@@ -353,6 +353,8 @@ std::string get_simple_expression_op_name(SimpleExpression::Kind kind) {
       return "pcypld";
     case SimpleExpression::Kind::VECTOR_PLUS:
       return "vector+!2";
+    case SimpleExpression::Kind::VECTOR_XYZ_PRODUCT:
+      return "vector-xyz-product!";
     case SimpleExpression::Kind::VECTOR_MINUS:
       return "vector-!2";
     case SimpleExpression::Kind::VECTOR_FLOAT_PRODUCT:
@@ -427,6 +429,7 @@ int get_simple_expression_arg_count(SimpleExpression::Kind kind) {
     case SimpleExpression::Kind::VECTOR_MINUS:
     case SimpleExpression::Kind::VECTOR_FLOAT_PRODUCT:
     case SimpleExpression::Kind::VECTOR_CROSS:
+    case SimpleExpression::Kind::VECTOR_XYZ_PRODUCT:
       return 3;
     case SimpleExpression::Kind::SUBU_L32_S7:
       return 1;

decompiler/IR2/AtomicOp.h

@@ -256,6 +256,7 @@ class SimpleExpression {
     VECTOR_PLUS,
     VECTOR_MINUS,
     VECTOR_FLOAT_PRODUCT,
+    VECTOR_XYZ_PRODUCT,  // vector*!, elementwise xyz, jak3 only
     VECTOR_CROSS,
     SUBU_L32_S7,  // use SUBU X, src0, s7 to check if lower 32-bits are s7.
     VECTOR_3_DOT,

decompiler/IR2/AtomicOpTypeAnalysis.cpp

@@ -238,6 +238,7 @@ TP_Type SimpleExpression::get_type(const TypeState& input,
     case Kind::VECTOR_PLUS:
     case Kind::VECTOR_MINUS:
     case Kind::VECTOR_CROSS:
+    case Kind::VECTOR_XYZ_PRODUCT:
       return TP_Type::make_from_ts("vector");
     case Kind::VECTOR_FLOAT_PRODUCT:
       return TP_Type::make_from_ts("vector");

decompiler/IR2/Form.cpp

@@ -1858,6 +1858,8 @@ std::string fixed_operator_to_string(FixedOperatorKind kind) {
       return "vector-!";
     case FixedOperatorKind::VECTOR_PLUS:
       return "vector+!";
+    case FixedOperatorKind::VECTOR_XYZ_PRODUCT:
+      return "vector*!";
     case FixedOperatorKind::VECTOR_CROSS:
       return "vector-cross!";
     case FixedOperatorKind::VECTOR_FLOAT_PRODUCT:
@@ -2146,6 +2148,15 @@ DerefToken to_token(const FieldReverseLookupOutput::Token& in) {
   }
 }
 
+std::vector<DerefToken> to_tokens(const std::vector<FieldReverseLookupOutput::Token>& in) {
+  std::vector<DerefToken> ret;
+  ret.reserve(in.size());
+  for (auto& x : in) {
+    ret.push_back(to_token(x));
+  }
+  return ret;
+}
+
 DerefElement::DerefElement(Form* base, bool is_addr_of, DerefToken token)
     : m_base(base), m_is_addr_of(is_addr_of), m_tokens({std::move(token)}) {
   m_base->parent_element = this;

decompiler/IR2/Form.h

@@ -1274,6 +1274,7 @@ class DerefToken {
 };
 
 DerefToken to_token(const FieldReverseLookupOutput::Token& in);
+std::vector<DerefToken> to_tokens(const std::vector<FieldReverseLookupOutput::Token>& in);
 
 class DerefElement : public FormElement {
  public: