Add VolumeMesher and VOLUME_MESH task type #2493

momchil-flex · 2025-05-20T13:09:19Z

Currently this depends on #2569 and has been rebased on it.

Introduces the VolumeMesher, VolumeMeshMonitor, and associated data types to be able to compute and inspect a HeatChargeSimulation mesh before running the solver. From the user perspective the workflow is:

mesher = td.VolumeMesher(
    simulation=HeatChargeSimulation,
    monitors=tuple[VolumeMeshmonitor, ...],
)
mesh_job = web.Job(simulation=mesher, folder_name="VOLUME_MESH", task_name="mesher")
mesh_data = mesh_job.run()

# Inspect mesh e.g. of monitor named "2D"
mesh_data["2D"].mesh.plot()

heat_sim = mesher.simulation
web.run(
    heat_sim,
    task_name="heat_after_mesh",
    parent_tasks=[mesh_job.task_id],
)

Greptile Summary

Introduces VolumeMesher functionality to compute and inspect HeatChargeSimulation meshes before running the solver, enabling better mesh quality verification and debugging.

Added new VolumeMesher class with associated VolumeMeshMonitor and data types to preview simulation meshes
Extended VTK file support in TetrahedralGridDataset and TriangularGridDataset with ignore_invalid_cells parameter for improved mesh handling
Implemented new VOLUME_MESH task type in web API with appropriate integrations
Created AbstractHeatChargeSimulationData base class to unify HeatChargeSimulation and VolumeMesher results
Added support for legacy VTK format (.vtk) files alongside existing .vtu support

github-actions · 2025-06-16T13:17:18Z

Diff Coverage

Diff: origin/develop...HEAD, staged and unstaged changes

tidy3d/init.py (100%)
tidy3d/components/base_sim/data/sim_data.py (75.0%): Missing lines 134
tidy3d/components/data/unstructured/base.py (42.9%): Missing lines 489-492,494,595-596,639
tidy3d/components/data/unstructured/tetrahedral.py (28.6%): Missing lines 115-118,122
tidy3d/components/data/unstructured/triangular.py (33.3%): Missing lines 151-153,157
tidy3d/components/tcad/data/monitor_data/mesh.py (78.9%): Missing lines 32,36,42-43
tidy3d/components/tcad/data/sim_data.py (64.5%): Missing lines 424-425,427-430,434,438-441
tidy3d/components/tcad/mesher.py (90.0%): Missing lines 24
tidy3d/components/tcad/monitors/mesh.py (100%)
tidy3d/components/tcad/simulation/heat_charge.py (100%)
tidy3d/web/api/tidy3d_stub.py (25.0%): Missing lines 101-102,163-164,217-218
tidy3d/web/api/webapi.py (100%)
tidy3d/web/core/http_util.py (0.0%): Missing lines 138
tidy3d/web/core/types.py (100%)

Summary

Total: 112 lines
Missing: 41 lines
Coverage: 63%

tidy3d/components/base_sim/data/sim_data.py

  130         return field_value
  131 
  132     def get_monitor_by_name(self, name: str) -> AbstractMonitor:
  133         """Return monitor named 'name'."""
! 134         return self.simulation.get_monitor_by_name(name)

tidy3d/components/data/unstructured/base.py

  485     @staticmethod
  486     @requires_vtk
  487     def _read_vtkLegacyFile(fname: str):
  488         """Load a grid from a legacy `.vtk` file."""
! 489         reader = vtk["mod"].vtkGenericDataObjectReader()
! 490         reader.SetFileName(fname)
! 491         reader.Update()
! 492         grid = reader.GetOutput()
  493 
! 494         return grid
  495 
  496     @classmethod
  497     @abstractmethod
  498     @requires_vtk

  591         -------
  592         UnstructuredGridDataset
  593             Unstructured data.
  594         """
! 595         grid = cls._read_vtkLegacyFile(file)
! 596         return cls._from_vtk_obj(
  597             grid,
  598             field=field,
  599             remove_degenerate_cells=remove_degenerate_cells,
  600             remove_unused_points=remove_unused_points,

  635             log.warning(
  636                 "No point data is found in a VTK object. '.values' will be initialized to zeros."
  637             )
  638             values_numpy = np.zeros(num_points)
! 639             values_coords = {"index": np.arange(num_points)}
  640             values_name = None
  641 
  642         else:
  643             field_ind = field if isinstance(field, str) else 0

tidy3d/components/data/unstructured/tetrahedral.py

  111         # verify cell_types
  112         cells_types = vtk["vtk_to_numpy"](vtk_obj.GetCellTypesArray())
  113         invalid_cells = cells_types != cls._vtk_cell_type()
  114         if any(invalid_cells):
! 115             if ignore_invalid_cells:
! 116                 cell_offsets = vtk["vtk_to_numpy"](vtk_obj.GetCells().GetOffsetsArray())
! 117                 valid_cell_offsets = cell_offsets[:-1][invalid_cells == 0]
! 118                 cells_numpy = cells_numpy[
  119                     np.ravel(valid_cell_offsets[:, None] + np.arange(4, dtype=int)[None, :])
  120                 ]
  121             else:
! 122                 raise DataError("Only tetrahedral 'vtkUnstructuredGrid' is currently supported")
  123 
  124         # pack point and cell information into Tidy3D arrays
  125         num_cells = len(cells_numpy) // cls._cell_num_vertices()
  126         cells_numpy = np.reshape(cells_numpy, (num_cells, cls._cell_num_vertices()))

tidy3d/components/data/unstructured/triangular.py

  147         # verify cell_types
  148         cell_offsets = vtk["vtk_to_numpy"](cells_vtk.GetOffsetsArray())
  149         invalid_cells = np.diff(cell_offsets) != cls._cell_num_vertices()
  150         if any(invalid_cells):
! 151             if ignore_invalid_cells:
! 152                 valid_cell_offsets = cell_offsets[:-1][invalid_cells == 0]
! 153                 cells_numpy = cells_numpy[
  154                     np.ravel(valid_cell_offsets[:, None] + np.arange(3, dtype=int)[None, :])
  155                 ]
  156             else:
! 157                 raise DataError(
  158                     "Only triangular 'vtkUnstructuredGrid' or 'vtkPolyData' can be converted into "
  159                     "'TriangularGridDataset'."
  160                 )

tidy3d/components/tcad/data/monitor_data/mesh.py

  28 
  29     @property
  30     def field_components(self) -> dict[str, UnstructuredFieldType]:
  31         """Maps the field components to their associated data."""
! 32         return {"mesh": self.mesh}
  33 
  34     def field_name(self, val: str) -> str:
  35         """Gets the name of the fields to be plot."""
! 36         return "Mesh"
  37 
  38     @property
  39     def symmetry_expanded_copy(self) -> VolumeMeshData:
  40         """Return copy of self with symmetry applied."""

  38     @property
  39     def symmetry_expanded_copy(self) -> VolumeMeshData:
  40         """Return copy of self with symmetry applied."""
  41 
! 42         new_mesh = self._symmetry_expanded_copy(property=self.mesh)
! 43         return self.updated_copy(mesh=new_mesh, symmetry=(0, 0, 0))

tidy3d/components/tcad/data/sim_data.py

  420     def data_monitors_match_sim(cls, val, values):
  421         """Ensure each :class:`AbstractMonitorData` in ``.data`` corresponds to a monitor in
  422         ``.simulation``.
  423         """
! 424         monitors = values.get("monitors")
! 425         mnt_names = {mnt.name for mnt in monitors}
  426 
! 427         for mnt_data in val:
! 428             monitor_name = mnt_data.monitor.name
! 429             if monitor_name not in mnt_names:
! 430                 raise DataError(
  431                     f"Data with monitor name '{monitor_name}' supplied "
  432                     f"but not found in the list of monitors."
  433                 )
! 434         return val
  435 
  436     def get_monitor_by_name(self, name: str) -> VolumeMeshMonitor:
  437         """Return monitor named 'name'."""
! 438         for monitor in self.monitors:
! 439             if monitor.name == name:
! 440                 return monitor
! 441         raise Tidy3dKeyError(f"No monitor named '{name}'")

tidy3d/components/tcad/mesher.py

  20         description="List of monitors to be used for the mesher.",
  21     )
  22 
  23     def _get_simulation_types(self) -> list[TCADAnalysisTypes]:
! 24         return [TCADAnalysisTypes.MESH]

tidy3d/web/api/tidy3d_stub.py

   97         elif type_ == "EMESimulation":
   98             sim = EMESimulation.from_file(file_path)
   99         elif type_ == "ModeSimulation":
  100             sim = ModeSimulation.from_file(file_path)
! 101         elif "VolumeMesher" == type_:
! 102             sim = VolumeMesher.from_file(file_path)
  103 
  104         return sim
  105 
  106     def to_file(

  159         if isinstance(self.simulation, EMESimulation):
  160             return TaskType.EME.name
  161         if isinstance(self.simulation, ModeSimulation):
  162             return TaskType.MODE.name
! 163         elif isinstance(self.simulation, VolumeMesher):
! 164             return TaskType.VOLUME_MESH.name
  165 
  166     def validate_pre_upload(self, source_required) -> None:
  167         """Perform some pre-checks on instances of component"""
  168         if isinstance(self.simulation, Simulation):

  213         elif type_ == "EMESimulationData":
  214             sim_data = EMESimulationData.from_file(file_path)
  215         elif type_ == "ModeSimulationData":
  216             sim_data = ModeSimulationData.from_file(file_path)
! 217         elif type_ == "VolumeMesherData":
! 218             sim_data = VolumeMesherData.from_file(file_path)
  219 
  220         return sim_data
  221 
  222     def to_file(self, file_path: str):

tidy3d/web/core/http_util.py

  134             if resp.status_code == ResponseCodes.NOT_FOUND.value:
  135                 raise WebNotFoundError("Resource not found (HTTP 404).")
  136             json_resp = resp.json()
  137             if "error" in json_resp.keys():
! 138                 print(json_resp)
  139                 raise WebError(json_resp["error"])
  140             resp.raise_for_status()
  141 
  142         if not resp.text:

marc-flex

Thanks Momchil for going through this.

I guess my main concern is that VolumeMesher doesn't add much. In fact, I think we don't need that class. Instead, we could create a new analysis type along the lines of CreateMesh() that can be passed to analysis_spec. Or even, it VolumeMesher could be an analysis type:

sim = td.HeatChargeSimulation(
     ...
     analysis_spec=td.VolumeMeshser(),
)

Even the parent task id and everything else could be used in the same way you were intended, right?

marc-flex · 2025-06-17T14:20:32Z

tidy3d/components/tcad/data/sim_data.py

+    simulation: HeatChargeSimulation = pd.Field(
+        title="Volume mesher",
+        description="Original :class:`VolumeMesher` associated with the data.",
+    )


This might as well go to the base class?

marc-flex · 2025-06-17T14:23:05Z

tidy3d/components/tcad/monitors/mesh.py

+class VolumeMeshMonitor(HeatChargeMonitor):
+    """Monitor for the volume mesh."""
+


I guess in the docstring we could add a couple of examples of monitors so that we make it clear that even if it is a "volume mesh monitor" the monitor itself can be planar?

dbochkov-flexcompute · 2025-06-18T05:39:22Z

we could probably close #2569 since it's included in here. As for missing tests from there, I think we can just get a 2d vtk and a 3d vtk from actual backend runs and add frontend unit tests that loads them (successfully with ignore_invalid_cells=True, and failing with ignore_invalid_cells=False)

greptile-apps

_{14 files reviewed, 8 comments}
_{Edit PR Review Bot Settings | Greptile}

greptile-apps · 2025-06-18T06:41:10Z

tidy3d/components/tcad/data/monitor_data/mesh.py

+        """Gets the name of the fields to be plot."""
+        return "Mesh"


syntax: parameter 'val' is unused in field_name implementation

Suggested change

"""Gets the name of the fields to be plot."""

return "Mesh"

def field_name(self, _: str) -> str:

greptile-apps · 2025-06-18T06:41:11Z

tidy3d/components/data/unstructured/tetrahedral.py

+                cell_offsets = vtk["vtk_to_numpy"](vtk_obj.GetCells().GetOffsetsArray())
+                valid_cell_offsets = cell_offsets[:-1][invalid_cells == 0]
+                cells_numpy = cells_numpy[
+                    np.ravel(valid_cell_offsets[:, None] + np.arange(4, dtype=int)[None, :])
+                ]


style: The hardcoded 4 in np.arange(4) assumes tetrahedral cells. Consider using cls._cell_num_vertices() instead for consistency with rest of the code

Suggested change

cell_offsets = vtk["vtk_to_numpy"](vtk_obj.GetCells().GetOffsetsArray())

valid_cell_offsets = cell_offsets[:-1][invalid_cells == 0]

cells_numpy = cells_numpy[

np.ravel(valid_cell_offsets[:, None] + np.arange(4, dtype=int)[None, :])

]

cell_offsets = vtk["vtk_to_numpy"](vtk_obj.GetCells().GetOffsetsArray())

valid_cell_offsets = cell_offsets[:-1][invalid_cells == 0]

cells_numpy = cells_numpy[

np.ravel(valid_cell_offsets[:, None] + np.arange(cls._cell_num_vertices(), dtype=int)[None, :])

]

greptile-apps · 2025-06-18T06:41:12Z

tidy3d/components/data/unstructured/triangular.py

+        invalid_cells = np.diff(cell_offsets) != cls._cell_num_vertices()
+        if any(invalid_cells):
+            if ignore_invalid_cells:
+                valid_cell_offsets = cell_offsets[:-1][invalid_cells == 0]
+                cells_numpy = cells_numpy[
+                    np.ravel(valid_cell_offsets[:, None] + np.arange(3, dtype=int)[None, :])
+                ]
+            else:
+                raise DataError(
+                    "Only triangular 'vtkUnstructuredGrid' or 'vtkPolyData' can be converted into "
+                    "'TriangularGridDataset'."
+                )


style: Be explicit about vectorization in the broadcasting. Change:

Suggested change

invalid_cells = np.diff(cell_offsets) != cls._cell_num_vertices()

if any(invalid_cells):

if ignore_invalid_cells:

valid_cell_offsets = cell_offsets[:-1][invalid_cells == 0]

cells_numpy = cells_numpy[

np.ravel(valid_cell_offsets[:, None] + np.arange(3, dtype=int)[None, :])

]

else:

raise DataError(

"Only triangular 'vtkUnstructuredGrid' or 'vtkPolyData' can be converted into "

"'TriangularGridDataset'."

)

invalid_cells = np.diff(cell_offsets) != cls._cell_num_vertices()

if np.any(invalid_cells):

greptile-apps · 2025-06-18T06:41:12Z

tidy3d/components/tcad/monitors/mesh.py

+class VolumeMeshMonitor(HeatChargeMonitor):
+    """Monitor for the volume mesh."""


style: Class docstring should detail the purpose, typical usage scenario, and relationship to HeatChargeMonitor. Consider adding an example.

greptile-apps · 2025-06-18T06:41:14Z

tidy3d/components/tcad/data/sim_data.py

@@ -218,7 +226,7 @@ def plot_field(
        # field.name = field_name
        field_data = self._field_component_value(field, val)

-        if isinstance(monitor_data, TemperatureData):
+        if isinstance(monitor_data, (TemperatureData, VolumeMeshData)):


logic: if block now includes VolumeMeshData but comment at line 175 only mentions TemperatureMonitorData

Suggested change

if isinstance(monitor_data, (TemperatureData, VolumeMeshData)):

field_monitor_name : str

Name of :class:`.TemperatureMonitorData` or :class:`.VolumeMeshData` to plot.

greptile-apps · 2025-06-18T06:41:15Z

tidy3d/components/tcad/data/sim_data.py

+    simulation: HeatChargeSimulation = pd.Field(
+        title="Volume mesher",
+        description="Original :class:`VolumeMesher` associated with the data.",
+    )


logic: incorrect docstring - references VolumeMesher but field shows HeatChargeSimulation

Suggested change

simulation: HeatChargeSimulation = pd.Field(

title="Volume mesher",

description="Original :class:`VolumeMesher` associated with the data.",

)

simulation: HeatChargeSimulation = pd.Field(

title="Heat-Charge Simulation",

description="Original :class:`HeatChargeSimulation` associated with the data.",

)

greptile-apps · 2025-06-18T06:41:16Z

tidy3d/components/data/unstructured/base.py

+    @requires_vtk
+    def from_vtk(
+        cls,
+        file: str,
+        field: Optional[str] = None,
+        remove_degenerate_cells: bool = False,
+        remove_unused_points: bool = False,
+        ignore_invalid_cells: bool = False,
+    ) -> UnstructuredGridDataset:
+        """Load unstructured data from a vtk file.
+
+        Parameters
+        ----------
+        fname : str
+            Full path to the .vtk file to load the unstructured data from.
+        field : str = None
+            Name of the field to load.
+        remove_degenerate_cells : bool = False
+            Remove explicitly degenerate cells.
+        remove_unused_points : bool = False
+            Remove unused points.
+


style: Parameter ignore_invalid_cells is documented but description is missing

Suggested change

@requires_vtk

def from_vtk(

cls,

file: str,

field: Optional[str] = None,

remove_degenerate_cells: bool = False,

remove_unused_points: bool = False,

ignore_invalid_cells: bool = False,

) -> UnstructuredGridDataset:

"""Load unstructured data from a vtk file.

Parameters

----------

fname : str

Full path to the .vtk file to load the unstructured data from.

field : str = None

Name of the field to load.

remove_degenerate_cells : bool = False

Remove explicitly degenerate cells.

remove_unused_points : bool = False

Remove unused points.

ignore_invalid_cells : bool = False

Whether to ignore invalid cells during loading.

greptile-apps · 2025-06-18T06:41:22Z

tidy3d/web/api/tidy3d_stub.py

+        elif "VolumeMesher" == type_:
+            sim = VolumeMesher.from_file(file_path)


style: Inconsistent string comparison style. Other type checks use type_ == 'String', but here it's flipped.

Suggested change

elif "VolumeMesher" == type_:

sim = VolumeMesher.from_file(file_path)

elif type_ == "VolumeMesher":

sim = VolumeMesher.from_file(file_path)

momchil-flex marked this pull request as draft May 20, 2025 13:28

momchil-flex force-pushed the momchil/separate_mesh_pipeline branch from ac85aa7 to 6b1c544 Compare May 21, 2025 06:50

momchil-flex force-pushed the momchil/separate_mesh_pipeline branch from 0e36855 to 1b29844 Compare June 12, 2025 09:32

ability to read unstructured data from legacy vtk format

e486b44

momchil-flex force-pushed the momchil/separate_mesh_pipeline branch 2 times, most recently from 1282a66 to d5e1b87 Compare June 16, 2025 13:02

momchil-flex requested a review from marc-flex June 16, 2025 13:07

momchil-flex marked this pull request as ready for review June 16, 2025 13:07

momchil-flex requested a review from dbochkov-flexcompute June 16, 2025 13:08

momchil-flex changed the title ~~Add VolumeMeshSpec and VOLUME_MESH task type~~ Add VolumeMesher and VOLUME_MESH task type Jun 16, 2025

Add VolumeMesher and VOLUME_MESH task type

169c6b4

momchil-flex force-pushed the momchil/separate_mesh_pipeline branch from d5e1b87 to 169c6b4 Compare June 17, 2025 11:14

marc-flex reviewed Jun 17, 2025

View reviewed changes

greptile-apps bot reviewed Jun 18, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add VolumeMesher and VOLUME_MESH task type #2493

Add VolumeMesher and VOLUME_MESH task type #2493

momchil-flex commented May 20, 2025 •

edited by greptile-apps bot

Loading

Uh oh!

github-actions bot commented Jun 16, 2025

Uh oh!

marc-flex left a comment

Uh oh!

marc-flex Jun 17, 2025

Uh oh!

marc-flex Jun 17, 2025

Uh oh!

dbochkov-flexcompute commented Jun 18, 2025

Uh oh!

greptile-apps bot left a comment

Uh oh!

greptile-apps bot Jun 18, 2025

Uh oh!

greptile-apps bot Jun 18, 2025

Uh oh!

greptile-apps bot Jun 18, 2025

Uh oh!

greptile-apps bot Jun 18, 2025

Uh oh!

greptile-apps bot Jun 18, 2025

Uh oh!

greptile-apps bot Jun 18, 2025

Uh oh!

greptile-apps bot Jun 18, 2025

Uh oh!

greptile-apps bot Jun 18, 2025

Uh oh!

Uh oh!

		class VolumeMeshMonitor(HeatChargeMonitor):
		"""Monitor for the volume mesh."""

	"""Gets the name of the fields to be plot."""
	return "Mesh"
	def field_name(self, _: str) -> str:

	if isinstance(monitor_data, (TemperatureData, VolumeMeshData)):
	field_monitor_name : str
	Name of :class:`.TemperatureMonitorData` or :class:`.VolumeMeshData` to plot.

		elif "VolumeMesher" == type_:
		sim = VolumeMesher.from_file(file_path)

Add VolumeMesher and VOLUME_MESH task type #2493

Are you sure you want to change the base?

Add VolumeMesher and VOLUME_MESH task type #2493

Conversation

momchil-flex commented May 20, 2025 • edited by greptile-apps bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Greptile Summary

Uh oh!

github-actions bot commented Jun 16, 2025

Diff Coverage

Diff: origin/develop...HEAD, staged and unstaged changes

Summary

tidy3d/components/base_sim/data/sim_data.py

tidy3d/components/data/unstructured/base.py

tidy3d/components/data/unstructured/tetrahedral.py

tidy3d/components/data/unstructured/triangular.py

tidy3d/components/tcad/data/monitor_data/mesh.py

tidy3d/components/tcad/data/sim_data.py

tidy3d/components/tcad/mesher.py

tidy3d/web/api/tidy3d_stub.py

tidy3d/web/core/http_util.py

Uh oh!

marc-flex left a comment

Choose a reason for hiding this comment

Uh oh!

marc-flex Jun 17, 2025

Choose a reason for hiding this comment

Uh oh!

marc-flex Jun 17, 2025

Choose a reason for hiding this comment

Uh oh!

dbochkov-flexcompute commented Jun 18, 2025

Uh oh!

greptile-apps bot left a comment

Choose a reason for hiding this comment

Uh oh!

greptile-apps bot Jun 18, 2025

Choose a reason for hiding this comment

Uh oh!

greptile-apps bot Jun 18, 2025

Choose a reason for hiding this comment

Uh oh!

greptile-apps bot Jun 18, 2025

Choose a reason for hiding this comment

Uh oh!

greptile-apps bot Jun 18, 2025

Choose a reason for hiding this comment

Uh oh!

greptile-apps bot Jun 18, 2025

Choose a reason for hiding this comment

Uh oh!

greptile-apps bot Jun 18, 2025

Choose a reason for hiding this comment

Uh oh!

greptile-apps bot Jun 18, 2025

Choose a reason for hiding this comment

Uh oh!

greptile-apps bot Jun 18, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

momchil-flex commented May 20, 2025 •

edited by greptile-apps bot

Loading