Adding custom layers

[ML-DL-AI-SS]

Hi everyone,

I'm new to the FINN and I would like to add custom neurons to the pipeline. Could you please advise which parts of the framework I need to modify to implement these neurons in a hardware-friendly way?

Additionally, I want to customize the convolutional and fully connected layers (specifically the MVTUs part). I was able to locate these layers in FINN-HLSLIB, but I'm unsure which files to modify in the FINN framework itself. Any guidance would be appreciated.

[fpjentzsch]

Hi,

we have this notebook, which demonstrates how to add a new CustomOp to FINN: https://github.com/Xilinx/finn/blob/dev/notebooks/advanced/2_custom_op.ipynb

Here you can see how the FINN HLS MVAU CustomOp calls the underlying function from FINN-hlslib:

finn/src/finn/custom_op/fpgadataflow/hls/matrixvectoractivation_hls.py

Line 291 in 1cb5877

def docompute(self):

You could modify that function in hlslib or make it call your own custom HLS function instead.