Skip to content

Commit 0de929f

Browse files
abdulrahman-10xetaimur-10xetalhaiqbal-10xe
authored
Release v1.1 (#3)
FPGA Binaries v1.1 with support for AR1335 and OV5647 image sensors * support for AR1335, OV5647 image sensors * video creation script from FPGA Binary ISP output Burst Capture * Kria KV260 Binary file for use with AR1335 IAS image sensor module * Kria KV260 Binary file for use with OV5647 RPi image sensor module * header and description added * header and description added * header and description added * Create sensor_bin_to_sensor_raw_burst_capture.py * header updated * updated video_creation.py * video_creation.py updated * verified with SD card dumps * how to use release v1.1 binaries added --------- Co-authored-by: taimur-10xe <taimur.bilal@10xengineers.ai> Co-authored-by: talhaiqbal-10xe <talha.iqbal@10xengineers.ai> Co-authored-by: taimur-10xe <92528065+taimur-10xe@users.noreply.github.com>
1 parent dd00e25 commit 0de929f

8 files changed

+293
-447
lines changed

README.md

+27
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,8 @@ Infinite-ISP is a full-stack ISP development platform - from algorithm developme
1414
Infinite-ISP Image Signal Processing Pipeline FPGA binaries for XCK26 Zynq® UltraScale+™ MPSoC present on Xilinx® Kria™ KV260 Vision AI Starter Kit. Each binary file includes an FPGA bitstream paired with its firmware executable.
1515

1616
# How to use the FPGA Binary Files
17+
18+
## v1.0
1719
1. Connect the AR1335 IAS image sensor module (included in Xilinx Kria KV260 Accessory Pack) to the IAS1 port on the Kria KV260 AI Starter Kit.
1820
2. Insert an SD Card (2GB or above) into the Kria KV260 board.
1921
3. Connect the Kria KV260 board with a monitor screen using an HDMI cable.
@@ -23,6 +25,31 @@ Infinite-ISP Image Signal Processing Pipeline FPGA binaries for XCK26 Zynq® Ult
2325
7. Connect the USB cable with the JTAG/USB port on Kria to read the messages over UART.
2426
8. You can remove the SD card from the Kria board, extract the dumped single RGB-RAW pair and 300 RGB frames (if applicable), and visualize them using the provided scripts.
2527

28+
## v1.1
29+
### For AR1335 IAS module:
30+
1. Connect AR1335 IAS image sensor module to the IAS1 port on Kria KV260 AI Starter Kit.
31+
2. Insert SD Card (2GB or above) into the Kria KV260 board.
32+
3. Connect the Kria KV260 board with a monitor screen using HDMI cable.
33+
4. Power up the Kria KV260 board and follow the [steps](https://docs.xilinx.com/r/en-US/ug1089-kv260-starter-kit/Ethernet-Recovery-Tool) for loading the on-board Xilinx Image Recovery Tool.
34+
5. Upload the desired binary file (e.g. Infinite-ISP_v1.1-AR1335-1080p.bin) provided in the release on the Kria KV260 AI Starter Kit.
35+
6. Reset the Kria KV260 board and visualize the Infinite-ISP output on your monitor screen.
36+
7. Connect the USB cable with the JTAG/USB port on Kria to read the messages over UART.
37+
8. RAW-ISPout image pair and Burst Capture frames dump in SD Card status will be displayed over UART.
38+
9. Once SD Card dumps are complete, you can remove the SD card from the Kria board.
39+
10. Extract the dumped single RAW-ISPout pair and Burst Capture frames from the SD card and visualize them using provided scripts.
40+
41+
### For OV5647 image sensor module:
42+
1. Connect OV5647 image sensor module to the RPi port on Kria KV260 AI Starter Kit.
43+
2. Insert SD Card (2GB or above) into the Kria KV260 board.
44+
3. Connect the Kria KV260 board with a monitor screen using HDMI cable.
45+
4. Power up the Kria KV260 board and follow the [steps](https://docs.xilinx.com/r/en-US/ug1089-kv260-starter-kit/Ethernet-Recovery-Tool) for loading the on-board Xilinx Image Recovery Tool.
46+
5. Upload the desired binary file (e.g. Infinite-ISP_v1.1-OV5647-1080p.bin) provided in the release on the Kria KV260 AI Starter Kit.
47+
6. Reset the Kria KV260 board and visualize the Infinite-ISP output on your monitor screen.
48+
7. Connect the USB cable with the JTAG/USB port on Kria to read the messages over UART.
49+
8. RAW-ISPout image pair and Burst Capture frames dump in SD Card status will be displayed over UART.
50+
9. Once SD Card dumps are complete, you can remove the SD card from the Kria board.
51+
10. Extract the dumped single RAW-ISPout pair and Burst Capture frames from the SD card and visualize them using provided scripts.
52+
2653
## Contact
2754
For any inquiries or feedback, feel free to reach out.
2855

7.99 MB
Binary file not shown.
7.99 MB
Binary file not shown.

scripts/isp_output_bin_to_isp_output_png.py

+12-2
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,26 @@
1+
"""
2+
File: isp_output_bin_to_isp_output_png.py
3+
Description: converts the ISP output memory dump (.bin) data
4+
from the FPGA Platform to output image frame (.png)
5+
as well as output pixel data frame (.bin) for
6+
verification
7+
Author: 10xEngineers
8+
------------------------------------------------------------
9+
"""
10+
111
import numpy as np
212
import matplotlib.image as mpimg
313
import matplotlib.pyplot as plt
414
from datetime import datetime
515

616

717
filepath = "./"
8-
filename = 'RTL_ImageTest_2048x1536_10bit_GRBG_1920x1080.bin'
18+
filename = 'ISPOut_TestImage_2592x1944_10bits_BGGR.bin'
919
with open(filepath + filename, 'rb') as f:
1020
arr = np.fromfile(f, dtype=np.uint8)
1121
f.close()
1222

13-
h, w, Format, CONV_STD = 1080, 1920, "BGR", 2
23+
h, w, Format, CONV_STD = 1080, 1920, "YUV444", 2 #copy string from support output formats here
1424

1525
SupportedFormats = {
1626
"BGR" : 1,

scripts/sensor_bin_to_sensor_raw.py

+11-9
Original file line numberDiff line numberDiff line change
@@ -1,25 +1,27 @@
1-
# -*- coding: utf-8 -*-
21
"""
3-
Created on Tue Feb 21 16:07:11 2023
4-
5-
@author: user3
2+
File: sensor_bin_to_sensor_raw.py
3+
Description: converts the image sensor memory dump (.bin) data
4+
from the FPGA Platform to Bayer RAW frame (.raw)
5+
containing valid pixel data
6+
Author: 10xEngineers
7+
------------------------------------------------------------
68
"""
9+
710
import numpy as np
8-
import cv2
911
import matplotlib.image as mpimg
1012
import matplotlib.pyplot as plt
1113

12-
# Supported Sensors
13-
14-
SENSOR = "AR1335"
14+
# Sensor selection for conversion to .raw
15+
SENSOR = "OV5647" #copy string from supported sensor type here
1516

17+
# Supported Sensors
1618
SupportedSensors = {
1719
"AR1335": 1,
1820
"OV5647": 2
1921
}
2022

2123
# reading the dumped binary file
22-
filename = 'ImageTest_2048x1536_10bit_GRBG_1920x1080.bin'
24+
filename = 'RAW_TestImage_2592x1944_10bits_BGGR.bin'
2325
with open(filename, 'rb') as f:
2426
# read the contents of the file into a new array
2527
arr = np.fromfile(f, dtype=np.uint8)
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,90 @@
1+
"""
2+
File: sensor_bin_to_sensor_raw_burst_capture.py
3+
Description: converts the image sensor memory dumps (.bin) of
4+
RAW Burst Capture from the FPGA Platform to
5+
Bayer RAW frames (.raw) containing valid pixel
6+
data.
7+
It also converts the Bayer RAW frames to
8+
equivalent grayscale .png for visualization.
9+
Author: 10xEngineers
10+
------------------------------------------------------------
11+
"""
12+
import numpy as np
13+
import matplotlib.image as mpimg
14+
import matplotlib.pyplot as plt
15+
from pathlib import Path
16+
17+
# Path of the directory containing "OV5467" and "AR1335" directories
18+
path = "./"
19+
20+
# Supported Sensors and selected sensor (SENSOR)
21+
SupportedSensors = {
22+
"AR1335": 1,
23+
"OV5647": 2
24+
}
25+
SENSOR = "AR1335"
26+
27+
# start and end index of burst capture frames for converting .bin to .raw
28+
start, end = 1, 250
29+
30+
# parent directory
31+
p = Path(path)
32+
parent_dir = p.resolve().joinpath(SENSOR)
33+
34+
# Making directories for saving .raw and .png files
35+
raw_dir = "Burst_Capture_RAW"
36+
png_dir = "Burst_Capture_PNG"
37+
bin_dir = "Burst_Capture" # Created by firmware, DO NOT MODIFY
38+
# joining paths with parent directory
39+
raw_path = parent_dir/ raw_dir
40+
Path.mkdir(raw_path, exist_ok=True)
41+
png_path = parent_dir / png_dir
42+
Path.mkdir(png_path, exist_ok=True)
43+
bin_path = parent_dir / bin_dir
44+
45+
# Selecting height and width based on selected sensor
46+
if(SupportedSensors[SENSOR] == SupportedSensors["AR1335"]):
47+
h, w = 1536, 2048
48+
49+
if(SupportedSensors[SENSOR] == SupportedSensors["OV5647"]):
50+
h, w = 1944, 2592
51+
52+
h = int(h)
53+
w = int(w)
54+
55+
# Images are stored in the form of rows where the size of each row in bytes
56+
# should be a multiple of 256, each such row size is called 'stride'
57+
# For raw10 format, 3 pixels are packed into 4 bytes
58+
stride = np.floor(np.floor(np.floor((w+2)/3) *4 +256 - 1) /256) * 256
59+
stride = stride.astype (np.uint16)
60+
pixelsInStride= int((stride/4)*3)
61+
62+
for index in range (start, end+1):
63+
# reading the dumped binary file
64+
filename = 'RAW' + str(index) + '.bin'
65+
filepath = bin_path / filename
66+
print('Processing ' + 'RAW'+ str(index) + ' ...')
67+
with open(filepath, 'rb') as f:
68+
# read the contents of the file into a new array
69+
arr = np.fromfile(f, dtype=np.uint8)
70+
71+
# Reshape the array into groups of 4 elements
72+
grouped_array = arr.reshape(-1, 4)
73+
flipped_array = np.flip(grouped_array, axis=1)
74+
result_list = []
75+
for inner_array in flipped_array:
76+
# Convert each element to binary and concatenate
77+
binary_concatenated = ''.join([format(x, '08b') for x in inner_array])
78+
result_list.append((int(binary_concatenated[2:12],2),int(binary_concatenated[12:22],2),int(binary_concatenated[22:32],2)))
79+
img = np.array(result_list).reshape((h, pixelsInStride))[:, 0:w].astype(np.uint16)
80+
81+
# dumping a .raw file for inf_isp
82+
filename = filename[:-4]
83+
extension = ".raw"
84+
85+
with open('{}{}'.format(str(raw_path/filename),extension),'wb') as f:
86+
img.tofile(f)
87+
88+
# dumping a numpy array
89+
img_norm = np.interp(img, (img.min(), img.max()), (0,1023 )).astype(np.uint8)
90+
plt.imshow(img_norm,cmap='gray').write_png(str(png_path/filename) + '.png')

0 commit comments

Comments
 (0)