Skip to content

Commit 6d2cada

Browse files
jchrisjchris
committed
feat: add image editing support with multiple input files to ImgGen component
1 parent e6f11f2 commit 6d2cada

File tree

7 files changed

+573
-16
lines changed

7 files changed

+573
-16
lines changed

‎README.md

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -58,11 +58,13 @@ function MyComponent() {
5858

5959
- `prompt`: Text prompt for image generation (required unless `_id` is provided)
6060
- `_id`: Document ID to load a specific image instead of generating a new one
61+
- `images`: Array of image files to edit or combine with AI (when provided, uses the image editing endpoint)
6162
- `options` (object, optional): Configuration options for image generation
6263

6364
- `model` (string, optional): Model to use for image generation, defaults to 'gpt-image-1'
6465
- `size` (string, optional): Size of the generated image (Must be one of 1024x1024, 1536x1024 (landscape), 1024x1536 (portrait), or 'auto' (default value) for gpt-image-1, and one of 256x256, 512x512, or 1024x1024 for dall-e-2.)
6566
- `quality` (string, optional): Quality of the generated image (high, medium and low are only supported for gpt-image-1. dall-e-2 only supports standard quality. Defaults to auto.)
67+
- `style` (string, optional): Style of the generated image (e.g., 'vivid', 'natural')
6668
- `debug` (boolean, optional): Enable debug logging, defaults to false
6769

6870
- `className`: CSS class name for the image element (optional)
@@ -105,6 +107,40 @@ When regenerating images, the component will:
105107

106108
This allows for iterative refinement of images while maintaining version history.
107109

110+
##### Image Editing
111+
112+
The component also supports editing or combining existing images:
113+
114+
```jsx
115+
import { ImgGen } from 'use-vibes';
116+
117+
function SketchEditor() {
118+
const [sketchFile, setSketchFile] = useState(null);
119+
120+
const handleFileChange = (e) => {
121+
if (e.target.files && e.target.files[0]) {
122+
setSketchFile(e.target.files[0]);
123+
}
124+
};
125+
126+
return (
127+
<div>
128+
<input type="file" accept="image/*" onChange={handleFileChange} />
129+
130+
{sketchFile && (
131+
<ImgGen
132+
prompt="Convert this rough sketch into a refined technical diagram"
133+
images={[sketchFile]}
134+
database="sketches-db"
135+
/>
136+
)}
137+
</div>
138+
);
139+
}
140+
```
141+
142+
When the `images` prop is provided, the component automatically uses the image editing endpoint instead of generating from scratch. The component handles all the details including file conversion, progress indicators, and version tracking.
143+
108144
#### Styling
109145

110146
The ImgGen component uses CSS custom properties (variables) for styling, making it easy to customize the appearance while maintaining consistency. There are two primary ways to customize styling:

‎examples/react-example/src/App.tsx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
import { useEffect, useState } from 'react';
1+
import { useState } from 'react';
22
import { ImgGen } from 'use-vibes';
33
import { useFireproof } from 'use-fireproof';
44
import type { DocBase, DocFileMeta } from 'use-fireproof';

‎notes/image-gen-llms.txt

Lines changed: 206 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,206 @@
1+
# call-ai Image Generation API Documentation
2+
3+
## Overview
4+
5+
The call-ai library includes image generation capabilities through the `imageGen` function, which integrates with a custom image generation API. This functionality supports both simple image generation from text prompts and image editing with multiple input images.
6+
7+
## Getting Started
8+
9+
Import the `imageGen` function:
10+
11+
```javascript
12+
import { imageGen } from 'call-ai';
13+
```
14+
15+
## Basic Usage
16+
17+
### Simple Image Generation
18+
19+
Generate an image from a text prompt:
20+
21+
```javascript
22+
const generateResponse = await imageGen(
23+
'A children\'s book drawing of a veterinarian using a stethoscope to listen to the heartbeat of a baby otter.'
24+
);
25+
26+
// Access the base64-encoded image
27+
const imageBase64 = generateResponse.data[0].b64_json;
28+
29+
// Use in an HTML element
30+
const imgElement = document.createElement('img');
31+
imgElement.src = `data:image/png;base64,${imageBase64}`;
32+
document.body.appendChild(imgElement);
33+
34+
// Error handling is built-in
35+
try {
36+
const result = await imageGen('Your prompt');
37+
} catch (error) {
38+
// All errors are proper Error objects with descriptive messages
39+
console.error(error.message);
40+
}
41+
```
42+
43+
### Image Editing with Multiple Input Images
44+
45+
Edit or combine multiple images with a text prompt:
46+
47+
```javascript
48+
// Get image files (e.g., from a file input)
49+
const imageFiles = [bathBombFile, bodyLotionFile, incenseKitFile, soapFile];
50+
51+
// Request image editing
52+
// Uses the same async pattern as callAI for consistency
53+
const editResponse = await imageGen(
54+
'Create a lovely gift basket with these four items in it',
55+
{
56+
model: 'gpt-image-1',
57+
images: imageFiles,
58+
size: '1024x1024', // optional
59+
quality: 'hd', // optional
60+
style: 'natural' // optional
61+
}
62+
);
63+
64+
// Access the base64-encoded edited image
65+
const editedImageBase64 = editResponse.data[0].b64_json;
66+
67+
// Use in an HTML element
68+
const editedImg = document.createElement('img');
69+
editedImg.src = `data:image/png;base64,${editedImageBase64}`;
70+
document.body.appendChild(editedImg);
71+
```
72+
73+
## API Reference
74+
75+
### `imageGen(prompt, options)`
76+
77+
Generates or edits images based on a text prompt and optional images.
78+
79+
#### Parameters
80+
81+
- `prompt` (string, required): Text prompt describing the desired image
82+
- `options` (object, optional): Configuration options
83+
- `model` (string, optional): Model to use for image generation, defaults to 'gpt-image-1'
84+
- `apiKey` (string, optional): API key, defaults to 'VIBES_DIY'
85+
- `images` (File[], optional): Array of File objects to edit, if provided uses the edit endpoint
86+
- `size` (string, optional): Size of the generated image (Must be one of 1024x1024, 1536x1024 (landscape), 1024x1536 (portrait), or 'auto' (default value) for gpt-image-1.)
87+
- `quality` (string, optional): Quality of the generated image ( high | medium | low . Defaults to auto.)
88+
- `style` (string, optional): Style of the generated image (e.g., 'vivid', 'natural'). Note: Style parameter may have limited effect depending on the model.
89+
- `debug` (boolean, optional): Enable debug logging, defaults to false
90+
91+
#### Returns
92+
93+
Returns a Promise that resolves to an `ImageResponse` object:
94+
95+
```typescript
96+
interface ImageResponse {
97+
created: number;
98+
data: {
99+
b64_json: string;
100+
url?: string;
101+
revised_prompt?: string;
102+
}[];
103+
}
104+
```
105+
106+
## Implementation Details
107+
108+
The image generation functionality uses two custom API endpoints:
109+
110+
- `/api/openai-image/generate`: For simple image generation with a text prompt
111+
- `/api/openai-image/edit`: For editing/combining multiple images with a text prompt
112+
113+
These endpoints return base64-encoded image data that can be directly used in HTML `<img>` tags or saved as files. All API calls use proper async/await patterns for consistent error handling across the library.
114+
115+
## Error Handling
116+
117+
The `imageGen` function includes proper error handling and will throw Error objects with descriptive messages, consistent with the rest of the call-ai library:
118+
119+
```javascript
120+
try {
121+
const result = await imageGen('An impossible prompt', { debug: true });
122+
// Use the result...
123+
} catch (error) {
124+
// Error objects include useful properties
125+
console.error('Image generation failed:', error.message);
126+
127+
// Access additional properties if available
128+
if ('status' in error) console.error('Status code:', error.status);
129+
if ('errorType' in error) console.error('Error type:', error.errorType);
130+
if ('details' in error) console.error('Details:', error.details);
131+
132+
// Handle the error appropriately
133+
}
134+
```
135+
136+
## Helper Functions
137+
138+
### Converting Base64 to File
139+
140+
To save the generated image as a file:
141+
142+
```javascript
143+
function base64ToFile(base64Data, filename = 'generated-image.png', mimeType = 'image/png') {
144+
// Remove data URL prefix if present
145+
const base64Content = base64Data.includes('base64,')
146+
? base64Data.split('base64,')[1]
147+
: base64Data;
148+
149+
// Convert base64 to binary
150+
const binaryStr = atob(base64Content);
151+
152+
// Create array buffer
153+
const bytes = new Uint8Array(binaryStr.length);
154+
for (let i = 0; i < binaryStr.length; i++) {
155+
bytes[i] = binaryStr.charCodeAt(i);
156+
}
157+
158+
// Create blob and file
159+
const blob = new Blob([bytes], { type: mimeType });
160+
return new File([blob], filename, { type: mimeType });
161+
}
162+
163+
// Usage
164+
const imageFile = base64ToFile(imageResponse.data[0].b64_json, 'my-image.png');
165+
```
166+
167+
## Browser Compatibility
168+
169+
This functionality is designed to work in modern browsers and requires:
170+
171+
- `fetch` API
172+
- `FormData` API
173+
- `File` and `Blob` APIs
174+
- Support for async/await
175+
176+
These are available in all modern browsers without polyfills.
177+
178+
## TypeScript Support
179+
180+
The `imageGen` function is fully typed with TypeScript interfaces:
181+
182+
```typescript
183+
// Input options interface
184+
interface ImageGenOptions {
185+
apiKey?: string;
186+
model?: string;
187+
images?: File[];
188+
size?: string;
189+
quality?: string;
190+
style?: string;
191+
debug?: boolean;
192+
}
193+
194+
// Response interface
195+
interface ImageResponse {
196+
created: number;
197+
data: {
198+
b64_json: string;
199+
url?: string;
200+
revised_prompt?: string;
201+
}[];
202+
}
203+
204+
// Function signature
205+
declare function imageGen(prompt: string, options?: ImageGenOptions): Promise<ImageResponse>;
206+
```

‎notes/imggen-llms.txt

Lines changed: 95 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,95 @@
1+
# ImgGen Component
2+
3+
## Generate from a prompt
4+
5+
Generate images with AI by suppplying a prompt.
6+
7+
```jsx
8+
import { useFireproof } from 'use-fireproof';
9+
import { ImgGen } from 'use-vibes';
10+
11+
function MyComponent() {
12+
const { database } = useFireproof("my-db-name");
13+
return (
14+
<div>
15+
<ImgGen prompt="A sunset over mountains" database={database} />
16+
</div>
17+
);
18+
}
19+
```
20+
21+
This will generate an image and display it in the component. It also provides a progress bar and the ability to edit the prompt or regenerate the image.
22+
23+
Prompt and image versions are automatically tracked in the Fireproof database with a `type` of `image`. If a database is not provided, the component uses the database named `"ImgGen"`.
24+
25+
## List by ID
26+
27+
Display stored images by their ID. Ensure you do this, so users can find the images they created.
28+
29+
```jsx
30+
import { useFireproof } from 'use-fireproof';
31+
import { ImgGen } from 'use-vibes';
32+
33+
function MyComponent() {
34+
const { database, useLiveQuery } = useFireproof("my-db-name");
35+
const { docs: imageDocuments } = useLiveQuery('type', {
36+
key: 'image',
37+
descending: true,
38+
});
39+
40+
return (
41+
<div>
42+
<ImgGen prompt="A sunset over mountains" database={database} />
43+
{imageDocuments.length > 0 && (
44+
<div className="history">
45+
<h3>Previously Generated Images</h3>
46+
<ul className="image-list">
47+
{imageDocuments.map((doc) => (
48+
<li key={doc._id} className="image-item">
49+
<ImgGen _id={doc._id} />
50+
</li>
51+
))}
52+
</ul>
53+
</div>
54+
)}
55+
</div>
56+
);
57+
}
58+
```
59+
60+
## Styling
61+
62+
ImgGen supports custom styling through CSS variables or custom class names:
63+
64+
```jsx
65+
// With CSS variables in your styles
66+
:root {
67+
--imggen-text-color: #222;
68+
--imggen-accent: #0088ff;
69+
--imggen-border-radius: 8px;
70+
}
71+
72+
// With custom class names
73+
<ImgGen
74+
prompt="A landscape"
75+
className="my-custom-image"
76+
classes={{
77+
root: 'custom-container',
78+
image: 'custom-img',
79+
overlay: 'custom-overlay'
80+
}}
81+
/>
82+
```
83+
84+
#### Props
85+
86+
- `prompt`: Text prompt for image generation (required unless `_id` is provided)
87+
- `_id`: Document ID to load a specific image instead of generating a new one
88+
- `database`: Database name or instance to use for storing images (default: `'ImgGen'`)- `options` (object, optional): Configuration options for image generation
89+
- `model` (string, optional): Model to use for image generation, defaults to 'gpt-image-1'
90+
- `size` (string, optional): Size of the generated image (Must be one of 1024x1024, 1536x1024 (landscape), 1024x1536 (portrait), or 'auto' (default value) for gpt-image-1, and one of 256x256, 512x512, or 1024x1024 for dall-e-2.)
91+
- `quality` (string, optional): Quality of the generated image (high, medium and low are only supported for gpt-image-1. dall-e-2 only supports standard quality. Defaults to auto.)
92+
- `debug` (boolean, optional): Enable debug logging, defaults to false
93+
- `onLoad`: Callback when image load completes successfully
94+
- `onError`: Callback when image load fails, receives the error as parameter
95+
- `className`: CSS class name for the image element (optional)- `classes`: Object containing custom CSS classes for styling component parts (see Styling section)

0 commit comments

Comments
 (0)