Skip to content

Commit ffc32ce

Browse files
refactor: simplify infinite scroll config by replacing scroll_options with number_of_scrolls parameter
1 parent dc95174 commit ffc32ce

File tree

1 file changed

+1
-6
lines changed

1 file changed

+1
-6
lines changed

scrapegraph-py/examples/async/async_smartscraper_infinite_scroll_example.py

Lines changed: 1 addition & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -14,12 +14,7 @@ async def scrape_companies(client: AsyncClient, url: str, batch: str) -> None:
1414
response = await client.smartscraper(
1515
website_url=url,
1616
user_prompt="Extract all company information from this page, including name, description, and website",
17-
infinite_scroll=True,
18-
scroll_options={
19-
"max_scrolls": 10, # Adjust based on page size
20-
"scroll_delay": 2, # Seconds between scrolls
21-
"scroll_to_bottom": True
22-
}
17+
number_of_scrolls=10,
2318
)
2419

2520
# Process the results

0 commit comments

Comments
 (0)