Skip to content

Add Option to Select Number of Websites to Scrape in FastAPI Frontend #1504

@keshav861

Description

@keshav861

Is your feature request related to a problem? Please describe.
Currently, the FastAPI frontend does not provide a way for users to control how many websites should be scraped during the research process. The scraping is handled automatically in the backend with no user input, which can sometimes result in either too much information or not enough, depending on the research context.

Describe the solution you'd like
I would like a feature added to the FastAPI frontend that allows users to select the number of websites to be scraped before starting the research. This could be implemented as:

A dropdown or numeric input field for selecting the number of websites (e.g., 3, 5, 10).

A configurable parameter that is passed to the backend when initiating the research task.

Clear integration into the existing research workflow so users can easily customize the scraping depth.

Describe alternatives you've considered

Manually editing backend configuration, but this is not practical or user-friendly.

Relying on the default scraping logic, but this may lead to inconsistent or undesired levels of information.

Additional context
This feature would make the FastAPI frontend more flexible and user-centric by giving control over how much data is collected. It ensures better alignment with diverse use cases—whether a user wants a quick, high-level overview (fewer sites) or a more in-depth, comprehensive analysis (more sites).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions