Configuration

Progrid AI Lead Research provides several configuration options that control caching behavior, rate limiting, and default values for research jobs. All settings are accessible from CRM ‣ AI Research ‣ Configuration ‣ Settings.

Important

Configuration settings require the AI Research Manager security group. See Security for details.

General settings

Navigate to CRM ‣ AI Research ‣ Configuration ‣ Settings to access all module configuration options.

Default provider

The Default Search Provider field sets which search provider is pre-selected when creating new research jobs through the wizard. Users can override this selection per job.

Options:

  • Brave – Default to Brave Search

  • Tavily – Default to Tavily Search

  • Mixed – Default to using both providers

Default max results

The Default Max Results field controls the default value for the Max Results field in the research job wizard. This determines how many search results are processed per query variation.

  • Recommended value: 10

  • Range: 1-50

  • Impact: Higher values consume more API credits but may discover more businesses

Default deliverables

These checkboxes set the default deliverable options for new research jobs:

  • Default Create Leads – Pre-check the Create Leads option in the wizard

  • Default Enrich Partners – Pre-check the Enrich Partners option

  • Default Export CSV – Pre-check the Export CSV option

Cache settings

The content cache (Progrid.fetch.cache) stores downloaded web page content to avoid redundant fetches. This reduces API costs and speeds up jobs that target overlapping URL sets.

Cache duration

The Cache Duration (Days) field controls how long cached content remains valid before being considered stale and re-fetched.

  • Default: 7 days

  • Recommended range: 3-14 days

  • Setting to 0: Disables caching entirely (every URL is fetched fresh every time)

Note

Cache entries use SHA256 hashing of the URL to create a unique key. This means the exact same URL always maps to the same cache entry, regardless of which research job fetched it.

Cache management

Navigate to CRM ‣ AI Research ‣ Configuration ‣ Cache Management to view and manage cached content.

From this view you can:

  • View cache statistics – See total cached entries, storage size, and cache hit rate

  • Clear expired entries – Remove entries older than the configured cache duration

  • Clear all cache – Remove all cached content (useful when you want to force fresh fetches)

  • Search cache entries – Find specific cached URLs to inspect their content

Warning

Clearing the entire cache means all subsequent research jobs will need to re-fetch every URL, which may significantly increase processing time and API usage.

Rate limiting

Rate limiting controls prevent the module from exceeding provider API limits, which could result in temporary blocks or additional charges.

Groq rate limits

Setting

Default

Description

Groq Requests Per Minute

30

Maximum LLM API calls per minute. Applies to both normalize and score phases.

Groq Daily Request Limit

14,400

Maximum LLM API calls per day across all research jobs.

Search provider rate limits

Setting

Default

Description

Brave Requests Per Second

1

Maximum Brave Search API calls per second.

Tavily Requests Per Second

1

Maximum Tavily API calls per second.

Tip

If you are on a paid plan with higher rate limits, you can increase these values to speed up research job processing. However, do not exceed your plan’s actual limits or you may receive HTTP 429 (Too Many Requests) errors.

Fetch settings

Setting

Default

Description

Fetch Timeout (Seconds)

30

How long to wait for a web page to respond before skipping it.

Max Concurrent Fetches

5

How many URLs can be downloaded simultaneously during the fetch phase.

System parameters

Advanced configuration options are available as system parameters (ir.config_parameter records) for administrators who need fine-grained control. These are accessible from CRM ‣ AI Research ‣ Advanced ‣ System Parameters.

Warning

System parameters are intended for advanced users. Incorrect values can cause research jobs to fail or produce poor results. Only modify these if you understand their impact.

Common system parameters include:

Parameter

Description

ai_research.max_query_variations

Maximum number of search query variations generated per job (default: 5)

ai_research.min_confidence_for_lead

Minimum confidence score for automatic lead creation (default: 0.5, meaning 50%)

ai_research.fetch_user_agent

The User-Agent string used when fetching web pages

ai_research.groq_temperature

LLM temperature setting for Groq API calls (default: 0.1, low for consistency)

ai_research.groq_max_tokens

Maximum token count for LLM responses