How Intelligence Sources are Updated

Updated 2 weeks ago by Elvis Hovor

External Intelligence Sources are classified by the way their information are updated:

  • Feed-based: Automatic polling of the source provider for new updates
  • Query-based: Submitting a new report and triggering queries to the source provider.

Feed-based Sources

An intelligence source that is feed-based has its enclave automatically and regularly updated by TruSTAR. Think of a feed-based source as similar to a news feed; all the information is streamed from the source provider (for example, Alienvault OTX Pulse) into an enclave without any need for you to request updates.

Reports in a feed-based enclave can focus on a single observable but they usually include multiple observables, their relationships to each other, and their relationships to security events or malware or threat-actors.

How It Works

When you submit a new report to a private enclave, TruSTAR extracts all observables and checks all feed-based enclaves available to you. The information from those enclaves is shown as nodes within an event analysis so that you can easily explore correlations between your own data and the subscribed feeds. You can click on any data point to reveal additional context and links directly to the associated report in a specific enclave.

Updating the Enclave

TruSTAR queries the partner's data source on a regular basis and updates the enclave with that information. The update interval can be anywhere from 10 minutes to 2 hours to 24 hours, based on how often the partner updates the source data at their end.

Query-Based Sources

An intelligence source that is query-based is only updated when a new report is submitted to that enclave. TruSTAR extracts the observables from the report and then requests enrichment from the enclave's provider. Information from the source is then added to the enclave and to the report itself.

Query-based source reports usually focus on a single observable and that observable is usually included in the title of the report. A report may contain multiple observables in the report body, usually to provide context about the relationship of those observables to the title (or main) observable.

How It Works

When you submit a new report, TruSTAR extracts the observables in that report. Those observables are then sent as queries to the partner and the results stored in the enclaves for that intelligence source. For example, if you subscribe to both VirusTotal and AlienVault , then observables from a new report are sent to VirusTotal and Alienvault for enrichment. The information VirusTotal sends back is stored in your VirusTotal (premium source) enclave and the AlienVault information is stored in your AlienVault (premium source) enclave.

The process of extracting the observables from a new report and querying sources can take 15-20 minutes. It can take up to 70 additional minutes for that enrichment to be available to workflow application integrations such as TruSTAR's integrations with Splunk Enterprise Security, IBM Resilient, and ServiceNow. This is why those integrations' documentation will mention that best practice is to enrich a Splunk ES notable event 90 minutes after it was initially sent to the user's Splunk ES Notable Events enclave, or re-enrich a Jira / ServiceNow ticket 90 minutes after it was initially created.

Updating the Enclave

Query-based enclaves are not automatically updated with new information from sources. Data is only added to these enclaves when a new report is submitted to the private enclave, observables found by Station in that report, sources queried for enrichment, and the sources' responses stored in their enclaves in Station. Note: if a query-based source does not have any information about a particular observable, no report will be created about that observable in the source's enclave. This is sometimes interpreted by the user as Station failing to fetch (query) information from the source about the observable; however, reality is that the source didn't have any information about that data.


How Did We Do?