Install: TruSTAR Unified App for Splunk Enterprise & Enterprise Security

Updated 1 week ago by TruSTAR

This article explains how to install and configure the TruSTAR Unified Workflow App for Splunk Enterprise and Enterprise Security (ES). Installing typically takes anywhere from 15-30 minutes, depending on your Splunk environment and the number of TruSTAR Enclaves you specify during configuration.

This App is designed to work together with TruSTAR Intel Workflows that specify Splunk as a destination for the workflow.

TruSTAR Docs

Splunk Docs

Splunk Dev Docs

Splunk API Docs

Use Cases.

  • Depending on which version of Splunk you are using, different use cases are available.
  • This table shows the relationship between this app's features and the TruSTAR use-case those features serve / support.

Application

TruSTAR Use-case

Associated Features

Splunk Enterprise (core) + ES

Detect

Modinput: TruSTAR Observables -> Kvstores

  • Download Indicators from TruSTAR Enclaves into Splunk KV Stores for use in searching or alerts.
  • Config enables user to specify Indicator types, tags, intelligence sources, age, and kvstore group relevant to user's organization.
  • Remove modinput's indicators when added to TruSTAR Company Safelist
  • automatically cleanup indicators downloaded by old, deleted modinputs.

ES (only)

Triage

Adaptive Response Action: Enrich

  • Prioritize Threat Activity Detected Notable Events (NE) by adjusting their Urgency scores according to TruSTAR's normalization of scores provided by intel sources about that indicator.
  • Enrichment comments provide deeper understanding of key tags, attributes, properties user's intel sources have associated with the NE's observable.

Core + ES

Disseminate

Adaptive Response Action: Submit

  • Submit saved-search result events to TruSTAR as a TruSTAR Report.
  • Send to a TruSTAR enclave shared with other groups (Ex: ISAC/ISAO, other teams in your company).
  • Optional: redact terms sent to TruSTAR.

Requirements - Splunk deployment.

Software

  • Splunk Enterprise version 8.0 or higher
    • Splunk 7+ is supported up to TruSTAR Unified 2.0.3
    • TruSTAR Unified 2.0.4 deprecates interoperability with Splunk 7 and Python 2.
  • Optional: Splunk Enterprise Security version 6.0 or higher. This also requires:

Data

  • CIM-compliant logs from relevant sources (ex: Proxy, Firewall, Antivirus, SSH logs, etc.)

User Accounts

You must have two Splunk user accounts on each search head:

  • admin required for installation
  • power recommended for Splunk Enterprise users
  • ess_admin (or greater) required for Splunk ES users
    • Must have the "modify_notable_event" permission to be able to manually run the "Enrich" adaptive response action.

For more information, see Check Splunk User Account Permissions in the FAQ.

Networking

  • Modinput & Modaction Code is started & executed by the Splunk application on search-heads.
  • The search head will need to be able to make REST API calls to its Splunk application on localhost, port 8089.
  • The search heads' Splunk application daemon must accept incoming traffic on port 8089, from "localhost" / loopback.
  • Modinput & modactions all make REST calls to TruSTAR's API. Search-head will need to be able to call to TruSTAR (https://api.trustar.co) through port 80.

Logs-to-CIM Mapping

For Splunk ES to generate notable events when it finds a TruSTAR Indicator in logs, those logs must be mapped to the Splunk Common Information Model. Splunk Professional Services can assist with this process. For more information see the Splunk ES CIM Overview.

Special Requirements.

SplunkCloud users

  • SplunkCloud has a policy that modinputs not be allowed to run on search heads.
  • This app has been granted an exception to the policy, but SplunkCloud personnel who install this app for customers often are not aware of that exception.
  • Notify TruSTAR TAM and send a support request to support@trustar.co if you encounter this.
  • Technical details regarding why this app must run on search heads (not heavy forwarders / IDMS) are explained on the app's Splunkbase page

Distributed Splunk Deployments

  • Many Splunk ES knowledge objects reside on search-heads only.
  • Correlation, threat-gen, and lookup-gen searches must process entirely on search-heads and cannot be distributed to an indexer or indexer cluster.
  • This app's modinput is intended to run on the search head, and will add cyber threat observables to kvstores on that searchhead.
  • Most likely, any searches you write that use the observables added to the kvstores by this app will need to process entirely on the searchhead.

For assistance, please contact Splunk Professional Services.

Related Link: Splunk Distributed Search documentation.

Search-head Clusters

Lots. see next table.

Search-Head Clusters:

Concept

Description

Related Links

Config File Replication

  • The TruSTAR Unified App ships with a custom server.conf file that should enable your cluster to replicate the necessary config files to all nodes in the cluster.
  • In some cases, your Splunk Administrator may need to manually copy its (server.conf) contents to a different server.conf file.

KV Store Replication

  • Downloading observables from TruSTAR to search-head can occur only on the cluster Captain node.
  • The SHC must be configured to replicate observables added to the Captain's kvstores to all nodes in the cluster.
  • Cluster Captain should also be the KV Store Captain.

Sticky Sessions

  • If a user's interaction with your search heads passes through a proxy or load balancer, you must enable sticky sessions. This is a Splunk requirement and ensures a consistent user experience.

passwords.conf files

  • Do not copy passwords.conf files from one Splunk host to another.
  • TruSTAR Unified does not ship with one, but other apps do.

Deployer

  • you should be able to use your deployer to distribute the TruSTAR app to SHC nodes.
  • ** caution: any Splunk app built using the Splunk Addon Builder will fail if your deployer distributes a "passwords.conf" file to the SHC nodes.
  • Do not configure the TruSTAR app on your deployer node and push the configs to the SHC nodes - you'll encounter the passwords.conf problem.

Requirements - TruSTAR account.

Enclaves

Enclave

Purpose

How Create?

Splunk Threat Activity

  • Submit ARA will send all events / notable events that match your threatintel-related correlation searches to this enclave for historical archiving, which enables TruSTAR to enrich the observables in that event.

TruSTAR Account Manager will create for you.

Workflow destination enclaves

  • Ideally each modinput will connect to and download from a single workflow destination enclave only.

Create Prioritized Indicator Intel Workflows.

Intel Source Enclaves

  • Prioritized Indicator Intel Workflows select subsets of indicators from your intel source enclaves and deposit them in Workflow Destination Enclaves.

Subscribe to intel sources.

Sharing Group Enclaves

  • share splunk events / alerts / notable events with sharing group using Submit ARA.

TruSTAR Account Manager will permission you to these enclaves.

User Accounts.

Your TruSTAR account manager will create the user accounts required for the TruSTAR Unified App, using "dummy" email accounts for each one. When you view these accounts in the TruSTAR Web App, you will see these names in the left column, the permissions in the right column, and the dummy email addresses.

If you are using Splunk Enterprise, your account manager will create only Download and Submit accounts. If you are using Splunk ES, your account manager will create all three of the accounts listed below.

Account Name

Enclave + Permissions

Download

(will be given during prioritized indicators intel workflow creation)

All Enclaves - view

Submit

All Enclaves - view

Splunk Threat Activity Enclave - full

Enrich

All Enclaves - view

This account is only for Splunk ES users.

Upgrading from "TruSTAR App for ES" to "TruSTAR Unified":

  1. De-configure Adaptive Response Actions.
  • TruSTAR Unified has the same Adaptive Response Actions that the TruSTAR App for ES did.
    • all correlation searches configured to use TruSTAR App for ES's ARAs will need those configs removed.
      • To find those searches, run this Spl search:
        To find the searches that need to be de-configured & re-configured, run this SPL query:

        | rest splunk_server=local count=0 /servicesNS/-/SplunkEnterpriseSecuritySuite/saved/searches

        | where match('action.correlationsearch.enabled', "1|[Tt]|[Tt][Rr][Uu][Ee]")

        | table title actions

        | where match('actions', "trustar")

  1. Note your modinput configs.
  • You will have to re-configure your modinputs. Make note of your configs.
  1. Delete TruSTAR App for ES entirely.
  2. Follow the "Installing TruSTAR Unified" section's instructions to install the new app.
  3. Re-configure correlation searches to use TruSTAR Unified's ARAs.

Installing TruSTAR Unified.

The TruSTAR App must be installed on search-heads. Do not install the App on indexers or heavy-forwarders.
  1. Select Apps -> Manage Apps from the Splunk main menu bar.
  2. Click the Browse More Apps button, then use the Search box to find the TruSTAR Unifed App.
    SplunkES_Install_Figure4
  3. Proceed with the installation of the TruSTAR App.

Configuring TruSTAR Unified.

The Configuration Options is where you enter API credentials and set up proxy server, logging, and other details for the TruSTAR App. You can also configure automatic submission of Notable Events to TruSTAR (Splunk ES only).

  1. Choose TruSTAR Unified App from the App pull-down menu on the top-level Splunk menu.
  2. Click Configuration on the blue submenu.

Account Settings.

The Account Settings tab sets up the API credentials for the integration.

You will now set up user accounts that match the TruSTAR accounts created for you by TruSTAR.

  1. For the first account, enter the Account name as DOWNLOAD.
  2. Enter the TruSTAR API Key for the Intel Workflow in the TruSTAR API Key field.
  3. Enter the TruSTAR API Secret for the Intel Workflow in the TruSTAR API Secret field.
  4. Click Add to save the account.
  5. For the second account, enter the Account name as SUBMIT.
  6. Enter the TruSTAR API Key for the Submit account in the TruSTAR API Key field.
  7. Enter the TruSTAR API Secret for the Submit account in the TruSTAR API Secret field.

If you are using Splunk ES, you need to create a third user account for enrichment.

  1. For the third account, enter the Account name as ENRICH.
  2. Enter the TruSTAR API Key for the Enrich account in the TruSTAR API Key field.
  3. Enter the TruSTAR API Secret for the Enrich account in the TruSTAR API Secret field.

Proxy Settings.

If your installation uses a proxy between search head(s) and the TruSTAR platform, you need to configure the proxy information as shown below.

SplunkES_Install_Figure8

Logging.

You can choose one of five logging levels: Debug, Info, Warning, Error, and Critical.

TruSTAR recommends leaving the level at Info (the default) unless instructed by TruSTAR Support.

Add-On Settings.

The Add-on Settings specify which Enclaves to use for submissions and for enrichment.

Explanation of Settings

  • Default Submit Enclave: The Enclave ID for Splunk Threat Activity.
  • Default Enrich Enclaves: The Enclaves to use when enriching events.
    • TruSTAR recommends: ALL
    • Alternatively, you can enter a comma-separated list of Enclave IDs.
You can override the default Enclave settings when running individual enrichment or submission actions.

Configure Notable Event Auto-Submission.

You can configure the TruSTAR App to automatically submit Notable Events. You can manually submit events, using the procedure described in the User Guide for the App.

This feature is only available with Splunk ES.
  1. Choose Enterprise Security from the App pull-down menu on the top-level Splunk menu.
  2. Click the Configure menu, then click the Content menu, and then click Content Management.
    SplunkES_Install_Figure15
  3. Search for "threat activity detected" and then click the correlation search named Threat Activity Detected. This opens a configuration window for that search.
    SplunkES_Install_Figure17
  4. In the configuration window, scroll to the Adaptive Response Actions section, then click the carat next to Notable.
    SplunkES_Install_Figure19
  5. In the Next Steps text box, add these lines, separated by 2 newline characters:
[[action|trustar_submit_event]]


[[action|trustar_enrich_threat_activity]]
  1. In the Recommended Actions section, select TruSTAR - Enrichment and TruSTAR - Submit. The configuration should now look like this example:
    SplunkES_Install_Figure20
  2. Go back to the Adaptive Response Actions section and choose Add New Response Action, then select the TruSTAR - Submit action.
  3. Configure the Submit action by adding a Report Title and any comments you want to add to the event. The configuration should now look like this:
    SplunkES_Install_Figure22
  4. Click the green Save button in the lower right corner to complete the configuration.

Configuring Notable Event Auto-Enrichment. (ES only).

  1. Choose Enterprise Security from the App pull-down menu on the top-level Splunk menu.
  2. Click Settings, then click Searches, reports, and alerts.
  3. Click New Alert.
  4. In the Create Alert window, use these settings:

Setting

Value

Title

TruSTAR Enrichment

Search

`notable` | search rule_title="*Threat Activity Detected*"

Permissions

Shared in App

Alert Type

Scheduled (recommend every 4 hours)

Trigger Alert When

Number of Results

is greater than

Trigger

For each result

  • Scroll to the Adaptive Response Actions section and click +Add Actions.
  1. In the list, select TruSTAR - Enrichment.
  2. Optionally, you can click the Custom Radio button and enter a Custom Enclave ID to send the results to a different Enclave than your default Enclave. You can also enable or disable the Urgency Adjustment for the search.
  3. Click Save in the bottom right corner to save the correlation search.

Configuring Inputs.

To help in detection, you can configure inputs to copy Indicators from TruSTAR Enclaves into Splunk KV Stores.

Creating an Input

  1. Choose TruSTAR Unified from the App pull-down menu on the top-level Splunk menu.
  2. Click Inputs on the blue submenu.
  3. Click Create New Input to start defining an input source.
  4. Fill out the configuration options as shown in this table.

Field

Value

Notes

Name

Name of the input

A unique input name. Valid characters are letters and underscores only. You cannot use spaces or special characters.

Destination KV Store Group

TruSTAR

or

Splunk ES Threatintel

Tells the app which KV Stores to copy observables to.

Global Account

DOWNLOAD

The account you created during the installation process.

Enclave IDs

Workflow Enclaves to download from

You must enter the Workflow Enclave ID. TruSTAR recommends configuring one mod input per Workflow. To specify multiple Enclave IDs, separate them with commas and no spaces. Finding Enclave IDs

IOC Types

Indicator Types

If you used Workflow Enclave ID, this is not needed because this filtering is configured during the creation of the Workflow.

The Indicator types you want to download from TruSTAR. The default is to include all Indicators supported by Splunk (Email Address, IP, MD5, SHA1, SHA256, Software, Registry Key, URL).

Click x on an Indicator to remove it.

Tags

List of Indicator tags

This list is used to filter Indicators when downloading from TruSTAR. Lowercase characters only.

The input will only download Indicators that match all other criteria (Enclaves, IOC types, etc.) AND include ALL the tags in the list.

Expiration

Number of days

When an Indicator has not been mentioned in any of the Enclaves this input downloads from in the specified number of days, that Indicator will no longer be detected on.

  1. Click Add to save these settings and create the input.

Examples of Input Configuration

When setting up the TruSTAR App, you can configure which types of Indicators to pull from which Enclaves. For example, you may want to only pull IPs from one Enclave and email addresses from another enclave. You can edit these inputs at any time by changing your configuration.

To help you understand the power of granular inputs, here are five inputs for a fictional company called Acme Corp.

Input utilizing Workflows

Acme user has setup a workflow to prioritize indicators. The workflow takes care of the sources, score filtering, IOC filtering and whitelisting.

Input Name

Enclave IDs

Indicator Types

Expiration

Prioritized_Indicators

<workflow Enclave ID>

All

Value depends on what type of IOCs the workflow is centered around

ex: 180 for hashes

7 for IPs

Input 1

Acme wants to watch for any Indicators that they have already investigated and determined are malicious. Acme stores these Indicators in a Vetted Indicators Enclave in TruSTAR.

Input Name

Enclave IDs

Indicator Types

Expiration

Vetted_Indicators

<vetted indicators Enclave ID>

All

360 days

Input 2

Acme is extremely concerned about file hashes reported on by Intelligence-X. They want to constrain this input to file hashes only, and only from that one Intelligence Source. 

Input Name

Enclave IDs

Indicator Types

Expiration

Intel-X_Source

<Intelligence-X Enclave ID>

SHA1, SHA256, MD5

180 days

Input 3

Acme wants to alert on IP addresses reported on by Intelligence Sources A,B and C, but only if the reporting is timestamped within the last 7 days. To do this, they configure an input that downloads IP addresses from Enclaves A,B, and C and retains that data for 7 days. 

Input Name

Enclave IDs

Indicator Types

Expiration

Malicious_IPs

<EnclaveA_ID, EnclaveB_ID, EnclaveC_ID>

IP

7 days

Input 4

Acme is a member of a sharing group named CyberSleuths. Acme wants to copy all Indicators from that sharing group Enclave and retain them for 90 days.  

Input Name

Enclave IDs

Indicator Types

Expiration

CyberSleuth_Intel

<CyberSleuthEnclave_ID>

All

90 days

Input 5

Acme Corporation runs a script that copies TruSTAR Reports and Indicators that meet certain criteria to an Enclave named ACME_CURATED that then contains very high-signal data. Acme wants to configure an input that copies all Indicator types from that Enclave and retains them for 180 days. 

Input Name

Enclave IDs

Indicator Types

Expiration

Curated_Intel

<AcmeCuratedEnclave_ID>

All

180 days

More Examples of Inputs.

You can reduce false-positive alerts by exercising fine-grained control over the Indicators that the app brings into the detection set. The table below suggests some inputs filtering Indicators.

Input Name

Enclave

Indicator Types

Tags

Expire

investigated_ip

Investigations

IP

malicious, detection

7

investigated_hash_email

Investigations

Email, MD5, SHA256

malicious, detection

180

investigated_phish_urls

Investigations

URL

malicious, phish

90

investigated_phish_ips

Investigations

IP

malicious, phish

7

isac_vetted_ip

Sharing Group Vetted Indicators

IP

7

isac_vetted_email_hash

Sharing Group Vetted Indicators

Email, MD5, SHA1, SHA256

180

isac_vetted_url

Sharing Group Vetted Indicators

URL

60

premium_sources_ip

- Source A

- Source B

- Source C

IP

7


How Did We Do?