Splunk Enterprise Security: Automatic Identity Lookup Tables

One of the critical components of configuring the Splunk Enterprise Security Suite involves building lookup tables for identities. This post will provide an overview of the steps required to get started with automatic identity lookup building using LDAP or Active Directory.

One of the critical components of configuring the Splunk Enterprise Security Suite involves building lookup tables for identities. This post will provide an overview of the steps required to get started with automatic identity lookup building using LDAP or Active Directory. This is by no means a comprehensive document of all possible configurations and options, but it should provide enough information to get you going in the right direction.

More often than not, you will already have much of this information already existing in another directory, such as Active Directory or LDAP. Fortunately, Splunk can be configured to automatically pull data from these sources and build the appropriate lookup tables for Enterprise Security.

Step 1: Configure the Splunk Supporting Add-on for Active Directory (SA-ldapsearch) to query your LDAP/Active Directory environment

Don’t be fooled by the name - just because this app is called the Splunk Support for Active Directory doesn’t mean that Active Directory is required. This app can be used to query any LDAP server. Download and install the Splunk Support for Active Directory app.

The following instructions were written for version 1.x of the app, which requires Java to function. The configuration should be similar on version 2.x, which no longer has the Java requirement.

As part of the app setup, configure ldap.conf for the domains you want to query:

Sample:

#Domain FQDN
[corp.domain.com]
#LDAP server
server = dc01.corp.domain.com
#Can be set to 389 for non-SSL LDAP
port = 636
#SSL should be set on Windows Server 2008 and above
ssl = true
#base OU that all objects will be in
basedn = DC=corp,DC=domain,DC=com
#base OU that all objects will be in
binddn = username@corp.domain.com
#password is in plaintext by default
password = <password>
#domain NETBIOS name
alternatedomain = DOMAIN

#Example for anonymous bind
[ldap.domain.com]
server = ldap.domain.com
port = 389
ssl = false
basedn = ou=users,dc=domain,dc=com
#If anonoymous, leave binddn and password fields blank
binddn =
password =
aternatedomain = LDAP

#Required to be configured as a fallback
[default]
server = dc01.corp.domain.com
port = 636
ssl = true

Step 2: Use ldapsearch to generate the lookup table

These instructions are based on the steps found in the Splunk Enterprise Security Installation manual.

Substitute the domain (configured in ldap.conf) in the search as follows:

|ldapsearch domain=corp.domain.com 
search="(&(objectclass=user)(!(objectClass=computer)))"
attrs="sAMAccountName,personalTitle,displayName,givenName,sn,mail,telephoneNumber,mobile,manager,department,whenCreated,userAccountControl"
|makemv userAccountControl
|search userAccountControl="NORMAL_ACCOUNT"
|eval suffix=""
|eval priority="medium"
|eval category="normal"
|eval watchlist="false"
|eval endDate=""
|table
sAMAccountName,personalTitle,displayName,givenName,sn,suffix,mail,telephoneNumber,mobile,manager,priority,department,category,watchlist,whenCreated,endDate
|rename sAMAccountName as identity, personalTitle as prefix, displayName as nick, givenName as first, sn as last, mail as email, telephoneNumber as phone, mobile as phone2, manager as managedBy, department as bunit, whenCreated as startDate

If referencing the Splunk documentation, leave the outputlookup command off since this lookup doesn’t exist yet

|outputlookup <lookup_name>

If you are using an LDAP directory that is not Active Directory, this can still be queried using a similar method, but the table will be using different values depending on the LDAP structure.

Step 3: Create the lookup table file

Step3A 1024X373
Lookup table

The base lookup table file can be created by uploading a skeleton file through SplunkWeb. You will want to make sure that the destination app is the SplunkEnterpriseSecuritySuite and that the sharing permissions are set to Global (object should appear in all Apps).

Step3B 1024X349
​Creating the lookup table file (uploading a skeleton file)

The header of the CSV should be as follows:

identity,prefix,nick,first,last,suffix,email,phone,phone2,managedBy,priority,bunit,category,watchlist,startDate,endDate
Step3C 1024X501
Setting the appropriate permissions
Step3D 1024X307
File saved successfully

Step 4: Create the lookup table definition

Step4A 1024X373
The second link is used for configuring lookup definitions

The lookup table definition will be used in a saved search to generate the lookup table. The destination app for this lookup table definition should again be the SplunkEnterpriseSecuritySuite. The name can be whatever you would like the lookup definition to be called (ad_identities generally works fine for Active Directory). The type should be file-based, and the lookup file should be set to the CSV you uploaded in step 3

Step4B 1024X407
Adding a new lookup definition
Step4C 1024X244
Lookup definition created, permissions set to private

You will want to make sure that the sharing permissions are set to Global on this as well.

Step4D 1024X500
Setting permissions
Step4C 1024X244 1
Correct permissions displayed

Step 5: Create the lookup table and verify that it is updated on the filesystem

Run the same search as Step 2, but add the | outputlookup <lookup_name> command to the search. This will output the results to the lookup table specified.

Step5
Splunk search outputting lookup table

Verify that this is functioning properly by checking the file on the filesystem, which should be $SPLUNK_HOME/etc/apps/SplunkEnterpriseSecuritySuite/lookups if you have followed the steps up to this point.

Step 6: save the search as a report

This is assuming that the lookup table is being generated properly. Also, this can be scheduled at a later point in time to automatically update the lookup table.

Step 7: Configure Identities in ES: Navigate to App: Enterprise Security -> Configuration -> Data Enrichment -> Identity Management

Step7A 1024X270
Identity Management Configuration

Create a new identity, being careful to identify the type as “identity” and the source to the lookup definition you created earlier, which will be lookup://<lookup_name>. This should be enabled by default once it’s created, but verify that this is the case in the web interface.

Step7 1024X472
Adding a new identities to the identity manager

Step 8: create the merged identity file

Before you create the merged identity file, wait approximately 5 minutes for Splunk to automatically detect the change in the identity configuration. The following search will show you the status: index=_internal source=*python_modular_input.log *identit*

Step 9: Verify that the identities_expanded.csv file was updated on the filesystem of the search head

This should be in $SPLUNK_HOME/etc/apps/SA-IdentityManagement/lookups by default

Step9
Current modification date/time on identities_expanded.csv

Step 10: Schedule the report you created earlier to run on a semi-regular basis

This is being done assuming all is working as expected. Also, as a final note, don't forget to disable the sample identities that are enabled by default Enterprise Security.

Original Post Date: March 19, 2015 by Tom Kopchak Updated: March 8, 2016 by Ryan O’Connor



Close off Canvas Menu