One of the critical components of configuring the Splunk Enterprise Security Suite involves building lookup tables for identities. This post will provide an overview of the steps required to get started with automatic identity lookup building using LDAP or Active Directory. This is by no means a comprehensive document of all possible configurations and options, but it should provide enough information to get you going in the right direction.
More often than not, you will already have much of this information already existing in another directory, such as Active Directory or LDAP. Fortunately, Splunk can be configured to automatically pull data from these sources and build the appropriate lookup tables for Enterprise Security.
Step 1: Configure the Splunk Supporting Add-on for Active Directory (SA-ldapsearch) to query your LDAP/Active Directory environment
Don’t be fooled by the name - just because this app is called the Splunk Support for Active Directory doesn’t mean that Active Directory is required. This app can be used to query any LDAP server. Download and install the Splunk Support for Active Directory app.
The following instructions were written for version 1.x of the app, which requires Java to function. The configuration should be similar on version 2.x, which no longer has the Java requirement.
As part of the app setup, configure ldap.conf for the domains you want to query:
#Domain FQDN [corp.domain.com] #LDAP server server = dc01.corp.domain.com #Can be set to 389 for non-SSL LDAP port = 636 #SSL should be set on Windows Server 2008 and above ssl = true #base OU that all objects will be in basedn = DC=corp,DC=domain,DC=com #base OU that all objects will be in binddn = email@example.com #password is in plaintext by default password = <password> #domain NETBIOS name alternatedomain = DOMAIN #Example for anonymous bind [ldap.domain.com] server = ldap.domain.com port = 389 ssl = false basedn = ou=users,dc=domain,dc=com #If anonoymous, leave binddn and password fields blank binddn = password = aternatedomain = LDAP #Required to be configured as a fallback [default] server = dc01.corp.domain.com port = 636 ssl = true
Step 2: Use ldapsearch to generate the lookup table
These instructions are based on the steps found in the Splunk Enterprise Security Installation manual.
Substitute the domain (configured in ldap.conf) in the search as follows:
|ldapsearch domain=corp.domain.com search="(&(objectclass=user)(!(objectClass=computer)))" attrs="sAMAccountName,personalTitle,displayName,givenName,sn,mail,telephoneNumber,mobile,manager,department,whenCreated,userAccountControl" |makemv userAccountControl |search userAccountControl="NORMAL_ACCOUNT" |eval suffix="" |eval priority="medium" |eval category="normal" |eval watchlist="false" |eval endDate="" |table sAMAccountName,personalTitle,displayName,givenName,sn,suffix,mail,telephoneNumber,mobile,manager,priority,department,category,watchlist,whenCreated,endDate |rename sAMAccountName as identity, personalTitle as prefix, displayName as nick, givenName as first, sn as last, mail as email, telephoneNumber as phone, mobile as phone2, manager as managedBy, department as bunit, whenCreated as startDate
If referencing the Splunk documentation, leave the outputlookup command off since this lookup doesn’t exist yet
If you are using an LDAP directory that is not Active Directory, this can still be queried using a similar method, but the table will be using different values depending on the LDAP structure.
Step 3: Create the lookup table file
The base lookup table file can be created by uploading a skeleton file through SplunkWeb. You will want to make sure that the destination app is the SplunkEnterpriseSecuritySuite and that the sharing permissions are set to Global (object should appear in all Apps).
The header of the CSV should be as follows:
Step 4: Create the lookup table definition
The lookup table definition will be used in a saved search to generate the lookup table. The destination app for this lookup table definition should again be the SplunkEnterpriseSecuritySuite. The name can be whatever you would like the lookup definition to be called (ad_identities generally works fine for Active Directory). The type should be file-based, and the lookup file should be set to the CSV you uploaded in step 3
You will want to make sure that the sharing permissions are set to Global on this as well.
Step 5: Create the lookup table and verify that it is updated on the filesystem
Run the same search as Step 2, but add the | outputlookup <lookup_name> command to the search. This will output the results to the lookup table specified.
Verify that this is functioning properly by checking the file on the filesystem, which should be $SPLUNK_HOME/etc/apps/SplunkEnterpriseSecuritySuite/lookups if you have followed the steps up to this point.
Step 6: save the search as a report
This is assuming that the lookup table is being generated properly. Also, this can be scheduled at a later point in time to automatically update the lookup table.
Step 7: Configure Identities in ES: Navigate to App: Enterprise Security -> Configuration -> Data Enrichment -> Identity Management
Create a new identity, being careful to identify the type as “identity” and the source to the lookup definition you created earlier, which will be lookup://<lookup_name>. This should be enabled by default once it’s created, but verify that this is the case in the web interface.
Step 8: create the merged identity file
Before you create the merged identity file, wait approximately 5 minutes for Splunk to automatically detect the change in the identity configuration. The following search will show you the status: index=_internal source=*python_modular_input.log *identit*
Step 9: Verify that the identities_expanded.csv file was updated on the filesystem of the search head
This should be in $SPLUNK_HOME/etc/apps/SA-IdentityManagement/lookups by default
Step 10: Schedule the report you created earlier to run on a semi-regular basis
This is being done assuming all is working as expected. Also, as a final note, don't forget to disable the sample identities that are enabled by default Enterprise Security.
Original Post Date: March 19, 2015 by Tom Kopchak Updated: March 8, 2016 by Ryan O’Connor