Talk to Splunk with Amazon Alexa v1.1.1

Overview

This is a Splunk App that enables your Splunk instance for interfacing with Amazon Alexa by way of a custom Alexa skill, thereby provisioning a Natural Language interface for Splunk.

You can then use an Alexa device such as Echo,Tap or Dot to tell or ask Splunk anything you want.

The App also allows you to train your Splunk instance to the conversational vocabulary for your specific use case.

Vision

The ultimate vision I foresee here is a future where you can completely do away with your keyboard, mouse , monitor & login prompt.

Even right now there are use cases where having to look at a monitor or operate an input device are simply counter productive, infeasible or unsafe , such as industrial operating environments.

You should be able to be transparently & dynamically authenticated based on your voice signature and then simply converse with your data like how you would talk to another person... asking questions or requesting to perform some action.This app is a step in the direction of this vision.

Video of this app in action with an Echo device

Video of this app in action with a Dot device

Advanced usage example with complex SPL and Predictive Analytics

Dependencies

Note on Splunk Cloud

Your internet accessible Splunk instance needs to be your own hosted Splunk instance on premise or perhaps in the cloud using a Splunk AMI. This App will not currently work in Splunk Cloud, but should in the next version when a Configuration UI is completed so you don't need to edit config files on the filesystem directly(Which requires SSH access that is not available at the time of writing this).

Setup

Activation Key

You require an activation key to use this App. Visit http://www.baboonbones.com/#activation to obtain a non-expiring key

Generate Your Crypto Assets

Place your crypto assets and Java Keystore file (java-keystore.jks) in the SPLUNK_HOME/etc/../apps/alexa/crypto directory.

Follow the docs here for creating a certificate and private key

Use the following openssl command to create a PKCS #12 archive file from your private key and certificate. Replace the private-key.pem and certificate.pem values shown here with the filenames for your key and certificate. Specify a password for the archive when prompted.

openssl pkcs12 -keypbe PBE-SHA1-3DES \ -certpbe PBE-SHA1-3DES \ -inkey private-key.pem \ -in certificate.pem \ -export \ -out keystore.pkcs12

Use the following keytool command to import the PKCS #12 file into a Java KeyStore, specifying a password for both the destination KeyStore and source PKCS #12 archive:

$JAVA_HOME/bin/keytool -importkeystore \ -destkeystore java-keystore.jks \ -srckeystore keystore.pkcs12 \ -srcstoretype PKCS12

Note , make sure the keystore and the key have the SAME password.

Configuration

In SplunkWeb , browse to Settings -> Data Inputs -> Alexa and create an input stanza. The fields are described in the web interface or you can read SPLUNK_HOME/etc/../apps/alexa/README/inputs.conf.spec

Upon saving this stanza , an HTTPs web server will be spawned to start listening for incoming requests from the Amazon Alexa Cloud Service.

alt text

Firewall

You will need to open your firewall to your internet accessible Splunk instance to accept incoming requests for the HTTPs port 443.

Setting up your Splunk Alexa Skill

The means by which you interface your Alexa device(Echo/Tap/Dot) to Splunk is by registering a custom Alexa Skill with the AWS Alexa Cloud Service.

This App is a web service based implementation of a custom Alexa Skill you can register (it does NOT use AWS Lambdas).

As we want this custom skill to be private and secure to your own usage , you are going to be registering the skill under your own free Developer account.This is in essence 100% functionally equivalent to hosting a private Alexa skill(not currently an Alexa feature offering) rather than a publicly published Alexa skill.

Because this is not a publicly published skill , we don't need to worry about implementing an authentication handler as detailed here, as this skill will be private to your own Alexa device.

alt text

Using Alexa for Business to host this skill privately

1 Dec 2017 UPDATE

The recently announced Alexa for Business offering now allows you to run this skill PRIVATELY so you don't have to use the above workaround by way of deploying the skill under your personal development account. Everything you need to know can be found in the Alexa docs here.

Let's get started

  1. Sign up for your free Developer Account

  2. Register the Splunk skill

Skill Information Tab

Application Id : this is generated for you , but can be then provided when you setup your Splunk App for more security

Name : anything you want ie: My Splunk Server

Invocation name : splunk , this is then used when you talk to your Echo (" Alexa .... ask splunk .....") .Doesn't have to be "splunk" , you can use any name you want.

Endpoint : https://YOURHOST/alexa. The value of YOURHOST should match what you have in the certificate you created and be a resolvable domain name, not an IP Address.

alt text

Interaction model Tab

Samples are in the SPLUNK_HOME/etc/../apps/alexa/alexa_assets directory from the Splunk App you installed. Just copy paste them into the appropriate boxes below. Whenever you add more slots/utterances/intents as you train up your Splunk instance , you will also have to update this interaction model tab.

alt text

SSL Certificate Tab

Select "I will upload a self-signed certificate in X.509 format” Copy paste your certificate.pem file contents(just open in a text editor) that you created in the Crypto instructions above.

alt text

Test Tab

Enable the skill. You should see "This skill is enabled for testing on this account."

alt text

Test that it is all working using the service simulator.

A few things you can ask :

  1. What is splunk
  2. What is the XXXX search command (any command on docs)
  3. What is the max cpu usage of server foo today
  4. What is the average cpu usage of server foo yesterday

alt text

Training your Splunk instance

This App ships with a simple example vocabulary to get you started , but you are soon going to want to extend this by training up your own Splunk instance. So this App is designed around the concept of a training model.

Every user of this App will want to voice interact with their Splunk instance differently , usually based on the domain of data they have indexed and the questions they want to ask that data by way of underlying Splunk searches.

So over time you will train up your Splunk instance to develop a rich conversational vocabulary.

There are 2 parts to training up your vocabulary :

  1. Editing JSON files in the SPLUNK_HOME/etc/../apps/alexa/intents directory.All JSON config files are monitored and dynamically reloaded every 10 secs if they have changed , so there is no need to restart the App or Splunk server when you make changes.

  2. Updating your Splunk Alexa Skill definition in your developer account under the Interaction Model tab with utterances, slots and your intent schema.

For convenience you can keep a copy of your utterances,slots and intent schema in the SPLUNK_HOME/etc/../apps/alexa/alexa_assets directory.

Configuring mapping.json

This JSON file is the heart of the training model. It is where you define the mapping of the incoming intent to some action to perform.

The actions that you can perform are :

Global Search Variables

Global Authentication Variables

By default , the Alexa App will connect to Splunk using the "splunk-system-user". However you can override this if you want to authenticate with another user.

Search actions

Saved Search actions

Search tips

It is a good idea to try and optimize your searches for the Alexa voice environment.

Static response actions

Create your own dynamic actions

You can easily extend the available set of built in actions by creating your own custom dynamic actions and plugging them in , all you need is some simple Java coding skills.

This App ships with an example dynamic action , DocsLookupAction , that responds to an incoming intent request to get information about a Splunk search command. The dynamic action simply makes an HTTP call out to the docs page , scrapes the search command description and returns this back to Alexa.

These are the steps for creating a new Dynamic Action :

  1. Create a class that extends the DynamicAction base class.
  2. Implement the executeAction() method , use the DocsLookupAction example as a guide.You can access slot values and custom arguments from within your code also.
  3. Compile the class , add it to a jar file and place in the SPLUNK_HOME/etc/../apps/alexa/dynamic_actions/lib directory.Also place any other dependent jars for your action in this directory.
  4. Update the SPLUNK_HOME/etc/../apps/alexa/dynamic_actions/dynamicactions.json file to map the class name to some action name that you can refer to from mapping.json

    { "name": "foo_action", "class": "com.foo.FooAction" }

  5. Add a mapping from an incoming intent request to this dynamic action in mapping.json

Response formatting

The JSON response field in mapping.json can be in plain text or SSML.

The response text or SSML can contain tokens to replace from the values of any slots that were passed in the request intent by wrapping the slot key in $ signs ie: $timeperiod$ or $servername$ (except for static responses)

For searches and saved searches the response text or SSML can contain tokens to replace from the results of the searches. These are declared in the format $resultfield_xxx$ , where xxx is the name of the field in the search result.

Dynamic action responses also have a special token $dynamic_response which is some dynamic text that the action returns ie: from the result of an HTTP lookup in the case of the example DocsLookupAction.This token can be used standalone or mixed in with plain text, SSML and slot tokens.

Response Examples :

Configuring timemappings.json

When you are communicating with your Alexa device you are going to use simple noun phrases to express time periods.So we need a way of mapping these human spoken times to Splunk time patterns.

This is accomplished with a combination of custom slots(TIME_PERIOD) and a mapping of the slot value(the noun phrase) in the timemappings.json mapping file.

To add more time noun phrases , you just update the TIME_PERIOD slots list for your Splunk Alexa skill in your Amazon developer console as shown above on the Interaction Model Tab screenshot.

To map these noun phrases you update the timemappings.json file in the Splunk App to map the time noun phrase to an earliest and latest Splunk time pattern. { "utterance": "yesterday", "earliest": "-1d@d", "latest": "@d" }
To use this time period slot , just refer to it in your Utterance definition for your Splunk Alexa skill * MaxCPUIntent what is the maximum cpu usage of server {servername} {timeperiod}

Then in your search and saved search actions in mapping.json you can simply refer to the name of the time slot key in the time_slot field and it will be applied to your search * "time_slot" : "timeperiod"

You can also completely omit any TIME_PERIOD slot in you utterances if you want , and the default time range values will then be determined from what you have configured in your mapping.json file.

Soundbites

You can include MP3 soundbites in your response by embedding the URL of an MP3 file in an SSML formatted response. * <speak>Splunk sounds like <audio src=\"https://www.myhost.com/soundbites/horse.mp3\"/></speak>

This App is able to serve up these MP3 files over HTTPs for you also. All you need to do is to put your MP3 file in the SPLUNK_HOME/etc/../apps/alexa/soundbites directory.

Please refer to these additional guidelines for creating your MP3 soundbite

Example walkthrough for setting up a new Intent

Let's walkthrough setting up a mock scenario where you want to ask Splunk how many error events for a particular error code there were for a particular host today.

  1. Login to your Amazon developer account and browse to the Interaction Model tab for your Splunk skill.
  2. Add a new intent to your intent schema, I'll call it ErrorsIntent and specify 2 slots that it can accept for the name of the host and the time period.

    { "intent": "ErrorsIntent", "slots": [ { "name": "servername", "type": "SERVER_NAME" }, { "name": "errorcode", "type": "AMAZON.NUMBER" }, { "name": "timeperiod", "type": "TIME_PERIOD" } ] }

  3. Add 1 or more Utterances for this intent. In this example I have used a combination of custom slots and built in AMAZON slots.

  4. Update the SERVER_NAME slot type with the name(s) of some hosts.
  5. Save everything. That's all there is to setup in the Alexa cloud.Now let's move over to your Splunk App.
  6. Open mapping.json in a text editor and add an action mapping for the ErrorsIntent intent.This is just mocked up , but you can get the idea if you had some events with a host and some errors.

    { "intent": "ErrorsIntent", "search": "index=_internal host=$servername$ code=$errorcode$ error| stats count as errorcount", "time_slot" : "timeperiod", "response": "host $servername$ has had $resultfield_errorcount$ errors for error code $errorcode$ $timeperiod$" }

  7. Save the file and it will be dynamically reloaded.

  8. Browse back to your Amazon developer account and test your new intent with the Service Simulator on the Test tab by typing in how many errors have there been for error code five oh three for host foo today
  9. If that worked , then fire up your Alexa device and speak away ... Alexa , ask splunk how many errors have there been for error code 503 for host foo today

Logging

Any errors can be searched for in SplunkWeb : index=_internal error ExecProcessor alexa.py

You can ignore any SLF4J errors

Troubleshooting

Support

BaboonBones.com offer commercial support for implementing and any questions pertaining to this App.