Idiots attempt of learning how to use Splunk's SimData

Splunk training has awesome 'fake' data to make it look like you are drilling down into a real network's pile of logs.

In the past I heard it was a program called EventGen that the Splunk team used, but an in-house version.

Now there has been a shift and the splunk dev website lists SimData.

https://dev.splunk.com/enterprise/docs/dataapps/simdata

One needs to install Java Run time Environment (JRE)

And the SimData Jar file:  They link to this:
https://dev.splunk.com/enterprise/downloads

Scroll down to the bottom to find the SimData Jar download.

The command to run SimData is java -jar simdata-<version>.jar -s <simulation_file> -c <scene_file>

https://dev.splunk.com/enterprise/docs/dataapps/simdata/runsimdatasimulation

so... we need a simulation file and a scene file.. also we need to set up HEC to get the data ingested into Splunk.

Simulation File (.simulation file extension) uses a Domain Specific Language (yeah right?  like intro CS brah)

https://dev.splunk.com/enterprise/docs/dataapps/simdata/simdatareference/simdatasimfileref/

And scene file (json format)

https://dev.splunk.com/enterprise/docs/dataapps/simdata/simdatareference/simdatascenefileref/

I...

Have ...

No...

Idea ...  Where to start.  But thankfully, there is an example file:

Example simulation and scene files here:
https://dev.splunk.com/enterprise/examples


DON'T FORGET to setup HEC:
https://docs.splunk.com/Documentation/Splunk/8.0.3/Data/UsetheHTTPEventCollector

From the Splunk SimData Examples README.md file:

# Splunk SimData Examples

This project is a collection of SimData example scenes and simulation files.
Each example has its own corresponding README file.

SimData is a tool that generates event data from a simulation of a user-defined scenario. Instead of using a sample set of data that is repetitive and unrealistic, SimData allows you to generate a rich and robust set of events from real-world situations by mimicking how multiple systems work together and affect the performance of your system.

## Get started

For details about installing, configuring, and running SimData, see the [Splunk Developer Portal](https://dev.splunk.com/enterprise/docs/dataapps/simdata/).

### Requirements

* Java 8+
* Download SimData the SimData JAR file: https://dev.splunk.com/enterprise/downloads

### Example usage

This example shows how to execute the SimData CLI:

```sh
java -jar <SimData JAR file> --simulation <path to simulation file> --scene <path to scene file>
```

## Contact
If you have questions, reach out to us on [Slack](https://splunkdevplatform.slack.com) in the **#simdata** channel or email us at _devinfo@splunk.com_.

At first you read, "this project is a collection of SimData example scenes and simulation files"  and you think, "awesome!  Maybe there are windows event log examples, RHEL, cisco!"

Nope.  There is one example set thus far.  "hello".

Hello's README.md

# Hello SimData example

To run this example, run:

```sh
java -jar <SimData JAR file> --simulation hello.simulation --scene hello.json
```

### Expected output

```sh
Starting simulation
"eventType"="Greeting" "text"="Hello, World!"
"eventType"="Greeting" "text"="Hello, World!"
"eventType"="Greeting" "text"="Hello, World!"
"eventType"="Greeting" "text"="Hello, World!"
"eventType"="Greeting" "text"="Hello, World!"

Thats no fun.  But it looks like on this page we can start to edit the files to make a webserver example:

https://dev.splunk.com/enterprise/docs/dataapps/simdata/examplesimulation

## quick note, don't follow the README's example of running simdata, follow this:

java -jar simdata-<version>.jar -s <simulation_file> -c <scene_file>

## another quick note, the output for options:

Usage: simdata [options]
  Options:
    --enable-debug, --debug
      Enables debug logging.
      Default: false
    -h, --help
      Show help information
    --no-rest, --no-web
      Disable the rest endpoints and web server.
      Default: false
    -p, --port
      The port to use for the REST endpoints and web server
      Default: 11013
  * -c, --scene
      The scene file
  * -s, --simulation
      The simulation file
    --start-time
      The absolute (ex: '2017-12-25 00:00:00') or relative (ex: '-1d', '-2h',
      '-3m') start time to backfill data from. Overrides the value set in the
      scene file.
    --validate
      Only validate the simulation and scene files, do not run the simulation.
      Default: false

## End note

The example link gets right to business, how to point this to our Splunk instance, input some stuff and get going.



``` break till next time ```




This is of some interest for a noob like me:

Use the simulation control UI

SimData provides a web-based user interface for you to update the value of variables at runtime at http://localhost:11013 or the port you specify using the SimData CLI. You can disable this web server by passing the --no-web flag. For more, see the SimData CLI reference.

This UI exposes controls for bots of entity types with runtime variable controls. Each entity type has a set of controls to modify the state of all bots of that type. Additionally, each bot has its own set of controls to modify the state of only that bot. The simulation control UI refreshes the state of bots every second.

Fixing the Splunk TA-pfsense stanza for sourcetype extraction

I'm not sure if its a setting due to me, be it the way my pfsense is setup or some other PEBCAK (Problem exists between chair and keyboard) issue but in the pursuit of maximum value extracted from pfsense logs I dare not tackle it myself, but taste test from people far smarter than me.

There are a few good sites to help explain what is found in pfsense filter logs:

Here is Netgates' guide:

https://docs.netgate.com/pfsense/en/latest/monitoring/filter-log-format-for-pfsense-2-2.html

Here is an attempt by a regex ninja at extracting fields:

https://forum.netgate.com/topic/102549/filterlog-field-extractions-for-log-analytics-products/2

I work with Splunk and like to use it whenever possible to keep my practice up.  There are a number of Splunk apps that tackle pfsense logs:

- homemonitor

- A3sec

- TA-pfsense

- Technology add-on for pfSense filterlog (looking at the download count and last date updated I passed over testing this add-on).

Looking 'under the hood', homemonitor is a one-size-fits-all app.  Its simply amazing- the installation GUI, the attempt to make it as comparable as possible across a broad range of brands of gear, the thought put into the various dashboards and panels.  If you are not running pfsense but some other consumer router, highly highly recommend running this app.

That said, it can only commit so much effort into extraction of the pfsense logs.  So onto another taste testing.

The TA and APP for pfSense by A3Sec is another full featured app.   It has lookup tables, tags, all kinds of higher level Splunking to make sense of and even add value to data.  But when I would look at the raw log streams (be it the pfsense webUI or clog -f a log in the  terminal) I was finding discrepancies.  I believe what has happened is as pfsense updates or changes log format slightly, it throws off the extractions.  Or again PEBCAK and I'm failing to see (but its likely) that A3Sec made a clever way of cranking down the noise and showing events that have more value/importance.  

But while looking into A3Sec app I found other posts online of people trying their hand at regex trying to extract every field possible, from event logs that can end up being very different in format due to the source of the log coming from IPv4 or IPv6 traffic, of OpenVPN traffic, webUI logs etc.

Add to the hair pulling, adding further data sources such as Snort or even Windows logs from one of my local machines would cause splunk to simply just stop ingesting.  What I found in this case is that a lot of these apps are not explicitly calling out a method of time extraction so Splunk uses it's default to attempt.  I theorize that when you have an assortment of apps and data feeds coming in where this is not specified, it breaks my free-license splunk instance's ability to field extract.  When investigating the internal logs you will see line break errors due to size, and some bogus dates being extracted, this was my sign.

So going over to this thread, someone provides a nice little regex to get pfsense time extracted:

https://answers.splunk.com/answers/626816/pfsense-event-date-time-wrong.html

With that set, you can have Snort logs, pfSense logs and other logs coming in.  For Snort I used a splunk stanza to simply put for time stamp the current time the data get ingested.  For line breaking (as many apps use time stamps as a method of knowing the end of one event and the beginning of a new log event) ensure the app that you use this time setting on has a stanza in the props that performs the line breaking.

Splunk configure timestamp recognition:

https://docs.splunk.com/Documentation/Splunk/8.0.3/Data/Configuretimestamprecognition

''' code test '''

So with different data inputs working at the same time, back to testing various pfSense extractions.  Onto the TA-pfsense add-on.

As a add-on, its simply (but not simple in-of-itself) the field extraction element and you are required to make dashboards.  I found porting over A3Sec's dashboards a great start.

Prepare yourself--- the extractions are amazing.  I was in awe what was coming in and getting caught.  Just wow.  A big part of this effectiveness is a great example of using a transforms.conf to look at the incoming data and with some regex output that data one sourcetype or another so that the subsequent props.conf stanzas can apply effective regex for the log format.

See example of this transforms using regex to get past the time data stamp, find pfsense and output the word right after that to give it a source type (sourcetyper).

And this is where I found issue.  In Splunk there are logs coming in without the time stamp in the raw, so it brakes TA-pfSense's ability to apply a sourcetype other than the default "pfsense", so something that should be sourcetype "pfsense:openvpn" and then accurately field extracted is instead coming out simply sourcetype "pfsense" and the openvpn props.conf field extractions are not applied.

When SSH'ing into the pfsense box and clog -f (linux's tail -f, pfsense is FreeBSD) the openvpn logs, there are dates and times.  Same in pfsense's WebUI log viewer.  

WHAT THE HECK?!

Example of raw and splunk, good sourcetype extraction

Example of raw without stamp, so sourcetyped only as "pfsense" but terminal has normal pfsense time format:

Issue with sourcetyping:

https://answers.splunk.com/answers/813296/ta-pfsense-sourcetyping-only-catching-filterlog.html

This was on a Splunk instance that also had the A3sec app installed, so just to be sure there wasn't some issue I was not finding with app interaction, little time extraction changes I had put in etc I built up a vanilla splunk server just for the TA-pfSense add-on.  There was some delay doing this as I was also moving over from ESXi free to XCP-ng in an attempt to get more hypervisor features.  Thats a whole nother blog post (and learning curve).

So now with a fresh vanilla CentOS7 box with Splunk 7.x installed, off we go.  Same issue with a vanilla setup (time extraction from that earlier splunk post not added, this is using all default).

Whats interesting is the lack of time in the raw logs in Splunk itself.  One thing to do is to spin up yet another server, A3Sec app and see if those same log types still come in the raw like that.  Another observation I had was the A3sec app's openvpn dashboard used to work, but seemed to stop.  I really suspect pfSense's syslog output to have possible changed.  Heck maybe a bug has been found?  Its not streaming out as its displayed in the terminal.

If I can't get pfSense to output correctly, or if this is an "at ingestion" issue, a band-aid might be to completely migrate away from the current transforms.conf method and use some other regex that just finds the first reference of pfsense, takes the word after the next dead space and use that for source typing.  I'm very weak on regex, but this is a reason to try.

UPDATE 7/18/2020

With fresh eyes I looked at apps/TA-pfsense/default/props.conf

I saw right at the top:

 SEDCMD-event_cleaner = s/^(\w{3}\s+\d{1,2}\s\d{2}:\d{2}:\d{2}\s)+\S+\.\S+\s+/\1/g

 SEDCMD-event_cleaner2 = s/^(\w{3}\s+\d{1,2}\s\d{2}:\d{2}:\d{2}\s)+(\w{3}\s+\d{1,2}\s\d{2}:\d{2}:\d{2}\s)+/\1/g

 SEDCMD-event_cleaner3 = s/^\w{3}\s+\d{1,2}\s\d{2}:\d{2}:\d{2}\s\S+\s(\S+\s)/\1/g

So has this dropping of the time stamps of certain source types (and some of the message itself) been a function?  Intentional to reduce sourcetype noise?  Not sure as the props has extractions for openvpn but those were not working because openvpn logs were not getting sourcetyped.

I commented them out and bam, the types of sourcetypes filling the splunk index started to increase in diversity.  Not sure if there is a negative long term effect with this, I wish the dev was reachable to ask.  

How to tail -f logs in FreeBSD pfsense

In BSD its not tail -f, its clog -f

LetsEncryp and NGINX for hass.io with docker

The goal was to use linuxserver.io's letsencrypt container that has both NGINX and Let's Encrypt combined together, and use it to provide a reverse proxy service to a homeassistant container on a CentOS machine.

TLDR- I didn't get this to work so I used Lawrence Systems YouTube vid on pfsense HAproxy and acme certs to accomplish the same goal, all within a GUI.
https://www.youtube.com/watch?v=gVOEdt-BHDY&t=1320s

Its more steps and I didn't get it right on the first go, but its easy to trouble shoot via GUI and re-reviewing Lawrence's video.

Onto the failed attempt(s) using LinuxServer.io:

Homeassistant has an NGINX pluging and Letsencrypt plugin but the documentation is very lacking (really, homeassistant documentation that is outdated and/or not written for noobs?  No way!)

So after reading a homeasistant user's tutorial thread:
https://community.home-assistant.io/t/nginx-reverse-proxy-set-up-guide-docker/54802

And then later in troubleshooting per the linuxserver.io team discord and the linuxserver.io instructions
https://blog.linuxserver.io/2019/04/25/letsencrypt-nginx-starter-guide/#authorizationmethod

- looks easy as cake right?

Firstly, I transfered my domain from Bluehost to Cloudflare.  This was overdue, but its mainly due to Cloudflare's API(s) being supported by a great many projects, this and acme (pfsense) being some of them.

I created a cname record for the subdomain I wanted to use as well.  I'm fairly decent with firewall rules, port forwarding and NAT settings on pfsense as well, thanks to Lawrence Systems on youtube and having already done a number of self hosting projects such as a wordpress server built on a LAMP stack facing the internet.

What followed was what feels like countless docker compose builds / nukes playing with config files, going 100% per linuxserver,io instructions, going 100% per homeassistant forum instruction, and mixtures of the two.

The linuxserver.io team was sure their default setup using the renamed homeassistant proxy conf would work (contrary to the homeassistant forum... and most other product specific forums running their container for things like Plex, Nextcloud on things like UnRaid etc).

But this was a great exercise in learning Docker, and a reminder there are still plenty of linux communities where if you are not at a dev's / grey beard's level, you are not welcome- but that is human nature and not going away anytime soon.

Bit thanks to Lawrence System's channel and discord not being one of these places, but fostering learning and community acceptance/training (go figure they are associated with the "Learn Linux" channel which is also amazing).  Together with places like Level1techs, we noobs aspiring to RTFM but need help along the way have great people and resources.

Useful docker links:

Docker Compose install, basic use:

https://docs.docker.com/compose/gettingstarted/

Correcting my machines timezone

https://linuxize.com/post/how-to-set-or-change-timezone-on-ubuntu-18-04/

Level1techs:
https://level1techs.com/
click on their forums, and check out their merch at their store

Installing TWRP on Samsung Galaxy Tab 8.9 SGH-i957

All files (TWRP, Danno ROM, Gapps) located here:

https://forum.xda-developers.com/showthread.php?p=76879448#post76879448

Instructions using Odin to flash TWRP on the tablet:
https://www.theandroidsoul.com/how-to-install-twrp-recovery-using-odin/

Moved the ROM and Gapps files over to the TWRP folder (in stock android GUI allow USB mode) from Windows

Install from zip file both the ROM and Gapps.  Reboot.

cannot start vmware USB arbitration service error 2

Had this issue, needed to run the manual process of installing hcmon to fix it.

https://kb.vmware.com/s/article/2043656

• VMware USB Arbitration service fails to start in VMware Workstation
• While powering on a virtual machine, you see the error:
  
  Host USB device connections disabled: The connection to the VMware USB Arbitration Service was unsuccessful. Please check the status of this service in the Microsoft Management Console.
   
• Attempting to start the service manually from Windows Services fails with the error:
  
  Windows could not start the VMware USB Arbitration Service on Local Computer. Error 2: the system cannot find the file specified.

 Cause

This issue occurs if some of the files that are required to run the service are missing. The files may be missing if the product was installed or upgraded using an account that does not have sufficient privileges/permissions.

 Resolution

To resolve this issue:

1. Launch a command prompt on the host machine using an Administrator account. For more information, see the Microsoft TechNet article Start a command prompt as an administrator.
   
   Note: The preceding link was correct as of January 25, 2013. If you find the link is broken, provide feedback and a VMware employee will update the link.
    
2. Run this command:
   
   net start hcmon
    
3. Open the Windows service console and start the service manually.

If the service fails to start, try installing the hcmon service manually.

To install the hcmon service manually:

1. Launch a command prompt on the host machine using an Administrator account.
2. Run this command:
    
   • In a 32-bit host operating system:
     
     "C:\Program Files\VMware\VMware Workstation\vnetlib.exe" -- install hcmon
     
     In a 64-bit host operating system:
     
     "C:\Program Files (x86)\VMware\VMware Workstation\vnetlib64.exe" -- install hcmon
     
      
3. Open the Windows service console and start the service manually. The service should now start successfully.

If the issue persists even after trying the preceding steps, reinstall/repair VMware Workstation as a user with Administrative rights on the Windows host. For more information, see the Getting Started with VMware Workstation guide.

4 channel Amazon wifi relay Tasmota flash

I had to take chunks from various tutorials, and honestly don't know if I had bad connections and other bumps in the road that made this more convoluted than necessary, but none the less, here we go:

- Reference URLs of tutorials- read through all of them:

https://notenoughtech.com/home-automation/nodered-home-automation/hacking-esp8285-geekgerit-4-way-relay-controller/

https://community.home-assistant.io/t/hacking-the-psf-b04-esp8285-4-channel-relay-board-and-with-tasmota-and-unable-to-drive-all-4-relays-concurrently/155919

https://www.hagensieker.com/wordpress/2019/02/21/hacking-unknown-2-channel-relay-with-tasmota/

https://www.youtube.com/watch?v=BUfWytrzrJ4&t=275s


- I tried the first URL without success, in hindsight I think the following would work

- Use the ground on the top left, not the GND mid left for grounding out GPIO14
- Press the top mid button to get into flash mode- press until mode where it repeatedly cycles the relays on/off in series.
- Then used ESPEasy_mega to flash the tasmota bin file
- After resetting power the board should become an access point to configure wifi, but you can also connect via FTDI adapter and Termite to input SSID1 and password1.

Use steps from hagensieker URL to go into WebUI and set to 4ch relay, and set relays to inching for garage door openers.

Follow various tutorials if integrating into Home Assistant (I have yet to do it)

Find commands to put only certain relays into inching mode, others into latching (For lights).

If cannot command multiple relays at once, do the chip mod found in the community.home-assistant.io URL.

- What I actually did.
- youtube URL, installed the ewelink app, connected to the board, updated the firmware to 3.3.x
- Did no use the windows flashing software from the youtube tutorial, I was still having a hard time putting the board into flashing mode and doubting my RX/TX soldering job.
- Starting using a different GND point for the GPIO10 lead, and then pressed the mode button until the relay would go into a mode where relays kept cycling on/off
- RX/TX FTDI adapter was now reading the board, it was in flash mode (remember tutorial mentions you must power the board with 5v microUSB, not just via FTDI adapter).
- Used ESPEasy_mega to flash over Tasmota bin
- reset power
- No need to use Termite to update SSID1 and Password1 info, tasmota went into wifi mode server 192.168.4.1, so used wifi to connect to "tasmota-5933" and used webUI to update wifi info.
- Tasmota flashed

- I appear to be having the problem the home assistant thread had, where I cannot power more than one relay at a time, looks like I will need to remove the 'unknown' chip and jumper the terminals in accordance to that thread's tutorial.

Still need to find commands to put certain relays into inching, others into latching (if possible), and integrate via MQTT into Home Assistant.