Friday, March 23, 2018

Adding Snort for Splunk app

Log into Snort, go to add app, search snort- add Snort for Splunk

Follow this instructions:

App Installation

1.) To install the app, download the app to a suitable download location.
2.) Open Splunk and click on the Manage Apps icon.
3.) Click on the Install app from file button.
4.) In the Upload app window, select the Browse button under File and locate the SnortforSplunk.spl file in the download location in step 1.
5.) Click the Upload button to install the app.
6.) Once the app is installed follow the next steps to setup the Data Input.
7.) Under Splunk -> Settings -> Data Inputs -> Local Inputs -> UDP -> Click the New button.
8.) In the Port field under Add Data -> Select Source, enter 514 for the port to be used.
9.) In the Only accept connection from field under Add Data -> Select Source, enter the IP address of the pfSense appliance
(in the format XXX.XXX.XXX.XXX)and click Next.
10.) From the Source Type dropdown under Add Data -> Input Settings, select Network and Security -> snort.
11.) From the App Context dropdown under Add Data -> Input Settings, select Snort for Splunk.
12.) Click the Review button.
13.) Once satisfied with the settings, click the Submit button.

pfSense Setup

1.) The setup assumes that pfSense version 2.3.2-RELEASE-p1 is being used as a firewall, along with pfSense-pkg-snort version 3.2.9.2_16 (which includes Barnyard2 version 1.13 and Snort version 2.9.8.3) and that this has been properly setup.
2.) Select Services -> Snort from the main menu and this will show the Snort Interfaces page.
3.) Select the Edit option (Pencil icon) under the Actions column on the page adjacent to the interface to be captured.
4.) Under the submenu, select the {Interface} Barnyard2 (substitute {interface} for either WAN or LAN or as has been setup on pfSense).
5.) Under General Barnyard2 Settings, make sure the following are checked:-
- Enable Barnyard2
- Show Year
- Archive Unified2 Logs
and leave the rest of these settings on their default values.
6.) Scroll down to Syslog Output Settings and select Enable Syslog
7.) Under Remote Host enter the IP address of the Splunk server that is receiving the log files from Barnyard2.
8.) Under Remote Port enter the port of the Splunk server that is receiving the log files from Barnyard2 (default is port 514). **
9.) Change Log Facility from default to LOG_AUTH.
10.) Change Log Priority from default to LOG_ALERT.
11.) With all the settings done click on the Save button at the bottom.
12.) Click on the Snort Interfaces menu item and under the Snort Status column, click on the icon to start/restart the Snort interface.
13.) Check on the Splunk server that the information logged by Barnyard2 is captured by the app.
** Exception, I already had PFsense syslogs going into UDP 514, so made another data input in splunk for port 992/UDP 
*** added port 992/UDP to public interface firewall on CentOS 7 server hosting Splunk indexer.
**** Specified a new index created, index = snort, used for snort app
* Is PFSsense blocking barnyard logs?

Monday, December 25, 2017

building a relatively secure raspberry pi server- save yourself the time

** This is an insanely unorganized clump of notes I've taken while setting up a server for my own personal re-use when I may have to do it again, so I store it in the cloud for ease of access- but hopefully there may be some helpful tidbits of others found within, as it was other people's blogs that really helped me get through it.

Years ago I built a working Wordpress server using my domain name following most of pestermeester's tutorial below:
https://pestmeester.nl/index.html#10.0

I had taken the server down sometime ago, played with some other Pi projects and upon trying to follow the tutorial again, there have been too many updates and the commands are no longer relevant.  Going to raspberry Pi forums is not much help, you cannot post without waiting on moderators, and their very own tutorials are also outdated.  Though the appeal to use a Pi remains strong due to no moving parts and low power draw, I wanted to get something up and working, so I leaned on my power hungry ESXi machine.

Rather than this be a tutorial that goes step by step and be invalid within months, this is just general links and notes as to the sites I used that were most current at the time for my particular setup (pfSense 2.4.2, Ubuntu headless server 16.x with LAMP, email and SSH selected during install).  Then notes on the various 'trip wires' I ran into that had to be figured out or worked around.

So after a very frustrating day with the kid/education friendly (ie: should have solid documentation) Pi, I spun up a VM with Ubuntu Server 16.x- wow- the install GUI asks you what packages you want installed- it installed LAMP, webmail, SSH, new user account, all in one shot!  Its not like this sidesteps the learning process, you will still learn what a LAMP stack is when you have to set it all up, but it saves you all the "apt-get" lines that inevitably gets outdated.

Since this is a VM, added:

sudo apt-get install open-vm-tools

** make a snapshot here so its easy to start over when one inevitably messes things up beyond what $ --purge can fix.

Then follow this tutorial that nails the basics that pestermeester had for account setup and SSH private key login:
https://www.digitalocean.com/community/tutorials/initial-server-setup-with-ubuntu-16-04

Follow the digitalocean pre-req for Lets Ecrypt by configuring the virtual host setup in Apache:
https://www.digitalocean.com/community/tutorials/how-to-set-up-apache-virtual-hosts-on-ubuntu-16-04
** to get lets encrypt to install correctly, the router/firewall needs to be setup correctly after having setup my domain hosting service to point to my IPS public IP.  I had to use port forwarding as I could not get 1:1 NAT to work.  Once all setup, lets encrypt does not suffer from verification issues or time-out issues.
* enable NAT reflection drop down to NAT + proxy so that viewing the server/site works from machines on your LAN.

To even test port forwarder working I had to use incongnito tabs in chrome so that chrome would not automatically force https.

Lets Encrypt will print a URL to test your cert after its completed, its actually really fun to see this URL validate your certificate and give it a grade:https://www.ssllabs.com/ssltest/analyze.html?d=yourdomainnamehere

Now I can access my apache test page even when chrome forces https.  Finally getting somewhere!

* remember to 'fix' any firewall rules in your router/firewall and local rules on the Ubuntu box (ufw rules) to tighten things down if you had made any/any rules and/or ufw disable for testing purposes.

* take another ESXi snapshot, because why not?

Then the below tutorial is pretty good for installing wordpress:

https://www.techrepublic.com/article/how-to-install-wordpress-on-ubuntu-16-04/

But I'm partial to digitalocean tutorials, so their wordpress install has a few more details:
https://www.digitalocean.com/community/tutorials/how-to-install-wordpress-with-lamp-on-ubuntu-16-04

A more applicable digital ocean wordpress tutorial as the lets encrypt tutorial had us setup apache/ubuntu to be able to host multible sites on one host.
https://www.digitalocean.com/community/tutorials/how-to-set-up-multiple-wordpress-sites-on-a-single-ubuntu-vps

* note, hate to say this but the digitalocean tutorial was no-joy, the techrepublic one gets me up and running.

*** tip in the digital ocean comments:  If using a one-click LAMP image, be sure to remove /var/www/html/index.html file (or change the configuration) because the server defaults to .html files before .php files.

*** key point #2 from the comments:  Turns out on MySQL 5.7 the line:
GRANT ALL ON wordpress.* TO 'wordpressuser'@'localhost' IDENTIFIED BY 'password';

*** still had database access issues, followed this site:
https://chartio.com/resources/tutorials/how-to-grant-all-privileges-on-a-database-in-mysql/
did not not work for me; this command seems to be deprecated in MySQL. I used:
CREATE USER 'wordpressuser';
Then:
ALTER USER 'wordpressuser'@'localhost' IDENTIFIED BY 'password';

* yet another digital ocean tidbit that helps explains the organization of apache2 and wordpress files to make your site appear:
https://www.digitalocean.com/community/tutorials/how-to-move-an-apache-web-root-to-a-new-location-on-ubuntu-16-04

You will be up and running in a minutes!!!

Follow the raspberry pi tutorial or this other linked tutorial to add the security aspects-
https://www.digitalocean.com/community/tutorials/initial-server-setup-with-ubuntu-16-04

sudo apt-get install php7.0 php7.0-mysql libapache2-mod-php7.0 php7.0-cli php7.0-cgi php7.0-gd
OMG, logging into mysql with a password works!
Added a virtual NIC to connect the Ubuntu server to the DMZ, followed this guide:
https://ubuntuforums.org/showthread.php?t=1400504
** after everything was installed, wordpress put into the write folder, the right directories listed in the right .confs etc, the site still would not come up.  Apache worked when I reverted to an apache only snapshot, but not when wordpress was installed (inside the LAN i did get the wordpress site, just not from outside).  DNS and firewall settings were good- the issue is due to setting up/installing wordpress before the DNS A record is done, so the private IP was inputted.  Go to the wordpress admin gui, general settings, and replace the IP addresses to your DNS.
https://www.digitalocean.com/community/questions/domain-redirects-to-ip-address-and-also-loading-from-ip-address
With wordpress and woocommerce running, the next issue is uploading and/or cropping 
images will become an issue, so php7-gd needs to be installed
$ sudo apt-get install php7.0-gd
$ sudo systemctl restart apache2.service
## remaining tasks
- change permission settings so that plugins can be updated and installed
- fine tune firewall and ufw rules
- install email client/GUI

Sunday, October 29, 2017

LG G3 Lineage OS install

- This is just a pointers thread, as most any google'd "install TWRP/Lineage OS" finding will give you a great tutorial- but specific to my goals and the LG G3 there were some key lessons.

- LG G3 official 3.0.2-0 and even the un-official 3.0.3.2 does not support updating via OS GUI (must download from website, manually update via TWRP) nor decryption (if you encrypted the phone) to update when updating manually.  3.0.3.2 does at least prompt you for the swipe/pin to decrypt, but it just doesn't work.

- If you have already encrypted and now stuck in a loop where you can't format the internal storage to start over (as you can't mount it to start with), in TWRP go to command line and:

"recovery --wipe_data --set_filesystem_encryption=off"

- Don't encrypt the phone, you will not be able to update the OS and the LG G3's snapdragon 801 does not have hardware optimization for encryption, you take a huge read/write speed hit.

- I did not choose the superuser/root option, I actually wanted to run un-rooted this time to use android pay.

- Android pay works!  Its great going to a store and not caring if their network or POS device is compromised, my info and CC is still safe!  And its convenient (and much faster than chip and pin).

- My LG G3 isn't super fast/snappy, but Lineage has breathed new life into my old phone- saving me from having to ditch it and buy a new phone.

- Part of that 'new breath' is SUPER fast security updates- faster than Nexus/Pixel!  I got the WPA2 Krack patch the first Monday after the announcement of the hack!

Friday, October 27, 2017

Enable trim on PFsense

I started to bookmark or save URLs of helpful tutorials, but started to find sometimes the sites/blogs etc would be taken down and the info lost.  So I found a great tutorial on how to enable trim for SSD in PFSense and want to save it for the long run, as it was a pain sifting through google results for a tutorial of this quality.

URL:   https://gist.github.com/mdouchement/853fbd4185743689f58c

Thank you mdouchement for the below:

nstallation

Use memstick-licecd to install pfsense on your SSD.

Enable TRIM

  • Initialize fstab:
[2.2.4-RELEASE][root@pfSense.localdomain]/root: /usr/local/sbin/ufslabels.sh
It may do nothing because all is already initialize but it's only in a case of
  • Enable AHCI by adding ahci_load="YES" in th following file:
[2.2.4-RELEASE][root@pfSense.localdomain]/root: vi /boot/loader.conf.local
  • Get your device ID by showing the fstab file:
[2.2.4-RELEASE][root@pfSense.localdomain]/root: cat /etc/fstab
# Device                Mountpoint      FStype  Options         Dump    Pass#
/dev/ufsid/563dee910aa6a80a             /               ufs     rw              1       1
/dev/label/swap0                none            swap    sw              0       0
Check the Device ID that is mounted on /, in this example it's /dev/ufsid/563dee910aa6a80a
  • Before enabling TRIM, the partition/device must be unmounted so you have to reboot pfsense in Single User mode.
[2.2.4-RELEASE][root@pfSense.localdomain]/root: reboot
  • Add your USB memstick-livecd and boot on it and select 2 then use tunefs to enable TRIM:
          __ ____
   _ __  / _/ ___|  ___ _ __  ___  ___
  | '_ \| |_\___ \ / _ \ '_ \/ __|/ _ \
  | |_) |  _|___) |  __/ | | \__ \  __/
  | .__/|_| |____/ \___|_| |_|___/\___|
  |_|


 +------------Welcome to pfSense-----------+
 |                                         |                 ______
 |  1. Boot Multi User [Enter]             |                /      \
 |  2. Boot [S]ingle User                  |          _____/    f   \
 |  3. [Esc]ape to loader prompt           |         /     \        /
 |  4. Reboot                              |        /   p   \______/  Sense
 |                                         |        \       /      \
 |  Options:                               |         \_____/        \
 |  5. [K]ernel: kernel (1 of 2)           |               \        /
 |  6. Configure Boot [O]ptions...         |                \______/
 |                                         |
 |                                         |
 |                                         |
 +-----------------------------------------+


...


Enter full pathname of shell or RETURN for /bin/sh:
# /sbin/tunefs -t enable /dev/ufsid/563dee910aa6a80a
tunefs: issue TRIM to the disk set
# /sbin/reboot
  • Boot pfsense in normal mode then go in shell and you can check that TRIM is now activated:
[2.2.4-RELEASE][root@pfSense.localdomain]/root: tunefs -p /
tunefs: POSIX.1e ACLs: (-a)                                disabled
tunefs: NFSv4 ACLs: (-N)                                   disabled
tunefs: MAC multilabel: (-l)                               disabled
tunefs: soft updates: (-n)                                 enabled
tunefs: soft update journaling: (-j)                       enabled
tunefs: gjournal: (-J)                                     disabled
tunefs: trim: (-t)                                         enabled
tunefs: maximum blocks per file in a cylinder group: (-e)  4096
tunefs: average file size: (-f)                            16384
tunefs: average number of files in a directory: (-s)       64
tunefs: minimum percentage of free space: (-m)             8%
tunefs: space to hold for metadata blocks: (-k)            6408
tunefs: optimization preference: (-o)                      time
tunefs: volume label: (-L)
[2.2.4-RELEASE][root@pfSense.localdomain]/root: tunefs -p /var
tunefs: POSIX.1e ACLs: (-a)                                disabled
tunefs: NFSv4 ACLs: (-N)                                   disabled
tunefs: MAC multilabel: (-l)                               disabled
tunefs: soft updates: (-n)                                 enabled
tunefs: soft update journaling: (-j)                       enabled
tunefs: gjournal: (-J)                                     disabled
tunefs: trim: (-t)                                         enabled
tunefs: maximum blocks per file in a cylinder group: (-e)  4096
tunefs: average file size: (-f)                            16384
tunefs: average number of files in a directory: (-s)       64
tunefs: minimum percentage of free space: (-m)             8%
tunefs: space to hold for metadata blocks: (-k)            6408
tunefs: optimization preference: (-o)                      time
tunefs: volume label: (-L)

Links that help me to write this tutorial

Saturday, March 25, 2017

ESXi 6 hangs/freezes up

Troubleshooting is so fun when you made tens of changes at once haha.  So my ole' trusty ESXi 6 HPz800 machine started to hang up after a day or so of operation.  Looking into the ESXi logs, they are not much help as you find what you would think are red alarms, but after lots of googling they seem to be normal logs.  But skipping over to vmdkwarning logs and I got a lot of (copied and pasted from another forum member's issues, too lazy to SSH into the system again):
Lost access to volume
4bcce772-3bfe7a35-dceb-001b21541d90 (1_5WD
1) due to connectivity issues. Recovery attempt
is in progress and outcome will be reported
shortly.

And then:
Successfully restored access to volume 4bcce772-
3bfe7a35-dceb-001b21541d90 (1_5WD_1_)
following connectivity issues.
info
4/21/2010 1:16:16 PM

Then according to a few forum posts, it might be HDD failing, or overheating, or get in and tweak heart beat settings (IMO the latter a band-aid to symptoms of something needing to get fixed).

In denial that it is already time to replace my WD Black 1TB I have the datastore on, I thought about what things have changed on my system:
- put some buffer material along my HDDs in software RAID to reduce noise, but this could be reducing their airflow too
- pulled out a Hauppauge HVR card from a defunct MythTV build
- pulled out 24 gigs of 'original' RAM and put in 48 gigs of Ebay ECC RAM

Software side
- upgraded to FreeNAS 10
- Ubuntu server with Plex media server, fstab FreeNAS10 CIF of Movies share
- New Splunk server on CentOS7

Oh joy, plenty to look into.  But after lots of forums and poking around I was finding the Ubuntu Plex server to be getting hung up, particularly the kswapd process eating all of the resources- typical Linux forums leads one on a chase is it the kernel?  Kind of known yet unknownish bug? You the installer is just dumb etc.  What I settled on is I didn't give the Ubuntu server enough RAM to run plex effectively (though I watched two movies without issue, it spins out of control randomly at idle)- hopefully that is the fix, or else I will just have to start over on a CentOS server build (Ubuntu is showing it's desktop user bias as it has not been good at 'services' jobs like running splunk server or now in this case Plex).  This thread doesn't give me much optimism:
http://serverfault.com/questions/316560/how-do-i-tell-what-process-is-causing-kswapd-to-be-in-use/316636

Sunday, March 19, 2017

Mounting FreeNAS share to Ubuntu and installing Plex on Ubuntu

The notes are for the goal of installing Plex into the headless Ubuntu server repository so 1) its easy to install and b) updates are easy- then install Plex, then mount the FreeNAS SMB share to the Plex server running in Ubuntu.  Updates to this thread should be made over time after getting the setup to 'work' to then work towards security.  For now, its just within the NAT.

Then something I find a lot harder to do (and scary from past instances of bricking machines), editing fstab to mount drives or shares in Linux.  Why Linux do you make this so hard?

So to get Plex going on Ubuntu server-

Add Plex to your repo for updates (I thought this would be of use to install as well, but I guess not)
echo deb https://downloads.plex.tv/repo/deb/ public main | sudo tee /etc/apt/sources.list.d/plexmediaserver.list
curl https://downloads.plex.tv/plex-keys/PlexSign.key | sudo apt-key add -
After that, it's just a matter of running the normal sudo apt-get update and the Plex Media Server repo will be enabled on the OS.
Then SSH into your server (ensure you have installed SSH) and in my case type: $ wget https://downloads.plex.tv/plex-media-server/1.4.4.3495-edef59192/plexmediaserver_1.4.4.3495-edef59192_amd64.deb
Then sudo dpkg -i plexmediaserver_1.4.4.3495-edef59192_amd64.deb
To prep Ubuntu to mount the samba/CIF share:
sudo apt-get install rpcbind nfs-common
sudo apt-get install cifs-utils
______
Now the fun part, mounting my FreeNAS share of movies to the Plex server running on Ubuntu server before logging into the Plex webUI to configure the library (this is all on a single box running ESXi).
Getting the path to the FreeNAS share-
my FreeNAS IP is 192.168.*.***, target is /mnt/ZFS/movies, but come to find out that path FreeNAS shows is not how you should go about this, I used smbclient
to poke around with random path tries until seeing its simply FreeNAS IP / share name.  With that info, nano fstab time following this format:
//<IP address of NAS box 192.168.x.y>/<shared folder> /<mount point> cifs guest,_netdev,uid=<your user name on Linux box> 0 0

More info from the site that helped- the end goal is to make it persistent with reboots and secure.  I am running this within a NAT and do not have the IPs locked down as static, so I will need to put in work to remedy this.
Helpful site:  https://wiki.samba.org/index.php/Mounting_samba_shares_from_a_unix_client


Wednesday, March 15, 2017

CentOS7 Server, Splunk, Homemonitor

After installing CentOS7 Live with GNOME, i finalized the install, then
yum update (as root).

After the update, installed Splunk 6.5 (download the RPM package on Splunk's website), install:
Downloads$ rpm -i splunk_package_name.rpm

Then needed to open up the firewall for some ports

firewall-cmd --permanent --zone=public --add-port=8000/tcp (this is for the webUI access)

firewall-cmd --permanent --zone=public --add-port=8089/tcp (this is for managing forwarders)

firewall-cmd --permanent --zone=public --add-port=9997/tcp (this is for data input from forwarders)

firewall-cmd --permanent --zone=public --add-port=514/udp (this is for the router's syslog to feed into splunk as an input)

Install homemonitor after installing Splunk via the app manager.  Follow the prompts for the setup, you may have to go into data inputs, add, port 514 and in my case ensure source is set to Asus.

Profit.