Main apps needed:
– WxToImg
– Predict
– Rtl-SDR
– Sox
– Dropbox bash uploader
– Some essential packages
– A script to link it all together, provided by a reddit user the2belo. Check out the sources at the bottom of this post.

apt-get update
apt-get install curl git-core git cmake at predict sox libusb-1.0-0-devx-fmt-all build-essential libasound-dev libpulse-dev -y
cat <<EOF >no-rtl.conf
blacklist dvb_usb_rtl28xxu
blacklist rtl2832
blacklist rtl2830
EOF
mv no-rtl.conf /etc/modprobe.d/
git clone git://git.osmocom.org/rtl-sdr.gi
cd rtl-sdr/
mkdir build
cd build
cmake ../ -DINSTALL_UDEV_RULES=ON
make
make install
ldconfig
cd ~
cp ./rtl-sdr/rtl-sdr.rules /etc/udev/rules.d/
reboot
rtl_test

Output should look similar to this:
root@rpi:~/wxsat# rtl_test
Found 1 device(s):
0: Realtek, RTL2838UHIDIR, SN: 00000001

Using device 0: Generic RTL2832U OEM
Found Rafael Micro R820T tuner
Supported gain values (29): 0.0 0.9 1.4 2.7 3.7 7.7 8.7 12.5 14.4 15.7 16.6 19.7 20.7 22.9 25.4 28.0 29.7 32.8 33.8 36.4 37.2 38.6 40.2 42.1 43.4 43.9 44.5 48.0 49.6
[R82XX] PLL not locked!
Sampling at 2048000 S/s.

Info: This tool will continuously read from the device, and report if
samples get lost. If you observe no further output, everything is fine.

Reading samples in async mode…

wget http://www.wxtoimg.com/beta/wxtoimg-armhf-2.11.2-beta.deb
dpkg -i wxtoimg-armhf-2.11.2-beta.deb
wxtoimg

Accept.

predict

Setup your QTH.

cd ~
mkdir wxsat
wget https://raw.githubusercontent.com/the2belo/wxsat-scheduler/master/noaa-scheduler.sh
chmod +x noaa-scheduler.sh
mkdir images recordings

Edit noaa-scheduler.sh to reflect your location.

Change -L parameter
/usr/local/bin/wxmap -T “NOAA ${bird}” -H ~/wxsat/weather.txt -L “35.47/136.76/20” -p0 -o “\$mapdate” ~/wxsat/noaa${bird}map.png

Format is: (lat/lon/alt meters)
Use this site to get lat and lon for your location http://www.latlong.net/lat-long-dms.html

Edit noaa-scheduler.sh for gain and SDR dongle ppm correction:

timeout $rectime /usr/local/bin/rtl_fm -d 0 -f ${freq}M -s 48000 -g 44.5 -p 1 -F 9 -A fast -E DC ~/wxsat/recordings/NOAA${bird}-\$recdate.raw

-g = gain

-p = ppm

Play with gain settings to get the best results.

You also need to add the timezone for the script, add

export TZ='UTC date'

in a new line after #!/bin/bash and after cat << EOF > ~/wxsat/noaa${bird}.at

This is how it looks like in a final version: https://pastebin.com/3uWcp9wi

Script generates MCIR and HCVT images by default, you can change this options. I also added -c option to crop telemetry from the sides of the pictures.

/usr/local/bin/wxtoimg -c -e MCIR -m ~/wxsat/noaa${bird}map.png ~/wxsat/recordings/NOAA${bird}-\$recdate.wav ~/wxsat/images/NOAA${bird}-MCIR-\$recdate.png

mkdir Dropbox-Uploader
cd Dropbox-Uploader
curl "https://raw.githubusercontent.com/andreafabrizi/Dropbox-Uploader/master/dropbox_uploader.sh" -o dropbox_uploader.sh
chmod +x dropbox_uploader.sh
./dropbox_uploader.sh

Follow the instructions.

Automate the proces. Commands bellow will be run every 12 hours.

crontab -e

00 */12 * * * atq | awk ‘{print $1}’ | sort -n | xargs atrm
01 */12 * * * ~/wxsat/noaa-scheduler.sh 15 137.620
02 */12 * * * ~/wxsat/noaa-scheduler.sh 18 137.9125
03 */12 * * * ~/wxsat/noaa-scheduler.sh 19 137.100
00 */12 * * * wget -qr https://www.celestrak.com/NORAD/elements/weather.txt -O ~/wxsat/weather.txt

Start the jobs immediately:

wget -qr https://www.celestrak.com/NORAD/elements/weather.txt -O ~/wxsat/weather.txt
~/wxsat/noaa-scheduler.sh 15 137.620
~/wxsat/noaa-scheduler.sh 18 137.9125
~/wxsat/noaa-scheduler.sh 19 137.100

If you are going to change the file noaa-scheduler.sh, you need to kill the existing jobs and restart them. Kill the jobs with:

atq | awk '{print $1}' | sort -n | xargs atrm

Sources:
https://www.reddit.com/r/RTLSDR/comments/5iuzbl/fully_automated_raspberry_pi_noaa_satellite/
https://gist.github.com/floehopper/99a0c8931f9d779b0998
https://github.com/the2belo/wxsat-scheduler

There is another piece of software to automatically receive NOAA and Meteor satellites, I have yet to check it out.

Link: https://github.com/cyber-atomus/autowx

EDIT: I have experienced some lag with webm streaming, scroll down for swf configuration.

Live stream camera (Foscam in my case) over the internet cheat sheet:

apt-get install software-properties-common
add-apt-repository ppa:mc3man/trusty-media
apt-get update
apt-get install ffmpeg

Create ffserver.conf file

nano /etc/ffserver.conf
HTTPPort 8090 # Port to bind the server to
HTTPBindAddress 0.0.0.0
MaxHTTPConnections 2000
MaxClients 15
MaxBandwidth 50000 # Maximum bandwidth per client
 # set this high enough to exceed stream bitrate
CustomLog -

<Feed feed1.ffm>
 File /tmp/feed1.ffm
 FileMaxSize 1g
 ACL allow localhost
 ACL allow 192.168.0.0 192.168.0.255
</Feed>

<Stream live.webm>
 Format webm
 Feed feed1.ffm

 VideoCodec libvpx
 VideoSize 640x480
 VideoFrameRate 30
 VideoBitRate 512
 VideoBufferSize 512
 NoAudio
 AVOptionVideo flags +global_header
 StartSendOnKey

</Stream>

<Stream status.html> # Server status URL
 Format status
 # Only allow local people to get the status
 ACL allow localhost
 ACL allow 192.168.0.0 192.168.0.255
</Stream>

<Redirect index.html> # Just an URL redirect for index
 # Redirect index.html to the appropriate site
 URL live.webm/
</Redirect>

Save, exit and start the services

ffserver > /dev/null 2>&1 &
ffmpeg -loglevel error -r 30 -i rtsp://user:pass@camera_IP:port/url http://LAN_IP_WHERE_FFSERVER_IS_RUNNING:8090/feed1.ffm /dev/null 2>/var/log/ffmpeg.log &

Visit http://LAN_IP_WHERE_FFSERVER_IS_RUNNING:8090/live.webm
For status go to http://LAN_IP_WHERE_FFSERVER_IS_RUNNING:8090/status.html

I have experienced some lag with the above configuration, here is the config for a swf streaming:

HTTPPort 8090 # Port to bind the server to
HTTPBindAddress 0.0.0.0
MaxHTTPConnections 2000
MaxClients 15
MaxBandwidth 50000 # Maximum bandwidth per client
 # set this high enough to exceed stream bitrate
CustomLog -

<Feed feed1.ffm>
 File /tmp/feed1.ffm
 FileMaxSize 1g
 ACL allow localhost
 ACL allow 192.168.0.0 192.168.0.255
</Feed>

<Stream live.ffm>
 Feed feed1.ffm

Format swf
VideoCodec flv
VideoFrameRate 30
VideoBufferSize 80000
VideoBitRate 100
# quality ranges - 1-31 (1 = best, 31 = worst)
VideoQMin 1
VideoQMax 5
VideoSize 1024x786
PreRoll 0
Noaudio

</Stream>

<Stream status.html> # Server status URL
 Format status
 # Only allow local people to get the status
 ACL allow localhost
 ACL allow 192.168.0.0 192.168.0.255
</Stream>

<Redirect index.html> # Just an URL redirect for index
 # Redirect index.html to the appropriate site
 URL live.ffm/
</Redirect>

Save and restart ffserver and ffmpeg.

Edit your index.html file to play swf file in the browser:

<html>
<head>
<meta charset="UTF-8">
<link rel="icon" href="favicon.ico" type="image/x-icon" />
<link rel="shortcut icon" href="favicon.ico" type="image/x-icon" />
<title>Site title</title>
<script type="text/javascript" src="swfobject.js"></script>
 <script type="text/javascript">
 swfobject.registerObject("myFlashContent", "9.0.0", "expressInstall.swf");
 </script>
 <script type="text/javascript">
 swfobject.embedSWF("http://FF_SERVER_IP:8090/live.ffm", "myContent", "1024", "786", "9.0.0");
 </script>
</head>
<body>
<p align="top">Some optional text.</p>
 <div id="myContent">
 <p>Live stream</p>
 </div>
</video>

</body>

Enable I2C with raspi-config and reboot

Check if the sensor is connected:

i2cdetect -y 1

The output should look like this:

root@rpikiosk:~# i2cdetect -y 1
     0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f
00:          -- -- -- -- -- -- -- -- -- -- -- -- --
10: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
20: -- -- -- 23 -- -- -- -- -- -- -- -- -- -- -- --
30: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
40: 40 -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
50: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
60: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
70: -- -- -- -- -- -- -- 77

Our HTU21D device is detected as the number 40 in the line 40, so the address is 0X40.

#!/usr/bin/python
import struct, array, time, io, fcntl

I2C_SLAVE=0x0703
HTU21D_ADDR = 0x40
CMD_READ_TEMP_HOLD = "\xE3"
CMD_READ_HUM_HOLD = "\xE5"
CMD_READ_TEMP_NOHOLD = "\xF3"
CMD_READ_HUM_NOHOLD = "\xF5"
CMD_WRITE_USER_REG = "\xE6"
CMD_READ_USER_REG = "\xE7"
CMD_SOFT_RESET= "\xFE"

class i2c(object):
   def __init__(self, device, bus):

      self.fr = io.open("/dev/i2c-"+str(bus), "rb", buffering=0)
      self.fw = io.open("/dev/i2c-"+str(bus), "wb", buffering=0)

      # set device address

      fcntl.ioctl(self.fr, I2C_SLAVE, device)
      fcntl.ioctl(self.fw, I2C_SLAVE, device)

   def write(self, bytes):
      self.fw.write(bytes)

   def read(self, bytes):
      return self.fr.read(bytes)

   def close(self):
      self.fw.close()
      self.fr.close()

class HTU21D(object):
   def __init__(self):
      self.dev = i2c(HTU21D_ADDR, 1) #HTU21D 0x40, bus 1
      self.dev.write(CMD_SOFT_RESET) #soft reset
      time.sleep(.1)

   def ctemp(self, sensorTemp):
      tSensorTemp = sensorTemp / 65536.0
      return -46.85 + (175.72 * tSensorTemp)

   def chumid(self, sensorHumid):
      tSensorHumid = sensorHumid / 65536.0
      return -6.0 + (125.0 * tSensorHumid)

   def crc8check(self, value):
      # Ported from Sparkfun Arduino HTU21D Library: https://github.com/sparkfun/HTU21D_Breakout
      remainder = ( ( value[0] << 8 ) + value[1] ) << 8
      remainder |= value[2]
      
      # POLYNOMIAL = 0x0131 = x^8 + x^5 + x^4 + 1
      # divsor = 0x988000 is the 0x0131 polynomial shifted to farthest left of three bytes
      divsor = 0x988000
      
      for i in range(0, 16):
         if( remainder & 1 << (23 - i) ): remainder ^= divsor divsor = divsor >> 1
      
      if remainder == 0:
         return True
      else:
         return False
   
   def read_tmperature(self):
      self.dev.write(CMD_READ_TEMP_NOHOLD) #measure temp
      time.sleep(.1)

      data = self.dev.read(3)
      buf = array.array('B', data)

      if self.crc8check(buf):
         temp = (buf[0] << 8 | buf [1]) & 0xFFFC
         return self.ctemp(temp)
      else:
         return -255
         
   def read_humidity(self):
      self.dev.write(CMD_READ_HUM_NOHOLD) #measure humidity
      time.sleep(.1)

      data = self.dev.read(3)
      buf = array.array('B', data)
      
      if self.crc8check(buf):
         humid = (buf[0] << 8 | buf [1]) & 0xFFFC
         return self.chumid(humid)
      else:
         return -255
         
if __name__ == "__main__":
   obj = HTU21D()
   print "Temp:", obj.read_tmperature(), "C"
   print "Humid:", obj.read_humidity(), "% rH"

Run the script:

root@rpikiosk:~# ./HTU21DF.py
Temp: 22.9382104492 C
Humid: 45.0101318359 % rH

If you connect the ground wire to the wrong Rpi pin, you’ll get this error:

IOError: [Errno 5] Input/output error

 

Pi-Hole is a DNS based ad blocker. You can install it on a Debian based Linux distributions. To use it together with a PfSense firewall you need to change a few firewall settings:

  • Login to your PfSense and go to System -> General Setup:

In the first DNS Server field insert the IP of your Pi-Hole computer and save changes

  • Go to Services -> DNS Resolver:

Tick Enable DNS resolver

Tick Enable DNSSEC Support

Tick Enable Forwarding Mode (this will tell the PfSense to use a DNS server that we configured in a general settings in the previous step)

Save and that’s it.

This post will show you how to generate weather data from LA Crosse WS2300 series weather stations for submission to the APRS network.

APRS software I’m using is Xastir, but you can use this setup to push the weather data to other APRS applications also. Digipeater mode is already set up in Xastir. This post will not cover digipeating WX data.

1. Install packages

apt-get update

apt-get install  bc ncat unzip gcc build-essential

2. Download open2300 and compile fetch2300

Fetch2300 is part of open2300, it’s used to connect to the weather station and return the data. Credit goes to Kenneth Lavrsen (http://www.lavrsen.dk/foswiki/bin/view/Open2300)

wget -O "open2300-1-10.zip" https://sourceforge.net/projects/open2300/files/open2300/1.10/open2300-1.10.zip/download"

unzip open2300-1-10.zip

cd open2300-1.10

make fetch2300

cp fetch2300 /usr/local/bin/fetch2300



3. Edit  and rename open2300 config file named open2300-dist.conf or copy mine to /etc/open2300/open2300.conf

mkdir -p /etc/open2300

nano /etc/open2300/open2300.conf
SERIAL_DEVICE /dev/ttyUSB1 #/dev/ttyS0, /dev/ttyS1, COM1, COM2 etc
TIMEZONE 1 # Hours Relative to UTC. East is positive, west is negative
WIND_SPEED km/h # select MPH (miles/hour), m/s, or km/h
TEMPERATURE F # Select C or F
RAIN IN # Select mm or IN
PRESSURE hPa # Select hPa, mb or INHG

Note: make sure you select the correct SERIAL_DEVICE in the config. The station in my setup

is connected via USB to RS232 converter so the device is ttyUSB0. Also, don’t change the units, bash script will automatically convert them to be compatible with APRS network.

4. Copy bash script to desired location

mkdir -p /root/ws2300/

nano wxdata_v1.6.sh
#!/bin/bash

#This script reads weather data via fetch program which is part of Open2300 suite written by Kenneth Lavrsen (http://www.
#lavrsen.dk/foswiki/bin/view/Open2300/WebHome).
#It outputs the right data needed to feed Xastir for APRS weather reports. The scripts utilizes Ncat utility as server to
#serve the fetched output to Xastir.
#Fetched Data is pushed to Ncat server and then to Xastir. (Fetched data -> Ncat server -> Xastir)
#Ncat is part of Nmap, get it by installing Nmap.
#This script should work for LaCrosse weather stations, WS23xx series. Testing was done with WS2307.
#Written by S55MA and S56IUL, May 2016

#DEFINE VARIABLES
host="127.0.0.1"
port="1234"

#Start the Ncat server
chkncat=$(netstat -ant | grep $host:$port | grep -c LISTEN)
if [ "$chkncat" -ge "1" ]
then
echo "ncat already running, nothing to do"
else
nohup ncat -k -l --broker $host $port &>/dev/null &
fi

#Start while loop
while true; do

echo "start `date`"

datetime=$(date '+%Y%m%d%H%M%S')
ws2300config="/etc/open2300/open2300.conf"
/usr/local/bin/fetch2300 $ws2300config > /tmp/wxdata-"$datetime".tmp
fetch_path="/tmp/wxdata-$datetime.tmp"
chkfile=$(ls -la $fetch_path | awk -F ' ' '{ print $5 }')

if [ "$chkfile" -le "43" ]
then
echo "No Data"
sleep 30
else
tempF=$(cat "$fetch_path" | grep To | grep -v 'min\|max\|DRtot\|TRtot' | awk '{print $2}')
temp1=$(echo "$tempF" | awk '{ printf ("%d\n",$1 + 0.5)}')
if [ "$temp1" -ge "99" ] || [ "$temp1" -le "-99" ]
then
temp="$temp1"
else
if [ "$temp1" -le "-1" ]
then
if [ "$temp1" -ge "-9" ]
then
temp2=$(echo "$temp1" | sed 's/[-]//g')
temp=$(echo -0"$temp2")
else
temp2=$(echo "$temp1" | sed 's/[-]//g')
temp=$(echo -"$temp1")
fi
else
temp=$(echo 0"$temp1")
fi
fi

windspeed2=$(cat "$fetch_path" | grep -m1 WS | grep -v 'min\|max\|DRtot\|TRtot'| awk '{print $2/1.609344}' | awk '{ printf ("%d\n",$1 + 0.5)}')
if [ "$windspeed2" -le "9" ]
then
windspeed=$(echo 00"$windspeed2")
else
if [ "$windspeed2" -le "99" ]
then
windspeed=$(echo 0"$windspeed2")
else
windspeed=$(echo "$windspeed2")
fi
fi

winddirection2=$(cat "$fetch_path" | grep DIR0 | awk '{print $2}' | sed 's/\..*$//')
if [ "$winddirection2" -le "9" ]
then
winddirection=$(echo 00"$winddirection2")
else
if [ "$winddirection2" -le "99" ]
then
winddirection=$(echo 0"$winddirection2")
else
winddirection=$(echo "$winddirection2")
fi
fi

rain1h=$(cat "$fetch_path" | grep R1h | grep -v 'min\|max' | awk '{print $2}' | sed 's/[.]//g')
rain24h=$(cat "$fetch_path" | grep R24h | grep -v 'min\|max' | awk '{print $2}' | sed 's/[.]//g')

airpressureR=$(cat "$fetch_path" | grep RP | grep -v 'min\|max' | awk '{print $2}')
airpressure2=$(echo "scale=1;$airpressureR / 1" | bc | sed 's/[.]//g')
if [ "$airpressure2" -le "9999" ]
then
airpressure=$(echo 0"$airpressure2")
else
airpressure=$(echo "$airpressure2")
fi

relhumidity=$(cat "$fetch_path" | grep RHo | grep -v 'min\|max' | awk '{print $2}' | sed 's/\..*$//')

#Combine variables to forge Xastir string
xastir="c${winddirection}s${windspeed}t${temp}r${rain1h}p${rain24h}b${airpressure}h${relhumidity}xDvs"
printf "%s\n" "$xastir" | ncat --send-only $host $port
echo "$xastir"
sleep 3

rm -f /tmp/wxdata-*.tmp

echo "stop `date`"
echo "-----------------------------------"
fi

done

#EOS



If there are some formatting mistakes, the script is also available on pastebin: http://pastebin.com/29q8epF8

5. Start the script

/root/ws2300/wxdata_v1.6.sh

You should see output similar to that:

start Tue May 3 00:36:47 CEST 2016
c112s013t048b10037h85xDvs

Leave the script running, open another terminal and check if ncat is getting data:

ncat localhost 1234

Wait a few seconds and you should get the result similar to that:

c112s018t048b10037h85xDvs



6. Go to Xastir and add the WX interface

Interface -> Interface Control -> Add -> Networked WX

WX Host: 127.0.0.1

WX Port: 1234

Save and start the interface.

wx_interface

7. Go to Xastir, View -> Own Weather Data

You should see your own weather data from the station

wx_data

8. Start the script at boot and run it in background

Open /etc/rc.local and add

screen -d -m /root/ws2300/wxdata_v1.6.sh

Save and exit

Note: make sure you set your own path of the script location

 

My 5V RaspberryPi fan was very loud and annoying, so I wanted a solution to regulate the fan depending on the Pi temperature.

To achieve that, I used a mosfet switch, pulled out from an old computer motherboard, followed by some easy bash scripting.

Almost any mosfet from an old computer motherboard should work.

motherboard_mosfet

Wiring:

mosfet_fan_schematics

You can use any not used GPIO to connect the gate pin, in the picture above it’s #23.

mosfet_fan_schematics2

I connected the mosfet pin to GPIO 19 in my case. You can choose  your own.

Software:

sudo apt-get update
sudo apt-get install bc

Enable selected GPIO pin, In my case, GPIO 19

echo "19" > /sys/class/gpio/export
echo "out" > /sys/class/gpio/gpio19/direction

If you wan’t to select another GPIO and you already exported one GPIO, you need to unexport previous GPIO.

echo "19" > /sys/class/gpio/unexport

Bash script for controlling fan “speed”:

pico rpipwm.sh

Paste the script bellow, edit your GPIO pin and save:

#!/bin/bash
#Poor man's PWM
#16.11.2015 by S55MA
#Quick and dirty script for controlling fan speed on RaspberryPI
#No rights reserved

#Define GPIO pin
pin="19"

while true; do

#Read temp
temp=$(cat /sys/class/thermal/thermal_zone0/temp | awk 'NR == 1 { print $1 / 1000}' | cut -c -4)

#If temperature is equal or lower than 39.99, the fan will stop spinning
if [[ $(bc <<< "$temp <= 39.99") == 1 ]] ;
then
$(echo "0" > /sys/class/gpio/gpio$pin/value)
fi

#If temperature is between 40 and 42.99, the fan will start with 1 second burst and 1 second sleep
if [[ $(bc <<< "$temp >= 40 && $temp <= 42.99") == 1 ]] ;
then
$(echo "1" > /sys/class/gpio/gpio$pin/value; sleep 1; echo "0" > /sys/class/gpio/gpio$pin/value; sleep 1)
fi

#If temperature is between 43 and 47.99, the fan will start with 1 second burst and 0.5 second sleep
if [[ $(bc <<< "$temp > 43 && $temp <= 47.99") == 1 ]] ;
then
$(echo "1" > /sys/class/gpio/gpio$pin/value; sleep 1; echo "0" > /sys/class/gpio/gpio$pin/value; sleep 0.5)
fi

#If temperature is equal or higher than 48, the fan will start spinning constantly
if [[ $(bc <<< "$temp >= 48") == 1 ]] ;
then
$(echo "1" > /sys/class/gpio/gpio$pin/value)
fi
sleep 0.1
done
chmod +x rpipwm.sh
./rpipwm.sh

Enable auto start at boot:

sudo cp rpipwm.sh /etc/init.d/rpipwm.sh
sudo chmod +x /etc/init.d/rpipwm.sh
sudo update-rc.d /etc/init.d/rpipwm.sh defaults

or you can use crontab:

crontab -e

Add and save:

@reboot /path/to/script/rpipwm.sh
sudo update-rc.d cron defaults

The rpisystat script will report the following system parameters to your LCD:

  • Internal IP
  • External IP
  • CPU temperature
  • GPU temperature
  • CPU usage
  • Memory usage
  • Free disk space
  • Incoming and outgoing network traffic

You need:

1. Breadboard with T-Cobbler (or connect the wires directly)
2. 16×2 LCD board
3. Adjustable resistor (potentiometer) for adjusting LCD contrast

1. Wiring (source: https://learn.adafruit.com/drive-a-16×2-lcd-directly-with-a-raspberry-pi/wiring):

Pin #1 of the LCD goes to ground (black wire)
Pin #2 of the LCD goes to +5V (red wire)
Pin #3 (Vo) connects to the middle of the potentiometer (orange wire)
Pin #4 (RS) connects to the Cobbler #25 (yellow wire)
Pin #5 (RW) goes to ground (black wire)
Pin #6 (EN) connects to Cobbler #24 (green wire)
Skip LCD Pins #7, #8, #9 and #10
Pin #11 (D4) connects to cobbler #23 (blue wire)
Pin #12 (D5) connects to Cobbler #17 (violet wire)
Pin #13 (D6) connects to Cobber #21 (gray wire)
Pin #14 (D7) connects to Cobber #22 (white wire)
Pin #15 (LED +) goes to +5V (red wire)
Pin #16 (LED -) goes to ground (black wire)

raspberry_pi_pi-char-lcd

This schematics is for RaspberryPi 1 version, but you can connect to the same pins on RaspberryPi 2 (picture bellow).

16x2_lcd_display_rpi2

2. Software

Download required packages:

sudo apt-get update
sudo apt-get install python-dev python-setuptools python-pip git
sudo easy_install -U distribute
sudo pip install rpi.gpio

Download my scripts:

sudo git clone git://github.com/s55ma/16-2-LCD-rpisystat.git
cd 16-2-LCD-rpisystat
sudo ./rpisystat.py

Make sure you edit rpisystat.py to match your GPIO pins (default is for the wiring above). Also edit rx.sh and tx.sh to match your network adapter.

Check the display in action: https://www.youtube.com/watch?v=5YkLTBd5-bw

Scripts: https://github.com/s55ma/16-2-LCD-rpisystat

References: https://learn.adafruit.com/drive-a-16×2-lcd-directly-with-a-raspberry-pi/wiring

In my previous post, I was writting about how to graph temperature and humidity from AM2302 sensor on RasberryPi. In addition, we will add dewpoint monitoring. We need two variables to calculate dewpoint, temperature and humidity. I took some already made scripts and combined them together to fit my needs. I wanted to calculate dewpoint completely with bash and bc, but since I’m to lazy, I just used the python script from this blog.

Get to the root shell (we don’t want to type sudo everytime):

sudo -s

Create python script (for dewpoint calculations):

pico /opt/dewpoint.py

Paste the code, save and exit (CTRL + C), Y, ENTER

import sys
import numpy as np

# approximation valid for
# 0 degC < T < 60 degC
# 1% < RH < 100%
# 0 degC < Td < 50 degC

# constants
a = 17.271
b = 237.7 # degC

# sys.argv[0] is program name
T=float(sys.argv[1])
RH=float(sys.argv[2])


def dewpoint_approximation(T,RH):

    Td = (b * gamma(T,RH)) / (a - gamma(T,RH))

    return Td


def gamma(T,RH):

    g = (a * T / (b + T)) + np.log(RH/100.0)

    return g


Td = dewpoint_approximation(T,RH)
print Td

Make the script executable:

chmod +x /opt/dewpoint.py

Create plugin file:

pico /etc/munin/plugins/dewpoint

Paste the code, save and exit (CTRL + C), Y, ENTER

#!/bin/sh

case $1 in
config)
cat <<'EOM'
graph_title Dewpoint
graph_vlabel Celsius
graph_category AM2302
dewpoint.label Temperature
dewpoint.draw AREASTACK
dewpoint.colour 403075
EOM
exit 0;;
esac

humidity=$(/opt/lol_dht22/loldht 7 | grep -i "humidity" | cut -d ' ' -f3)
temperature=$(/opt/lol_dht22/loldht 7 | grep -i "temperature" | cut -d ' ' -f7)

printf "dewpoint.value "
python /opt/dewpoint.py $temperature $humidity
chmod +x /etc/munin/plugins/dewpoint

Open munin-node file:

pico /etc/munin/plugin-conf.d/munin-node

Add the line at the end of the file, save and exit:

[dewpoint*]

Restart services:

munin-node-configure
/etc/init.d/munin-node restart

20151021_150632

Hardware:

  • RasberryPi 2
  • AM2302 humidity/temperature sensor
  • Some wires from old PCs to connect sensor with RaspberryPi

Software:

  • Raspbian OS
  • Nginx
  • Munin
  • WiringPi
  • Lol_dht22

1. Solder wires to the sensor like on the picture above, and connect them to the correct pins:

Pin 1 on the AM2302 to pin 1 (+3.3V) on the GPIO connector (labeled P1 on the raspi)
Pin 2 on the AM2302 to pin 7 (GPIO 4) on the GPIO connector
Pin 4 on the AM2302 to pin 9 (Ground) on the GPIO connector

For detailed instructions, check this blog up to step 4: https://hackaday.io/project/3766/instructions

All shell commands will be run as root, so I will not use sudo.

2. Install Nginx (web server)

apt-get update
apt-get install nginx php5-fpm

3. Install Munin

Muning is a monitoring tool for sysadmins. It creates graphs to monitor various parameters. We will configure munin to display AM2302 sensor in graphs.

apt-get install munin munin-node munin-plugins-extra

Edit munin configuration file:

pico /etc/munin/munin.conf
[server.name]
 address 127.0.0.1
 use_node_name yes

4. Configure Nginx virtual host. Dynazoom will work with this config.

pico /etc/nginx/sites-enabled/default or pico /etc/nginx/sites-enabled/your.domain.com
server {
        listen 443 ssl;
        ssl_certificate /etc/nginx/ssl/your.domain.com.crt;
        ssl_certificate_key /etc/nginx/ssl/your.domain.com.key;
        server_name your.domain.com;
        root "/var/cache/munin/www/";
        auth_basic            "Private access";
        auth_basic_user_file  /etc/munin/munin_htpasswd;

        location ^~ /munin-cgi/munin-cgi-graph/ {
                fastcgi_split_path_info ^(/munin-cgi/munin-cgi-graph)(.*);
                fastcgi_param PATH_INFO $fastcgi_path_info;
                fastcgi_pass unix:/var/run/munin/spawn-fcgi-munin-graph.sock;
                include fastcgi_params;
        }

        location /static/ {
                alias /etc/munin/static/;
        }
}

5. Generate SSL cert

mkdir /etc/nginx/ssl
openssl req -subj '/CN=your.domain.com' -x509 -nodes -days 365 -newkey rsa:2048 -keyout /etc/nginx/ssl/your.domain.com.key -out /etc/nginx/ssl/your.domain.com.crt

6. Generate website password

apt-get install apache2-utils
htpasswd -c /etc/munin/munin_htpasswd admin

You will be promted to enter a new password.

7. Add common modules to munin

cd /usr/share/munin/plugins
wget -O pisense_ https://raw.github.com/perception101/pisense/master/pisense_
chmod a+x pisense_
ln -s /usr/share/munin/plugins/pisense_ /etc/munin/plugins/pisense_temp
ln -s /usr/share/munin/plugins/pisense_ /etc/munin/plugins/pisense_clock
pico /etc/munin/plugin-conf.d/munin-node
[pisense_*]
user root

8. Configure AM2302 prerequisites

apt-get install git-core
cd /opt/
git clone git://git.drogon.net/wiringPi
cd wiringPi
./build
cd /opt/
git clone https://github.com/technion/lol_dht22
cd lol_dht22
./configure
make

9. Create plugins for munin

pico /etc/munin/plugins/DHT22-humidity
#!/bin/sh

case $1 in
 config)
 cat <<'EOM'
graph_title Relative humidity
graph_vlabel Percent
graph_category AM2302
humidity.label RH
humidity.draw AREASTACK
humidity.colour 3E9BFB
EOM
 exit 0;;
esac

printf "humidity.value "
/opt/lol_dht22/loldht 7 | grep -i "humidity" | cut -d ' ' -f3
chmod +x /etc/munin/plugins/DHT22-humidity
pico /etc/munin/plugins/DHT22-temperature
#!/bin/sh

case $1 in
 config)
 cat <<'EOM'
graph_title Temperature
graph_vlabel Celsius
graph_category AM2302
temperature.label Celsius
temperature.label Temperature
temperature.draw AREASTACK
temperature.colour 00FF00
EOM
 exit 0;;
esac

printf "temperature.value "
/opt/lol_dht22/loldht 7 | grep -i "temperature" | cut -d ' ' -f7
chmod +x /etc/munin/plugins/DHT22-temperature
pico /etc/munin/plugin-conf.d/munin-node

Add this to the end of the file:

[DHT22-*]
user root

10. Enable Dynazoom for graphs

apt-get install spawn-fcgi libcgi-fast-perl
pico /etc/init.d/munin-fastcgi
#! /bin/sh

### BEGIN INIT INFO
# Provides: spawn-fcgi-munin-graph
# Required-Start: $all
# Required-Stop: $all
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Description: starts FastCGI for Munin-Graph
### END INIT INFO
# --------------------------------------------------------------
# Munin-CGI-Graph Spawn-FCGI Startscript by Julien Schmidt
# eMail: munin-trac at julienschmidt.com
# www: http://www.julienschmidt.com
# --------------------------------------------------------------
# Install: 
# 1. Copy this file to /etc/init.d
# 2. Edit the variables below
# 3. run "update-rc.d spawn-fcgi-munin-graph defaults"
# --------------------------------------------------------------
# Special thanks for their help to:
# Frantisek Princ
# Jérôme Warnier
# --------------------------------------------------------------
# Last Update: 14. February 2013
#
# Please change the following variables:

PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
NAME=spawn-fcgi-munin-graph
PID_FILE=/var/run/munin/$NAME.pid
SOCK_FILE=/var/run/munin/$NAME.sock
SOCK_USER=www-data
FCGI_USER=www-data
FCGI_GROUP=www-data
FCGI_WORKERS=2
DAEMON=/usr/bin/spawn-fcgi
DAEMON_OPTS="-s $SOCK_FILE -F $FCGI_WORKERS -U $SOCK_USER -u $FCGI_USER -g $FCGI_GROUP -P $PID_FILE -- /usr/lib/munin/cgi/munin-cgi-graph"

# --------------------------------------------------------------
# No edits necessary beyond this line
# --------------------------------------------------------------

if [ ! -x $DAEMON ]; then
 echo "File not found or is not executable: $DAEMON!"
 exit 0
fi

status() {
 if [ ! -r $PID_FILE ]; then
 return 1
 fi
 
 for FCGI_PID in `cat $PID_FILE`; do 
 if [ -z "${FCGI_PID}" ]; then
 return 1
 fi
 
 FCGI_RUNNING=`ps -p ${FCGI_PID} | grep ${FCGI_PID}`
 if [ -z "${FCGI_RUNNING}" ]; then
 return 1
 fi
 done;
 
 return 0
}
 
start() {
 if status; then
 echo "FCGI is already running!"
 exit 1
 else
 $DAEMON $DAEMON_OPTS
 fi
}

stop () { 
 if ! status; then
 echo "No PID-file at $PID_FILE found or PID not valid. Maybe not running"
 exit 1
 fi
 
 # Kill processes
 for PID_RUNNING in `cat $PID_FILE`; do
 kill -9 $PID_RUNNING
 done
 
 # Remove PID-file
 rm -f $PID_FILE
 
 # Remove Sock-File
 rm -f $SOCK_FILE
}

case "$1" in
 start)
 echo "Starting $NAME: "
 start
 echo "... DONE"
 ;;

 stop)
 echo "Stopping $NAME: "
 stop
 echo "... DONE"
 ;;

 force-reload|restart)
 echo "Stopping $NAME: "
 stop
 echo "Starting $NAME: "
 start
 echo "... DONE"
 ;;
 
 status)
 if status; then
 echo "FCGI is RUNNING"
 else
 echo "FCGI is NOT RUNNING"
 fi
 ;;
 
 *)
 echo "Usage: $0 {start|stop|force-reload|restart|status}"
 exit 1
 ;;
esac

exit 0
chmod 755 /etc/init.d/munin-fastcgi
update-rc.d munin-fastcgi defaults
/etc/init.d/munin-fastcgi start

11. Restart daemons and visit your munin site

munin-node-configure

/etc/init.d/nginx restart

/etc/init.d/munin-node restart

Go to https://your.domain.com/munin/

 

Check how to add dewpoint graph on my next post.

References:

 

This is duplicate of my original post: https://troubleshoot-coltpython.blogspot.com/2015/05/ubuntu-nginx-php5-fpm-mariadb-varnish.html

This is my cheat sheet for future reference in case I forget. It’s composed from various blogs around the interwebz and modified to suit my needs. I thought someone might find this useful.
I won’t go into much details about config definitions, it’s cheat sheet after all 🙂

We’ll setup Varnish caching engine with Nginx. Nginx will also serve as reverse proxy for SSL (https) requests.

My working environment:

Ubuntu 14.04.2 LTS | Trusty
Nginx 1.4.6
PHP 5.5.9
Varnish: 4.0.3-2
MariaDB: 5.5.43

Important config sections are highlighted. You need to change this.

Varnish will listen on port 80 and forward requests to backend (Nginx) listening on port 8080.

We are doing this as root:

sudo -s

 

1. Install Varnish

 

apt-get install apt-transport-https

curl https://repo.varnish-cache.org/GPG-key.txt | apt-key add -

echo "deb https://repo.varnish-cache.org/ubuntu/ trusty varnish-4.0" &amp;amp;amp;gt;&amp;amp;amp;gt; /etc/apt/sources.list.d/varnish-cache.list

apt-get update

apt-get install varnish

 

2. Install Nginx + Naxsi

 

echo "deb http://ppa.launchpad.net/nginx/stable/ubuntu $(lsb_release -sc) main" | sudo tee /etc/apt/sources.list.d/nginx-stable.list

apt-key adv --keyserver keyserver.ubuntu.com --recv-keys C300EE8C

apt-get update

apt-get install nginx

apt-get install nginx-naxsi

apt-get install libpcre3-dev libssl-dev

 

3. Install PHP5-FPM

 

apt-get install php5-fpm

 

4. Install MariaDB server

 

apt-get install mariadb-server

 

5. Configure Nginx (I also use CloudFlare in front)


Generate certificates for SSL sessions:

mkdir -p /etc/nginx/ssl

openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout /etc/nginx/ssl/domain.example.key -out /etc/nginx/ssl/domain.example.crt


Edit nginx.conf:

nano /etc/nginx/nginx.conf


user www-data;
worker_processes 4;
pid /run/nginx.pid;

events {
worker_connections 768;
multi_accept on;
use epoll;
}

http {

#Basic Settings

#Cloudflare real IP
set_real_ip_from 199.27.128.0/21;
set_real_ip_from 173.245.48.0/20;
set_real_ip_from 103.21.244.0/22;
set_real_ip_from 103.22.200.0/22;
set_real_ip_from 103.31.4.0/22;
set_real_ip_from 141.101.64.0/18;
set_real_ip_from 108.162.192.0/18;
set_real_ip_from 190.93.240.0/20;
set_real_ip_from 188.114.96.0/20;
set_real_ip_from 197.234.240.0/22;
set_real_ip_from 198.41.128.0/17;
set_real_ip_from 162.158.0.0/15;
set_real_ip_from 104.16.0.0/12;
set_real_ip_from 172.64.0.0/13;
set_real_ip_from 2400:cb00::/32;
set_real_ip_from 2606:4700::/32;
set_real_ip_from 2803:f800::/32;
set_real_ip_from 2405:b500::/32;
set_real_ip_from 2405:8100::/32;
real_ip_header CF-Connecting-IP;

#Varnish get real IP
real_ip_header X-Forwarded-For;
set_real_ip_from 127.0.0.1;

#TCP and buffer settings
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 20;
client_max_body_size 15m;
client_body_timeout 60;
client_header_timeout 60;
#client_body_buffer_size 1K;
#client_header_buffer_size 1k;
#large_client_header_buffers 4 8k;
send_timeout 60;
reset_timedout_connection on;
types_hash_max_size 2048;
server_tokens off;

include /etc/nginx/mime.types;
default_type application/octet-stream;

#Logging Settings

access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log;

#Gzip Settings
gzip on;
gzip_disable “msie6”;
gzip_vary on;
gzip_proxied any;
gzip_comp_level 6;
gzip_min_length 512;
gzip_buffers 16 8k;
gzip_http_version 1.1;
#gzip_types text/css text/javascript text/xml text/plain text/x-component application/javascript application/x-javascript application/json application/xml application/rss+xml font/truetype application/x-font-ttf font/opentype application/vnd.ms-fontobject image/svg+xml;


#Uncomment it if you installed nginx-naxsi
include /etc/nginx/naxsi_core.rules;

#Virtual Host Configs
include /etc/nginx/conf.d/*.conf;
include /etc/nginx/sites-enabled/*;
}

nano /etc/nginx/sites-enabled/example.domain.conf


server {
listen 127.0.0.1:8080;
server_name example.domain.com www.example.domain.com;
root “/home/example.domain/public_html”;

index index.php;
client_max_body_size 10m;

access_log /home/example.domain/_logs/access.log;
error_log /home/example.domain/_logs/error.log;

if ($http_user_agent ~* (Baiduspider|webalta|nikto|wkito|pikto|scan|acunetix|morfeus|webcollage|youdao) ) {
return 401;
}

if ($http_user_agent ~* (HTTrack|clshttp|archiver|loader|email|harvest|extract|grab|miner) ) {
return 401;
}

location ~ /(\.|wp-config.php|readme.html|license.txt) {
return 404;
}

# Add trailing slash to */wp-admin requests.
rewrite /wp-admin$ $scheme://$host$uri/ permanent;

location / {
include /etc/nginx/naxsi.rules;
try_files $uri $uri/ /index.php$uri?$args;
}

#Needed by Naxsi
location /RequestDenied {
return 403;
}

location ~ “^(.+\.php)($|/)” {
fastcgi_split_path_info ^(.+\.php)(.*)$;

fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_param SCRIPT_NAME $fastcgi_script_name;
fastcgi_param PATH_INFO $fastcgi_path_info;
fastcgi_param SERVER_NAME $host;

if ($uri !~ “^/uploads/”) {
fastcgi_pass unix:/var/run/example.domain_fpm.sock; #You can use TCP connection instead or single sock for all sites. You can define this (pools) in /etc/php5/fpm/pool.d/ 
}
include fastcgi_params;
}

location ~* \.(js|css|png|jpg|jpeg|gif|ico)$ {
expires max;
log_not_found off;
access_log off;
}

location ~* \.(html|htm)$ {
expires 30m;
}

location ~* /\.(ht|git|svn) {
deny all;
}

#Add compression
gzip on;
gzip_comp_level 2;
gzip_min_length 1000;
gzip_proxied expired no-cache no-store private auth;
gzip_types text/plain application/x-javascript text/xml text/css application/xml;

}

server {
listen 443 ssl;
server_name example.domain.com www.example.domain.com;
ssl_certificate /etc/nginx/ssl/example.domain.crt;
ssl_certificate_key /etc/nginx/ssl/example.domain.key;

location / {

## Pass the request on to Varnish.
location / {
proxy_pass http://127.0.0.1:80;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto https;
proxy_set_header X-Forwarded-Port 443;
proxy_set_header Host $host;
proxy_redirect off;
                }
         }
}

6. Configure naxsi for Nginx (Optimized for WordPress):

 

nano /etc/nginx/naxsi.rules


# Sample rules file for default vhost.

#LearningMode;
SecRulesEnabled;
#SecRulesDisabled;
DeniedUrl “/RequestDenied”;

## check rules
CheckRule “$SQL >= 8” BLOCK;
CheckRule “$RFI >= 8” BLOCK;
CheckRule “$TRAVERSAL >= 4” BLOCK;
CheckRule “$EVADE >= 4” BLOCK;
CheckRule “$XSS >= 8” BLOCK;

# WordPress naxsi rules

### HEADERS
BasicRule wl:1000,1001,1005,1007,1010,1011,1013,1100,1200,1308,1309,1310,1311,1315 “mz:$HEADERS_VAR:cookie”;
# xmlrpc
BasicRule wl:1402 “mz:$HEADERS_VAR:content-type”;

### simple BODY (POST)
BasicRule wl:1001,1015,1009,1311,1310,1101,1016 “mz:$URL:/|$BODY_VAR:customized”;
# comments
BasicRule wl:1000,1010,1011,1013,1015,1200,1310,1311 “mz:$BODY_VAR:post_title”;
BasicRule wl:1000 “mz:$BODY_VAR:original_publish”;
BasicRule wl:1000 “mz:$BODY_VAR:save”;
BasicRule wl:1008,1010,1011,1013,1015 “mz:$BODY_VAR:sk2_my_js_payload”;
BasicRule wl:1001,1009,1005,1016,1100,1310 “mz:$BODY_VAR:url”;
BasicRule wl:1009,1100 “mz:$BODY_VAR:referredby”;
BasicRule wl:1009,1100 “mz:$BODY_VAR:_wp_original_http_referer”;
BasicRule wl:1000,1001,1005,1008,1007,1009,1010,1011,1013,1015,1016,1100,1200,1302,1303,1310,1311,1315,1400 “mz:$BODY_VAR:comment”;
BasicRule wl:1100 “mz:$BODY_VAR:redirect_to”;
BasicRule wl:1000,1009,1315 “mz:$BODY_VAR:_wp_http_referer”;
BasicRule wl:1000 “mz:$BODY_VAR:action”;
BasicRule wl:1001,1013 “mz:$BODY_VAR:blogname”;
BasicRule wl:1015,1013 “mz:$BODY_VAR:blogdescription”;
BasicRule wl:1015 “mz:$BODY_VAR:date_format_custom”;
BasicRule wl:1015 “mz:$BODY_VAR:date_format”;
BasicRule wl:1015 “mz:$BODY_VAR:tax_input%5bpost_tag%5d”;
BasicRule wl:1015 “mz:$BODY_VAR:tax_input[post_tag]”;
BasicRule wl:1100 “mz:$BODY_VAR:siteurl”;
BasicRule wl:1100 “mz:$BODY_VAR:home”;
BasicRule wl:1000,1015 “mz:$BODY_VAR:submit”;
# news content matches pretty much everything
BasicRule wl:0 “mz:$BODY_VAR:content”;
BasicRule wl:1000 “mz:$BODY_VAR:delete_option”;
BasicRule wl:1000 “mz:$BODY_VAR:prowl-msg-message”;
BasicRule wl:1100 “mz:$BODY_VAR:_url”;
BasicRule wl:1001,1009 “mz:$BODY_VAR:c2c_text_replace%5btext_to_replace%5d”;
BasicRule wl:1200 “mz:$BODY_VAR:ppn_post_note”;
BasicRule wl:1100 “mz:$BODY_VAR:author”;
BasicRule wl:1001,1015 “mz:$BODY_VAR:excerpt”;
BasicRule wl:1015 “mz:$BODY_VAR:catslist”;
BasicRule wl:1005,1008,1009,1010,1011,1015,1315 “mz:$BODY_VAR:cookie”;
BasicRule wl:1101 “mz:$BODY_VAR:googleplus”;
BasicRule wl:1007 “mz:$BODY_VAR:name”;
BasicRule wl:1007 “mz:$BODY_VAR:action”;
BasicRule wl:1100 “mz:$BODY_VAR:attachment%5burl%5d”;
BasicRule wl:1100 “mz:$BODY_VAR:attachment_url”;
BasicRule wl:1001,1009,1100,1302,1303,1310,1311 “mz:$BODY_VAR:html”;
BasicRule wl:1015 “mz:$BODY_VAR:title”;
BasicRule wl:1001,1009,1015 “mz:$BODY_VAR:recaptcha_challenge_field”;
BasicRule wl:1011 “mz:$BODY_VAR:pwd”;
BasicRule wl:1000 “mz:$BODY_VAR:excerpt”;

### BODY|NAME
BasicRule wl:1000 “mz:$BODY_VAR:delete_option|NAME”;
BasicRule wl:1000 “mz:$BODY_VAR:from|NAME”;

### Simple ARGS (GET)
# WP login screen
BasicRule wl:1100 “mz:$ARGS_VAR:redirect_to”;
BasicRule wl:1000,1009 “mz:$ARGS_VAR:_wp_http_referer”;
BasicRule wl:1000 “mz:$ARGS_VAR:wp_http_referer”;
BasicRule wl:1000 “mz:$ARGS_VAR:action”;
BasicRule wl:1000 “mz:$ARGS_VAR:action2”;
# load and load[] GET variable
BasicRule wl:1000,1015 “mz:$ARGS_VAR:load”;
BasicRule wl:1000,1015 “mz:$ARGS_VAR:load[]”;
BasicRule wl:1015 “mz:$ARGS_VAR:q”;
BasicRule wl:1000,1015 “mz:$ARGS_VAR:load%5b%5d”;

### URL
BasicRule wl:1000 “mz:URL|$URL:/wp-admin/update-core.php”;
BasicRule wl:1000 “mz:URL|$URL:/wp-admin/update.php”;
# URL|BODY
BasicRule wl:1009,1100 “mz:$URL:/wp-admin/post.php|$BODY_VAR:_wp_http_referer”;
BasicRule wl:1016 “mz:$URL:/wp-admin/post.php|$BODY_VAR:metakeyselect”;
BasicRule wl:11 “mz:$URL:/xmlrpc.php|BODY”;
BasicRule wl:11 “mz:$URL:/wp-cron.php|BODY”;
BasicRule wl:2 “mz:$URL:/wp-admin/async-upload.php|BODY”;
# URL|BODY|NAME
BasicRule wl:1100 “mz:$URL:/wp-admin/post.php|$BODY_VAR:_wp_original_http_referer|NAME”;
BasicRule wl:1000 “mz:$URL:/wp-admin/post.php|$BODY_VAR:metakeyselect|NAME”;
BasicRule wl:1000 “mz:$URL:/wp-admin/user-edit.php|$BODY_VAR:from|NAME”;
BasicRule wl:1100 “mz:$URL:/wp-admin/admin-ajax.php|$BODY_VAR:attachment%5burl%5d|NAME”;
BasicRule wl:1100 “mz:$URL:/wp-admin/post.php|$BODY_VAR:attachment_url|NAME”;
BasicRule wl:1000 “mz:$URL:/wp-admin/plugins.php|$BODY_VAR:verify-delete|NAME”;
BasicRule wl:1310,1311 “mz:$URL:/wp-admin/post.php|$BODY_VAR:post_category[]|NAME”;
BasicRule wl:1311 “mz:$URL:/wp-admin/post.php|$BODY_VAR:post_category|NAME”;
BasicRule wl:1310,1311 “mz:$URL:/wp-admin/post.php|$BODY_VAR:tax_input[post_tag]|NAME”;
BasicRule wl:1310,1311 “mz:$URL:/wp-admin/post.php|$BODY_VAR:newtag[post_tag]|NAME”;
BasicRule wl:1310,1311 “mz:$URL:/wp-admin/users.php|$BODY_VAR:users[]|NAME”;
# URL|ARGS|NAME
BasicRule wl:1310,1311 “mz:$URL:/wp-admin/load-scripts.php|$ARGS_VAR:load[]|NAME”;
BasicRule wl:1000 “mz:$URL:/wp-admin/users.php|$ARGS_VAR:delete_count|NAME”;
BasicRule wl:1000 “mz:$URL:/wp-admin/users.php|$ARGS_VAR:update|NAME”;

# plain WP site
BasicRule wl:1000 “mz:URL|$URL:/wp-admin/update-core.php”;
BasicRule wl:1000 “mz:URL|$URL:/wp-admin/update.php”;
# URL|BODY
BasicRule wl:1009,1100 “mz:$URL:/wp-admin/post.php|$BODY_VAR:_wp_http_referer”;
BasicRule wl:1016 “mz:$URL:/wp-admin/post.php|$BODY_VAR:metakeyselect”;
BasicRule wl:11 “mz:$URL:/xmlrpc.php|BODY”;
BasicRule wl:11 “mz:$URL:/wp-cron.php|BODY”;
# URL|BODY|NAME
BasicRule wl:1100 “mz:$URL:/wp-admin/post.php|$BODY_VAR:_wp_original_http_referer|NAME”;
BasicRule wl:1000 “mz:$URL:/wp-admin/post.php|$BODY_VAR:metakeyselect|NAME”;
BasicRule wl:1000 “mz:$URL:/wp-admin/user-edit.php|$BODY_VAR:from|NAME”;
BasicRule wl:1100 “mz:$URL:/wp-admin/admin-ajax.php|$BODY_VAR:attachment%5burl%5d|NAME”;
BasicRule wl:1310,1311 “mz:$URL:/wp-admin/admin-ajax.php|$BODY_VAR:data[wp-auth-check]|NAME”;
BasicRule wl:1310,1311 “mz:$URL:/wp-admin/admin-ajax.php|$BODY_VAR:data[wp-check-locked-posts][]|NAME”;
BasicRule wl:1310,1311 “mz:$URL:/wp-admin/update-core.php|$BODY_VAR:checked[]|NAME”;
# URL|ARGS|NAME
BasicRule wl:1310,1311 “mz:$URL:/wp-admin/load-scripts.php|$ARGS_VAR:load[]|NAME”;
BasicRule wl:1000 “mz:$URL:/wp-admin/users.php|$ARGS_VAR:delete_count|NAME”;
BasicRule wl:1000 “mz:$URL:/wp-admin/users.php|$ARGS_VAR:update|NAME”;

### Plugins
#WP Minify
BasicRule wl:1015 “mz:$URL:/wp-content/plugins/bwp-minify/min/|$ARGS_VAR:f”;

7. Configure Varnish:

 

nano /etc/varnish/default


# Configuration file for varnish
#
# /etc/init.d/varnish expects the variables $DAEMON_OPTS, $NFILES and $MEMLOCK
# to be set from this shell script fragment.
#
# Note: If systemd is installed, this file is obsolete and ignored. You will
# need to copy /lib/systemd/system/varnish.service to /etc/systemd/system/ and
# edit that file.

# Should we start varnishd at boot? Set to “no” to disable.
START=yes

# Maximum number of open files (for ulimit -n)
NFILES=131072

# Maximum locked memory size (for ulimit -l)
# Used for locking the shared memory log in memory. If you increase log size,
# you need to increase this number as well
MEMLOCK=82000

# Default varnish instance name is the local nodename. Can be overridden with
# the -n switch, to have more instances on a single server.
# INSTANCE=$(uname -n)

# This file contains 4 alternatives, please use only one.

## Alternative 1, Minimal configuration, no VCL
#
# Listen on port 6081, administration on localhost:6082, and forward to
# content server on localhost:8080. Use a 1GB fixed-size cache file.
#
# DAEMON_OPTS=”-a :6081 \
# -T localhost:6082 \
# -b localhost:8080 \
# -u varnish -g varnish \
# -S /etc/varnish/secret \
# -s file,/var/lib/varnish/$INSTANCE/varnish_storage.bin,1G”


## Alternative 2, Configuration with VCL
#
# Listen on port 6081, administration on localhost:6082, and forward to
# one content server selected by the vcl file, based on the request.
# Use a 256MB memory based cache.
#
DAEMON_OPTS=”-a :80 \
-T localhost:6082 \
-f /etc/varnish/default.vcl \
-S /etc/varnish/secret \
-s malloc,256m”


## Alternative 3, Advanced configuration
#
# See varnishd(1) for more information.
#
# # Main configuration file. You probably want to change it 🙂
# VARNISH_VCL_CONF=/etc/varnish/default.vcl
#
# # Default address and port to bind to
# # Blank address means all IPv4 and IPv6 interfaces, otherwise specify
# # a host name, an IPv4 dotted quad, or an IPv6 address in brackets.
# VARNISH_LISTEN_ADDRESS=
# VARNISH_LISTEN_PORT=6081
#
# # Telnet admin interface listen address and port
# VARNISH_ADMIN_LISTEN_ADDRESS=127.0.0.1
# VARNISH_ADMIN_LISTEN_PORT=6082
#
# # The minimum number of worker threads to start
# VARNISH_MIN_THREADS=1
#
# # The Maximum number of worker threads to start
# VARNISH_MAX_THREADS=1000
#
# # Idle timeout for worker threads
# VARNISH_THREAD_TIMEOUT=120
#
# # Cache file location
# VARNISH_STORAGE_FILE=/var/lib/varnish/$INSTANCE/varnish_storage.bin
#
# # Cache file size: in bytes, optionally using k / M / G / T suffix,
# # or in percentage of available disk space using the % suffix.
# VARNISH_STORAGE_SIZE=1G
#
# # File containing administration secret
# VARNISH_SECRET_FILE=/etc/varnish/secret
#
# # Backend storage specification
# VARNISH_STORAGE=”file,${VARNISH_STORAGE_FILE},${VARNISH_STORAGE_SIZE}”
#
# # Default TTL used when the backend does not specify one
# VARNISH_TTL=120
#
# # DAEMON_OPTS is used by the init script. If you add or remove options, make
# # sure you update this section, too.
# DAEMON_OPTS=”-a ${VARNISH_LISTEN_ADDRESS}:${VARNISH_LISTEN_PORT} \
# -f ${VARNISH_VCL_CONF} \
# -T ${VARNISH_ADMIN_LISTEN_ADDRESS}:${VARNISH_ADMIN_LISTEN_PORT} \
# -t ${VARNISH_TTL} \
# -p thread_pool_min=${VARNISH_MIN_THREADS} \
# -p thread_pool_max=${VARNISH_MAX_THREADS} \
# -p thread_pool_timeout=${VARNISH_THREAD_TIMEOUT} \
# -S ${VARNISH_SECRET_FILE} \
# -s ${VARNISH_STORAGE}”
#


## Alternative 4, Do It Yourself
#
# DAEMON_OPTS=””

nano /etc/varnish/default.vcl


# Configuration file for varnish (optimized for WordPress)

vcl 4.0;

##########BACKEND##########
backend server1 {
.host = “127.0.0.1”;
.port = “8080”;
.connect_timeout = 600s;
.first_byte_timeout = 600s;
.between_bytes_timeout = 600s;
.max_connections = 800;
}

#Only allow purging from specific IPs
acl purge {
“localhost”;
“127.0.0.1”;
}

# This function is used when a request is send by a HTTP client (Browser)
##########START SUB VCL_RECV##########
sub vcl_recv {
#Normalize the header, remove the port (in case you’re testing this on various TCP ports)
set req.http.Host = regsub(req.http.Host, “:[0-9]+”, “”);

#Which pages to cache
if(!(req.http.host ~ “domain1.com“) &&
!(req.http.host ~ “domain2.com“) &&
!(req.http.host ~ “domain3.com“)) {
return (pass);
}

#FORCE SSL
if ( (req.http.host ~ “^(?i)domain1.com” || req.http.host ~ “^(?i)www.domain1.com“) && req.http.X-Forwarded-Proto !~ “(?i)https”) {
return (synth(751, “”));
}

if ( (req.http.host ~ “^(?i)domain2.com” || req.http.host ~ “^(?i)www.domain2.com“) && req.http.X-Forwarded-Proto !~ “(?i)https”) {
return (synth(750, “”));
}

if ( (req.http.host ~ “^(?i)domain3.com” || req.http.host ~ “^(?i)www.domain3.com“) && req.http.X-Forwarded-Proto !~ “(?i)https”) {
return (synth(752, “”));
}


#Set client IP to headers
if (req.restarts == 0) {
if (req.http.x-forwarded-for) {
set req.http.X-Forwarded-For =
req.http.X-Forwarded-For + “, ” + client.ip;
} else {
set req.http.X-Forwarded-For = client.ip;
}
}


#Allow purging from ACL
if (req.method == “PURGE”) {
#If not allowed then a error 405 is returned
if (!client.ip ~ purge) {
return(synth(405, “This IP is not allowed to send PURGE requests.”));
}
#If allowed, do a cache_lookup -> vlc_hit() or vlc_miss()
return (purge);
}

#Post requests will not be cached
if (req.http.Authorization || req.method == “POST”) {
return (pass);
}

#———-Wordpress specific configuration———-“

#Do not cache the RSS feed
if (req.url ~ “/feed”) {
return (pass);
}

#Blitz hack
if (req.url ~ “/mu-.*”) {
return (pass);
}


#Do not cache the admin and login pages
if (req.url ~ “/wp-(login|admin)”) {
return (pass);
}

#Do not cache the WooCommerce pages
### REMOVE IT IF YOU DO NOT USE WOOCOMMERCE ###
if (req.url ~ “/(cart|my-account|checkout|addons|/?add-to-cart=)”) {
return (pass);
}

#Remove the “has_js” cookie
set req.http.Cookie = regsuball(req.http.Cookie, “has_js=[^;]+(; )?”, “”);

#Remove any Google Analytics based cookies
set req.http.Cookie = regsuball(req.http.Cookie, “__utm.=[^;]+(; )?”, “”);

#Remove the Quant Capital cookies (added by some plugin, all __qca)
set req.http.Cookie = regsuball(req.http.Cookie, “__qc.=[^;]+(; )?”, “”);

#Remove the wp-settings-1 cookie
set req.http.Cookie = regsuball(req.http.Cookie, “wp-settings-1=[^;]+(; )?”, “”);

#Remove the wp-settings-time-1 cookie
set req.http.Cookie = regsuball(req.http.Cookie, “wp-settings-time-1=[^;]+(; )?”, “”);

#Remove the wp test cookie
set req.http.Cookie = regsuball(req.http.Cookie, “wordpress_test_cookie=[^;]+(; )?”, “”);

#Are there cookies left with only spaces or that are empty?
if (req.http.cookie ~ “^ *$”) {
unset req.http.cookie;
}

#Cache the following files extensions
if (req.url ~ “\.(css|js|png|gif|jp(e)?g|swf|ico)”) {
unset req.http.cookie;
}

#Normalize Accept-Encoding header and compression
if (req.http.Accept-Encoding) {
#Do not compress compressed files …
if (req.url ~ “\.(jpg|png|gif|gz|tgz|bz2|tbz|mp3|ogg)$”) {
unset req.http.Accept-Encoding;
} elsif (req.http.Accept-Encoding ~ “gzip”) {
set req.http.Accept-Encoding = “gzip”;
} elsif (req.http.Accept-Encoding ~ “deflate”) {
set req.http.Accept-Encoding = “deflate”;
} else {
unset req.http.Accept-Encoding;
}
}

#Check the cookies for WordPress-specific items
if (req.http.Cookie ~ “wordpress_” || req.http.Cookie ~ “comment_”) {
return (pass);
}
if (!req.http.cookie) {
unset req.http.cookie;
}

#———-End of WordPress specific configuration———-#

#Do not cache HTTP authentication and HTTP Cookie
if (req.http.Authorization || req.http.Cookie) {
# Not cacheable by default
return (pass);
}

#Cache all others requests
return (hash);
}
#########END SUB_VCL_RECV##########

#PART OF FORCE SSL
sub vcl_synth {
if (resp.status == 751) {
set resp.status = 301;
set resp.http.Location = “https://domain1.com” + req.url;
return(deliver);
}

if (resp.status == 750) {
set resp.status = 301;
set resp.http.Location = “https://domain2.com” + req.url;
return(deliver);
}

if (resp.status == 752) {
set resp.status = 301;
set resp.http.Location = “https://domain3.com + req.url;
return(deliver);
}


}


sub vcl_pipe {
return (pipe);
}

sub vcl_pass {
return (fetch);
}

#The data on which the hashing will take place
sub vcl_hash {
hash_data(req.url);
if (req.http.host) {
hash_data(req.http.host);
} else {
hash_data(server.ip);
}

#Include the X-Forward-Proto header, since we want to treat HTTPS
# requests differently, and make sure this header is always passed
# properly to the backend server.
if (req.http.X-Forwarded-Proto) {
hash_data(req.http.X-Forwarded-Proto);
}

#If the client supports compression, keep that in a different cache
if (req.http.Accept-Encoding) {
hash_data(req.http.Accept-Encoding);
}
return (lookup);
}

#This function is used when a request is sent by our backend (Nginx server)
##########START VCL_BACKEND_RESPONSE##########
sub vcl_backend_response {
#Remove some headers we never want to see
unset beresp.http.Server;
unset beresp.http.X-Powered-By;

#For static content strip all backend cookies
if (bereq.url ~ “\.(css|js|png|gif|jp(e?)g)|swf|ico”) {
unset beresp.http.cookie;
}

#Only allow cookies to be set if we’re in admin area
if (beresp.http.Set-Cookie && bereq.url !~ “^/wp-(login|admin)”) {
unset beresp.http.Set-Cookie;
}

#Do not cache response to posted requests or those with basic auth
if ( bereq.method == “POST” || bereq.http.Authorization ) {
set beresp.uncacheable = true;
set beresp.ttl = 120s;
return (deliver);
}

#Do not cache search results
if ( bereq.url ~ “\?s=” ){
set beresp.uncacheable = true;
set beresp.ttl = 120s;
return (deliver);
}

#Only cache status ok
if ( beresp.status != 200 ) {
set beresp.uncacheable = true;
set beresp.ttl = 120s;
return (deliver);
}

#A TTL of 24h
set beresp.ttl = 24h;
#Define the default grace period to serve cached content
set beresp.grace = 30s;

return (deliver);
}
##########END VCL_BACKEND_RESPONSE##########

##########START VCL_DELIVER##########
sub vcl_deliver {
if (obj.hits > 0) {
set resp.http.X-Cache = “cached”;
} else {
set resp.http.x-Cache = “uncached”;
}

#Remove some headers: PHP version
unset resp.http.X-Powered-By;

#Remove some headers: Apache\Nginx version & OS
unset resp.http.Server;

#Remove some headers: Varnish
unset resp.http.Via;
unset resp.http.X-Varnish;

return (deliver);
}
##########END VCL_DELIVER##########

sub vcl_init {
return (ok);
}

sub vcl_fini {
return (ok);
}

8. Set other config files:

 

nano /etc/php5/fpm/php.ini


[PHP]
display_errors = off
log_errors: on
max_input_time: 60
output_buffering: 4096
register_argc_argv: off
request_order: GP
session.bug_compat_42: off
session.bug_compat_warn: off
session.gc_divisor: 1000
session.hash_bits_per_character: 5
short_open_tag: off
variables_order: GPCS
engine = On
short_open_tag = Off
asp_tags = Off
precision = 14
output_buffering = 4096
zlib.output_compression = Off
implicit_flush = Off
unserialize_callback_func =
serialize_precision = 17
disable_functions=pcntl_alarm,pcntl_fork,pcntl_waitpid,pcntl_wait,pcntl_wifexited,pcntl_wifstopped,pcntl_wifsignaled,pcntl_wexitstatus,pcntl_wtermsig,pcntl_wstopsig,pcntl_signal,pcntl_signal_dispatch,pcntl_get_last_error,pcntl_strerror,pcntl_sigprocmask,
pcntl_sigwaitinfo,pcntl_sigtimedwait,pcntl_exec,pcntl_getpriority,pcntl_setpriority,system,exec,shell_exec,passthru,proc_close,proc_get_status,proc_nice,proc_open,proc_terminate,popen,pclose,phpinfo,eval
disable_classes =
ignore_user_abort = Off
zend.enable_gc = On
expose_php = Off
max_execution_time = 120
max_input_time = 60
memory_limit = 64M
error_reporting = E_ALL & ~E_DEPRECATED & ~E_STRICT
display_errors = Off
display_startup_errors = Off
log_errors = On
log_errors_max_len = 1024
ignore_repeated_errors = Off
ignore_repeated_source = Off
report_memleaks = On
track_errors = Off
html_errors = On
variables_order = “GPCS”
request_order = “GP”
register_argc_argv = Off
auto_globals_jit = On
post_max_size = 15M
auto_prepend_file =
auto_append_file =
default_mimetype = “text/html”
default_charset = “UTF-8”
doc_root =
user_dir =
enable_dl = Off
cgi.fix_pathinfo=1
file_uploads = On
upload_max_filesize = 15M
max_file_uploads = 20
allow_url_fopen = Off
allow_url_include = Off
default_socket_timeout = 30
[CLI Server]
cli_server.color = On
pdo_mysql.cache_size = 2000
pdo_mysql.default_socket=
[mail function]
SMTP = localhost
smtp_port = 25
mail.add_x_header = On
[SQL]
sql.safe_mode = Off
[ODBC]
odbc.allow_persistent = On
odbc.check_persistent = On
odbc.max_persistent = -1
odbc.max_links = -1
odbc.defaultlrl = 4096
odbc.defaultbinmode = 1
[Interbase]
ibase.allow_persistent = 1
ibase.max_persistent = -1
ibase.max_links = -1
ibase.timestampformat = “%Y-%m-%d %H:%M:%S”
ibase.dateformat = “%Y-%m-%d”
ibase.timeformat = “%H:%M:%S”
[MySQL]
mysql.allow_local_infile = On
mysql.allow_persistent = Off
mysql.cache_size = 2000
mysql.max_persistent = -1
mysql.max_links = -1
mysql.default_port =
mysql.default_socket =
mysql.default_host =
mysql.default_user =
mysql.default_password =
mysql.connect_timeout = 60
mysql.trace_mode = Off
[MySQLi]
mysqli.max_persistent = -1
mysqli.allow_persistent = On
mysqli.max_links = -1
mysqli.cache_size = 2000
mysqli.default_port = 3306
mysqli.default_socket =
mysqli.default_host =
mysqli.default_user =
mysqli.default_pw =
mysqli.reconnect = Off
[mysqlnd]
mysqlnd.collect_statistics = On
mysqlnd.collect_memory_statistics = Off
[PostgreSQL]
pgsql.allow_persistent = On
pgsql.auto_reset_persistent = Off
pgsql.max_persistent = -1
pgsql.max_links = -1
pgsql.ignore_notice = 0
pgsql.log_notice = 0
[Sybase-CT]
sybct.allow_persistent = On
sybct.max_persistent = -1
sybct.max_links = -1
sybct.min_server_severity = 10
sybct.min_client_severity = 10
[bcmath]
bcmath.scale = 0
[Session]
session.save_handler = files
session.use_strict_mode = 0
session.use_cookies = 1
session.use_only_cookies = 1
session.name = PHPSESSID
session.auto_start = 0
session.cookie_lifetime = 0
session.cookie_path = /
session.cookie_domain =
session.cookie_httponly =
session.serialize_handler = php
session.gc_probability = 0
session.gc_divisor = 1000
session.gc_maxlifetime = 1440
session.bug_compat_42 = Off
session.bug_compat_warn = Off
session.referer_check =
session.cache_limiter = nocache
session.cache_expire = 180
session.use_trans_sid = 0
session.hash_function = 0
session.hash_bits_per_character = 5
url_rewriter.tags = “a=href,area=href,frame=src,input=src,form=fakeentry”
[MSSQL]
mssql.allow_persistent = On
mssql.max_persistent = -1
mssql.max_links = -1
mssql.min_error_severity = 10
mssql.min_message_severity = 10
mssql.compatibility_mode = Off
mssql.secure_connection = Off
[Tidy]
tidy.clean_output = Off
[soap]
soap.wsdl_cache_enabled=1
soap.wsdl_cache_dir=”/tmp”
soap.wsdl_cache_ttl=86400
soap.wsdl_cache_limit = 5
[ldap]
ldap.max_links = -1

nano /etc/php5/fpm/pool.d/domain.example.pool.conf


; Use this config only if you want to run separate php5-fpm process per user
; Pool name, the variable $pool can be used in any directive and will be replaced by the
; pool name (‘www’ here)
[user]

; Per pool prefix
; It only applies on the following directives:
; – ‘slowlog’
; – ‘listen’ (unixsocket)
; – ‘chroot’
; – ‘chdir’
; – ‘php_values’
; – ‘php_admin_values’
; When not set, the global prefix (or /usr) applies instead.
; Note: This directive can also be relative to the global prefix.
; Default Value: none
;prefix = /path/to/pools/$pool

; The address on which to accept FastCGI requests.
; Valid syntaxes are:
; ‘ip.add.re.ss:port’ – to listen on a TCP socket to a specific address on
; a specific port;
; ‘port’ – to listen on a TCP socket to all addresses on a
; specific port;
; ‘/path/to/unix/socket’ – to listen on a unix socket.
; Note: This value is mandatory.
listen = /var/run/example.domain_fpm.sock

; Set listen(2) backlog. A value of ‘-1’ means unlimited.
; Default Value: 128 (-1 on FreeBSD and OpenBSD)
;listen.backlog = -1

; List of ipv4 addresses of FastCGI clients which are allowed to connect.
; Equivalent to the FCGI_WEB_SERVER_ADDRS environment variable in the original
; PHP FCGI (5.2.2+). Makes sense only with a tcp listening socket. Each address
; must be separated by a comma. If this value is left blank, connections will be
; accepted from any ip address.
; Default Value: any
;listen.allowed_clients = 127.0.0.1

; Set permissions for unix socket, if one is used. In Linux, read/write
; permissions must be set in order to allow connections from a web server. Many
; BSD-derived systems allow connections regardless of permissions.
; Default Values: user and group are set as the running user
; mode is set to 0666
listen.owner = user
listen.group = user
listen.mode = 0660

; Unix user/group of processes
; Note: The user is mandatory. If the group is not set, the default user’s group
; will be used.
user = user
group = user

; Choose how the process manager will control the number of child processes.
; Possible Values:
; static – a fixed number (pm.max_children) of child processes;
; dynamic – the number of child processes are set dynamically based on the
; following directives:
; pm.max_children – the maximum number of children that can
; be alive at the same time.
; pm.start_servers – the number of children created on startup.
; pm.min_spare_servers – the minimum number of children in ‘idle’
; state (waiting to process). If the number
; of ‘idle’ processes is less than this
; number then some children will be created.
; pm.max_spare_servers – the maximum number of children in ‘idle’
; state (waiting to process). If the number
; of ‘idle’ processes is greater than this
; number then some children will be killed.
; Note: This value is mandatory.
pm = dynamic

; The number of child processes to be created when pm is set to ‘static’ and the
; maximum number of child processes to be created when pm is set to ‘dynamic’.
; This value sets the limit on the number of simultaneous requests that will be
; served. Equivalent to the ApacheMaxClients directive with mpm_prefork.
; Equivalent to the PHP_FCGI_CHILDREN environment variable in the original PHP
; CGI.
; Note: Used when pm is set to either ‘static’ or ‘dynamic’
; Note: This value is mandatory.
pm.max_children = 1

; The number of child processes created on startup.
; Note: Used only when pm is set to ‘dynamic’
; Default Value: min_spare_servers + (max_spare_servers – min_spare_servers) / 2
pm.start_servers = 1

; The desired minimum number of idle server processes.
; Note: Used only when pm is set to ‘dynamic’
; Note: Mandatory when pm is set to ‘dynamic’
pm.min_spare_servers = 1

; The desired maximum number of idle server processes.
; Note: Used only when pm is set to ‘dynamic’
; Note: Mandatory when pm is set to ‘dynamic’
pm.max_spare_servers = 1

; The number of requests each child process should execute before respawning.
; This can be useful to work around memory leaks in 3rd party libraries. For
; endless request processing specify ‘0’. Equivalent to PHP_FCGI_MAX_REQUESTS.
; Default Value: 0
pm.max_requests = 500

; The URI to view the FPM status page. If this value is not set, no URI will be
; recognized as a status page. By default, the status page shows the following
; information:
; accepted conn – the number of request accepted by the pool;
; pool – the name of the pool;
; process manager – static or dynamic;
; idle processes – the number of idle processes;
; active processes – the number of active processes;
; total processes – the number of idle + active processes.
; max children reached – number of times, the process limit has been reached,
; when pm tries to start more children (works only for
; pm ‘dynamic’)
; The values of ‘idle processes’, ‘active processes’ and ‘total processes’ are
; updated each second. The value of ‘accepted conn’ is updated in real time.
; Example output:
; accepted conn: 12073
; pool: www
; process manager: static
; idle processes: 35
; active processes: 65
; total processes: 100
; max children reached: 1
; By default the status page output is formatted as text/plain. Passing either
; ‘html’, ‘xml’ or ‘json’ as a query string will return the corresponding output
; syntax. Example:
; http://www.foo.bar/status
; http://www.foo.bar/status?json
; http://www.foo.bar/status?html
; http://www.foo.bar/status?xml
; Note: The value must start with a leading slash (/). The value can be
; anything, but it may not be a good idea to use the .php extension or it
; may conflict with a real PHP file.
; Default Value: not set
;pm.status_path = /status

; The ping URI to call the monitoring page of FPM. If this value is not set, no
; URI will be recognized as a ping page. This could be used to test from outside
; that FPM is alive and responding, or to
; – create a graph of FPM availability (rrd or such);
; – remove a server from a group if it is not responding (load balancing);
; – trigger alerts for the operating team (24/7).
; Note: The value must start with a leading slash (/). The value can be
; anything, but it may not be a good idea to use the .php extension or it
; may conflict with a real PHP file.
; Default Value: not set
;ping.path = /ping

; This directive may be used to customize the response of a ping request. The
; response is formatted as text/plain with a 200 response code.
; Default Value: pong
;ping.response = pong

; The access log file
; Default: not set
;access.log = log/$pool.access.log

; The access log format.
; The following syntax is allowed
; %%: the ‘%’ character
; %C: %CPU used by the request
; it can accept the following format:
; – %{user}C for user CPU only
; – %{system}C for system CPU only
; – %{total}C for user + system CPU (default)
; %d: time taken to serve the request
; it can accept the following format:
; – %{seconds}d (default)
; – %{miliseconds}d
; – %{mili}d
; – %{microseconds}d
; – %{micro}d
; %e: an environment variable (same as $_ENV or $_SERVER)
; it must be associated with embraces to specify the name of the env
; variable. Some exemples:
; – server specifics like: %{REQUEST_METHOD}e or %{SERVER_PROTOCOL}e
; – HTTP headers like: %{HTTP_HOST}e or %{HTTP_USER_AGENT}e
; %f: script filename
; %l: content-length of the request (for POST request only)
; %m: request method
; %M: peak of memory allocated by PHP
; it can accept the following format:
; – %{bytes}M (default)
; – %{kilobytes}M
; – %{kilo}M
; – %{megabytes}M
; – %{mega}M
; %n: pool name
; %o: ouput header
; it must be associated with embraces to specify the name of the header:
; – %{Content-Type}o
; – %{X-Powered-By}o
; – %{Transfert-Encoding}o
; – ….
; %p: PID of the child that serviced the request
; %P: PID of the parent of the child that serviced the request
; %q: the query string
; %Q: the ‘?’ character if query string exists
; %r: the request URI (without the query string, see %q and %Q)
; %R: remote IP address
; %s: status (response code)
; %t: server time the request was received
; it can accept a strftime(3) format:
; %d/%b/%Y:%H:%M:%S %z (default)
; %T: time the log has been written (the request has finished)
; it can accept a strftime(3) format:
; %d/%b/%Y:%H:%M:%S %z (default)
; %u: remote user
;
; Default: “%R – %u %t \”%m %r\” %s”
;access.format = %R – %u %t “%m %r%Q%q” %s %f %{mili}d %{kilo}M %C%%

; The timeout for serving a single request after which the worker process will
; be killed. This option should be used when the ‘max_execution_time’ ini option
; does not stop script execution for some reason. A value of ‘0’ means ‘off’.
; Available units: s(econds)(default), m(inutes), h(ours), or d(ays)
; Default Value: 0
request_terminate_timeout = 30s

; The timeout for serving a single request after which a PHP backtrace will be
; dumped to the ‘slowlog’ file. A value of ‘0s’ means ‘off’.
; Available units: s(econds)(default), m(inutes), h(ours), or d(ays)
; Default Value: 0
;request_slowlog_timeout = 0

; The log file for slow requests
; Default Value: not set
; Note: slowlog is mandatory if request_slowlog_timeout is set
;slowlog = log/$pool.log.slow

; Set open file descriptor rlimit.
; Default Value: system defined value
;rlimit_files = 1024

; Set max core size rlimit.
; Possible Values: ‘unlimited’ or an integer greater or equal to 0
; Default Value: system defined value
;rlimit_core = 0

; Chroot to this directory at the start. This value must be defined as an
; absolute path. When this value is not set, chroot is not used.
; Note: you can prefix with ‘$prefix’ to chroot to the pool prefix or one
; of its subdirectories. If the pool prefix is not set, the global prefix
; will be used instead.
; Note: chrooting is a great security feature and should be used whenever
; possible. However, all PHP paths will be relative to the chroot
; (error_log, sessions.save_path, …).
; Default Value: not set
;chroot =

; Chdir to this directory at the start.
; Note: relative path can be used.
; Default Value: current directory or / when chroot
chdir = /

; Redirect worker stdout and stderr into main error log. If not set, stdout and
; stderr will be redirected to /dev/null according to FastCGI specs.
; Note: on highloaded environement, this can cause some delay in the page
; process time (several ms).
; Default Value: no
;catch_workers_output = yes

; Pass environment variables like LD_LIBRARY_PATH. All $VARIABLEs are taken from
; the current environment.
; Default Value: clean env
;env[HOSTNAME] = $HOSTNAME
;env[PATH] = /usr/local/bin:/usr/bin:/bin
;env[TMP] = /tmp
;env[TMPDIR] = /tmp
;env[TEMP] = /tmp

; Additional php.ini defines, specific to this pool of workers. These settings
; overwrite the values previously defined in the php.ini. The directives are the
; same as the PHP SAPI:
; php_value/php_flag – you can set classic ini defines which can
; be overwritten from PHP call ‘ini_set’.
; php_admin_value/php_admin_flag – these directives won’t be overwritten by
; PHP call ‘ini_set’
; For php_*flag, valid values are on, off, 1, 0, true, false, yes or no.

; Defining ‘extension’ will load the corresponding shared extension from
; extension_dir. Defining ‘disable_functions’ or ‘disable_classes’ will not
; overwrite previously defined php.ini values, but will append the new value
; instead.

; Note: path INI options can be relative and will be expanded with the prefix
; (pool, global or /usr)

; Default Value: nothing is defined by default except the values in php.ini and
; specified at startup with the -d argument
;php_admin_value[sendmail_path] = /usr/sbin/sendmail -t -i -f www@my.domain.com
;php_flag[display_errors] = off
;php_admin_value[error_log] = /var/log/fpm-php.www.log
;php_admin_flag[log_errors] = on
;php_admin_value[memory_limit] = 32M
php_admin_value[session.save_path] = “/home/user/_sessions”
php_admin_value[open_basedir] = “/home/user:/usr/share/pear:/usr/share/php:/tmp:/usr/local/lib/php”

php_flag[display_errors] = off
php_admin_value[error_reporting] = 0
php_admin_value[error_log] = /var/log/php5-fpm.log
php_admin_flag[log_errors] = on
php_admin_value[memory_limit] = 128M

9. Add SSL to WordPress:


Go to your WordPress web folder, open file wp-config.php and add:

define(‘FORCE_SSL_ADMIN’, true);

define(‘FORCE_SSL_LOGIN’, true);

if ($_SERVER[‘HTTP_X_FORWARDED_PROTO’] == ‘https’)

$_SERVER[‘HTTPS’]=’on’;


Important! Add this before the line “require_once(ABSPATH . ‘wp-settings.php’);”, otherwise you’ll get “You do not have sufficient permissions to access this page” message when trying to login into admin (wp-admin).

10. Restart services

 

service nginx restart

service php5-fpm restart

service varnish restart

My original post is at: https://troubleshoot-coltpython.blogspot.com/2013/11/measuring-raspberrypi-cpu-and-gpu.html

We’ll create a bash script:

sudo nano temp.sh

Paste this and save:

#!/bin/bash cpuTemp0=$(cat /sys/class/thermal/thermal_zone0/temp) cpuTemp1=$(($cpuTemp0/1000)) cpuTemp2=$(($cpuTemp0/100)) cpuTempM=$(($cpuTemp2 % $cpuTemp1)) cpuFreq=`cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_cur_freq | sed 's/.\{3\}$//'`Mhz echo CPU temp"="$cpuTemp1"."$cpuTempM"'C" echo GPU $(/opt/vc/bin/vcgencmd measure_temp) echo CPU frequency=$cpuFreq

Set permission and run it:

sudo chmod +x temp.sh;./temp.sh

Duplicate of my original post: https://troubleshoot-coltpython.blogspot.com/2013/11/bash-script-domain-expiration-check.html

This script checks if domain has expired and also supports e-mail notice. 

Original script is from Matt (domain-check). I modifed already modified script from Vivek (http://www.cyberciti.biz/tips/domain-check-script.html).


List of changes:


– Added support for Slovenian (.si) domains.

– Added VAR for “mail from:”.
– Increased time between whois queries to 7 seconds (Slovenian whois only allows 10 queries per minute).
– Mail subject and mail data are translated to Slovene language.
– Changed default VAR path for mail (/bin/mail/ to /usr/bin/mail)

Get the script:

wget http://krejzi.si/files/domain-check;mv domain-check domain-check.sh;chmod +x domain-check.sh

Mirror: https://pastebin.com/pQe3RmDz


Usage:

Usage: ./domain-check.sh [ -e email ] [ -x expir_days ] [ -q ] [ -a ] [ -h ] {[ -d domain_namee ]} || { -f domainfile}

  -a                                   : Send a warning message through email
  -d domain                            : Domain to analyze (interactive mode)
  -e email address                     : Email address to send expiration notices
  -f domain file                       : File with a list of domains
  -h                                   : Print this screen
  -s whois server                      : Whois sever to query for information
  -q                                   : Don't print anything on the console
  -x days                              : Domain expiration interval (eg. if domain_date < days)


Check single domain:

./domain-check.sh -d domain.si


Check single domain and send e-mail notice if domain will expire in less than defined in WARNDAYS:

./domain-check.sh -a -d domain.si


You can also check multiple domains. Create a file domains.txt and add your domains like:


domain1.com

domain2.com
domain3.com

Check multiple domains:

./domain-check.sh -f domains.txt


Check multiple domains and send e-mail notice if any domain will expire in less than defined in WARNDAYS:

./domain-check.sh -a -f domains.txt


Set your own parameters from command line. Next command will check domains in domains.txt with expiration date less than 60 days and send e-mail notice to admin@domain.si

./domain-check.sh -a -f domains.txt -e admin@domain.si -x 60 


If you are going to use this script with cron, use -q option or change QUIET=”FALSE” to QUIET=”TRUE”

Let’s say we want to run this script every day at 4 AM. Open crontab with crontab -e (Ubuntu) and add the following line:

0 4 * * * /path/to/your/script/domain-check.sh -q -a -f /path/to/your/list/domains.txt

Duplicate of original post: https://troubleshoot-coltpython.blogspot.com/2013/11/phpsysinfo-loading-slow-on-raspberrypi.html

Default installation of PhpSysInfo on RaspberryPI takes a long time to load due some missing programs and hardware components.

My system info:


Hardware: RaspberryPI Model B with 512MB RAM

OS: Raspbian GNU/Linux 7

Nginx version: 1.2.1-2.2+wheezy1

PHP version: 5.4.4-14+deb7u5

PhpSysInfo version: 3.0.17-1

Enable debug mode in /etc/phpsysinfo/config.php

define('PSI_DEBUG', true);

Open http://yoursitephpsysinfo/xml.php

I found the following errors:

<Error Function="find_program(lsb_release)">

<![CDATA[

program not found on the machine ./xml.php on line 45 ./includes/output/class.WebpageXML.inc.php on line 138 in function run() ./includes/xml/class.XML.inc.php on line 456 in function getXml() ./includes/xml/class.XML.inc.php on line 435 in function _buildXml() ./includes/os/class.OS.inc.php on line 70 in function getSys() ./includes/os/class.Linux.inc.php on line 576 in function build() ./includes/os/class.Linux.inc.php on line 527 in function _distro() ./includes/class.CommonFunctions.inc.php on line 117 in function executeProgram( "lsb_release", "-a 2>/dev/null", "", true )

]]>

</Error>

<Error Function="/usr/bin/lspci">

<![CDATA[

pcilib: Cannot open /proc/bus/pci lspci: Cannot find any working access method. Return value: 1 ./xml.php on line 45 ./includes/output/class.WebpageXML.inc.php on line 138 in function run() ./includes/xml/class.XML.inc.php on line 456 in function getXml() ./includes/xml/class.XML.inc.php on line 435 in function _buildXml() ./includes/os/class.OS.inc.php on line 70 in function getSys() ./includes/os/class.Linux.inc.php on line 583 in function build() ./includes/os/class.Linux.inc.php on line 306 in function _pci() ./includes/class.Parser.inc.php on line 36 in function lspci() ./includes/class.CommonFunctions.inc.php on line 142 in function executeProgram( "lspci", "", "", true )

]]>

</Error>

<Error Function="find_program(lsscsi)">

<![CDATA[

program not found on the machine ./xml.php on line 45 ./includes/output/class.WebpageXML.inc.php on line 138 in function run() ./includes/xml/class.XML.inc.php on line 456 in function getXml() ./includes/xml/class.XML.inc.php on line 435 in function _buildXml() ./includes/os/class.OS.inc.php on line 70 in function getSys() ./includes/os/class.Linux.inc.php on line 585 in function build() ./includes/os/class.Linux.inc.php on line 367 in function _scsi() ./includes/class.CommonFunctions.inc.php on line 117 in function executeProgram( "lsscsi", "-c", "", true )

]]>

</Error>

<Error Function="file_exists(/proc/scsi/scsi)">

<![CDATA[

the file does not exist on your machine ./xml.php on line 45 ./includes/output/class.WebpageXML.inc.php on line 138 in function run() ./includes/xml/class.XML.inc.php on line 456 in function getXml() ./includes/xml/class.XML.inc.php on line 435 in function _buildXml() ./includes/os/class.OS.inc.php on line 70 in function getSys() ./includes/os/class.Linux.inc.php on line 585 in function build() ./includes/os/class.Linux.inc.php on line 367 in function _scsi() ./includes/class.CommonFunctions.inc.php on line 191 in function rfts( "/proc/scsi/scsi", "", 0, 4096, true )

]]>

</Error>

First, third and fourth error are because some programs are not present on the system, let’s install them.

sudo apt-get install lsb-release lsscsi -y

Second error is due missing PCI bus on raspberryPI.

# lspci

pcilib: Cannot open /proc/bus/pci

lspci: Cannot find any working access method.

We can’t do anything about that but disable that function in PhpSysInfo PHP code. Open the file/your/path/to/phpsyinfo/includes/class.Parser.inc.php and find the lspci function.

/**

     * parsing the output of lspci command

     *

     * @return Array

     */

    public static function lspci()

        {

        $arrResults = array();

        if (CommonFunctions::executeProgram("lspci", "", $strBuf, PSI_DEBUG)) {

            $arrLines = preg_split("/\n/", $strBuf, -1, PREG_SPLIT_NO_EMPTY);

            foreach ($arrLines as $strLine) {

                list($strAddr, $strName) = preg_split('/ /', trim($strLine), 2);

                $strName = preg_replace('/\(.*\)/', '', $strName);

                $dev = new HWDevice();

                $dev->setName($strName);

                $arrResults[] = $dev;

            }

        }

        return $arrResults;

    }

Change the code above to match the one bellow:

/**

     * parsing the output of lspci command

     *

     * @return Array

     */

    public static function lspci()

        {

        return array();

            $arrResults = array();

        if (CommonFunctions::executeProgram("lspci", "", $strBuf, PSI_DEBUG)) {

            $arrLines = preg_split("/\n/", $strBuf, -1, PREG_SPLIT_NO_EMPTY);

            foreach ($arrLines as $strLine) {

                list($strAddr, $strName) = preg_split('/ /', trim($strLine), 2);

                $strName = preg_replace('/\(.*\)/', '', $strName);

                $dev = new HWDevice();

                $dev->setName($strName);

                $arrResults[] = $dev;

            }

        }

        return $arrResults;

    }

Disable debug mode in phpsysinfo config, save, reload, drink beer.

Duplicate of original post: https://troubleshoot-coltpython.blogspot.com/2013/11/ubuntu-fail2ban-fails-to-parse-apache.html

I was trying to setup Fail2ban to block WordPress login bruteforce attacks, but Fail2ban somehow failed to parse access.log

When parsing log file with command:

fail2ban-regex /var/log/apache2/access.log /etc/fail2ban/filter.d/apache-wp-login.conf

CPU rises to 100% usage until I kill the process. I made a quick fix – workaround with redirecting needed content from access.log to another log file.

My setup is as follows:

OS: Ubuntu Server 12.04 LTS
Fail2ban: 0.8.6-3wheezy2build0.12.04.1
Python: 2.7.3-0ubuntu2.2

Fail2ban configuration:

 nano /etc/fail2ban/fail2ban.conf 
# Fail2Ban configuration file
# Author: Cyril Jaquier
# $Revision$


[Definition]

# Option: loglevel
# Notes.: Set the log level output.
# 1 = ERROR
# 2 = WARN
# 3 = INFO
# 4 = DEBUG
# Values: NUM Default: 3
#

loglevel = 4

# Option: logtarget
# Notes.: Set the log target. This could be a file, SYSLOG, STDERR or STDOUT.
# Only one log target can be specified.
# Values: STDOUT STDERR SYSLOG file Default: /var/log/fail2ban.log
#

logtarget = /var/log/fail2ban.log

# Option: socket
# Notes.: Set the socket file. This is used to communicate with the daemon. Do
# not remove this file when Fail2ban runs. It will not be possible to
# communicate with the server afterwards.
# Values: FILE Default: /var/run/fail2ban/fail2ban.sock
#
socket = /var/run/fail2ban/fail2ban.sock

We add a new entry for WordPress

Add configuration to /etc/fail2ban/jail.conf

[apache-wp-login]

enabled = true
port = http,https
filter = apache-wp-login
logpath = /var/log/apache2/apache-wp-login.log
maxretry = 3
findtime = 60

Now me make new filter for WordPress. Some servers logs are in different format so we have to make different regex entries. These are two most common configurations:

Log format example 1:

www.domain.si:80 188.65.115.90 - - [08/Nov/2013:13:20:46 +0100] "POST /en/wp-login.php HTTP/1.1" 200 1784 "http://www.domain.si/en/wp-login.php" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1623.0 Safari/537.36"

Log format example 2:

89.222.15.152 - - [08/Nov/2013:13:40:12 +0000] "POST /wp-login.php HTTP/1.1" 200 1756 "http://domain.wordpress.com/wp-login.php" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:25.0) Gecko/20100101 Firefox/25.0"

Create /etc/fail2ban/filter.d/apache-wp-login.conf

For log format example 1 use:

[Definition]
failregex = ^[^\:]+\:80 <HOST> .* "POST
ignoreregex =

ATTENTION! Don’t apply this filter to default access.log, it would block all POST request, not only WordPress. Only use it with modified log file we will create bellow.

For log format example 2 use:

failregex = <HOST>.*] "POST /wp-login.php

ATTENTION! Don’t apply this filter to log format example 1, it would block your own server, because <HOST> regex would match your domain (www.domain.si).

Now we create new log file for parsing (/var/log/apache2/apache-wp-login.log). We only want to filter out POST requests for wp-login.php and write them to new log file called apache-wp-login.log

tail --follow=name /var/log/apache2/access.log | grep --line-buffered wp-login.php &amp;amp;gt; /var/log/apache2/apache-wp-login.log &amp;amp;amp;

Add this command to /etc/rc.local so it would run at reboot:

 

#!/bin/sh -e
#
# rc.local
#
# This script is executed at the end of each multiuser runlevel.
# Make sure that the script will "exit 0" on success or any other
# value on error.
#
# In order to enable or disable this script just change the execution
# bits.
#
# By default this script does nothing.

# Fail2ban WordPress Login
tail -f /var/log/apache2/access.log | grep --line-buffered wp-login.php > /var/log/apache2/apache-wp-login.log &

exit 0
Restart Fail2ban and you’re done.

/etc/init.d/fail2ban restart