Processing Apache log files using implode, explode

Posted under » PHP » IOT on 13 November 2022

I wanted to use MQTT of Kafka but this is is a simple project, so I just use what I already have, LAMP.

You have to set up an Apache custom filter.

The default date looks like [27/Sep/2022:14:10:01]. You should change the default date format so that it is compatible to MySQL datetime.

LogFormat "%{%Y-%m-%d %H:%M:%S}t"

We then pick the things we want to log other than the time to minimize complications

%h The remote host (IP)

\"%r\" The first line of the request

So it would look like this if I put x to seperate the time.

192.168.1.10 2022-11-19x23:20:01 "GET /whypapy HTTP/1.1"

I want to process Apache log file using PHP. First I will tail the log to a file so that I only process the latest 10 rows at a time. Saving it to a location so PHP is able to read it.

Alternatively you can apache lograte to make the log files manageable.

Another way is to use CRON, Rsync and detect if there is a change by comparing the files. Once thre is a change, we use substr instead of tail to insert the database.

To open a text file using PHP I use implode.

Now we have ten lines of log. To convert the lines into 10 arrays, I use EOF and explode.

$mendel = implode('', file('taik.txt'));
$pieces  = explode(PHP_EOL, $mendel);
echo $pieces[1];

We will get the 2nd line. We need to split it into array with " " as the delimiter.

$pieces = explode(" ", $pizza);

Let us see the pieces.

foreach ($pizza as $key => $val) {
  echo "pizza[" . $key . "] = " . $val . "\n";
  }

We get

pizza[0] = 192.168.1.10
pizza[1] = 2022-11-20x00:10:01
pizza[2] = "GET
pizza[3] = /whypapy
pizza[4] = HTTP/1.1"

The Date and time need to be separated with 'x' as the delimiter.

$pissa = explode("x", $pizza[1]);
echo $pissa[0];
echo $pissa[1];

We get...

2022-11-20
00:10:01

Alternatively, date can now be converted from x to the real MySQL datetime format.

$date = str_replace("x",' ',$pizza[1]);

We now want to remove the / from the request so we trimm it using substr.

$trimmed = substr($pizza[3], 1);

Now we create a MySQL database to record the data. Read further on how to insert the log into MySQL.

mysqli_query($dbhandle, "INSERT INTO `apache` (`happen`, `ip`, `sms`) VALUES ('$mate', '$ip', '$sms')");