GOA Systems

Category: Bash

Wait for host in BaSH

Okay, this one is a little specific, but I recently had this issue and I wanted to share it because knowing this would have saved me one scripting language and a lot of time. I have previously implemented this in PowerShell, but it can easily be done in BaSH as well.

The premise was to setup a new virtual machine and before I could work with the machine I had to wait until it was created and powered up. The installation was done in a pipeline and so there was no interaction. The following script waits until the machine is available and can be pinged and then terminates. This is not a finished solution but a small part of the pipeline that allowed me to continue with the regular installation via SSH after the machine was available.

2023-10-05
Get latest version from GitHub API
I am writing automated updaters for certain applications but I don’t want the update to be executed every day. Instead it should only run in case a new version is available. One key component to achieve this is to find out what the latest version is. If a software distributor is using GitHub and if the releases are maintained there the versions can be accessed via API. I wrote a small BaSH snippet to parse the latest version string into a BaSH variable. I demonstrate this I am using the repository of a really great piece of software: Kimai time-tracker. This is a time tracking software I also use for my projects, customers and invoices. It features a well maintained documentations, a great API to connect it to 3rd party applications and a more than usable interface!
```
TAGNAME=$(curl -H "Accept: application/vnd.github+json" https://api.github.com/repos/kevinpapst/kimai2/releases \
 | grep tag_name \
 | head -1 \
 | sed -E 's/^.*"tag_name": "(.*)",$/\1/g')
```
So what does this code do? It fetches the releases from the API, looks for lines containing “tag_name” (the member that holds the version number), takes only the first line and extracts the value from the line using sed with a regular expression. The output is assigned to the variable “TAGNAME” and can be used for further evaluation.
2022-09-12
Escaping in shells
First things first: A string with spaces is enclosed in either single quotes ‘ or double quotes “. The difference you ask? Single quotes are treated “as is”: No variable replacement, no parsing, just a string. If you want to embed a variable value, you have to concatenate the string. Period. Double quotes allow dynamic values via variables:
```
NAME="Andreas"
echo "The name is $NAME"
# Output: The name is Andreas
```
This would print my name in the echo line.

Okay, so now what if you need a quote in a echo line? Exactly, the same that is used to initiate and terminate the string? Well, you have to escape it. Let me make this clear: We are still working on a default Linux Bash now. Double quotes are relatively easy:
```
echo "A string with a \" is fine."
# Output: A string with a " is fine.
```
Same goes for variables:
```
NAME="Name with a \" character."
echo "Value: $NAME"
# Output: Value: Name with a " character.
```
Single quotes are a little more tricky:
```
echo $'String with a \' in it.'
# Output: String with a ' in it.
```
And as a variable:
```
NAME=$'Name with a \' in it.'
echo 'Value: '$NAME
# Output: Value: Name with a ' in it.
```
Okay, so now you can have quotes in strings that are framed by the same quotes. This is the beginning and I’d start practicing this for bash. I will continue this tutorial with PowerShell embedding:
```
Write-Host "A string containing a `" character."
# Output: A string containing a " character.
```
As you can see, the escaping character is different from bash: PowerShell uses a ` to escape newline (`n) or quotes (`”).

As we all know, MS is known for their consistency. Single quotes are treated differently:
```
Write-Host 'A string containing a '' character.'
# Output: A string containing a ' character.
```
Isn’t that awesome?

So why am I explaining this?

Try to imagine the following scenario: You are working on a Windows-Machine (with PowerShell) and would like to run a command via SSH (basically a Linux tool) on a remote Linux server that runs BaSH. I’d suggest working from bottom (the command on the Linux server) to top (the command being written in the PowerShell environment). First execute it on Bash, then via SSH and finally embed it in the PowerShell command. This way you always know on which level you are and which escapes you are going to need.

I will continue this tutorials because this always gave me a headache. Especially considering variables parsed on the executing host, the remote machine or coming from a command line. One has to work very concentrated here and always visualize where a certain part of a command is executed.
2022-04-13
Zabbix agent deployment and configuration
Installation and configuration of the Zabbix agent, the counterpart of the Zabbix server, that collects and sends data to the server to be shown, is quite easy and can be done with a simple bash script.
```
#!/bin/bash
# File: client.sh

SERVER=$1
HOSTNAME=$2

# Zabbix version
ZV=5.4

# File name version
FNV=${ZV}-1

if [ -z $HOSTNAME ]
then
        HOSTNAME=$(cat /etc/hostname)
fi

if [ -z $SERVER ]
then
        SERVER=zabbix.example.com
fi

wget https://repo.zabbix.com/zabbix/${ZV}/ubuntu/pool/main/z/zabbix-release/zabbix-release_${FNV}+ubuntu20.04_all.deb
dpkg -i zabbix-release_${FNV}+ubuntu20.04_all.deb
apt update
apt install -y zabbix-agent

sed -i "s/^Server=127\.0\.0\.1$/Server=${SERVER}/g" /etc/zabbix/zabbix_agentd.conf
sed -i "s/^Hostname=Zabbix server$/Hostname=${HOSTNAME}/g" /etc/zabbix/zabbix_agentd.conf

systemctl restart zabbix-agent.service
```
Now let’s imagine you have 1000 hosts and would have to do that on each and every one of them. That’s going to be a long day of work.

My assumption here is, that you as the administrator have ssh root access to all these machines via a ssh public key and each host has a individual hostname (For example: vm1, vm2, vm3, …, vmN or GUIDs).

If this is given the shown script can be transferred to the host via “scp” and executed via “ssh”. I will show this with 4 hosts but with a little bash magic this can be extended to a lot more.
```
#!/bin/bash
# File: server.sh

SERVER=zabbix.example.com

for HOST in vm1 vm2 vm3 vm4
do
        scp client.sh root@$host:/root/client.sh
        ssh root@$host "chmod u+x ./client.sh; ./client.sh $SERVER $HOST; rm ./client.sh;"
done
```
In this script the server securely copies the file “client.sh” shown above to the hosts one by one and executes it to install and configure Zabbix agent. The agent is restarted afterwards and can be added to the Zabbix server. I am planing to write another tutorial on how to do this via API. This can then be integrated into the server script.

The client script can also be used to add the Zabbix agent manually. Just check the current version and modify the script if necessary.

As always: I hope this saves somebody some time and makes administration a little bit easier.
2022-01-15
Parse JSON output from an API with Python
As great as PowerShell is with JSON objects you can’t always use it because it is not available on each and every machine with Linux out there. And I don’t know a way of parsing JSON code with plain bash properly. Fortunately a lot of servers come with “python” installed and it can be used to parse JSON and retrieve values from the object with a comprehensible notation.

For example, this is the response from a API call to create a new DNS record in a self managed zone:
```
{
    "record": {
        "id": "4711",
        "type": "A",
        "name": "testentry",
        "value": "0.0.0.0",
        "ttl": 3600,
        "zone_id": "2",
        "created": "1970-01-01 00:00:00.00 +0000 UTC",
        "modified": "1970-01-01 00:00:00.00 +0000 UTC"
    }
}
```
What I would need from this call is the record id. To show how this works, I have to explain and extend the example: Such a response comes most of the times when calling an API via “curl” for example. It is not just lying around on the disk. This in mind let’s assume that the shown JSON code is the result of the following call:
```
curl -s https://exampleapi.goa.systems/path/to/endpoint
```
Now curl just puts out the response on stdout and we can pipe it in the following command and read it there via stdin:
```
curl -s https://exampleapi.goa.systems/path/to/endpoint | python3 -c "import sys,json; print(json.load(sys.stdin)['record']['id'])"
```
With this code the value of “id” (['record']['id'] – compare with the source JSON, in the example “4711”) is printed (println) on the command line. Now, to store it in a variable one could do this:
```
$RECORDID=$(curl -s https://exampleapi.goa.systems/path/to/endpoint | python3 -c "import sys,json; print(json.load(sys.stdin)['record']['id'])")
```
And then save it on the disk and use it for your purposes. In my case for example I’d use it for deleting the record later in a process I’ll describe in another tutorial.
2022-01-11
Change date modified of files and delete files other than n days
I am working on a backup script on Linux and for that I need test data: Files that are modified on certain dates and that I can scan for. Of course I could create a file every day for the next month, but that would take a little too long. I found out, that one can use the command “touch” to change the dates for “modified” and “last access”
```
touch -m -t 202107200100.00 test.txt
```
This would change the modified date of the file “test.txt” to the 20th of July 20201 01:00:00 at night. This could be integrated in a loop for example. To create 10 files with one modified each day the following script does the job:
```
#!/bin/bash
i=20
while [ $i -ge 10 ]
do
  touch test_$i.txt
  echo "Hello World" > test_$i.txt
  touch -m -t 202107${i}0100.00 test_$i.txt
  ((i--))
done
```
Of course, this is pretty simple, but it does the job for my task.

To scan and delete files older than 5 days I can then use …
```
find . -mtime +5 -delete
```
… in the same folder.

Pretty neat, huh?
2021-07-21