How can I perform automatic database backups and cleanups of Polyspace Access from version R2020b?

How can I perform automatic database backups and cleanups of Polyspace Access from version R2020b?

Answer 1

[MathWorks Support Team]

Here is an example of how to do both backup and cleanup as once-a-week cronjobs:

1) cron is a job scheduler on Unix-like operating systems. 'crontab' should look like the following (run 'crontab -e' as root):

0 0 * * 0 /var/lib/polyspace/cleanup_access.sh > /var/backups/polyspace/cleanup_access.log 2>&1
0 1 * * 0 /var/lib/polyspace/backup_access.sh > /var/backups/polyspace/backup_access.log 2>&1

 

2) If /var/backups/polyspace does not exist, create this folder using the shell command:

mkdir /var/backups/polyspace

Warning: the folder /var/backups/polyspace should contain enough free disk space to store the copy of the database. Otherwise, you could experience problems during the backup.

To know the size of the database, enter the shell command:

For Polyspace R2022a and beyond:

docker exec -i polyspace-access-db-0-main psql -a -b -U postgres prs_data -t -c "SELECT pg_size_pretty(pg_database_size('prs_data'))"

For Polyspace R2020b through R2021b:

docker exec -i polyspace-access-db-main psql -a -b -U postgres prs_data -t -c "SELECT pg_size_pretty(pg_database_size('prs_data'))"

 

To know the disk space available for /var/backups/polyspace, enter the shell command:

df -h /var

If there is not enough space on /var, instead of /var/backups/polyspace you could choose another folder for the backup file and update the cronjob accordingly.

3) Create and put the following 3 files in /var/lib/polyspace:

 

cleanup_access.sh:

#!/bin/sh
set -e
BASEDIR=$(dirname "$0")
# Set the ACCESS_STORAGE_DIR variable according your Polyspace installation path.
# The path of the Storage directory is available in the Cluster Settings of the Cluster Admin interface.
# Alternatively, you can find this path in the settings.json file by searching for etlStorageDir.
ACCESS_STORAGE_DIR=/local/home/dtstrunner/ps_access/appdata/polyspace-access/storage
date
echo ""
echo "Cleanup starts..."
$BASEDIR/make_clean_commands.pl
echo ""
echo "To clean:"
cat $BASEDIR/cleanup.pscauto
echo ""
echo "Requesting clean in $ACCESS_STORAGE_DIR..."
cp $BASEDIR/cleanup.pscauto $ACCESS_STORAGE_DIR/.

backup_access.sh:

For Polyspace R2022a and beyond:

#!/bin/sh
set -e
date
echo "Pausing services..."
docker stop polyspace-access-etl-0-main polyspace-access-web-server-0-main
echo "Backing up database..."
time docker exec polyspace-access-db-0-main pg_dumpall -U postgres | gzip > /var/backups/polyspace/backup_access.gz
echo "Resuming services..."
docker start polyspace-access-etl-0-main polyspace-access-web-server-0-main

For Polyspace R2020b through R2021b:

#!/bin/sh
set -e
date
echo "Pausing services..."
docker stop polyspace-access-etl-main polyspace-access-web-server-main
echo "Backing up database..."
time docker exec polyspace-access-db-main pg_dumpall -U postgres | gzip > /var/backups/polyspace/backup_access.gz
echo "Resuming services..."
docker start polyspace-access-etl-main polyspace-access-web-server-main

make_clean_commands.pl:

#!/usr/bin/perl
# 1. Please change the command to list projects according to your needs (e.g. path to binary, hostname, login, protocol, password)
my $ps_access_command = '"/usr/local/Polyspace_Server/R2020b/polyspace/bin/polyspace-access" -host access.mycompany.org -protocol http -list-project -login admin -encrypted-password ABABABABABABABABABA';
# 2. Please change the location of the output file
my $pscauto_filename = './cleanup.pscauto';
# 3. 
$value = qx($ps_access_command);
@lines = grep(/ProjectsWaitingForDeletion\/.*/, split('\n', $value));  # use this for recursive delete
#@lines = grep(/ProjectsWaitingForDeletion\/.* RUN_ID \d+$/, split('\n', $value));  # use this to only delete leaf nodes
@lines = map { s/(.+?)( RUN_ID .*)?$/delete_project \"\1\"/g; $_ } @lines; 
open(FH, '>', $pscauto_filename) or die $!;
print FH join("\n", @lines);
close(FH);
print "\n$pscauto_filename successfully written!\n\n";

Details that must be adjusted to the installation:

1. in make_clean_commands.pl: path, username, and password (note: encrypted password as in other server commands)
2. in cleanup_access.sh: ACCESS_STORAGE_DIR according to the Polyspace Access installation path
3. in case any paths are changed, backup_access.sh and crontab must be adjusted, too

Please note:

• There should be several GB of free disk space in /var/backups since the database (even compressed) can be quite large.
• A backup can take quite a while and the database is unavailable during the operation, therefore this should be done overnight or at the weekends.

Note that this is very simplistic, but it works (tested with Polyspace R2020b on Debian and Ubuntu). For advanced use, log rotation and rotating backups might be interesting (https://rotate-backups.readthedocs.io/en/latest), as well as NFS/AFS/Samba/... backups instead of putting them locally into /var/backups. Users might also want to distribute these files via puppet/ansible/chef, in case they have multiple Access instances. For more information about 'puppet', please see: https://www.puppet.com/blog/cron-job-management

More information regarding the 'crontab' syntax can be found in the following Wikipedia page: 

https://en.wikipedia.org/wiki/Cron