Refreshing Federation Metadata for Shibboleth 1.3

Authors: Brusten Philip & Van der Velpen Jan
Last modified: Wednesday, 17-Oct-2007 16:21:19 CEST

Table of contents

Introduction

Every time a new Service provider joins the federation Associatie K.U.Leuven, the metadata used by Shibboleth has to be updated on the Identity providers. We have provided a script for the Identity provider that automatically updates this metadata.

The metadata provided by the federation Associatie K.U.Leuven will be digitally signed by a certificate (CN=metadata.associatie.kuleuven.be) that is signed by Cybertrust Educational CA. In order to verify the identity and integrity of the metadata file, you must download this public signing certificate or a Java Keystore that contains the public signing certificate.

Identity provider

The Shibboleth Identity provider comes with a tool called metadatatool which can update (and sign) the IdP's metadata with the one provided by the federation.

The metadatatool makes use of a Java Keystore to verify the metadata. You can download this pre-loaded Java Keystore here:

metadata.associatie.kuleuven.be.crt.jks

Store this certificate at c:\pki\metadata.associatie.kuleuven.be.crt.jks or /etc/pki/metadata.associatie.kuleuven.be.crt.jks

We also provide a wrapper script for Linux and Windows Server 2003 that will automaticaly update and backup your old metadata on the Identity provider. You can run this script at regular times using a cronjob (Linux) or using Scheduled Tasks (Windows Server 2003).

Linux

You can download this script here: http://shib.kuleuven.be/download/metadata/metadatatool/updatemetadata.sh
Save the script at /usr/local/shib-idp/bin/updatemetadata.sh

You need to adjust the highlighted lines to your system specifications.

#!/bin/bash

#
# metadatarefresh: automatically download and verify the federation metadata
#
# Valery Tschopp - SWITCH - 20050805
# Dominique Petitpierre - University of Geneva - 20050810
#
# Put this script in $IDP_HOME/bin/metadatarefresh.sh
# and a add a crontab entry like this:
# 30 4 * * * /usr/local/shib-idp/bin/metadatarefresh.sh
#
#set -x

##
# Configure directories and path
##
IDP_HOME="/usr/local/shib-idp"
IDP_ETC="/usr/local/shib-idp/etc"
JAVA_HOME="/usr/local/java"
LOG_FILE="/var/log/shibboleth/metadatarefresh.log"
#CLASSPATH=

##
# Configure metadata download URL and local name
##
METADATA_URL="http://shib.kuleuven.be/download/metadata/idp/metadata-kulassoc-sp.xml"
METADATA_FILE=$IDP_ETC"/metadata-kulassoc-sp.xml"

##
# Configure signer certificate java keystore
##
SIGNER_KEYSTORE="/etc/pki/metadata.associatie.kuleuven.be.crt.jks"
SIGNER_ALIAS="metadata-signer"
SIGNER_PASSWORD="secret"

if [ ! -r "$SIGNER_KEYSTORE" ] ; then
echo "ERROR: Metadata Signer java keystore $SIGNER_KEYSTORE not found"
    exit 1
fi


############################################
# Environment variables
export IDP_HOME
export JAVA_HOME

############################################
# Refresh metadata
NOW=`date +%Y%m%d%H%M`
# logging
echo "$NOW: $0 starts" >> $LOG_FILE

if [ -f "$METADATA_FILE" ]; then
    $IDP_HOME/bin/metadatatool \
        --in $METADATA_URL \
        --out $METADATA_FILE.$NOW \
        --keystore "$SIGNER_KEYSTORE" \
        --alias "$SIGNER_ALIAS" \
        --password "$SIGNER_PASSWORD" \
        >> $LOG_FILE 2>&1
    ret=$?
    if [ "$ret" -ne "0" ] || [ ! -f "$METADATA_FILE.$NOW" ]; then
        sed -n -e "/^$NOW/"',$p' $LOG_FILE
        echo "ERROR: refresh of $METADATA_URL failed ($ret)"
        echo "$NOW: ERROR: refresh from $METADATA_URL failed ($ret)" \
        >> $LOG_FILE
        exit $ret
    fi
fi

############################################
# Make backups
if [ ! -d "$IDP_ETC/backup" ]; then
        mkdir $IDP_ETC/backup
fi

# Compare new to existing.  If same, delete new, if not replace existing
# and backup new.
if [ -r $METADATA_FILE.$NOW ]; then
    if cmp -s $METADATA_FILE.$NOW $METADATA_FILE; then
        rm -f $METADATA_FILE.$NOW
        echo "$NOW: unmodified $METADATA_FILE" >> $LOG_FILE
    else
        echo "$NOW: new $METADATA_FILE"
        METADATA_BASENAME=`basename $METADATA_FILE`
        cp -p $METADATA_FILE $IDP_ETC/backup/$METADATA_BASENAME.$NOW
        mv -f $METADATA_FILE.$NOW $METADATA_FILE
        echo "$NOW: new $METADATA_FILE installed" >> $LOG_FILE
    fi
fi

# logging
echo "$NOW: $0 done." >> $LOG_FILE

After you modified this script you can run it as a cronjob on a daily basis.

Windows Server 2003

You can download this script here: http://shib.kuleuven.be/download/metadata/metadatatool/updatemetadata.bat
Save the script at c:\shib-idp\bin\updatemetadata.bat

You need to adjust the highlighted lines to your system specifications.

Schedule the following batch file to run on a daily basis. In Windows Server 2003, you can use "Start">"Program Files">"Accessories">"System Tools">"Scheduled Tasks".

@echo off
SETLOCAL
REM Made by Philip Brusten 2005-09-20
REM ########################################################################
REM 
REM  Welcome to the automated metadata update tool for Shibboleth IdP software on Windows.
REM  This tool helps you to keep your metadata synchronised with the one provided by the 
REM  Associatie KULeuven Federation
REM 
REM  Please send any remarks or updates/improvements to shib@kuleuven.net
REM 
REM 
REM  DISCLAIMER: 
REM  KULassoc cannot be held responsible for anything that might be caused by running this file.
REM ########################################################################

REM ############################SETTING DATE#################################

For /f "tokens=1-7 delims=:/-, " %%i in ('echo exit^|cmd /q /k"prompt $D $T"') do (
	For /f "tokens=2-4 delims=/-,() skip=1" %%a in ('echo.^|date') do (
		set dow=%%i
		set %%a=%%j
		set %%b=%%k
		set %%c=%%l
		set hh=%%m
		set min=%%n
		set ss=%%o
	)
)
REM #########################################################################

REM ############################START CONFIG#################################

REM Set the variables to the right directory or filename
SET IDP_HOME=c:\shib-idp
SET IDP_ETC=%IDP_HOME%\etc
SET JAVA_HOME=c:\jdk
SET LOG_FILE=%IDP_HOME%\logs\metadatatool.log
REM SET CLASSPATH=

REM Provide the metadata url and file

SET METADATA_URL=http://shib.kuleuven.be/download/metadata/idp/metadata-kulassoc-sp.xml
SET METADATA_FILE=metadata-kulassoc-sp.xml
SET TMP_METADATA_FILE=metadata-kulassoc-sp.tmp.xml
SET METADATA_BACKUP_FILE=%yy%%mm%%dd%_%hh%h%min%m%ss%s-%METADATA_FILE%

REM Provide the location of the java keystore used for verification of the signed metadata
SET SIGNER_KEYSTORE=c:\pki\metadata.associatie.kuleuven.be.crt.jks
SET SIGNER_ALIAS=metadata-signer
SET SIGNER_PASSWORD=secret
REM #############################END CONFIG##################################

echo ########## Start wrapper script %yy%%mm%%dd%_%hh%h%min%m%ss%s ########### >> %LOG_FILE%

IF EXIST %SIGNER_KEYSTORE% GOTO logging
IF NOT EXIST %SIGNER_KEYSTORE% GOTO nokeystore

:nokeystore
echo %dd%/%mm%/%yy% %hh%:%min%:%ss% - Error: No Metadata Signer Java keystore found. >> %LOG_FILE%
GOTO end

:logging
echo %dd%/%mm%/%yy% %hh%:%min%:%ss% - Succeed: Metadata Signer Java keystore found. >> %LOG_FILE%

:synchronise
echo %dd%/%mm%/%yy% %hh%:%min%:%ss% - Attempting to synchronise with Associatie KULeuven Metadata (%METADATA_URL% to %IDP_ETC%\%METADATA_FILE%) >> %LOG_FILE%

REM CALL %IDP_HOME%\bin\metadatatool.bat --in %METADATA_URL% --out %IDP_ETC%\%TMP_METADATA_FILE% --noverify 2>> %LOG_FILE%
CALL %IDP_HOME%\bin\metadatatool.bat --in %METADATA_URL% --out %IDP_ETC%\%TMP_METADATA_FILE% --keystore %SIGNER_KEYSTORE% --alias %SIGNER_ALIAS% --password %SIGNER_PASSWORD% 2>> %LOG_FILE%
IF ERRORLEVEL 1 GOTO error
IF ERRORLEVEL 0 GOTO succes

:error
echo %dd%/%mm%/%yy% %hh%:%min%:%ss% - ERROR: Couldn't synchronise with %METADATA_URL% >> %LOG_FILE%
GOTO end

:succes
echo %dd%/%mm%/%yy% %hh%:%min%:%ss% - Metadata succesfully synchronised to %IDP_ETC%\%TMP_METADATA_FILE% >> %LOG_FILE%

IF EXIST %IDP_ETC%\%METADATA_FILE% GOTO compare
:nometadata
echo %dd%/%mm%/%yy% %hh%:%min%:%ss% - No metadata file (%IDP_ETC%\%METADATA_FILE%) present >> %LOG_FILE%
goto movemetadata

:compare
echo %dd%/%mm%/%yy% %hh%:%min%:%ss% - Comparing old (%METADATA_FILE%) with new metadata (%TMP_METADATA_FILE%) >> %LOG_FILE%
echo n|comp %IDP_ETC%\%METADATA_FILE% %IDP_ETC%\%TMP_METADATA_FILE% 
IF ERRORLEVEL 1 GOTO differentsize
IF ERRORLEVEL 0 GOTO samesize

:samesize
echo %dd%/%mm%/%yy% %hh%:%min%:%ss% - Metadata is the same as previously backupped>> %LOG_FILE%
goto movemetadata

:differentsize
IF EXIST %IDP_ETC%\backup GOTO dobackup
mkdir %IDP_ETC%\backup
echo %dd%/%mm%/%yy% %hh%:%min%:%ss% - Folder %IDP_ETC%\backup created >> %LOG_FILE%

:dobackup
echo %dd%/%mm%/%yy% %hh%:%min%:%ss% - Metadata is newer and the old one will be backupped>> %LOG_FILE%
copy %IDP_ETC%\%METADATA_FILE% %IDP_ETC%\backup\%METADATA_BACKUP_FILE% 1>> %LOG_FILE%
IF ERRORLEVEL 1 GOTO copyerror
IF ERRORLEVEL 0 GOTO copysucces

:copyerror
echo %dd%/%mm%/%yy% %hh%:%min%:%ss% - ERROR: Couldn't backup %IDP_ETC%\%TMP_METADATA_FILE% >> %LOG_FILE%
GOTO movemetadata

:copysucces
echo %dd%/%mm%/%yy% %hh%:%min%:%ss% - %IDP_ETC%\%METADATA_FILE% copied to %IDP_ETC%\backup\%METADATA_BACKUP_FILE% >> %LOG_FILE%

:movemetadata
echo %dd%/%mm%/%yy% %hh%:%min%:%ss% - Saving the new metadata %IDP_ETC%\%TMP_METADATA_FILE% to %IDP_ETC%\%METADATA_FILE% >> %LOG_FILE%
move /Y %IDP_ETC%\%TMP_METADATA_FILE% %IDP_ETC%\%METADATA_FILE% 

:end

Troubleshooting

If you installed your IdP with the K.U.Leuven install script, you will probably get the following error message in %IDP_HOME%\logs\metadatatool.log:
########## Start wrapper script 20070424_10h17m45s ########### 
24/04/2007 10:17:45 - Succeed: Metadata Signer Java keystore found. 
24/04/2007 10:17:45 - Attempting to synchronise with Associatie KULeuven Metadata (http://shib.kuleuven.be/download/metadata/idp/metadata-kulassoc-sp.xml to c:\shib-idp\etc\metadata-kulassoc-sp.xml) 
java.lang.NoClassDefFoundError: org/apache/log4j/Appender
Exception in thread "main" 24/04/2007 10:17:45 - ERROR: Couldn't synchronise with http://shib.kuleuven.be/download/metadata/idp/metadata-kulassoc-sp.xml

To solve this problem you have to add the log4j library to your classpath.
The K.U.Leuven install script removed this library from the shibboleth-idp install directory because it configures tomcat to do all the logging with log4j. The additional log4j libraries (for Shibboleth and CAS) gave trouble, so they were removed. The metadatatool needs this library, that's why we will provide it in the CLASSPATH variable.


After that run the updatemetadata.bat script again.