Validate User Data Values in an XML Config File with Powershell
This blog discusses how I decided to validate user data in an XML Config file.
I’ve been creating a rather involved Powershell program that automates the creation of videos and posting them to our team’s video site, in order to create points that could potentially generate a nice part-time income.
As I wrote the program, I externalized any variables that I thought the user might want to customize to an XML config file. I provide documentation and videos to show the user what to change in the config file; but as of yet, I have not had time to make a GUI configuration utility. Thus, the user could make mistakes, typos etc…, and I need to validate that 1) the file is still really XML, and 2) the user has entered valid data values for certain variables.
<?xml version="1.0"?> <configuration> <appSettings> <add key="EmailRecipient" value="somebody@somebody.com" /> <add key="GmailUser" value="somebody-somebody.com" /> <add key="GmailPassword" value="abcdabcdabcd" /> <add key="WukarUser" value="somebody@somebody.com" /> <add key="WukarPassword" value="abcdabcdabcd" /> <add key="WukarSiteLoginUrl" value="http://www.teamwukar.com/login/" /> <add key="WukarSiteCustomerVideosUrl" value="http://www.teamwukar.com/members/area/jobs_customervideos.php" /> <add key="WukarSiteBAVideosUrl" value="http://www.teamwukar.com/members/area/jobs_bavideos.php" /> <add key="WukarSiteCustomerUploadUrl" value="http://www.teamwukar.com/members/area/jobs_customervideos_nokeyword.php" /> <add key="WukarSiteBAUploadUrl" value="http://www.teamwukar.com/members/area/jobs_bavideos_nokeyword.php" /> <add key="YNPlayTuneOnChangeVideo" value="N" /> <add key="MaxScanDetailRows" value="500" /> <add key="SlideWaitSeconds" value="1" /> <add key="KeywordSourceFile" value="c:\Amerisoft\trunk\WukarVideoTools\Data\KeywordFilePastCityStateZip.txt" /> <!-- Logging level. 0 – Off, 1 – Error, 2 – Warning, 3 – Info, 4 – Verbose --> <add key="LoggingLevel" value="2" /> <add key="MaxReuseOfSameVideoStyle" value="12" /> <add key="MonitorStartX" value="0" /> <add key="WatinDotNet" value="Net40" /> <add key="VideoDirectory" value="C:\Videos\GeneratedCustomerVideos\" /> <add key="VideoDirectoryShort" value="C:\Videos\GeneratedCustomerVideos\" /> <add key="VideoDirectoryLaptop" value="c:\Videos\DubliBiz\GeneratedCustomerVideos\" /> <add key="VideoDirectoryRDP" value="C:\Users\Administrator\Documents\VideoMakerFX\Exported Videos\" /> <add key="TemplateConfigDirectory" value="C:\Amerisoft\trunk\WukarVideoTools\Data\" /> <add key="TemplateDirectory" value="C:\Amerisoft\trunk\WukarVideoTools\Data\" /> <add key="CSVHistoryFilename" value="C:\Amerisoft\trunk\WukarVideoTools\Data2\SummaryResults.csv" /> <add key="DesiredHotelTemplate" value="Neal_Hotel.prj" /> <add key="DesiredAppliancesTemplate" value="CustomerTemplate - Appliances 3.prj" /> <add key="DesiredBATemplate" value="Neal_BA_1.prj" /> <add key="DesiredKeywordType" value="Customer" /> <add key="VideoProductionWaitSeconds" value="60" /> <add key="PostInitialWaitSeconds" value="30" /> <add key="CustomKeywordPostWaitSeconds" value="30" /> <add key="PostMaxSeconds" value="150" /> <add key="MaxWaitSecondsNewKeyword" value="45" /> <add key="IsPostOnOff" value="Off" /> <add key="UploadVideoDirectory" value="c:\Videos\DubliBiz\GeneratedCustomerVideos\Upload\" /> <add key="UploadVideoMoveToDirectory" value="c:\Videos\DubliBiz\GeneratedCustomerVideos\Uploaded\" /> <add key="UploadKeywordType" value="Customer" /> </appSettings> </configuration>
Retrieving single values from the config is done using these two functions:
function LoadConfigString ($configFile, $appSettingsKey)
{
$configXml = [xml](get-content $configFile)
[string]$lookupValue = $($configXml.configuration.appSettings.add | where { $_.key -eq $appSettingsKey }).value
return $lookupValue
}
function LoadConfigInt ($configFile, $appSettingsKey)
{
$configXml = [xml](get-content $configFile)
[int]$lookupValue = $($configXml.configuration.appSettings.add | where { $_.key -eq $appSettingsKey }).value
return $lookupValue
}
### Example calls to functions above ###
$slideWaitSeconds = LoadConfigInt $configFilename "SlideWaitSeconds"
$videoDirectory = LoadConfigString $configFilename "VideoDirectory"
Below are my data validation routines.
Function Test-XMLFile ($xmlFilePath)
{
# Returns $true if file exists and is valid XML
# Check the file exists
if (!(Test-Path -Path $xmlFilePath))
{
throw "$xmlFilePath is not valid. Please provide a valid path to the .xml file"
}
# Check for Load or Parse errors when loading the XML file
$xml = New-Object System.Xml.XmlDocument
try
{
$xml.Load((Get-ChildItem -Path $xmlFilePath).FullName)
return $true
}
catch [System.Xml.XmlException]
{
#this message may help user debug or at least see the detailed issue of the file.
Write-Trace "$xmlFilePath : $($_.toString())" Y
#TODO - why did line above have Write-Verbose, what does that do?
return $false
}
}
function Is-Numeric-Int ($Value)
{
#use Regular Expression to verify if a string is a number
return $Value -match "^[\d]+$"
}
function Is-Email ($testEmail)
{
#use Regular Expression to verify if a string is an email
$EmailRegex = '^[_a-z0-9-]+(\.[_a-z0-9-]+)*@[a-z0-9-]+(\.[a-z0-9-]+)*(\.[a-z]{2,4})$';
if ($testEmail -match $EmailRegex) {
return $true
}
else {
return $false
}
}
function IsXMLConfigFileDataValid
{
param($configFilename = $(throw "You must specify a config file"))
$intVarNames = @("MaxScanDetailRows","SlideWaitSeconds","LoggingLevel","MonitorStartX",
"PostInitialWaitSeconds", "CustomKeywordPostWaitSeconds",
"PostMaxSeconds", "MaxWaitSecondsNewKeyword", "MaxReuseOfSameVideoStyle"
)
$YNVarNames = @("YNPlayTuneOnChangeVideo")
$OnOffVarNames = @("IsPostOnOff")
$PathVarNames = @("VideoDirectory", "VideoDirectoryShort", "TemplateConfigDirectory",
"TemplateDirectory")
$PathVarNamesBlankOK = @("KeywordSourceFile", "UploadVideoDirectory", "UploadVideoMoveToDirectory")
$PathVarNamesWithoutFile = @("CSVHistoryFilename")
$templateNames = @("DesiredHotelTemplate","DesiredAppliancesTemplate","DesiredBATemplate")
$emailVarNames = @("EmailRecipient","GmailUser")
$global:appSettings = @{}
$config = [xml](get-content $configFilename)
foreach ($addNode in $config.configuration.appsettings.add) {
if ($addNode.Value.Contains(','))
{
# Array case
$value = $addNode.Value.Split(',')
for ($i = 0; $i -lt $value.length; $i++)
{
$value[$i] = $value[$i].Trim()
}
}
else
{
# Scalar case
$value = $addNode.Value
}
$global:appSettings[$addNode.Key] = $value
}
#$global:appSettings
#enumerate the hash table in code - maybe we want to write to trace?
$enum = "N"
if ($enum -eq "Y")
{
foreach($key in $($global:appSettings.keys)){
Write-Host "$key=$($global:appSettings[$key])"
}
}
$IsValid = $true #start by assuming valid, unless reset to $false below in any one of the tests
Write-Host "*** Starting Validation ***"
#enumerate the hash table in code and validate integers
foreach($key in $($global:appSettings.keys)){
if ($intVarNames -contains $key)
{
if (!(Is-Numeric-Int($global:appSettings[$key])))
{
Write-Host "Variable must be an Numeric Integer: $key=$($global:appSettings[$key])"
$IsValid = $false
}
}
if ($YNVarNames -contains $key)
{
if ($global:appSettings[$key] -ne "Y" -and $global:appSettings[$key] -ne "N")
{
Write-Host "Variable must be have value of "Y" or "N": $key=$($global:appSettings[$key]) "
$IsValid = $false
}
}
if ($PathVarNames -contains $key)
{
if (!(Test-Path $global:appSettings[$key]))
{
Write-Host "Path does not exist on your disk: $key=$($global:appSettings[$key]) "
$IsValid = $false
}
}
if ($PathVarNamesBlankOK -contains $key)
{
if ($global:appSettings[$key] -ne "")
{
if (!(Test-Path $global:appSettings[$key]))
{
Write-Host "Non-Blank value, and Path does not exist on your disk: $key=$($global:appSettings[$key]) "
$IsValid = $false
}
}
}
if ($PathVarNamesWithoutFile -contains $key)
{
$testJustThePath = split-path $global:appSettings[$key] #get just the pathname part
#$filename = split-path "C:\Docs\*.xls" -leaf #this returns just the filename part
if (!(Test-Path $testJustThePath))
{
Write-Host "Path does not exist on your disk: $key=$($global:appSettings[$key]) (file does not have to exist, just the path: $testPath)"
$IsValid = $false
}
}
if ($OnOffVarNames -contains $key)
{
if ($global:appSettings[$key] -ne "on" -and $global:appSettings[$key] -ne "off")
{
Write-Host "Expected value of 'on' or 'off': $key=$($global:appSettings[$key]) "
$IsValid = $false
}
}
if ($emailVarNames -contains $key)
{
if (!(Is-Email($global:appSettings[$key])))
{
Write-Host "Not a valid email address: $key=$($global:appSettings[$key]) "
$IsValid = $false
}
}
if ($templateNames -contains $key)
{
$templateDirectoryName = $global:appSettings["TemplateDirectory"]
$testTemplateFilename = $templateDirectoryName + $global:appSettings[$key] #get just the pathname part
$templateExtension = [System.IO.Path]::GetExtension($testTemplateFilename)
#write-host "testTemplateFilename=$testTemplateFilename"
#$filename = split-path "C:\Docs\*.xls" -leaf #this returns just the filename part
if (!(Test-Path $testTemplateFilename))
{
Write-Host "Template filename does not exist on your disk: $key=$($global:appSettings[$key]) (in your TemplateDirectory: $templateDirectoryName)"
$IsValid = $false
}
if ($templateExtension -ne ".prj")
{
Write-Host "File extension for template must be .prj: $key=$($global:appSettings[$key])"
$IsValid = $false
}
}
}
### special hard-coded validations ###
if (!("Net40Net35Net20".Contains($global:appSettings["WatinDotNet"])))
{
Write-Host "WatinDotNet must have value of: Net40, Net35, Net20 (depending of the version of .NET on your computer) $key=$($global:appSettings[$key])"
$IsValid = $false
}
if ($global:appSettings["UploadKeywordType"] -ne "Customer" -and $global:appSettings["UploadKeywordType"] -ne "BA")
{
Write-Host "UploadKeywordType must have value of 'Customer' or 'BA' $key=$($global:appSettings[$key])"
$IsValid = $false
}
if ($global:appSettings["DesiredKeywordType"] -ne "Customer" -and $global:appSettings["DesiredKeywordType"] -ne "BA")
{
Write-Host "DesiredKeywordType must have value of 'Customer' or 'BA' $key=$($global:appSettings[$key])"
$IsValid = $false
}
##### final logic #####
if (!($IsValid))
{
Write-Host "***********************************************************************************"
Write-Host "**** Error VAL-01: Config file has errors, see above."
Write-Host "**** configFileName=$configFilename"
Write-Host "***********************************************************************************"
}
return $IsValid
}
cls
$configFilename = "$($PSScriptRoot)\Data\Wukar_Config.xml"
$result1 = Test-XMLFile $configFilename
Write-Host "Validation Result1 = $result1"
if ($result1)
{
$result2 = isXMLConfigFileDataValid $configFilename
Write-Host "Validation Result2 = $result2"
}
The general idea is that in the documentation, I will advise the user to run the validation program before starting the application. The application itself will probably also run the validation and fail to start if any issues have. The idea is that it is better to fail immediately, then to run for 5 or 10 minutes, then fail later, only when some variable is used for the first time. My application is made to actually run 24 hours a day, creating hundreds of spin-off VideoMakerFX videos from a video template.
The downside of building this application in Powershell vs C# is that it becomes basically “Open Source”, the users can see all my code, and even share it with others. I will be adding a license key to the config file, but with the source code, that could be easily skipped over. Thus, I’m counting on honest customers who don’t know how to code. I may call a few C# routines here and there just to make hide a few features.
Here were some of my ideas on the above validation routine.
1. I categorized the config parms into categories: integers, paths, YN Variables, Emails, Templates, etc…
I then loop through all the Key= values in the app config file, looking to see if each config variable-name matches an array an array of one of the validation. If it is a match, that validation is done against the value of that key.
2. Some of the tricky parts were with the disk paths. Some of them I will allow to be blank, i.e. they are not always used.
Some require folder names only, and some require fully qualified file names.
3. I used two function Is-Numeric and Is-Email that do Regular Expression (Reg Ex) matches on the data.
Here is the video that describes what my application does: