I was evaluating what I thought was a string read from a file, then doing an .indexOf on that string. Took me over an hour to debug something so simple.
Trying to get the position of a string within a larger string is pretty basic stuff; yet I was getting the mystery result of -1.
I then printed the file to console, copied into a separate program as a string, and it worked fine.
I was also puzzling why the .LastIndexOf variable didn’t appear in the auto-complete of ISE. That should have been my clue.
When I did the following:
$templateContent = Get-Content $inputTemplate
this creates an array in $templateContent, and arrays have a similar method called .IndexOf.
The solution was to add the “-raw tag”. I had tried “-encoding ASCII” and various encodings, and that not getting me anywhere.
Have you ever wanted to make a new Visual Studio solution (even a BizTalk one) as a copy of another, except just change the program names? And it would automatically rename everything, including the solutions, project files, assembly files and so on? I’ve needed this program for years, and finally wrote it!
Remember that the project name is often the Assembly name and the Default Namespace (in the AssemblInfo.cs file). This program renames the files, the folders, and updates the .sln and .csproj/.btproj files (and any other file). The “-exclude” parm on the Get-ChildItem is used to avoid binaries and other files that you do not want to change.
In BizTalk, the project name can also become your “TargetNamespace” for schemas.
Another trick with BizTalk is that some of the files are Unicode and some aren’t. So the rename program has to be conscious of that, and preserve the file type. I’m doing this by testing if the first tow characters of a file are are Byte Order Mark.
NOTE: Use with caution. Make a backup of your project before letting any program rip through and rename and change files!
Make sure your old/string new string are quite unique, so as not to change other random parts of your code. For example, if you use a 20-40 character project name like I have in $oldstring below, you are generally safe.
Similar for \Cust2
Similar for \Cust3 etc
I was looking for a sample 210 EDI Invoice to use in testing; and I wanted the most recent ones.
It turns out you can put a wildcard in the variable passed on the -Path parameter. In that case, I don’t really need the -Recurse option, but I included it anyway (just in case someone created additional subfolders).
$Startpath = "D:\Application\DEV\*\Outbound\210"
#If you just want THE Most Recent File:
#$latest = Get-ChildItem -Recurse -Path $Startpath | Sort-Object LastAccessTime -Descending | Select-Object -First 1
#Write-Host "Latest= $($lastest.Fullname)"
#Most recent 12 files (or whatever number you chose in the -First ## parameter)
$latestFiles = Get-ChildItem -Recurse -Path $Startpath | Sort-Object LastAccessTime -Descending | Select-Object -First 12
Write-Host "Outbound Latest 12 files"
foreach ($file in $latestFiles)
Use the .FullName property to show the complete folder name in the results.
With the SFTP ports in BizTalk 2013, we are using the log function. BizTalk appends each SFTP log to the end of the existing log file.
So I created a Powershell Script to take an array of directory names, and run the same re-run and delete logic for each.
I also include the logic to delete a file over a certain retention period (see also shorter code for that in prior post: “Powershell to Delete Old Files“).
# Neal Walters - 09/05/2017
# Script: RenameSFTPLogsToAddDate.ps1
# Purpose: Rename a file such as "receive_internal.txt" to
# so that each day has it's own separate file of SFTP messages.
# Also purge files over $retentionDays
# Use Task Scheduler to schedlue this script
# to run once late a night or early morning.
$fullname = $_.FullName.ToString();
$dirname = $_.Directory.ToString();
$filename = $_.Name.ToString();
$filenameOnly = [io.path]::GetFileNameWithoutExtension($filename)
$ext = [io.path]::GetExtension($filename) # this includes the ., for example .txt
#If filename contains an underscore, then we think it has a date in it.
#If no date found, we want to rename the file, otherwise leave it as it was.
# looking for 20 as the century… part of date 2017, 2018, 2019 etc…
# if (-Not ($filenameOnly.toString().Contains(“_20”) ) )
if (-Not ($filenameOnly -Match “_20”) )
Write-Host “OldName $fullname”
$content = Get-Content $_.FullName
$formatDate = get-date -f _yyyy_MM_dd
The one thing I didn’t account for (yet), is the case where different files can have different EDI delimiters. Technically, you should look for the end of the ISA segment to find the delimiters, and use those in the RegEx match. For now, I’m assuming the field delimiter is * and the segment delimiter is the tilda (~).
I was archiving the EDI files in BizTalk with the filename set to “%datetime%_%MessageID%_EDI.txt”. I decided it would be better to name the files COMPANYNAME_DOCTYPE_ORDERNO_ORDERDATE_%datetime%_%MessageID%_EDI.txt.
NOTE: I could have done this logic in a custom C# BizTalk Pipeline, but decided to do it after the fact with a more simple Powershell than would be easier for administrative staff to maintain and update.
Sample 1 – Just test the parsing
With this sample, you can copy the contents of a file into the $ediText string, and test.
#Note subsituted " with `" in the string to escape the quotes within quotes issue
$ediText = "ISA*00* *00* *ZZ*MYCUSTOMER*ZZ*MYCOUNTRY*170823*1610*U*00401*000000117*0*T*:~GS*PO*BTS-SENDER*RECEIVE-APP*170823*1610*117*T*00401~ST*850*0117~BEG*00*NE*391949**20170828~N1*BY*DELIVERY-ADDRESS~N1*ST*DELIVERY-ADDRESS~N3*1420 MAINSTREET DR~N4*DALLAS*TX*12345~PO1*1*5.00*EA*4.350**IN*106889~PID*F****SAND MIX ( SSM80 )~PO1*2*1.00*etc...~";
$CompanyID = [regex]::match($ediText,'.*ISA\*.*?\*.*?\*.*?\*.*?\*.*?\*(.*?)\*.*').Groups.Value
$OrderNum = [regex]::match($ediText,'.*BEG\*.*?\*.*?\*(.*?)\*.*').Groups.Value
$OrderDate = [regex]::match($ediText,'.*BEG\*.*?\*.*?\*.*?\*.*?\*(.*?)[~\*].*').Groups.Value
$EdiDocType = [regex]::match($content,'.~ST\*(.*?)[~\*].*').Groups.Value
Write-Host "CompanyID = $CompanyID";
Write-Host "OrderNum = $OrderNum";
Write-Host "OrderDate= $OrderDate";
Write-Host "EdiDocType= $EdiDocType";
Having a filename like this will make it faster to search the archives for certain types of orders or files from a certain partner, or do do quick counts, based on the filename alone. For example, how many files did we get from XYZ company yesterday and today? This could be done in BizTalk with BAM as well, but my current client opted out of the overhead and complexity of BAM, especially since BizTalk was (for the most part), just passing the files around, not creating them.
The variable $EdiDocType above represents something like and 850, 855, 856, 810, 997, etc…
I might add one more feature. Many of the trading partner don’t use name, but some Dun number, phone number, or other ID number. I might have a lookup table to translate the code to a shortname that represents that trading partner.
It selects and executes any command on a file or set of files.
With forfiles, you can run a command on or pass arguments to multiple files. For example, you could run the type command on all files in a tree with the .txt file name extension. Or you could execute every batch file (*.bat) on drive C, with the file name “Myinput.txt” as the first argument.
With forfiles, you can do any of the following:
Select files by an absolute date or a relative date by using the /d parameter.
Build an archive tree of files by using variables such as @FSIZEand @FDATE.
Differentiate files from directories by using the @ISDIRvariable.
Include special characters in the command line by using the hexadecimal code for the character, in 0xHH format (for example, 0x09 for a tab).
I recently showed a VBScript to Archive/Move xml files to a subfolder. This is often needed when you have been archive or storing 1000s of XML files, and the the directory/folder is very slow to open due to the large number of files. Now, we will do it in Powershell.
Create the subfolders ahead of time. With some minor improvements we could do that in the code, but I was in a hurry today when I needed this…
This does the following:
1) Select all files starting with “201604” (in my case, the files began with yyyymmdd.
2) pipe that into a ForEach loop
3) Run the “Move” commandlet
4) the filename in the loop is the $_ symbol
5) Then you build the destination directory $_ again is the iterator of the loop, i.e. the FileName object, so we can get it’s directory and “Name”. There, we insert the 2016_xx for the month.
So yes, you can make this a lot fancier, but it’s a start…
I had a .bat file with 50 lines or more, and many of them had disk paths. We were migrating this to production, so I did “replace all” commands to change all the paths to production SAN/Server names. But then, I knew some of the paths existed, and some didn’t. So I wanted to find all the paths that didn’t exist, so either:
1) I could fix the filename, or
2) Create the path on the disk
So I needed to parse the file looking for file/path names. At first I tried RegEx, but then decided that just using “Split” was faster in my case. (Sometimes you just want to get the job done in the shortest amount of time.)
The following works when you have a prefix on each directory path. I’m sure there are variations you could make on this depending on your filenames. I’m only looking for lines that have .exe, because the .bat file is running various C# program to process the files.
Sample file Test.bat:
line1 Small.exe \\MyServer\Messages\Dir1 and more words
line2 Biggertest2.exe \\MyServer\Messages\Dir1 parm2 \\MyServer\Messages\Dir2 parm4
With BizTalk, we often archive files in a sub directory. Personally, I would rather archive to a SQL database with a database column, but that takes a little more architecture and sales. So in the meantime, many clients continue to write files to disk. There are frequently clean-up jobs that delete files over x days old.
However, when there are multiple thousands of a files in any directory, it can take a long time to open that directory, and display the files, especially when you are accessing it from a remote computer.
The $DaysBack parameter can be set to non-zero if you want to only group files over x days old into sub-directories.
</script></noindex> files (for example BizTalk archive XML files)
#into subfolders based on date
$SourceDir = "C:\TestFiles\"
$DestinationDir = "C:\TestFiles\"
$DaysBack = 0
$files = get-childitem $SourceDir *.*
Write-Host "File COunt= $($files.Count)"
foreach ($file in $files)
$NewSubDirectory = $DestinationDir + "" + $file.LastWriteTime.Date.ToString('yyyy-MM-dd')
#Create $NewSubDirectory if it doesn't already exist
if (!(Test-Path $NewSubDirectory))
New-Item $NewSubDirectory -type directory
if ($DaysBack -gt 0)
If($file.LastWriteTime -lt (Get-Date).adddays(-1 * $DaysBack).date)
Move-Item $file.fullname $NewSubDirectory
Move-Item $file.fullname $NewSubDirectory
You will need full write/rename access to run this script. You can specify a UNC name in the $SourceDir and $DestinationDir variables.