One table per section covering every key: allowed values, defaults when
null, the sftp://-ftps:// host prefixes, the {practice} token, null=ignore
semantics, and _-prefixed comment keys.
CAMBS's partner turned out to receive insurance files over SFTP - Melissa
pasted "SFTP://sft.polluxsystems.com/" as ftps.host, which the engine fed to
WinSCP as a literal FTP hostname. Host values are now parsed: an sftp://
scheme switches the upload to the SFTP protocol (default port 22), ftps://
forces explicit TLS, a bare hostname behaves as before per the tls flag, and
the scheme/trailing slash are stripped from the hostname either way.
Also fixes a real bug this exposed: GiveUpSecurityAndAcceptAnyTlsHostCertificate
was set unconditionally, and WinSCP refuses that combination when FtpSecure
is None - exactly the "TlsHostCertificateFingerprint ... is set, but neither
FtpSecure nor Secure is enabled" error from her run. The flag is now only set
when TLS is actually on.
pdf_path being null blocked ALL insurance uploads even when the date folder
had no PT STMT files at all - CAMBS's first partners send only TIFs, so
tiff_path is filled and pdf_path intentionally stays null. Validation is now
per-category: connection fields (host/username/password) are needed whenever
anything would upload, tiff_path only when non-PT-STMT files exist, pdf_path
only when PT STMT files exist. If some files upload but others were ignored
via a null path, the date folder is NOT archived (so ignored files are never
buried in the archive unsent) with an explicit NOT-archived notice.
On each run the engine compares entity.json against the full set of known
keys, adds any that are absent with a null value, and writes the file back -
so nobody has to hand-type key names (the typo class of config error goes
away). A null/blank value now means "this entity doesn't use this": files
that would need it are ignored with a yellow notice and left in place,
instead of counting as failures. Insurance-unconfigured no longer skips the
patient step for that practice (was continue, now else-branch).
Renamed in engine.ps1, both EXAMPLE configs, and the README. Existing
entity.json/practice.json files using insurance_path keep working - the
engine copies it to tiff_path after the merge when tiff_path isn't set.
- Full inline entity.json example plus a Required? column in the field table
- Explicit JSON editing rules (exact key names, comma placement, straight
quotes, no quotes on numbers/booleans)
- insurance_file_type placement called out (top level, not inside ftps)
- practice.json can add fields entity.json lacks, with a different-paths
example; note that workflow/insurance_file_type are entity-wide only
- Troubleshooting entries for CONFIG ERROR, wrong-file-type "No .pdf files",
all-fields-missing (bad section name), and the startup banner check
Merge-Config assigned override values with '=', which throws
SetValueInvocationException when the property doesn't exist on the base
object - so a practice.json could only override fields already present in
entity.json. AJMATS's practice.json supplies insurance_path/pdf_path that
CAMBS's entity.json lacks entirely, which crashed the merge. Use Add-Member
-Force (handles both add and replace), copy whole sections missing from the
base, skip _comment keys, and replace scalar sections directly.
CAMBS's run showed host= blank, an empty upload path, and WinSCP's
"Password is set, but UserName is not" - i.e. entity.json's ftps section
had a readable password but host/username/insurance_path came up empty,
which points at misspelled or misplaced keys. Instead of letting WinSCP
throw a cryptic connect error per practice, check the merged config up
front and print exactly which entity.json fields are missing or blank.
Makes it obvious at a glance whether entity.json's insurance_file_type was
actually read (e.g. 'File type: .tif') instead of silently defaulting to pdf.
The Get-WinSCP.ps1 unblock-on-extract fix wasn't enough - if the WinSCP
folder is ever copied again afterward (network share, zip, sync tool),
Windows can reapply Mark-of-the-Web and silently re-block it. Unblock the
whole folder at the top of every engine.ps1 run instead of relying on
setup.bat timing. Also wrapped Add-Type in try/catch so a real load failure
now stops the run with a clear message instead of silently continuing into
a script that would only fail later once it actually tried to upload
something.
CAMBS, RMI, CONSENSIO, and INLAND always export TIFs into the daily date
folder instead of PDFs, so engine.ps1's hardcoded "*.pdf" filter silently
found nothing to upload (no error, just "No PDFs" for every practice).
Added optional entity.json field "insurance_file_type" (defaults to "pdf"
when omitted, so existing entities are unaffected); the PT STMT split logic
already worked on filename regardless of extension, so no other changes
needed.
- Unblock-File the extracted WinSCP files after unzip: Windows tags
downloaded files with Mark-of-the-Web, which makes Add-Type refuse to
load WinSCPnet.dll (FileLoadException 0x80131515) until unblocked.
- run.bat passed -EntityDir "%~dp0", but %~dp0 always ends in a trailing
backslash. cmd's argument parser treats a lone backslash before a closing
quote as an escaped literal quote rather than the string terminator, so
PowerShell received the path with a stray " character baked in
(Entities\CAMBS" instead of Entities\CAMBS), breaking Test-Path. Strip the
trailing backslash before quoting it.
Root cause found: winscp.net's download URL serves a human "please wait"
landing page (18KB HTML) instead of redirecting to the real file, unless
the User-Agent contains a known download-tool signature like curl or Wget.
PowerShell's own default UA always gets the landing page - confirmed by
reproducing locally with various UAs against the live URL. Not antivirus,
not TLS, not a network block; overriding -UserAgent to a curl string on
Invoke-WebRequest resolves it.
The 1MB size-check catches a bad download but gave no way to see why - on a
locked-down network something is very likely substituting a small HTML page
(proxy/AV block page) for the real zip. Print the first lines of whatever
was actually saved so the content is visible without another round trip.
Expand-Archive was failing with a cryptic "End of Central Directory record
could not be found" on Melissa's PC - the inline one-liner in setup.bat had
no error handling, so a blocked/truncated download (antivirus, firewall, or
an older TLS default) surfaced as a raw .NET exception instead of a useful
message. Moved the logic to Get-WinSCP.ps1: forces TLS 1.2, validates the
downloaded file size before unzipping, and reports what actually went wrong.
Windows PowerShell (5.1) reads .ps1 files without a UTF-8 BOM using the
system codepage, which mangles multi-byte UTF-8 characters like -> and --
into garbage that includes smart-quote characters PowerShell treats as
string terminators - this broke parsing on Melissa's PC (error at
engine.ps1:283, cascading missing-brace errors above it). Sticking to plain
ASCII avoids the whole encoding-dependent failure class.
Practice.json already supported overriding insurance_path/pdf_path/patient_path
via Merge-Config, but the README and example only mentioned login overrides —
document that practices can have different folder paths, not just credentials.
blance