melissa-uploader/README.md

# Melissa Uploader

Uploads each practice's daily files to the right server, then archives what was
sent. There are three buttons and no files to edit by hand.

| Double-click this | When |
|---|---|
| **setup.bat** | Once, on a new PC. Downloads the uploader's helper program. |
| **setup-entity.bat** | Whenever a partner is added or a password/folder changes. Asks questions and writes the settings for you. |
| **run-all.bat** | Every day, after the PT STMT rename script. |

---

## Setting up a partner

Double-click **setup-entity.bat** and pick "Set up a new entity". It asks:

- **Entity name** - the folder name for this group of practices (CAMBS, RMI, ...)
- **Server address** - just the name, like `sft.polluxsystems.com`
- **How the server receives files** - pick SFTP, FTPS, or FTP from a list.
  If you don't know, ask the partner which one they use. Guessing wrong is the
  single most common reason an upload fails, and the test at the end catches it.
- **Username and password**
- **Where the files go on the server** - type `{practice}` where the practice
  name belongs and it gets filled in for each one, so a single line covers
  every practice:

  ```
  /home/clients/CAMBS/{practice}
  ```

  For CAMBS that sends AJMATS's files to `/home/clients/CAMBS/AJMATS`,
  CIIR's to `/home/clients/CAMBS/CIIR`, and so on.

- **Whether PT STMT files show up in the dated folders**, and where those go
- **Whether this entity sends patient files** from `Patient\Export\`
- **The practice names** - it creates each practice folder for you

Then it offers to test. Say yes.

---

## The test is the important part

At the end of setup (or any time, via "Test an entity's connection and
folders") it logs in and checks that every practice's folder actually exists
on the server:

```
  Daily upload
    Connecting to sft.polluxsystems.com over SFTP as cambs-user ... OK
    /home/clients/CAMBS/AJMATS ... found
    /home/clients/CAMBS/CIIR ... found
    /home/clients/CAMBS/EMMD ... NOT THERE
```

**OK** means the address, protocol, and login are all right.
**found** means files sent there will land.
**NOT THERE** means that practice will fail - either the path is wrong (fix it
in setup) or the partner hasn't created the folder yet (ask them).

Running this after setup means you find out about a problem now, in ten
seconds, instead of in tomorrow's upload.

---

## Daily run

1. Run the **PT STMT rename script** as usual
2. Double-click **run-all.bat**
3. Watch for red lines

Every file that uploads prints `ok`. A date folder only gets archived when
**everything** in it went up - if anything fails or gets skipped, the folder
stays put so tomorrow's run picks it up again. Nothing is ever deleted.

You do not have to tell it whether the files are PDFs or TIFs. It sends
whatever the imaging system put in the dated folder.

---

## When something goes wrong

The run tells you what to do in plain words. The common ones:

| What you see | What to do |
|---|---|
| `This server wants SFTP...` | Run setup-entity.bat, change that entity's server, pick the protocol it names |
| `The username or password was rejected` | Run setup-entity.bat and retype the login |
| `That server address could not be found` | Typo in the address - fix it in setup |
| `No answer from the server` | Ask IT whether the firewall allows it |
| `... not configured (...) - N file(s) ignored, left in place` | That upload has no folder path set yet. Run setup-entity.bat and fill it in, or ignore the message if that entity genuinely doesn't send those files |
| `NOT archived - N file(s) were ignored` | Some files went up and some didn't, so the folder was left alone on purpose. Fix the setting above and run again |
| `WinSCP error on startup` | Run setup.bat again |

Full detail for every run is in `Logs\YYYYMMDD.log`.

---

## Folder layout

```
run-all.bat            <- daily
setup-entity.bat       <- add or change a partner
setup.bat              <- once per PC
engine.ps1
WinSCP\                <- created by setup.bat
Logs\                  <- one log per day
Entities\
  CAMBS\
    entity.json        <- written by setup-entity.bat, don't edit by hand
    run.bat            <- runs just CAMBS
    Export1\
      AJMATS\
        20260826\                 <- today's files, from the imaging system
        Archive sent to FTP\      <- where they go once sent
        Patient\
          Export\
            PT STMT_20260826\     <- from the rename script
            Archive Sent to FTP\
      CIIR\
        ...
```

---

---

# Reference

Everything below is for whoever maintains the tool. **setup-entity.bat writes
all of it** - you should never need this to use the uploader.

## entity.json

Written into each entity folder. `setup-entity.bat` produces:

```json
{
  "_note": "Written by setup-entity.bat ...",
  "workflow": "insurance+patient",
  "ftps": {
    "host": "sft.polluxsystems.com",
    "protocol": "sftp",
    "port": null,
    "username": "shared-username",
    "password": "shared-password",
    "tiff_path": "/home/clients/CAMBS/{practice}",
    "pdf_path": null
  },
  "sftp": {
    "host": "sftp.example.com",
    "protocol": "sftp",
    "port": null,
    "username": "shared-username",
    "password": "shared-password",
    "patient_path": "/{practice}/Patient"
  }
}
```

The section names are historical: **`ftps` is the daily date-folder upload**
(over whatever `protocol` says, SFTP included) and **`sftp` is the patient-file
upload**.

### Top level

| Setting | Values | What it does |
|---|---|---|
| `workflow` | `"insurance"` or `"insurance+patient"` | `insurance+patient` also processes `Patient\Export\PT STMT_<date>\` via the `sftp` section |
| `insurance_file_type` | `"pdf"`, `"tif"`, or absent | Optional. Absent (the normal case) sends every file in the date folder except `Thumbs.db`, `desktop.ini`, `.DS_Store`, and `.tmp/.lnk/.ini/.db`. Set it to pin the upload to one extension |

### `ftps` (daily upload) and `sftp` (patient files)

| Setting | Values | What it does |
|---|---|---|
| `host` | server name | Bare hostname. A pasted `sftp://` / `ftps://` prefix is still honored and stripped |
| `protocol` | `"sftp"`, `"ftps"`, `"ftp"` | How to connect. Wins over any prefix on `host` and over `tls`. Null falls back to the `host` prefix, then `tls`, then FTP for the daily section / SFTP for the patient section |
| `port` | number, no quotes | Null = 22 for SFTP, 21 for FTP/FTPS |
| `tls` | `true`/`false` | Older equivalent of `protocol: "ftps"`. Only read when `protocol` is null |
| `username` / `password` | text | The login |
| `tiff_path` | path | `ftps` only. Where the regular daily files go. The older name `insurance_path` still works |
| `pdf_path` | path | `ftps` only. Where PT STMT-named files *from the date folder* go. Null = ignore them |
| `patient_path` | path | `sftp` only. Where the `.txt`/`.tif` patient files go |

### Values that work anywhere

| Value | Meaning |
|---|---|
| `{practice}` in a path | Replaced with the practice's folder name |
| `null` | "Not used - ignore." Affected files are left in place with a notice, never uploaded, never archived, never counted as failures |
| any key starting with `_` | Ignored - use it for notes |

On every run the engine adds any keys it knows about that are missing from
`entity.json`, as `null`, and saves the file.

## practice.json

Optional, inside one practice's folder. Same keys and sections; whatever it
contains replaces (or adds to) `entity.json` for that practice only. Use it
when one practice has its own login or a remote path that doesn't fit the
`{practice}` pattern.

```json
{
  "ftps": {
    "tiff_path": "/SpecialFolder/Claims"
  }
}
```

`workflow` and `insurance_file_type` are entity-wide and ignored here.