melissa-uploader/README.md

# Melissa Uploader

Automated upload tool for practice exports. Uploads insurance PDFs and patient files to the correct FTP/SFTP servers for each practice, then archives what was sent.

---

## First-time setup

1. Double-click **setup.bat** — this downloads WinSCP (needed for uploading)
2. Set up your entity folders (see below)
3. Double-click **run-all.bat** each day after running the PT STMT rename script

---

## Daily workflow

1. Run the **PT STMT rename script** manually (as usual)
2. Double-click **run-all.bat** to upload everything
3. Check the window for any failures — failed files stay in place for retry

---

## Folder layout

```
run-all.bat                        <- double-click to run everything
engine.ps1
setup.bat
WinSCP\                            <- created by setup.bat automatically
Entities\
  CAMBS\
    entity.json                    <- server details for this entity
    run.bat                        <- double-click to run just CAMBS
    Export1\
      AJMAT\
        practice.json              <- only needed if AJMAT has its own FTP login
        20260629\                  <- today's insurance PDFs go here
        Archive sent to FTP\       <- uploaded date folders move here
        Patient\
          Export\
            PT STMT_20260629\      <- created by rename script
            Archive Sent to FTP\   <- uploaded patient folders move here
      CIIR\
        ...
  CONSensio\
    entity.json
    ...
```

---

## The two config files

There are two JSON files, and they work together — one sets the defaults for
the whole entity, the other overrides just what's different for one practice.

- **`entity.json`** — lives in the entity folder (e.g. `Entities\CAMBS\entity.json`).
  Required. This is the default server info used by every practice under that
  entity: which server, which login, which folder paths.
- **`practice.json`** — lives inside one practice's folder (e.g.
  `Entities\CAMBS\Export1\AJMAT\practice.json`). Optional — only add it if a
  practice needs something *different* from `entity.json`. Whatever fields you
  put in here replace the matching field from `entity.json` for that practice
  only; anything you leave out still comes from `entity.json`.

Common reasons to add a `practice.json`:
- The practice has its own separate FTP/SFTP login instead of the shared one
- The practice's files need to go to a **different folder path** on the
  server than the standard `{practice}` pattern (e.g. the imaging vendor set
  it up with a different folder name, or a nonstandard structure) — override
  `tiff_path`, `pdf_path`, and/or `patient_path` to whatever that
  practice actually needs

You can mix and match — e.g. override just `pdf_path` for one practice and
leave its login and every other path alone.

### `entity.json`

1. Create a folder under `Entities\` with the entity name (e.g. `Entities\CAMBS\`)
2. Copy `run.bat` into it
3. Create `entity.json` in that folder — copy from `Entities\EXAMPLE\entity.json`
   and fill in your real values. A complete file looks like this:

```json
{
  "workflow": "insurance+patient",
  "insurance_file_type": "pdf",

  "ftps": {
    "host": "ftp.example.com",
    "port": 21,
    "tls": true,
    "username": "shared-username",
    "password": "shared-password",
    "tiff_path": "/{practice}/Insurance",
    "pdf_path": "/{practice}/PDF"
  },

  "sftp": {
    "host": "sftp.example.com",
    "port": 22,
    "username": "shared-username",
    "password": "shared-password",
    "patient_path": "/{practice}/Patient"
  }
}
```

| Setting | Required? | What to put |
|---------|-----------|------------|
| `workflow` | Yes | `insurance` if no patient files, `insurance+patient` if both |
| `insurance_file_type` | No (defaults to `pdf`) | Set to `tif` for an entity whose imaging system exports the daily files as TIFs instead of PDFs (CAMBS, RMI, CONSENSIO, INLAND). Goes at the top level, next to `workflow` — NOT inside `ftps` |
| `ftps.host` | Yes | FTP server address |
| `ftps.port` | No (defaults to 21) | FTP port, only if not 21 |
| `ftps.tls` | Yes | `true` for FTPS (secure), `false` for plain FTP |
| `ftps.username` | Yes | FTP username (shared across all practices) |
| `ftps.password` | Yes | FTP password |
| `ftps.tiff_path` | Yes | Remote folder for insurance files — keep `{practice}` in it. (The older name `insurance_path` still works too) |
| `ftps.pdf_path` | Yes | Remote folder for PT STMT files — keep `{practice}` in it |
| `sftp.host` | Only if `insurance+patient` | SFTP server address |
| `sftp.port` | No (defaults to 22) | SFTP port, only if not 22 |
| `sftp.username` | Only if `insurance+patient` | SFTP username |
| `sftp.password` | Only if `insurance+patient` | SFTP password |
| `sftp.patient_path` | Only if `insurance+patient` | Remote folder for patient files — keep `{practice}` in it |

4. Create the `Export1\` folder and a subfolder for each practice inside the entity folder

`{practice}` in any path gets automatically replaced with that practice's
folder name — so one `entity.json` can serve every practice as long as they
all follow the same folder-naming pattern on the server.

**Every "Yes" field must be present in `entity.json`** unless *every single
practice* under the entity supplies it in its own `practice.json`. If a
required field is missing or blank, the run stops that practice with a
`CONFIG ERROR` message naming exactly which fields it couldn't find — nothing
uploads and nothing archives for that practice until it's fixed.

**Editing JSON — the rules that bite:**
- Key names must match **exactly** (all lowercase, underscores): `host`, not
  `hostname`; `username`, not `user`; `tiff_path`, not `tiffpath`
- Every `"key": "value"` pair ends with a comma **except the last one in its
  block** — a missing or extra comma breaks the whole file
- Keep values inside straight double quotes (`"`). Avoid editing in Word or
  anything that turns quotes curly — use Notepad
- `port` numbers and `tls` true/false do **not** get quotes; everything else does

---

## `practice.json` — overriding one practice

If one practice needs a different login, a different folder path, or both:

1. Create a `practice.json` file inside that practice's folder (next to its
   date folders, e.g. `Entities\CAMBS\Export1\AJMAT\practice.json`)
2. Copy from `Entities\EXAMPLE\Export1\PRACTICE_NAME\practice.json`
3. Delete every line you *don't* need to change — keep only the fields that
   differ for this practice. Everything you delete falls back to `entity.json`.

The field names are the same ones as `entity.json`, wrapped in the same
`ftps` / `sftp` sections. A `practice.json` can also **add** a field that
`entity.json` doesn't have at all — e.g. if `entity.json` has no
`tiff_path` because every practice's path is different, each practice's
`practice.json` can carry its own.

Example — this practice only has its own PDF folder name, nothing else differs:

```json
{
  "ftps": {
    "pdf_path": "/{practice}/PDF_Statements_Custom"
  }
}
```

Example — this practice has its own SFTP login but uses the same paths as everyone else:

```json
{
  "sftp": {
    "username": "ajmat-sftp-user",
    "password": "ajmat-sftp-pass"
  }
}
```

Example — this practice has completely different remote folders for everything:

```json
{
  "ftps": {
    "tiff_path": "/SpecialFolder/Claims",
    "pdf_path": "/SpecialFolder/Statements"
  },
  "sftp": {
    "patient_path": "/SpecialFolder/Patients"
  }
}
```

Note: a `practice.json` never needs `workflow` or `insurance_file_type` —
those are entity-wide only.

---

## Something went wrong?

- **Files failed to upload** — they stay in the date folder. Fix the issue and run again.
- **A date folder is still there after running** — one or more files failed. Check the output window.
- **WinSCP error on startup** — run `setup.bat` again to re-download WinSCP.
- **`CONFIG ERROR: entity.json is missing or has blank: ...`** — the fields it
  lists couldn't be found in that entity's `entity.json` (or the practice's
  `practice.json`). Open the file and check those exact key names against the
  example above — a typo in the key name counts as missing.
- **`No .pdf files in <date>\` but the folder isn't empty** — the files in the
  date folder are probably TIFs. Add `"insurance_file_type": "tif",` at the top
  level of that entity's `entity.json` (right under `workflow`).
- **Every field shows missing at once** — the section name itself is probably
  wrong (must be exactly `ftps` / `sftp`), or the file's structure got damaged
  while editing. Compare the overall shape against `Entities\EXAMPLE\entity.json`.
- **The startup banner shows the wrong file type** — the banner line
  (`Entity: ... | File type: .pdf | ...`) shows what was actually read from
  `entity.json`, so it's the quickest way to confirm your edit took effect.