Building & Training an Intervention Targeting Model

Intervention Targeting Model Overview

An intervention targeting model is a combination of machine learning models and evaluation metrics to help an organization allocate intervention resources for maximum benefit.

The targeting method may be based on predicted un-treated risk (outcome if the individuals do not receive the intervention) or predicted impact (change in outcome because of the intervention.) Calculating predicted impact utilizes causal inference machine learning.

To train an intervention model, it is necessary to identify a list of past individuals who received the intervention and did not receive the intervention, the outcome to maximize, and the relevant time periods and features.

Each intervention is represented in the platform as a list of individual_id (patient id), year, and month.

Creating an Intervention Targeting Model

  1. Navigate to the folder where you want your model to be stored
  2. Click the “+” icon located to the right of “Models”
  3. The following box will pop-up:
  • Type: Select "Intervention Targeting"
  • Define Model and Dataset
    – Build in Curia App: Select if you want Curia to generate the dataset or have a custom component of a dataset (such as an independent custom treatment, outcome, or cohort)
    – Upload pre-compiled data: If you have a completely custom training dataset that you wish to use
  1. Enter a name for your model
  2. (Optional) Write a description of your model
  3. Click the "Create Model" button, which will open a new page
85008500

Building in the Curia App

Outcome and Intervention

You will first be prompted to select an Outcome and Intervention

Outcome

  1. Select Outcome
    – The Point & Click workflow allows you to define Diagnosis, Procedure, Prescription, DME, Metric, and Events outcomes by selecting specific codes corresponding to the desired outcome you'd like to model
    – To model a custom outcome, select “Custom Outcome” and choose the outcome dataset you wish to use.

  2. Select Outcome Type:
    – To predict the likelihood a given binary event occurs, select Occurrence
    – To predict a continuous outcome, select Regression

Intervention

  1. Select Intervention
    – The Point & Click workflow allows you to define the intervention by selecting specific codes
    – To model a custom outcome, select “Custom Outcome”
    – Click "Select Dataset" and choose the outcome dataset you wish to use.
85008500

Generate Cohorts

  1. Set Evidence: Number of months that the covariates (features) for training are built on. This is always 12.

  2. Set Data Delay: Number of months between the end of the evidence period and the rest of the time periods

  3. Set Intervention: Number of months the intervention is being done.

  4. Pre-Outcome Delay: Number of months between the end of the evidence period and the start of the outcome period

  5. Outcome: Number of months that information is aggregated over to generate the modeling outcomes

Window

Rolling (Multiple Cohorts): Select if you wish to create more data from a rolling window of several different incremental time periods. Then select the date when the evidence period starts and the date you want your outcome period to end across your windows.

Fixed (Single Cohort): If you’re certain about your exact start and end dates for the evidence and outcome period

Require full outcome period data for individuals

– It is often the case that some individuals will die before the end of the outcome period (or switch to a new healthcare organization), meaning that if we are modeling on the occurrence of a specific code, this individual could have had this code, but we ran out of information on them since they are no longer in our dataset.

– Checking this option counteracts this by ensuring only individuals with data after the outcome period ends are included

85008500

Population

New Code Filter: Add a filter that either only includes or excludes patients with specific code data in our modeling analysis

85008500

New Demographic Filter: Add a filter that either only includes or excludes patients that share some specific demographic information in our modeling analysis

85008500
  1. Once you have configured all of these elements, click "Preview Model Data" to run queries that generate the relevant dataset and output summary statistics

  2. Click the "Train Model" button

Upload Precompiled Data (WIP)

You must have already uploaded a “training” dataset (see the “Create Custom Dataset” section)

If you are building an impact model, the training dataset must have a “treatment” column, whereas a risk model does not need this column in the data.

  1. Click “Select Dataset” and choose the relevant dataset

    • The user interface will show a preview of the columns/features read from the training dataset.
  2. Then click train model