Modelling Module

Modelling Module

Overview 

The Modelling Module is a key component of the Demand Drivers tool, specifically designed to help users create, configure, and run marketing mix models to assess the impact of marketing activities on business outcomes. Here’s a detailed look at the module’s functionality:

Getting Started

Once the user clicks on next from the review module, the user will be navigated to the Modelling module on 'In Progress' page. 

'In Progress' page, allows the user to view the log of completed, running, errored, and draft batches, providing a comprehensive overview of the status of all live jobs for the project. Additionally, if it is the user's first time on this page, they can create a new model. 

To create a model, user must click on ‘Create New Model’ from the bottom right of the page.

                                             A screenshot of a computer Description automatically generated
'Model Results’ is another page that contains a log of all the batches run in the project.

Create New Model

To create new model, user must click on ‘Create New Model’ from bottom right of the 'In Progress' or 'Model Results' page.

On clicking on ‘Create New Model’, user will be displayed two options,

1. Create New Model – This option allows users to create a model from scratch by providing all necessary configurations.
2. Edit Model – If users already have a batch configuration, they can use this option to upload the existing configuration and run batches.

                                                                      

Create New Model Option 

When user clicks on ‘Create New Model’, user is navigated to model configuration page.

Model Configuration Page

The model configuration page allows users to set the model type and form, define the model and holdout duration, choose variables, apply transformations, specify priors, and set criteria and weights. Users can also select between manual or auto model and run the batch process.

Model Setup

By default, the model setup popover displays the following settings:

1. Dependent Variable – Users can classify multiple dependent variables, but they can only model one dependent variable at a time. For example, if both 'Sales Units' and 'Revenue' are classified as dependent variables, they must choose one for each batch run.

2. Model duration – by default entire datacube period is considered.

3. Holdout duration – The holdout duration is off by default but automatically set to 20% of the model duration. It acts as a testing period to check how well the model’s predictions match actual results. For instance, if the data spans from January 2021 to December 2023, the holdout period will default to May 2023 through December 2023. You can adjust this period, but it must not overlap with the training data. The model only trains on data up to the start of the holdout period, and the coefficients are applied to the holdout data.

4. Advance options, the advance options when expanded contains below settings of the metrics-
  1. Model Type – allows users to select the modelling approach that defines how data is handled across different dimensions. For single-dimension data, only the Unpooled Model is available. For multidimensional data, users can choose between the Unpooled and Pooled models. The default is Unpooled, which runs separate models for each dimension. The Pooled Model, on the other hand, combines data across dimensions, providing a shared set of coefficients with different intercepts for each dimension.
  2. Model Form – The platform defaults to the additive model form and currently supports only this model type.
  3. Dimension– For a single dimension project, this field does not get displayed in model setup but if it is a multi-dimension project, by default all the granular or the lowest level dimensions will be selected. User has an option to deselect the dimension for which they do not wish to model.
                                                              
                                                                         Image 1: Single dimension model setup

                                                                  
                                                                  Image 2: Multi-dimension model setup

After user sets the model setup, they need to click on ‘Confirm’ to save the changes.

Variable Selection

After saving the model setup, the user is directed to the variable selection page. Here, the user selects the necessary variables for the model. Variables can be chosen from the left pane and will then appear on the right. The user can utilize checkboxes to select variables and has the option to switch the view to display all variables from the data classification hierarchy or use the search box to locate specific variables.

The variable selection page allows users to manage variables with the following features:

1. Mandatory () – To make a variable mandatory, user must select lock icon and by default all the shortlisted variables be considered mandatory, and all are considered essential for the model.

2. Optional () – To mark a variable as optional, the user needs to click the lock icon, which toggles the variable to an unlocked state. An optional variable will be included in some iterations and excluded in others. For example, if your model has 5 variables, and you make one of them optional while the others remain mandatory, the system will run one iteration with all 5 variables and another without the optional variable.

Note: Optional variables can be used in the model if needed, but they may increase computation complexity.

3. Switch off ()- When a variable is switched off, it is excluded from the model but can still have its configuration set. If the user later decides to use this variable, they can make it mandatory or optional without needing to reconfigure it.

Advance Settings

Bare minimum steps for a novice user to run a model is to select variables and click on ‘Run’. Mutually exclusive and report format is an optional step. For the users who wishes to provide transformations, priors and add some criteria for the model to meet, user must click on ‘Advance Settings’ placed at the top right corner denoted as 

Advance settings contain following options for the advanced users to configure their model better,

1. Custom Transformation (optional: with saturation)
2. Priors
3. Qualifying Criteria (optional: with additional weights)
4. Run Model Options

                                                             

Custom Transformation

Custom transformation allows users to apply specific adjustments to their variables. Here’s how to use it:

  1. Range: This option is suitable for testing multiple parameter values. For example, if the optimal decay parameter is uncertain, users can set a range (e.g., Min: 0.2, Max: 0.8, Increment: 0.2). The system will test values such as 0.2, 0.4, 0.6, and 0.8.

  2. Values: This option is for users who already know the key parameters. They can enter the exact values they wish the model to use.

  3. Show Saturation: Enabling this checkbox applies saturation curves to the model. For S-curve transformations, users can specify the Alpha and Beta parameters to define the shape and scale of the curve. Default values will be applied if they are uncertain. Users should click Save to apply these settings, and default values will populate if left blank

By enabling ‘Custom Transformations,’ an additional tab appears next to Report Format, labeled 'Variable Transformations,' as shown below,

Single Dimension:




Multi-Dimension:



For multidimensional data, a dimension column is visible, as shown in the screenshot above, where users can apply different transformation parameters for each dimension. An option called ‘All’ allows them to apply a transformation to all dimensions at once, which can be useful. For example, if a decay parameter of 50% is applied, this parameter will be replicated across each dimension.

Depending on the selected transformation type, the corresponding parameters are enabled such as decay for adstock, build, decay and period for gamma etc.
If ‘Show Saturation’ is activated, users can choose between S-curve or Hill curve from the list.

The platform allows users to enter the following transformations along with their parameters:

Transformation Type

Parameters

Gamma

Build

Decay

Period

Carryover

 

Decay

 

Exponential Decay

 

Decay

Period

Adstock

 

Decay

 

Lag

 

 

Period

Log

 

 

 

Moving Average

 

 

Period


If there is no transformation, then it's type will be selected as ‘Direct’. 

Gamma Transformation and its parameters–  

  • Build or Degrees of Freedom: The minimum time required for the advertisement to run and impact the viewers to make purchase decision 

  • Period:  The time for the impact to reach maximum 

  • Decay:  The amount of current period activity that effects the future period. 


Adstock and its parameters –  

  • Decay: The amount of current period activity that effects the future period. 


Exponential Decay and its parameters –  

  • Decay:  The amount of current period activity that affects the future period.  Higher values decay more quickly (most impact in the earlier weeks), lower values decay more slowly (impact more similar across weeks) 

  • Period:  The period at which the decay impact is spread


Priors  

Priors allow users to incorporate prior knowledge about the coefficients or contributions of variables.

Setting Priors: 

When user enables the priors from advance settings, by default, the system assigns priors as contribution based on the classification of the variable. The options for priors as contributions are: 

  • -80% to 80%: When the relationship with the dependent variable is unknown or marked as don't know in variable properties. 

  • 0% to 80%: When a positive relationship is expected. 

  • -80% to 0%: When a negative relationship is expected. 


Priors as Coefficient: If users prefer to set coefficients instead of contribution percentages, they can select the coefficient option from the dropdown menu under Priors, which allows them to enter the coefficient and its standard deviation %.

Note: Users should be mindful when entering coefficients with an S-curve transformation, as variable scaling can significantly influence the coefficients.

Qualifying Criteria  

Qualifying Criteria help in evaluating the iterations generated by the system. To provide the criteria, users have two options,

  1. 1. Setting Criteria: Define threshold for statistical conditions like R-Square, MAPE, Holdout MAPE, and Durbin-Watson. Specify the threshold for business conditions like incremental contributions percentage and spend index. 

  1. 2. Ranking and Weights: 'Set Weights for Ranking' checkbox allows the user to rank iterations based on statistical and business criteria. 


Note: all the iterations meeting all criteria fall under the Qualified tab, while those missing at least one criterion are listed under the Disqualified tab in the model summary output page. 

All the above model configurations can be performed using the export from the variable transformation or priors or qualifying criteria page and import it back to the system.

Except for model setup, variable selection, and report format, other model configurations—such as custom transformation, priors, and qualifying criteria—can be set using the export or batch configuration file from the variable transformation tab. To apply these configurations, the variables must be selected from the platform before exporting.

Note: In the export file or the batch configuration file, if users enter parameters only in the min column and upload it, the system will automatically populate the same value in the max column, with increments set to 0 when the custom transformation is selected as a range in the advanced settings. If the custom transformation is selected as a value, the values column will be updated with the entries made in the min column.

Run Model Options  

The Run Model option lets you choose between Manual and Auto model types, 

  1. 1. Manual: This requires more manual intervention and does not apply hyper parameterization. You can generate a single model or multiple models depending on the transformations and variables selected. 


  1. 2. Auto: This feature uses hyper parameterization to find the best model configuration. It generally generates 60 or more iterations per batch, optimizing parameters based on the selected range or values.


Additional Options 

Mutually Exclusive Variables  

Once all the variables are selected, user will be notified if there are any mutually exclusive variables, if yes then user needs to click on Yes which navigates the user from variable selection page to mutually exclusive page. 

 

User has an option to change their response back to no and continue configuring they accidentally clicked on Yes. 

 

Mutually exclusive variables allow the user to handle multi-collinearity between alike variables.  

For instance, consider the 'Media' category with 'FB' variables presented in two forms: 'FB_Impressions' and 'FB_Clicks.' These variables cannot be utilized in the same iteration; therefore, by using this feature, two separate iterations can be generated—one for 'FB_Impressions' and one for 'FB_Clicks.'


Below are the steps to segregate the alike variables on the platform- 

Step 1: click on ‘Add Custom group’ 

A screenshot of a computer Description automatically generated 

Step 2: Enter a group name, such as Meta_group. After creating this group, users will see sub-groups labeled #1 and #2 underneath it. 

A screenshot of a computer Description automatically generated 

Step 3: Users should now assign FB_Impressions to bucket #1 and FB_Clicks to bucket #2. This allows the system to run iterations without including these two variables together.

Note:  

1. Users have the option to create up to four sub-groups and can create more custom groups based on their requirements.
2. This feature can be utilized in the model if necessary; however, it may increase computational complexity.

Report Format  

The Report Format is specifically used to preload the Due-to chart by defining two subset time periods within the model's duration. All other charts will utilize the full model duration. While setting the Report Format is optional, the platform provides default settings for report formats that can be edited if necessary. Even without specifying a report format, the Due-to chart will display with default settings. 

Users can create custom time periods for the Due-to chart. To do this, click the periodicity dropdown in the left pane of the report format. 

 A screenshot of a report Description automatically generated 

 

This dropdown allows users to select time periods such as monthly, quarterly, yearly, or custom. For the custom period option, users must specify both the date range and the number of weeks for each custom period.

A screenshot of a report Description automatically generated


For example, if a user selects January 5th, 2020 as the start date and wants to create periods of 2 weeks each until the end of the model duration, Period 1 would span from January 5th, 2020 to January 19th, 2020. Period 2 would then cover from January 19th, 2020 to February 2nd, 2020 and so on until end date of model duration.

After generating the desired periods, users will see them in the right pane under the period 1 and period 2 dropdowns. By selecting the desired periods and clicking ‘Add,’ users create a new set of periods. They can set this new set as the default for the Due-to chart by enabling the corresponding radio button.

                                                       A screenshot of a report Description automatically generated

Model Output 

Once the user clicks on ‘Run’, it automatically redirects the user to the ‘In Progress’ tab within the modelling module. 

InProgress Page  

On this page, the user can view the log of completed, running, errored, and draft batches, providing a comprehensive overview of the status of all live jobs for the project.

A screenshot of a computer Description automatically generated 

 

Here's a breakdown of the information that is in the Inprogress page: 

     

Name 

Description 

Batch Name 

 The unique name given for the batch. 

Model Type 

 Indicates whether it's an Auto Model or Manual Model. 

Date 

 When the model was triggered. 

Time 

 Estimated time left for batch completion. 

Status 

 The current status of the batch, such as "Draft", "100%", or "Error". 

View Batch 

 A button to view the output of the batch. 

View Details 

 A button to view more details about an error that occurred during the training process. 

Resume Setup 

 A button that allows user to access the latest draft of model configuration. 

         

 Clicking on cancel icon will clear the job from the row on Inprogress page 

          

 Clicking on delete will delete the entire batch irrespective of its status (Inprogress,   completed, errored etc.) 

 

Model Results Page

The Model Results tab provides a centralized view of all completed batches or models within the project, irrespective of passed or failed. It displays each batch as a tile, offering a concise summary of the model's key attributes.

Screens screenshot of a computer Description automatically generated 

 

Key Features 

  • 1. Batch Tiles: 


  • a) Display: Each batch is represented as a tile, providing a visual overview. 

  • b) Indicators: 

  • I. Model Type: Indicates whether the model was built using an Auto or Manual approach. 

  • II. Progress Bar: Shows the completion percentage of the model building process. 


  • 2. Additional Information: 


  • a) Inputs Overview: Displays the number of mandatory and optional variables used in the model, the selected dependent variable, and the duration of the model building process. 

  • b) Saved Iterations: Provides information on how many saved iterations are available for quick review. 


  • 3. Tile Actions: 


  • a) Click Anywhere on the Tile: Navigates to the Saved Iterations page for detailed inspection. 

  • b) Info Icon : Provides a comprehensive summary of the batch, including model setup, completion time, the number of outputs, transformations used, if spends are uploaded then the spend tag will show blue else it will show grey.


  •                                                        A screenshot of a computer Description automatically generated


The summary popup has two buttons, 

  • I. View Batch – clicking on view batch will re-direct the model output summary page where user can see all the recommended, qualified and disqualified iterations. 

  • II. Download Configuration – This will allow the user to download the batch configuration. 

  • c) Copy Icon : Creates a copy of the existing batch, allowing you to edit configurations without affecting the original. 

  • d) Delete Icon : Permanently removes the batch, regardless of its status (in progress or completed). 

  • e) Output Icon : Opens the model output page directly for single-dimension models. For multidimensional models, redirects to the recommended iterations page. 

       

  • 4. Hide Unsaved Iterations: User can access this feature at the top right corner of the model results page. On enabling this feature, user can view the batch tile card in tile view or list view of saved batches only. 

 

A blue and white screen Description automatically generated  

 

From the list view, user will be able to compare two models of different batches as well (refer to the image below), 


A screenshot of a computer Description automatically generated 

User must select the two iterations of two batches after which the ‘Compare’ button will be enabled to view the compared or multi-view of the outputs. Maximum user can compare 3 iterations at a time.


Model Output Summary 

The Model Output Summary page provides a comprehensive overview of the performance and configuration details for a specific batch. It displays key metrics, model iterations, and options for further analysis.

Key Features

  • Batch Information:
    • Batch Name: The unique identifier for the batch with an edit option to edit system generated batch name.
    • Model Count: The total number of models within the batch is displayed next to the batch name within brackets.

  • Model Iterations:

For an auto model, system can generate 60-70 iterations with top rankings and for a manual model, a single iteration is generated.

    • Saved: Displays iterations that have been saved for future reference.
    • Recommended: Lists the number of recommended iterations based on qualifying criteria and assigned weights. In recommended tab, there can be maximum of 9 ranked iterations.
                   For a multi-dimensional project, there can be 9 ranks per dimension.

    • Qualified: The number of iterations that meet the specified qualifying criteria will fall under qualified bucket.
    • Disqualified: The number of iterations that did not meet the qualifying criteria will fall under disqualified bucket.
    • Errored: This tab only appears if there are iterations with errors during model building.
             


Iteration Details: 

  • Rank: First row of summary page shows the ranking of each iteration based on its performance.


  • Model Name: Below the ranking number, the system-generated model name is shown.


  • Statistical Comparison: Compare metrics like R-Squared, Adjusted R-Squared, and MAPE across iterations.


  • Model Configuration: The summary will show co-efficients, transformations, absolute contributions, percentage contributions for each variable in an expanded view, User an option to collapse each of these for better view.

  • A screenshot of a computer Description automatically generated 

 

Actions and Features 

  • View icon A blue eye in a square Description automatically generated: Users can click the checkboxes next to the desired iterations and then select the view icon to access the model output. This feature also allows users to compare iterations by selecting up to three iterations and clicking the view icon to enable comparison mode. 


  • Save icon : Users have the option to save specific iterations by selecting them and clicking the save icon. The saved iterations will be reflected under the "Saved" tab.

  •  

  • Edit Model Configuration: Allows users to modify the configuration settings for a specific model. 


  • Download Summary: Provides the option to download a summary report of the batch's performance metrics. 


  • Download Configuration: Enables users to download the configuration details for a specific model.  

  • Filtering & Sorting : 


  • Change Qualifying Criteria & Weights: Adjust criteria and weights to re-rank iterations. 


  • Sort Outputs: Prioritize iterations based on specific criteria (e.g., higher R-squared, lower MAPE). 

  • I. Primary Sort: Choose the main criterion. 

  • II. Secondary Sort: Define a secondary criterion for further ranking. 


  • Dimension Dropdown: For multidimensional datasets, filter outputs by specific dimensions. A screenshot of a chat Description automatically generated 

 

Model Dashboard View 

The Dashboard View provides a comprehensive visualization of model outputs through various charts and interactive elements. It offers insights into model fit, variable contributions, response curves, ROI, Due-To chart, effectiveness, and spend vs. contribution with ROI. 

Chart Details 

  • Model Fit: Provides a visual representation of how well a predicted model fits the actual data.

  •  

  • Decomposition Chart: Breaks down the model results into base and incremental contributions. 


  • Spend vs. Contribution with ROI: Compares spending of the incremental variables against the contributions and ROI.

  •  

  • A graph on a white background Description automatically generated

 

  • Contribution Charts: Displays the contribution of each variable to the model. 


  • Response Curves: Illustrates the relationship between variables and the dependent variable.

  •  

  • A screen shot of a graph Description automatically generated

 

  • Due-To: Allows the user to compare the change in contribution of the variables between two periods defined (Period 1 and Period 2).


 

  • ROI: Displays the return on investment obtained for the variable. 

  • A screenshot of a video Description automatically generated

 

  •  Effectiveness: Helps the user to analyze how impactful is a variable per support to the model. By default, the chart is loaded for impression variables if user has set the units in variable properties. 


 

The order of precedence of units for the default to generate are, 

1. Impressions 

2. GRP 

3. TRP 

4. Clicks 

5. USD 

6. Custom units (if they have created in project or global settings) 

7. Views 

8. Users 

9. Kilogram 

10. Liters 

11. Pounds 

12. Count 

13. Percentage 

14. Score 

15. Coupons 

16. Other 

17. NA 

Note: If in project settings, user selects effectiveness as KPI/Spend then the unit’s dropdown must be set to All. In this case the precedence is not required since the units are unique. 

Interactive Features: 

  • 1. View Mode Indicators: 

  • a) Single Model View: Shows detailed charts for one iteration. 

  • b) Multi-Model Comparison View: Limits dashboard display; individual charts are expanded separately. 

 

Note: user can compare upto maximum of three iterations. 

  • 2. Drill-Down Functionality: 

  • Click on specific parts of charts (e.g., incremental segment) to view more granular data (e.g., media vs. non-media). 


  • 3. Expanded view: On clicking on the title of the chart or the pills at the top will navigate the user to the expanded view of the chart. 


  • A screenshot of a graph Description automatically generated



  • Compare Within Charts: Users can compare up to four periods at the same time only in the expanded view.
  • Dimension Selection: For a multi-dimensional dataset, click the Select Dimension  to choose specific dimensions for focused analysis.
  • Export Features:

(1)    Export Model: Model export gives a summary of the model.

(2)    Export Data: Data export provides the user below details,

      • Raw Data: Access untransformed data for detailed analysis.
      • Transformed Data: Obtain data that has undergone model-specific transformations.

 

  • Save and Publish:
    • Save Model: Click the Save button to store the model for downstream use.
    • Publish to Report: Check the Published to Report option to make the model available in reporting modules like Simulation and Planning.
                                                                           

 

·         Update Model: User is allowed to update the model using this functionality. Clicking on this will open up the configuration of the existing model with priors having 10% standard deviation on the netted out co-efficient. This 10% standard deviation defaults can be changed from project settings under modelling.


                                                                                 

 

In the project settings, under the modeling module, users have 2 options -

1.      User defined: By opting for this, user will see the co-efficients inputted to the existing model. For example, for the variable TV_GRP the co-efficient provided was 12678 and standard deviation is 0.001% then on using this option user will see the same settings on priors page.

2.      Default deviation percentage: user can use this option to change the defaults from 10% to custom value.



 Batch Detail View Page 

After clicking on the batch tile card, user is navigated to ‘Saved Iterations’ page where user can view all the saved batches.

For multi-dimensional datasets, along with ‘Saved Iterations’ user will another tab called ‘Aggregated Models’.

This aggregated models page contains all the aggregate models. By default, an aggregated model is generated by combines all dimension-level Rank 1 models.                                           A screenshot of a computer Description automatically generated


User also has an option to create a custom aggregated model. The steps to create an aggregated model are:

Step1: Save all the models of interest from model output summary page (recommended and qualified page)

Step 2: Select the radio buttons from Saved iterations page as shown in the image below. 

                              

Note: one iteration per dimension must be selected.

Step 3: Click on the ‘Create Aggregated Model’ as shown in the image below. This button gets enabled only when one iteration per dimension is selected and to verify this, at the top user can view if all the dimensions are selected. In the example below all the models for 12 dimensions are selected to create an aggregated model.

                                              A white screen with text Description automatically generated with medium confidence

Step4: To view this aggregated model, user must switch the tab from Saved iterations to Aggregated Model.

o    Dimension Selection from Saved iterations page: Use the dropdown at the top to drill down into specific dimensions (e.g., regions).

                                    

 Additional Features

  1. Editing Configurations: User can click the Edit Model Configuration button to return to the model setup page and adjust configuration settings and run a new batch.
                                          A blue and white striped background Description automatically generated with medium confidence
  1. Download Configuration: The user can download the configuration of the batch using this feature.

·         View Summary: Provides a comprehensive summary of the batch, including model setup, completion time, the number of outputs, transformations used, if spends are uploaded then the spend tag will show blue else it will show grey.


                                                                                           A screenshot of a computer Description automatically generated

The summary popup has two buttons,

  1.  View Batch – clicking on view batch will re-direct the model output summary page where user can see all the recommended, qualified and disqualified iterations.
  2. Download Configuration – This will allow the user to download the batch configuration.

·         Publish the model to report: To publish the model to the report, user must click on the check box for the respective model under ‘View in Report’ as shown in the image below. 

                                           

·         Compare saved iterations: user can click on the check boxes next to the iterations which will enable the compare option at the top and will be able to compare the iterations. User can compare maximum upto 3 iterations. 

                                     A screenshot of a computer Description automatically generated
              


    • Related Articles

    • Report Module

      Overview The Report Module serves as a comprehensive tool for reviewing and analyzing the outputs from the Modeling Module. It enables users to delve into various analytical charts, including contribution, decomposition, ROI, spend vs contribution ...

    • Simulation Module

      Overview The Simulation module allows users to run different "what-if" scenarios or algorithm-driven budget optimizations to understand the best way to allocate the available marketing budget across different marketing channels. By analyzing ...

    • Introduction to Input Module

      Introduction The Input Module provides users with essential features for the data ingestion process, including support for various file types and formats. After creating a project and selecting the project title, users will enter the Input Module, ...

    • Frequently Asked Questions

      All Projects Page What is the All Projects page? The All Projects page is the landing screen after logging in, showing all your projects. Initially, you may see a blank page with a ‘Create New’ button if you have no projects. Projects are displayed ...

    • Guide to the All Projects Screen and Project Creation Process

      All Projects Screen When you log in to the URL, you will land on the All Projects page. When you log in for the first time, you may see a blank page with only a ‘Create New’ button at the top right-hand side of the screen. If you have created ...