Line Evaluation

Line Evaluation lets you evaluate traffic data on line objects (routes). The evaluation compares flow between an analysis period and a reference period or target value, and calculates KPI scores based on the configured method.

Evaluation overview

The page consists of:

Title bar: At the top, the title bar shows the project name, template name, and KPI method (separated by ·)
Configuration panel: Select periods, time windows, and display mode (route speed / travel time)
Map or KPI table: Shows line objects and KPI scores per time window
Charts: Flow analyses and comparisons

KPI methods

The KPI method is chosen when configuring the evaluation template. Line object evaluation supports two KPI methods. For both methods, the day pattern (the average across days) of the analysis period is compared against a target value. The methods differ in how that target value is determined. Then, for both methods, the basic calculation is how many time intervals of the analysis period fall below/above the target value relative to the total number of time intervals. That results in a percentage that determines the KPI score.

Below is a short overview of how the methods differ. After that, the exact KPI calculation method is described.

Fixed target value (Absolute)

With this method, a fixed route speed (km/h) is specified as the target value:

Measured travel times are converted to route speeds
Compared with the configured target value independently of a reference period
Target values can be set per line object
No distinction by time window
KPI thresholds are predefined (hard-coded) for 6 KPI scores and cannot be configured by the project owner

Target value relative to reference period (Relative)

With this method, the target value is determined relative to values from the reference period per time window:

Two calculation methods are available to determine target values:
- Extra travel time: Reference travel time ± a number of minutes (e.g. +5 or -5 minutes)
- Factor (percentage): Reference travel time × factor (e.g. 1.05 = +5%, 0.95 = -5%)
Target values are set per configured time window
Target values can be set per line object
Additionally, (global) target value sets can be configured by the project owner (1 value per time window for all objects)
KPI thresholds can be configured by the project owner for 4 KPI scores

KPI calculation

The KPI calculation differs per method:

Fixed target value method

With this method there is one fixed target value per object that applies for the whole day (independent of time window and reference value). A reference period of your choice is still shown, but only for comparison purposes.

Step-by-step calculation process:

Day pattern calculation:
- For the analysis period, a day pattern is calculated per object
- The day pattern contains for each time interval the median of the travel times across all days
Route speed calculation per interval:
- The measured travel time is converted to route speed per time interval
- Formula: Route speed (km/h) = Line object length (m) / Travel time (s) × 3.6
- Example: Line object of 5000 m with measured travel time of 360 seconds (6 minutes)
  - Route speed = 5000 / 360 × 3.6 = 13.89 × 3.6 = 50 km/h
Comparison per interval:
- For each time interval within the selected time window, the calculated route speed is compared with the configured target speed
- Only measurements within the time window are included
KPI percentage calculation:
- The KPI percentage is the percentage of measurements where the calculated route speed is lower than the target speed
- Formula: KPI = (Number of measurements with route speed < target speed / Total number of measurements) × 100
Optional weighting by largest deviation (enabled by default):
- If weighting is enabled, the KPI percentage is adjusted based on the largest deviation within the time window
- Of all measurements where the route speed is lower than the target speed, the lowest route speed (largest deviation) is determined
- The relative deviation is calculated: Relative deviation = (Target speed - Lowest route speed) / Target speed
- The adjusted KPI percentage is calculated: KPI = KPI × Relative deviation
- This balances the KPI score when many measurements are below the target speed, but only slightly below. Two time windows with an equal number of undershoots then still differ, with the time window that has a limited deviation scoring better than the time window where the deviation is larger.

Example (without weighting):

Line object length: 5000 metres
Target speed: 50 km/h
Time window: 07:00 - 09:00
Total number of measurements within time window: 24
Number of measurements with route speed < 50 km/h: 6
KPI = (6 / 24) × 100 = 25%

Example (with weighting):

Same data as the example above
Of the 6 measurements below the target speed, the lowest route speed is: 40 km/h
Relative deviation = (50 - 40) / 50 = 0.20 (20%)
Adjusted KPI = 25% × 0.20 = 5%

Interpretation:

Low KPI (0-10%): The route speed is usually above the target speed (good, green)
High KPI (> 50%): The route speed is often below the target speed (poor, red)
With weighting, the KPI is attenuated when the undershoot is limited

Target value relative to reference period method

With this method, the target value is derived from the reference period. The target value therefore moves with the pattern of the reference period.

Step-by-step calculation process:

Reference day pattern calculation:
- For the reference period, a day pattern is calculated per object
- The day pattern contains for each time interval the median of the travel times across all days in the reference period
Target value calculation per interval:
- For each time interval within the selected time window, a target travel time is calculated based on the reference travel time of that specific time interval
- Within the time window there is therefore not one fixed target value; the target value depends on the reference travel time of each time interval
- The calculation method determines how the target value per time interval is calculated:
Extra travel time calculation method (addition):
- Formula per time interval: Target travel time = Reference travel time (of this time interval) + (Configured offset in minutes × 60)
- Example: Reference travel time of time interval = 300 seconds, Offset = +5 minutes
  - Target travel time for this time interval = 300 + (5 × 60) = 600 seconds
- Each time interval has its own reference travel time, so each time interval gets its own target travel time
Factor calculation method (multiplication):
- Formula per time interval: Target travel time = Reference travel time (of this time interval) × Configured factor
- Example: Reference travel time of time interval = 300 seconds, Factor = 1.10
  - Target travel time for this time interval = 300 × 1.10 = 330 seconds
- Each time interval has its own reference travel time, so each time interval gets its own target travel time
Analysis day pattern calculation:
- For the analysis period, a day pattern is calculated in the same way with average travel times per time interval
Comparison per interval:
- For each time interval within the selected time window, the measured travel time from the analysis period is compared with the calculated target travel time
- Only measurements within the time window are included
KPI percentage calculation:
- The KPI percentage is the percentage of measurements where the measured travel time is higher than the target travel time. Or, conversely, where the measured route speed is lower than the target speed.
- Formula: KPI = (Number of measurements with travel time > target travel time / Total number of measurements within time window) × 100

Example (Extra travel time calculation method):

Reference period: 1–7 January 2023
Analysis period: 1–7 January 2024
Time window: 07:00 - 09:00
Configured offset: +5 minutes (+300 seconds)
Time interval 07:00-07:05: Reference travel time = 280 seconds → Target travel time = 280 + 300 = 580 seconds
Time interval 07:05-07:10: Reference travel time = 290 seconds → Target travel time = 290 + 300 = 590 seconds
Time interval 07:10-07:15: Reference travel time = 300 seconds → Target travel time = 300 + 300 = 600 seconds
(and so on for all time intervals within the time window)
Total number of time intervals within time window: 24 (2 hours × 12 intervals of 5 minutes per hour)
Number of time intervals where measured travel time > target travel time: 9
KPI = (9 / 24) × 100 = 37.5%

Example (Factor calculation method):

Time window: 07:00 - 09:00
Configured factor: 1.10 (10% slower allowed)
Time interval 07:00-07:05: Reference travel time = 280 seconds → Target travel time = 280 × 1.10 = 308 seconds
Time interval 07:05-07:10: Reference travel time = 290 seconds → Target travel time = 290 × 1.10 = 319 seconds
Time interval 07:10-07:15: Reference travel time = 300 seconds → Target travel time = 300 × 1.10 = 330 seconds
(and so on for all time intervals within the time window)
Total number of time intervals within time window: 24 (2 hours × 12 intervals of 5 minutes per hour)
Number of time intervals where measured travel time > target travel time: 6
KPI = (6 / 24) × 100 = 25%

Interpretation:

Low KPI (0-25%): The travel time is usually shorter than the target value derived from the reference (good)
High KPI (75-100%): The travel time is usually longer than the target value derived from the reference (poor)

Influence of time windows:

With the relative method, target values are calculated per time window
This means that different time windows can have different target values
The KPI is always calculated for the selected time window

Colour coding: Colour coding is determined by the configured KPI thresholds:

Green: KPI ≤ 10% - Flow is good (low KPI means good flow)
Yellow: 10% < KPI ≤ 50% - Flow is moderate
Orange: 50% < KPI ≤ 75% - Flow is poor
Red: KPI > 75% - Flow is very poor (high KPI means poor flow)
The thresholds can be configured by the project owner. More than one threshold set can be configured. Via the configuration panel under Target value set you can switch between them.

Adjust evaluation

An analysis and reference period are already preconfigured by the project owner. If more periods are configured, you can switch between them via the left configuration panel. In that way other periods can be compared with each other for more insight.

Analysis period

Select the period you want to analyse:

Choose from configured project periods
Or use the button to configure a custom period (see below)
The analysis period is used as the period under evaluation

Reference period

Select the reference period for comparison:

Required when KPI method “Relative” is selected
Choose from configured project periods
Or use the button to configure a custom period (see below)
The reference period serves as the reference for the KPI calculation

Configure custom period

Click the calendar button next to the period selection
Choose a static or dynamic period
Choose which days within this period are included (specific weekdays or the same weekday as the selected date)
Save the custom period

The custom period now appears as Custom in the dropdown menu. You can switch between the preconfigured periods and this custom period.

See period type for more information on setting up periods.

Time window selection

Select the time window you want to evaluate:

Choose from one of the configured time windows
Objects on the map get the colour of the KPI score for this time window
Charts that show data per time window use this time window
The time window determines the sorting of the KPI table (from high to low)

KPI set selection

This option is only available with the target value relative to reference (Relative) method. Select the KPI set you want to use for the evaluation:

Choose from one of the configured KPI sets
The KPI set determines the thresholds (lower and upper bound) for the KPI colour score
Each KPI set has its own threshold values that determine when a KPI score is shown as green, yellow, orange, or red
You can switch between different KPI sets to view the evaluation with different threshold values

Target value set

This option is only available with the target value relative to reference (Relative) method. Select which target values should be used for the evaluation:

Object-specific: Uses the target values configured per line object. Each line object can have its own target values per time window.
Global target value sets: Choose from one of the configured global target value sets. These sets contain target values per time window that apply to all line objects.
Global target value sets are configured by the project owner and contain one target value per time window that is applied to all objects.
You can switch between different target value sets to view the evaluation with different target values.

Day selection

Select a specific day for additional functionality:

Dynamic periods: When using a dynamic period, the selected day determines the reference point for the period calculation
Chart comparison: The selected day is (optionally) shown additionally in the following charts:
- Flow (24 hours): The selected day is shown as an extra line, so you can compare a specific day with the averages of the analysis and reference period.
- Flow per time window: The selected day is shown as an extra data point next to the averages of the analysis and reference period.
- Same weekday: The 6 weekdays in this chart are determined by the 5 same weekdays preceding the selected day (e.g. if you select Monday, all Mondays are compared).
- Last 4 weeks: The days in this chart are determined by the 4 weeks preceding the selected day.

Weighted KPI score

This option is only available with the fixed target value (Absolute) method. Enable or disable the weighted KPI score:

Weighted KPI score enabled (default): The KPI percentage is adjusted based on the largest deviation within the time window. This balances the KPI score when many measurements are below the target speed, but only slightly below. Two time windows with an equal number of undershoots then still differ, with the time window that has a limited deviation scoring better than the time window where the deviation is larger.
Weighted KPI score disabled: The KPI percentage is based only on the number of measurements below the target speed, without taking the size of the deviation into account.

See the section KPI calculation - Fixed target value method for more details on how the weighting is calculated.

Open selection in Analysis

When the Analysis module is available for the project, the configuration panel shows the Open selection in Analysis button.

First select a line object on the map or in the KPI table
Click Open selection in Analysis to open Analysis in a new tab
The linked measurement locations of the selected object are prefilled in the Analysis selection list
The button is disabled when there is no suitable selection (for example when no object is selected yet, or for data that Analysis does not support)

See From Monitor or Evaluation for what happens in Analysis.

View results

The evaluation results based on the settings are made available via the map, the KPI table, and various charts. These are explained below.

Map view

The map shows:

Line objects: Routes being evaluated
Colour coding: KPI scores visually displayed for the selected time window
Popup: Hovering a line object shows the object name, selected time window, KPI percentage, and the number of measurements above the target value
Selection: Click an object to view the charts for that object

KPI table

The KPI table shows for each line object:

Object name: Name of the line object
Comparison selection: Checkbox to add the object to the comparison chart. When checked, you can choose which period is shown:
- Analysis: Shows the analysis period data for this object
- Reference: Shows the reference period data for this object
- Day: Shows the selected day data for this object
KPI score: Calculated score (colour and percentage) per time window based on the selected method
Tooltip: A tooltip when hovering the colour score with the relevant information for the corresponding time window

The table is sorted from high to low based on the KPI score for the selected time window. By selecting a different time window, the sorting can be adjusted.

Charts

Flow (24 hours)

This chart shows the flow for the selected object:

Analysis period: Average flow per time interval across all days in the analysis period
Reference period: Average flow per time interval across all days in the reference period (with relative method)
(optional) Day (date): The data of the day selected in the configuration panel. With this, a specific day can be compared with the other periods. This line can be toggled on/off by clicking the day in the legend.
Time window: A shaded area in the chart indicating the time window

Use this chart to identify patterns and see differences between periods.

Compare line objects

Tab that compares all selected line objects:

Shows flow for all objects selected in the KPI table in one chart
Objects are added via the checkbox in the KPI table
Per object you can choose which period is shown (Analysis, Reference, or Day)
Useful for identifying problematic routes and comparing different objects
You can select multiple objects to compare them side by side

Flow per time window

Shows the flow for the selected object for each time window:

Average flow per time window for the analysis period
The same for the reference period (with relative method)
(optional) Day (date): The data of the day selected in the configuration panel. With this, a specific day can be compared with the other periods. This can be toggled on/off by clicking the day in the legend.

Flow within time window (tabs)

Two tabs with totals per time window:

Same weekday: Compares the selected time window for the same weekday (e.g. all Mondays) based on and ending on the selected day in the configuration panel. Days without data remain visible on the x-axis without a bar.
Last 4 weeks: Shows the last 4 calendar weeks for the selected time window ending on the selected day in the configuration panel. Days without data remain visible on the x-axis without a bar.

Run evaluation

The KPI method is preset in the evaluation template (shown in the title bar)
Select the analysis period
Select the reference period (with relative method)
Choose the desired time window
The evaluation is performed automatically
View the results on the map, in the KPI table, and in the charts
Select objects via the map or table to view detailed results
Optionally compare detailed data between objects via the compare objects function (see Compare line objects)