Retrieve a forecast's analysis

gethttps://api.faraday.ai/v1/forecasts/{forecast_id}/analysis
This endpoint is experimental and may change without notice.

How the forecast's current model performed on entries it did not train on, the level correction applied to delivered values, how the training labels were handled, and the readings taken on each daily refresh. Written by the forecast's refresh after a build finishes; until then the request returns 404.

Authentication

Bearer Authentication

Provide your API key in the Authorization header. You can find your API key in the Settings page of the dashboard.

Authorization: Bearer YOUR_TOKEN

path parameters

forecast_idstring<uuid>required

The UUID of a forecast

Responses

200The forecast's analysis
calibrationobject

The level correction the latest refresh applies to delivered values. Absent when the refresh recorded none.

Properties of object
applies_below_ageinteger

Entries younger than this many days have factor applied to their forecast value remaining; older entries are delivered as the model forecasts them.

base_entriesinteger

Training entries the recent cohort is compared with.

base_mean_earlynumber

The training population's mean value realized within the window.

factornumber

The multiplier applied to the forecast value remaining of entries younger than applies_below_age. 1 when no correction could be trusted.

Example: 1.04
methodstring

How the factor is read; drift compares the newest observed cohort with what the model implied for it.

ran_atstring<date-time>

The refresh's data cutoff, the end of the last full day with data or the pinned observation date if earlier.

reasonstring

Why the factor is 1 or was held inside its band, when it was.

recent_entered_beforestring<date>

The recent cohort's entries are before this date.

recent_entered_fromstring<date>

First entry date of the recent cohort.

recent_entriesinteger

Entries in the recent cohort.

recent_mean_earlynumber

The recent cohort's mean value realized within the window.

refreshed_atstring<date-time>

When the refresh ran.

window_daysinteger

Days after entry the recent cohort is observed through.

yoy_factornumber

The damped year-over-year ratio of the two early means, recorded for comparison and not applied.

driftobject

The drift reading of the latest refresh, on the newest cohort whose window is fully observed. Absent when the refresh recorded none.

Properties of object
rationumber

The cohort's realized value at the window age over what the model implied for it, minus one. Positive means the business runs above what the model learned.

reasonstring

Why no ratio could be read, when none was.

recent_entered_beforestring<date>

The cohort's entries are before this date.

recent_entered_fromstring<date>

First entry date of the cohort.

recent_entriesinteger

Entries in the cohort.

window_daysinteger

Days after entry the cohort is observed through.

by_windowarray[object]

The same reading at shorter windows, each on its own newest cohort. They react sooner and overshoot, so they are recorded and not applied.

Items of array[object]
rationumber

The cohort's realized value at the window age over what the model implied for it, minus one. Positive means the business runs above what the model learned.

reasonstring

Why no ratio could be read, when none was.

recent_entered_beforestring<date>

The cohort's entries are before this date.

recent_entered_fromstring<date>

First entry date of the cohort.

recent_entriesinteger

Entries in the cohort.

window_daysinteger

Days after entry the cohort is observed through.

dataobject

How far the streams' data reaches.

Properties of object
high_water_markstring<date-time>

The newest event on either stream.

lag_daysinteger

Days from the newest event to the refresh.

pinnedboolean

Whether the data cutoff is held by predictors.max_observation_date.

development_sharenumber

The share of its value the training population had realized by the window age.

expected_realized_meannumber

What the cohort was expected to have realized by the window age.

predicted_total_meannumber

The model's mean forecast for the cohort at entry.

ran_atstring<date-time>

The refresh's data cutoff.

realized_meannumber

What it did realize.

refreshed_atstring<date-time>

When the refresh ran.

generationobject

The model build the analysis describes.

Properties of object
active_entriesinteger

Entries whose horizon was still open at training.

balancenumber

Factor applied to the model's raw predictions so that they average to the training value.

boosting_roundsinteger

Boosting rounds the model kept.

entriesinteger

Entries on the entry stream at training.

feature_columnsarray[string]

Identifiers of the model's inputs. Entry properties appear with an entry_prop_ prefix; the rest describe the entry's age, its realized value and the person's earlier entries.

finished_atstring<date-time>

When the build finished.

holdout_entriesinteger

Entries held out from the fit.

holdout_rowsinteger

Observations held out from the fit, on which every performance number is measured.

matured_entriesinteger

Entries whose horizon had ended at training.

trained_as_ofstring<date-time>

The data cutoff the model was trained with.

trained_entriesinteger

Entries the model fit on.

trained_rowsinteger

Observations the model fit on. An entry observed at one age is one observation.

training_entries_fromstring<date>

The earliest entry the model learned from.

training_entries_tostring<date>

The latest entry the model learned from.

unmatched_value_event_sharenumber

With an entry_key_property_name, the share of value events whose key matched no entry.

value_eventsinteger

With an entry_key_property_name, events on the value stream that passed stream_conditions. Absent for a forecast without an entry key.

label_handlingobject

What was done to the matured values before the model learned from them.

Properties of object
agesarray[integer]

The ages at which each matured entry is observed as a training row.

cap_quantilenumber

The quantile of matured values the cap was set at.

Example: 0.999
cappedinteger

Entries whose value was capped.

excluded_datesinteger

Matured entries left out for an implausible entry date.

flooredinteger

Entries whose value was negative after deductions and floored at zero.

label_capnumber

Values above this were capped at it.

training_entriesinteger

Entries that produced training observations.

training_rowsinteger

Training observations after label handling.

performanceobject

Performance of the current model on held-out entries.

Properties of object
acceptanceobject

Whether the fit cleared the floor on its own held-out entries. Recorded, not enforced; the model is in use either way. Absent when the model did not record it.

Properties of object
passedbooleanrequired
reasonsarray[string]required

Why the fit did not clear the floor, empty when it did.

by_agearray[object]

Every held-out entry, scored at each age the model was trained at. These rows carry the decile calibration, the bias interval and the constant baseline when the model recorded them.

Items of array[object]
gininumber

Normalized Gini of the ranking by forecast. 1 means entries are perfectly ordered by value, 0 means no better than random order, and a negative value means the ranking is inverted.

Example: 0.3
maenumber

Mean absolute error per entry. A descriptive number; it favors forecasting low when most entries realize nothing, so the comparison with the constant baseline is on rmse.

mean_actualnumber

Mean value actually realized.

mean_forecastnumber

Mean forecast of the value remaining.

ninteger

Held-out observations the numbers are measured on.

rmsenumber

Root mean squared error per entry, in the value's unit. The error score the model and the constant baseline are compared on; it is minimized by the mean, which is what the model forecasts.

top_decile_sharenumber

Share of all actual value found in the tenth of entries with the highest forecasts.

Example: 0.25
total_biasnumber

Sum of the forecasts over the sum of the actual values, minus one. What a platform summing the delivered values sees; +0.1 means the forecasts added up to 10% more than reality.

Example: 0.04
ageintegerrequired

Days after the entry event at which the entries were scored.

constant_baselineobject

What forecasting the training rows' mean remaining value for every entry would have read on the same held-out observations. The model is compared with it on rmse, the error the mean minimizes; it ranks nothing.

Properties of object
maenumber

Mean absolute error of the constant forecast.

mean_forecastnumber

The constant forecast, the training rows' mean remaining value at this age.

rmsenumber

Root mean squared error of the constant forecast. A model whose rmse is lower explains some of the spread between entries; one whose rmse is higher does worse than forecasting the same value for everyone.

total_biasnumber

Total bias of the constant forecast.

decilesarray[object]

Calibration by forecast decile.

Items of array[object]
decileintegerrequired

1 is the tenth with the highest forecasts, 10 the lowest.

mean_actualnumber

Mean value actually realized in the decile.

mean_forecastnumber

Mean forecast of the value remaining in the decile.

nintegerrequired

Held-out observations in the decile.

share_of_actualnumber

This decile's share of all actual value in the held-out entries.

total_biasnumber

Sum of the forecasts over the sum of the actual values in this decile, minus one.

total_bias_intervalobject

Where the total bias lands over bootstrap resamples of the held-out observations. It measures how much of the reading is the particular entries drawn; it is not a forecast interval.

Properties of object
highnumber

Upper end of the band.

levelnumber

The band's coverage, as a share of the resamples it contains.

Example: 0.9
lownumber

Lower end of the band.

by_prior_entriesarray[object]

Held-out entries at age 0, by how many entries the person had before. Metrics only, as for newest_entries. Absent when the model did not record the slice.

Items of array[object]
gininumber

Normalized Gini of the ranking by forecast. 1 means entries are perfectly ordered by value, 0 means no better than random order, and a negative value means the ranking is inverted.

Example: 0.3
maenumber

Mean absolute error per entry. A descriptive number; it favors forecasting low when most entries realize nothing, so the comparison with the constant baseline is on rmse.

mean_actualnumber

Mean value actually realized.

mean_forecastnumber

Mean forecast of the value remaining.

ninteger

Held-out observations the numbers are measured on.

rmsenumber

Root mean squared error per entry, in the value's unit. The error score the model and the constant baseline are compared on; it is minimized by the mean, which is what the model forecasts.

top_decile_sharenumber

Share of all actual value found in the tenth of entries with the highest forecasts.

Example: 0.25
total_biasnumber

Sum of the forecasts over the sum of the actual values, minus one. What a platform summing the delivered values sees; +0.1 means the forecasts added up to 10% more than reality.

Example: 0.04
prior_entriesstringrequired

How many entries the person had before this one.

Allowed values: 0, 1, 2-3, 4+
cold_startobject

The day-0 cold start, what a scope delivers for a population member with no entry on the forecast's entry stream (fdy_forecast_<id>_basis of attributes or prior). The prior is always recorded; t...

Properties of object
attributes_offeredinteger

Third-party attributes the attributes model was offered.

entries_fromstring<date>

Entries from this date on were offered to the attributes model, the earliest date the account's third-party data covers. Absent when every training entry qualified.

modelobject

The attributes model, a second regression on the value of an entry given only the person's third-party attributes as of the entry date. Its performance is measured on the held-out age-0 entries that t...

Properties of object
attributesinteger

Attributes the model reads, those of the offered ones that any training entry filled.

balancenumber

Factor applied to the model's raw predictions so that they average to the training value.

boosting_roundsinteger

Boosting rounds the model kept.

fitobject

The same numbers on the entries the model fit on, for the gap to holdout.

Properties of object
gininumber

Normalized Gini of the ranking by forecast. 1 means entries are perfectly ordered by value, 0 means no better than random order, and a negative value means the ranking is inverted.

Example: 0.3
maenumber

Mean absolute error per entry. A descriptive number; it favors forecasting low when most entries realize nothing, so the comparison with the constant baseline is on rmse.

mean_actualnumber

Mean value actually realized.

mean_forecastnumber

Mean forecast of the value remaining.

ninteger

Held-out observations the numbers are measured on.

rmsenumber

Root mean squared error per entry, in the value's unit. The error score the model and the constant baseline are compared on; it is minimized by the mean, which is what the model forecasts.

top_decile_sharenumber

Share of all actual value found in the tenth of entries with the highest forecasts.

Example: 0.25
total_biasnumber

Sum of the forecasts over the sum of the actual values, minus one. What a platform summing the delivered values sees; +0.1 means the forecasts added up to 10% more than reality.

Example: 0.04
holdoutobject

Performance on the matched held-out age-0 entries; the number to set against by_age at age 0.

Properties of object
gininumber

Normalized Gini of the ranking by forecast. 1 means entries are perfectly ordered by value, 0 means no better than random order, and a negative value means the ranking is inverted.

Example: 0.3
maenumber

Mean absolute error per entry. A descriptive number; it favors forecasting low when most entries realize nothing, so the comparison with the constant baseline is on rmse.

mean_actualnumber

Mean value actually realized.

mean_forecastnumber

Mean forecast of the value remaining.

ninteger

Held-out observations the numbers are measured on.

rmsenumber

Root mean squared error per entry, in the value's unit. The error score the model and the constant baseline are compared on; it is minimized by the mean, which is what the model forecasts.

top_decile_sharenumber

Share of all actual value found in the tenth of entries with the highest forecasts.

Example: 0.25
total_biasnumber

Sum of the forecasts over the sum of the actual values, minus one. What a platform summing the delivered values sees; +0.1 means the forecasts added up to 10% more than reality.

Example: 0.04
holdout_rowsinteger

Matched entries held out from the fit, on which holdout is measured.

matched_rowsinteger

Offered entries the attributes describe well enough to fit or score.

matched_sharenumber

matched_rows over rows. How much of this population the attributes model, rather than the prior, speaks for.

mean_attribute_fillnumber

Mean share of the offered attributes that were present on an offered entry.

rowsinteger

Age-0 training entries offered to the fit, after entries_from.

trained_rowsinteger

Matched entries the model fit on.

priornumber

The average matured entry, the mean value of the training entries at age 0 after label handling. What a person Faraday's data does not describe is delivered, before the level correction.

prior_entriesinteger

Entries the prior averages.

skippedstring

Why no attributes model was fit, when none was; every person with no entry then gets the prior.

newest_entriesobject

The held-out entries of the newest year of the training population, the closest the training rows come to a read on the current business. Metrics and, when the model recorded it, the slice's own `tota...

Properties of object
by_agearray[object]
Items of array[object]
gininumber

Normalized Gini of the ranking by forecast. 1 means entries are perfectly ordered by value, 0 means no better than random order, and a negative value means the ranking is inverted.

Example: 0.3
maenumber

Mean absolute error per entry. A descriptive number; it favors forecasting low when most entries realize nothing, so the comparison with the constant baseline is on rmse.

mean_actualnumber

Mean value actually realized.

mean_forecastnumber

Mean forecast of the value remaining.

ninteger

Held-out observations the numbers are measured on.

rmsenumber

Root mean squared error per entry, in the value's unit. The error score the model and the constant baseline are compared on; it is minimized by the mean, which is what the model forecasts.

top_decile_sharenumber

Share of all actual value found in the tenth of entries with the highest forecasts.

Example: 0.25
total_biasnumber

Sum of the forecasts over the sum of the actual values, minus one. What a platform summing the delivered values sees; +0.1 means the forecasts added up to 10% more than reality.

Example: 0.04
ageintegerrequired

Days after the entry event at which the entries were scored.

constant_baselineobject

What forecasting the training rows' mean remaining value for every entry would have read on the same held-out observations. The model is compared with it on rmse, the error the mean minimizes; it ranks nothing.

Properties of object
maenumber

Mean absolute error of the constant forecast.

mean_forecastnumber

The constant forecast, the training rows' mean remaining value at this age.

rmsenumber

Root mean squared error of the constant forecast. A model whose rmse is lower explains some of the spread between entries; one whose rmse is higher does worse than forecasting the same value for everyone.

total_biasnumber

Total bias of the constant forecast.

decilesarray[object]

Calibration by forecast decile.

Items of array[object]
decileintegerrequired

1 is the tenth with the highest forecasts, 10 the lowest.

mean_actualnumber

Mean value actually realized in the decile.

mean_forecastnumber

Mean forecast of the value remaining in the decile.

nintegerrequired

Held-out observations in the decile.

share_of_actualnumber

This decile's share of all actual value in the held-out entries.

total_biasnumber

Sum of the forecasts over the sum of the actual values in this decile, minus one.

total_bias_intervalobject

Where the total bias lands over bootstrap resamples of the held-out observations. It measures how much of the reading is the particular entries drawn; it is not a forecast interval.

Properties of object
highnumber

Upper end of the band.

levelnumber

The band's coverage, as a share of the resamples it contains.

Example: 0.9
lownumber

Lower end of the band.

entered_fromstring<date-time>

Entries on or after this date are the newest year.

refreshesarray[object]

The newest daily refreshes of the current model, newest first, each with the level readings it recorded.

Items of array[object]
calibration_factornumber

The level correction the refresh applied.

drift_rationumber

The refresh's drift reading.

entriesinteger

Entries whose horizon was open at the refresh.

ran_atstring<date-time>

The refresh's data cutoff.

refreshed_atstring<date-time>required

When the refresh ran.

unmatched_value_event_sharenumber

With an entry_key_property_name, the share of value events whose key matched no entry at the refresh.

versionstring

Short hash of the forecast's definition, the version delivered with each entry. It changes when a defining member changes, not on a routine retrain.

400The request was invalid.
errorstringrequired

A Faraday error code.

Some possible values include:

Generic HTTP Errors

  • BAD_REQUEST: The request could not be validated.
  • FORBIDDEN: You do not have permission to access the specified resour...
Example: "ERROR_TYPE"
Allowed values: BAD_REQUEST, FORBIDDEN, MAX_RESOURCES_REACHED, INTERNAL_SERVER_ERROR, INVALID_AUTHORIZATION, NOT_FOUND, MALFORMED_API_KEY, MISSING_API_KEY, EXPIRED_API_KEY, VALIDATION_FAILED, CONFLICT, QUOTA_EXCEEDED
idstring<uuid>required

A unique ID for this error. Please include this in bug reports.

Example: "082f9513-901c-4308-8081-902a8fe22d7e"
notestringrequired

A human-readable description of the error.

Example: "An error occurred"
validationErrorsarray[object]

JSON Schema validation errors, if any.

Items of array[object]
contextobjectrequired

More information about the error.

Properties of object
errorTypestringrequired

The type of validation error which occurred.

messagestringrequired

A human-readable error message.

pathstringrequired

The location in the document that failed validation.

suggestionstring

A suggestion for fixing this error.

401No API key was supplied.
errorstringrequired

A Faraday error code.

Some possible values include:

Generic HTTP Errors

  • BAD_REQUEST: The request could not be validated.
  • FORBIDDEN: You do not have permission to access the specified resour...
Example: "ERROR_TYPE"
Allowed values: BAD_REQUEST, FORBIDDEN, MAX_RESOURCES_REACHED, INTERNAL_SERVER_ERROR, INVALID_AUTHORIZATION, NOT_FOUND, MALFORMED_API_KEY, MISSING_API_KEY, EXPIRED_API_KEY, VALIDATION_FAILED, CONFLICT, QUOTA_EXCEEDED
idstring<uuid>required

A unique ID for this error. Please include this in bug reports.

Example: "082f9513-901c-4308-8081-902a8fe22d7e"
notestringrequired

A human-readable description of the error.

Example: "An error occurred"
validationErrorsarray[object]

JSON Schema validation errors, if any.

Items of array[object]
contextobjectrequired

More information about the error.

Properties of object
errorTypestringrequired

The type of validation error which occurred.

messagestringrequired

A human-readable error message.

pathstringrequired

The location in the document that failed validation.

suggestionstring

A suggestion for fixing this error.

403Access to this resource was forbidden.
errorstringrequired

A Faraday error code.

Some possible values include:

Generic HTTP Errors

  • BAD_REQUEST: The request could not be validated.
  • FORBIDDEN: You do not have permission to access the specified resour...
Example: "ERROR_TYPE"
Allowed values: BAD_REQUEST, FORBIDDEN, MAX_RESOURCES_REACHED, INTERNAL_SERVER_ERROR, INVALID_AUTHORIZATION, NOT_FOUND, MALFORMED_API_KEY, MISSING_API_KEY, EXPIRED_API_KEY, VALIDATION_FAILED, CONFLICT, QUOTA_EXCEEDED
idstring<uuid>required

A unique ID for this error. Please include this in bug reports.

Example: "082f9513-901c-4308-8081-902a8fe22d7e"
notestringrequired

A human-readable description of the error.

Example: "An error occurred"
validationErrorsarray[object]

JSON Schema validation errors, if any.

Items of array[object]
contextobjectrequired

More information about the error.

Properties of object
errorTypestringrequired

The type of validation error which occurred.

messagestringrequired

A human-readable error message.

pathstringrequired

The location in the document that failed validation.

suggestionstring

A suggestion for fixing this error.

404The requested resource ID was not found.
errorstringrequired

A Faraday error code.

Some possible values include:

Generic HTTP Errors

  • BAD_REQUEST: The request could not be validated.
  • FORBIDDEN: You do not have permission to access the specified resour...
Example: "ERROR_TYPE"
Allowed values: BAD_REQUEST, FORBIDDEN, MAX_RESOURCES_REACHED, INTERNAL_SERVER_ERROR, INVALID_AUTHORIZATION, NOT_FOUND, MALFORMED_API_KEY, MISSING_API_KEY, EXPIRED_API_KEY, VALIDATION_FAILED, CONFLICT, QUOTA_EXCEEDED
idstring<uuid>required

A unique ID for this error. Please include this in bug reports.

Example: "082f9513-901c-4308-8081-902a8fe22d7e"
notestringrequired

A human-readable description of the error.

Example: "An error occurred"
validationErrorsarray[object]

JSON Schema validation errors, if any.

Items of array[object]
contextobjectrequired

More information about the error.

Properties of object
errorTypestringrequired

The type of validation error which occurred.

messagestringrequired

A human-readable error message.

pathstringrequired

The location in the document that failed validation.

suggestionstring

A suggestion for fixing this error.

500An internal server error occurred.
errorstringrequired

A Faraday error code.

Some possible values include:

Generic HTTP Errors

  • BAD_REQUEST: The request could not be validated.
  • FORBIDDEN: You do not have permission to access the specified resour...
Example: "ERROR_TYPE"
Allowed values: BAD_REQUEST, FORBIDDEN, MAX_RESOURCES_REACHED, INTERNAL_SERVER_ERROR, INVALID_AUTHORIZATION, NOT_FOUND, MALFORMED_API_KEY, MISSING_API_KEY, EXPIRED_API_KEY, VALIDATION_FAILED, CONFLICT, QUOTA_EXCEEDED
idstring<uuid>required

A unique ID for this error. Please include this in bug reports.

Example: "082f9513-901c-4308-8081-902a8fe22d7e"
notestringrequired

A human-readable description of the error.

Example: "An error occurred"
validationErrorsarray[object]

JSON Schema validation errors, if any.

Items of array[object]
contextobjectrequired

More information about the error.

Properties of object
errorTypestringrequired

The type of validation error which occurred.

messagestringrequired

A human-readable error message.

pathstringrequired

The location in the document that failed validation.

suggestionstring

A suggestion for fixing this error.

Tags

forecasts