Skip to content

Datastore Fetcher

Fetch dataset resource data from the DataStore.

This fetcher is used to fetch data from the DataStore using the resource ID.

__init__(resource_id, resource_view_id=None, settings=None, limit=1000, cache_strategy=None)

Initialize the DatastoreDataFetcher.

PARAMETER DESCRIPTION
resource_id

The ID of the resource to fetch data for.

TYPE: str

settings

The settings for the chart.

TYPE: dict[str, Any] DEFAULT: None

limit

The maximum number of rows to fetch.

TYPE: int DEFAULT: 1000

cache_strategy

The cache strategy to use. If not provided, the configured cache strategy will be used.

TYPE: str DEFAULT: None

build_sort_clauses()

Build sort clauses for SQL query based on settings.

cache_column_metadata(columns)

Cache the complete column list for the resource.

PARAMETER DESCRIPTION
columns

List of column names to cache

TYPE: list[str]

fetch_data()

Fetch data from the DataStore.

RETURNS DESCRIPTION
DataFrame

pd.DataFrame: Data from the DataStore

get_all_column_names()

Get all column names from the table, excluding system columns.

This method doesn't format the columns, just returns their names. It's used both internally and can be used by form builders.

RETURNS DESCRIPTION
list[str]

List of all column names in the resource

get_needed_columns()

Extract a set of all column names required for chart generation.

This includes: - Fields explicitly defined in the chart settings (e.g., 'x', 'y', 'color', etc) - Filter columns used in the 'filter' expression (e.g., 'column:value|...')

invalidate_cache()

Invalidate the cache for the data fetcher.

make_cache_key()

Generate a cache key for the DataStore data fetcher.

Uses both resource_id and resource_view_id (if available) as part of the cache key.

RETURNS DESCRIPTION
str

The cache key

TYPE: str

make_metadata_cache_key()

Generate a cache key for the resource metadata (column list).

RETURNS DESCRIPTION
str

The metadata cache key

parse_filters()

Parse filter string from settings into SQLAlchemy query.

RETURNS DESCRIPTION
list[ClauseElement]

List of SQLAlchemy filter query (e.g., column == value).

update_view_id(resource_view_id)

Update the resource view ID after view creation.

If data was cached with just the resource_id, this method will move the cached data to a key that includes the view_id.

PARAMETER DESCRIPTION
resource_view_id

The new resource view ID

TYPE: str