Skip to main content

Snowflake Dynamic Tables as virtual assets

In this example, you'll build a pipeline with Dagster that:

  • Represents Snowflake Dynamic Tables as AssetSpec(is_virtual=True) — unexecutable in Dagster, refreshed automatically by Snowflake
  • Preserves full asset lineage through virtual assets to real upstream sources
  • Uses AutomationCondition.eager().resolve_through_virtual() so a downstream asset re-runs whenever source data changes, even though its direct upstreams are virtual
  • Monitors Dynamic Table freshness with a sensor that emits AssetObservation events

Prerequisites​

To follow the steps in this guide, you'll need:

  • Basic Python knowledge
  • Python 3.10+ installed on your system. For more information, see the Installation guide.
  • A Snowflake account with a warehouse

Step 1: Set up your Dagster environment​

First, set up a new Dagster project.

  1. Clone the Dagster repo and navigate to the project:

    cd examples/docs_projects/project_snowflake_dynamic_tables

  2. Install the required dependencies with uv:

    uv sync

  3. Activate the virtual environment:

    source .venv/bin/activate

  4. Copy .env.example to .env and fill in your Snowflake credentials:

    cp .env.example .env

Step 2: Create the Snowflake tables​

Run the DDL in sql/create_dynamic_tables.sql against your Snowflake account to create the source tables and Dynamic Tables used in this example.

Step 3: Launch the Dagster webserver​

Navigate to the project root directory and start the Dagster webserver:

dg dev

Next steps​

Continue this example by defining virtual assets.