Structuring a large dbt (data build tool) project requires shifting from a monolithic "dump-everything-here" mindset to a clean, modular software-engineering approach. The gold standard for achieving this scalability is the layers-based architecture popularized by fishtown analytics (dbt Labs) and refined by the community.

Here is a blueprint to structure your project for maximum maintainability and scale:

1. The Multi-Layered Architecture

Divide your models into clear, sequential schemas or directories. Data flows strictly forward from one layer to the next.

• Staging Layer (staging/ or stg/):
  • Purpose: Your first-line cleanup. One model per source table.
  • Rules: Do not join across sources here. Rename columns to your internal conventions, recast data types (e.g., string to timestamp), and handle light null-safety or filtering. Keep it 1:1 with source data.
• Intermediate Layer (intermediate/ or int/ - Optional):
  • Purpose: The heavy lifting. This is where you handle complex business logic, re-usable joins, unions, and grain-shifting before building final entities.
  • Rules: Models here are usually built on top of staging models. They should be modular building blocks, not final reporting tables.
• Marts Layer (marts/ or fct_ / dim/):
  • Purpose: Business-ready output. Structured as dimensional models (facts and dimensions) or wide reporting tables tailored for BI tools.
  • Rules: Named clearly (fct_orders, dim_customers ). This is the only layer downstream users and BI tools (like Looker, Tableau, or Preset) should query.

2. Recommended Directory Layout

Organize your folders either by domain/source in staging and by business entity in marts. A clean layout looks like this:

text``` my_dbt_project/ ├── analyses/ ├── macros/ ├── models/ │ ├── intermediate/ │ │ ├── finance/ │ │ │ (_int_finance_models.yml, int_payments_combined.sql) │ │ └── marketing/ │ └── marts/ │ ├── core/ │ │ (dim_customers.sql, fct_orders.sql, core.yml) │ └── finance/ │ (fct_revenue.sql, finance.yml) ├── seeds/ ├── snapshots/ ├── tests/ └── dbt_project.yml

Use code with caution.

3. Key Scaling Best Practices

- *Use Source Declarations (`sources.yml`):* Never hardcode raw table names in your staging models using `from source_db.schema.table` . Always define them in a `sources.yml` file and call them using the `{{ source('source_name', 'table_name') }}` function. This ensures lineage tracking works from day one.
- *Leverage Groups and Contracts (Enterprise):* For massive multi-team dbt projects, use **dbt mesh / contracts** to enforce data types and primary key constraints on public models, preventing upstream changes from breaking downstream BI dashboards.
- *DRY Up Code with Packages:* Don't reinvent the wheel. Use community-loved packages like [dbt_utils](https://google.com/goto?url=CAESaQHrOzAVz7SpA5YDNKGup-akzmYvWV6E6Hn8cS0hDxTis9vFLdRAcR3DEqHGL_tmbZm9X3YjW313Li7JDSkwe0eR5DO77VtmC4vxMZdAKwRX4CNpRiwSSTu2Wslu5t5QoG965riHwFs-Vg) for cross-database general macros and dbt_expectations for advanced data testing.
- *Enforce Naming Conventions:* Standardize prefixes so anyone on the team knows what a model is just by looking at its filename:
	- `stg_` for staging
	- `int_` for intermediate
	- `fct_` for fact tables (events, transactions)
	- `dim_` for dimension tables (entities, people, places)

Would you like me to dive deeper into **how to configure `dbt_project.yml`** for materializations, or do you want an example of a **staging-to-mart model workflow** for a specific domain like e-commerce or SaaS?

For a large dbt project, optimize for three things: clear ownership, predictable dependencies, and small units of change. A good structure should let someone unfamiliar with the project answer “where does this logic belong?” without asking another engineer.

The classic staging → intermediate → marts pattern is still a strong foundation, and dbt Labs continues to recommend it for separating source preparation, reusable transformations, and business-facing models.

1. Organize the repository by responsibility

A scalable starting point:

dbt_project/
├── dbt_project.yml
├── packages.yml
├── macros/
├── models/
│   ├── staging/
│   │   ├── salesforce/
│   │   │   ├── _salesforce__sources.yml
│   │   │   ├── stg_salesforce__accounts.sql
│   │   │   ├── stg_salesforce__contacts.sql
│   │   │   └── salesforce.yml
│   │   ├── stripe/
│   │   │   ├── _stripe__sources.yml
│   │   │   ├── stg_stripe__customers.sql
│   │   │   └── stg_stripe__payments.sql
│   │   └── app/
│   │
│   ├── intermediate/
│   │   ├── customers/
│   │   │   ├── int_customers__enriched.sql
│   │   │   └── int_customers__orders.sql
│   │   └── orders/
│   │       └── int_orders__enriched.sql
│   │
│   └── marts/
│       ├── core/
│       │   ├── dim_customers.sql
│       │   ├── dim_products.sql
│       │   └── fct_orders.sql
│       ├── finance/
│       ├── marketing/
│       └── product/
│
├── seeds/
├── snapshots/
├── tests/
└── analyses/

The important distinction is that staging is organized around source systems, while marts are organized around business domains. This makes ownership and discovery much easier as the number of sources and models grows.

2. Give each layer a strict contract

Staging: source-conformed

Think:

“Make this source usable and consistent.” A staging model should generally:

• Rename columns.
• Cast data types.
• Standardize timestamps.
• Normalize obvious values.
• Handle source-specific quirks.
• Perform basic deduplication when appropriate.

Avoid putting business logic here.

select
    id                 as customer_id,
    email              as customer_email,
    created_at::timestamp as created_at,
    status
from {{ source('stripe', 'customers') }}

The key benefit is that if Stripe changes created_at, you fix it once, rather than fixing dozens of downstream models. That's one of the main reasons dbt recommends a staging layer.

Use a predictable naming convention such as:

stg_<source>__<entity>

For example:

stg_salesforce__accounts
stg_salesforce__contacts
stg_stripe__customers
stg_stripe__payments

Intermediate: reusable transformation logic

This is where complicated work goes:

stg_customers
       │
stg_orders ──→ int_customers__orders
       │              │
stg_payments ────────┘
                      │
                      ↓
                dim_customers

Use intermediate models for:

• Multi-table joins.
• Complex transformations.
• Reusable business logic.
• Window calculations.
• Entity construction.
• Expensive transformations shared by multiple downstream models.

Name them around what they accomplish, e.g.:

int_orders__enriched
int_customers__orders
int_subscription__status

Don't let your marts become 800-line SQL queries simply because there's nowhere else to put the complexity.

dbt Labs explicitly recommends modularizing shared logic into its own models to keep the DAG and transformations manageable.

Marts: business-facing data products

Marts should answer:

“What data should analysts and applications actually consume?” Organize these by domain:

marts/
├── finance/
├── marketing/
├── product/
├── sales/
└── core/

And use recognizable model names:

dim_customers
dim_products
fct_orders
fct_payments

The dim_ / fct_ convention makes the intended grain and model type immediately recognizable.

3. Treat the DAG as an architecture, not just a dependency graph

A healthy large project tends to look roughly like:

                RAW SOURCES
                     │
          ┌──────────┴──────────┐
          ↓                     ↓
      STAGING                 STAGING
     Salesforce                Stripe
          │                     │
          └──────────┬──────────┘
                     ↓
                INTERMEDIATE
                     │
          ┌──────────┼──────────┐
          ↓          ↓          ↓
       FINANCE    PRODUCT    MARKETING
        MART       MART        MART

Avoid patterns like:

fct_orders
   ├── raw_orders
   ├── raw_customers
   ├── raw_payments
   ├── raw_products
   ├── another_mart
   └── some_random_report

That creates hidden coupling and makes changes increasingly dangerous.

A useful rule is:

Dependencies should generally flow downward through the layers, never sideways or backward. For example, a staging model shouldn't depend on a marketing mart.

4. Separate "shared" from "domain-specific"

This becomes particularly important around 100+ models.

Suppose both Finance and Marketing need customer lifetime value.

Don't create:

finance/int_customer_ltv.sql
marketing/int_customer_ltv.sql

with subtly different implementations.

Instead, establish a shared canonical model:

intermediate/
└── customers/
    └── int_customers__lifetime_value.sql

Then:

finance ──────┐
              ├──> int_customers__lifetime_value
marketing ────┘

This prevents business definitions from diverging.

But don't over-generalize everything into a giant "common" layer. Shared logic should actually be shared.

5. Make dbt_project.yml enforce the architecture

Don't rely solely on engineers remembering conventions.

For example:

models:
  my_project:

    staging:
      +materialized: view
      +schema: staging

    intermediate:
      +materialized: view
      +schema: intermediate

    marts:
      +materialized: table
      +schema: marts

Then override individual expensive models when necessary:

models:
  my_project:
    intermediate:
      +materialized: view

      orders:
        +materialized: table

    marts:
      +materialized: table

      finance:
        +schema: finance

Materialization should be driven by workload and reuse, not ideology. Staging views are common; heavily reused or expensive intermediate models may deserve tables/incremental materialization; marts are often persisted for consumers.

6. Keep tests and documentation next to the models

Don't create a giant:

models.yml

containing 500 models.

Instead:

staging/
└── stripe/
    ├── stg_stripe__customers.sql
    ├── stg_stripe__payments.sql
    └── stripe.yml

marts/
└── finance/
    ├── fct_payments.sql
    ├── dim_customers.sql
    └── finance.yml

That keeps ownership localized and makes PRs much easier to review. The dbt Project Evaluator similarly recommends keeping tests/documentation close to the models they describe.

For important models, document at least:

• What the model represents.
• Its grain.
• Ownership.
• Important business definitions.
• Primary/business keys.
• Known limitations.

For example:

models:
  - name: fct_orders
    description: >
      One row per customer order.

    columns:
      - name: order_id
        description: "Unique identifier for an order."
        tests:
          - unique
          - not_null

7. Treat model grain as a first-class concept

One of the biggest sources of dbt pain is ambiguous grain.

Every fact model should effectively answer:

What does one row represent? For example:

fct_orders
    grain = one row per order

fct_order_items
    grain = one row per order line item

fct_daily_customer_activity
    grain = one row per customer per day

Put that information in the model description or SQL comment and test the key.

A model called customer_metrics that could mean one row per customer, customer-day, customer-product, or customer-month is a maintenance problem waiting to happen.

8. Use macros sparingly

Macros are excellent for repeated technical patterns:

macros/
├── generate_surrogate_key.sql
├── safe_cast.sql
├── date_spine.sql
└── cents_to_dollars.sql

But don't put business logic into enormous macros just to avoid writing SQL.

Prefer:

int_customer_ltv.sql

over:

{{ calculate_everything_about_customers() }}

Models are much easier to discover, lineage-track, test, and debug.

9. Design CI around domains and selectors

At scale, you don't want every PR rebuilding the entire warehouse.

Use tags, selectors, state comparison, and DAG-aware CI so a change to:

models/marts/marketing/

doesn't unnecessarily rebuild unrelated Finance models.

For example, conceptually:

PR changes
    ↓
affected models
    ↓
their downstream dependencies
    ↓
tests

This becomes increasingly important as projects move from dozens to hundreds or thousands of models.

10. Establish ownership

Every important domain should have an owner:

Finance       → Finance Analytics
Marketing     → Marketing Analytics
Product       → Product Analytics
Core customer → Data Platform

You can encode ownership through metadata/tags and reinforce it with CODEOWNERS or your team's review system.

The goal is that when someone sees:

fct_revenue

they know:

• Who owns it?
• What is its grain?
• Who consumes it?
• What upstream sources does it depend on?
• What tests protect it?
• How frequently does it update?

11. Avoid premature micro-projects

I generally wouldn't split a large dbt codebase into many separate projects just because it has become large.

A single well-organized project often gives you:

• One dependency graph.
• Shared macros.
• Consistent conventions.
• Easier refactoring.
• Centralized CI.
• Easier lineage.

Split projects when you have genuine boundaries such as independent teams, security boundaries, release cycles, or operational requirements—not merely because the model count is high.

A practical architecture for a 500+ model project

If I were starting a large project today, I'd aim for something like:

models/
│
├── staging/
│   ├── salesforce/
│   ├── stripe/
│   ├── postgres/
│   ├── app/
│   └── ad_platforms/
│
├── intermediate/
│   ├── customers/
│   ├── orders/
│   ├── subscriptions/
│   └── marketing/
│
└── marts/
    ├── core/
    ├── finance/
    ├── marketing/
    ├── product/
    ├── sales/
    └── operations/

With these rules:

The most important principle is not the exact folder names. It's establishing a small number of architectural rules and making them hard to violate.

The rules I'd put in the README

1. Raw sources are referenced through source().
2. Staging models represent individual source entities.
3. Staging contains no business logic.
4. Complex/reusable transformations belong in intermediate.
5. Marts are organized by business domain.
6. Every mart has an explicit grain.
7. fct_ = fact; dim_ = dimension; int_ = intermediate; stg_ = staging.
8. Tests and documentation live close to their models.
9. Shared business logic has one canonical implementation.
10. CI should build only the affected DAG wherever practical.

That combination—layered architecture + domain ownership + explicit grain + modular DAG + automated standards—is what makes a dbt project scale. The directory structure itself is only the visible part of the architecture.

If you're dealing with a project that's already large and messy, the best strategy is usually incremental refactoring rather than a rewrite: establish the conventions, migrate one domain at a time, and use the DAG to identify duplicated or overly coupled logic.

Dependencies should generally flow downward through the layers, never sideways or backward. For example, a staging model shouldn't depend on a marketing mart.

4. Separate "shared" from "domain-specific"

This becomes particularly important around 100+ models.

Suppose both Finance and Marketing need customer lifetime value.

Don't create:

finance/int_customer_ltv.sql
marketing/int_customer_ltv.sql

with subtly different implementations.

Instead, establish a shared canonical model:

intermediate/
└── customers/
    └── int_customers__lifetime_value.sql

Then:

finance ──────┐
              ├──> int_customers__lifetime_value
marketing ────┘

This prevents business definitions from diverging.

But don't over-generalize everything into a giant "common" layer. Shared logic should actually be shared.

5. Make dbt_project.yml enforce the architecture

Don't rely solely on engineers remembering conventions.

For example:

Structuring a large dbt (data build tool) project requires moving away from the default "throw everything into the models folder" approach and adopting a modular, tiered architecture . The goal is to enforce clear boundaries, predictable lineage, and easy onboarding for new team members.

Here is a proven blueprint to structure a large-scale dbt project for long-term maintainability and scalability.

1. The Layered Directory Architecture

Organize your models/ directory into distinct operational layers. Data flows strictly downstream: Staging → Intermediate → Marts.

Layer 1: Staging (models/staging/)

The foundation. This is where you connect to your raw sources 1-to-1.

• Purpose: Rename columns to a standard convention, cast data types, handle light parsing (e.g., extracting JSON fields), and apply basic filters (like removing soft-deleted rows if necessary).
• Rules: No heavy business logic, no complex joins between different entities, and no aggregations. Every raw table should have one corresponding staging model (prefixed usually with stg_).
• Example path: models/staging/stripe/stg_stripe__payments.sql

Layer 2: Intermediate (models/intermediate/)

The transformation and preparation zone.

• Purpose: Handle complex business logic, heavy joins, un-nesting arrays, and bridging staging data into entity-centric shapes.
• Rules: Models here should be modular (e.g., joining orders with order items). They are usually ephemeral (built as CTEs in downstream models rather than physical tables/views in the warehouse) to save compute and storage, unless they are reused by multiple downstream marts.
• Example path: models/intermediate/finance/int__orders_enriched.sql

Layer 3: Marts (models/marts/)

The business-facing layer. This is what your BI tools (Looker, Tableau, etc.) and stakeholders query.

• Purpose: Present clean, highly aggregated, or dimensionalized data (Fact and Dimension tables).
• Rules: Grouped by business domain or department (e.g., finance, marketing, product ). Models here are materialized as tables or incremental models. They consume from staging or intermediate layers—never raw sources directly.
• Example path: models/marts/finance/fct_revenue.sql

2. File and Naming Conventions

Consistency is key to readability when a project hits hundreds of models.

• Prefixes: Use clear prefixes to denote the type of model at a glance:
  • stg_ for Staging
  • int_ for Intermediate
  • fct_ for Fact tables (Marts)
  • dim_ for Dimension tables (Marts)
• File Structure by Source/Domain: Under staging , group by source system (e.g., salesforce, zendesk, stripe ). Under marts , group by business domain (e.g., core, finance, product).

3. Essential Scaling Strategies

• Leverage dbt-expectations or Core Tests: Define unique and not_null tests on every primary/foreign key at the staging and mart layers. Use the dbt Package Hub to pull in community-tested assertion packages.
• Use Groups and Contracts (for Enterprise Scale): If multiple teams contribute to the same dbt repo, use dbt's Model Contracts and Access/Groups features to define public vs. protected interfaces, preventing upstream changes from silently breaking downstream team reports.
• Master Incremental Materializations Wisely: For large event/fact tables in your marts, switch to materialized='incremental' early, but ensure you write robust is_incremental() logic to handle late-arriving data and deduplication cleanly.
• Isolate Environment Variables & Secrets: Never hardcode database names or credentials. Use target.name or environment variables via {{ env_var('DBT_ENV') }} in your profiles.yml.

If you'd like, let me know:

• What data warehouse you are using (Snowflake, BigQuery, Databricks, etc.)
• Roughly how many models or data sources you currently have
• Whether you are dealing with a single team or multi-pod/cross-functional teams contributing to the project

I can dive deeper into incremental strategies or how to set up access controls for your specific setup.

For a large dbt project, I’d optimize for clear ownership, predictable dependencies, small models, and cheap selective builds rather than trying to minimize the number of files.

A good default architecture is:

dbt_project/
├── dbt_project.yml
├── packages.yml
├── models/
│   ├── staging/
│   │   ├── salesforce/
│   │   │   ├── src_salesforce.yml
│   │   │   ├── stg_salesforce.yml
│   │   │   ├── stg_salesforce__accounts.sql
│   │   │   └── stg_salesforce__opportunities.sql
│   │   ├── stripe/
│   │   │   ├── src_stripe.yml
│   │   │   ├── stg_stripe.yml
│   │   │   └── ...
│   │   └── product/
│   │
│   ├── intermediate/
│   │   ├── finance/
│   │   ├── marketing/
│   │   └── product/
│   │
│   └── marts/
│       ├── finance/
│       │   ├── dim_accounts.sql
│       │   ├── fct_revenue.sql
│       │   └── finance.yml
│       ├── marketing/
│       ├── product/
│       └── core/
│
├── macros/
├── tests/
├── snapshots/
├── seeds/
└── analyses/

This broadly follows dbt's recommended staging → intermediate → marts pattern.

1. Give each layer one job

Staging: normalize sources

A staging model should usually correspond to one source table:

source → stg_salesforce__accounts
source → stg_stripe__customers

Use it for:

• Renaming columns
• Type casting
• Standardizing timestamps
• Basic cleaning
• Light deduplication
• Establishing consistent keys

Avoid putting business logic or cross-source joins here. The goal is to create a stable interface between your raw systems and the rest of the project. dbt specifically recommends staging as a modular layer that reduces downstream duplication and code drift.

Intermediate: do the complicated work

This is where I'd put:

stg_orders
    +
stg_customers
    ↓
int_orders__enriched

Examples:

• Multi-table joins
• Attribution logic
• Sessionization
• Complex window functions
• Reusable business transformations
• Preparing data for multiple marts

An intermediate model should have one clear purpose. Don't create int_model_47 that becomes a dumping ground.

Marts: expose business concepts

Marts are the interface consumers should care about:

marts/
├── finance/
│   ├── fct_revenue
│   └── dim_customer
├── marketing/
│   ├── fct_campaign_performance
│   └── dim_campaign
└── product/
    ├── fct_events
    └── dim_user

Use fct_ and dim_ consistently where appropriate; dbt also uses this fact/dimension convention for mart models.

The important distinction is:

Layers describe transformation responsibility; domains describe business ownership. That's why I prefer marts/finance/, marts/marketing/, etc. over one enormous marts/ directory.

---

2. Organize staging by source, marts by domain

This is one of the most useful scaling rules.

Staging:

staging/
├── salesforce/
├── stripe/
├── postgres/
└── app/

Marts:

marts/
├── finance/
├── marketing/
├── product/
└── operations/

That makes two questions easy to answer:

"Where does this data come from?" → staging/stripe/

"Who owns this analytical dataset?" → marts/finance/

dbt's project-evaluator guidance similarly recommends source-specific staging directories and keeping model tests/documentation close to their models.

---

3. Treat naming conventions as an API

Establish conventions early and enforce them.

For example:

stg_<source>__<entity>
int_<entity>__<transformation>
fct_<business_event>
dim_<business_entity>

Examples:

stg_stripe__payments
stg_salesforce__opportunities

int_orders__enriched
int_customers__first_purchase

fct_orders
fct_subscription_revenue

dim_customers
dim_products

The prefixes communicate the model's role without opening the SQL file.

I'd also make grain explicit.

For example, at the top of fct_orders.sql:

-- Grain: one row per order

Then test that grain:

columns:
  - name: order_id
    data_tests:
      - not_null
      - unique

A model's grain is arguably more important than its filename.

---

4. Keep YAML next to the models

Avoid this:

models/
├── 400 SQL files
└── schema.yml

Instead:

finance/
├── fct_revenue.sql
├── dim_customer.sql
└── finance.yml

And:

stripe/
├── stg_stripe__customers.sql
├── stg_stripe__payments.sql
├── src_stripe.yml
└── stg_stripe.yml

This keeps documentation, tests, and code discoverable together and reduces giant YAML files and merge conflicts.

For very large directories, splitting YAML further by model is perfectly reasonable.

---

5. Be deliberate about materializations

Don't blindly make everything a table—or everything ephemeral.

A reasonable starting point is:

models:
  my_project:

    staging:
      +materialized: view

    intermediate:
      +materialized: view

    marts:
      +materialized: table

Then override based on actual workload.

For example:

{{ config(
    materialized='incremental',
    unique_key='order_id'
) }}

for a very large fact table.

Staging views are particularly useful because they provide a reusable normalization layer without unnecessarily duplicating the raw data.

I'd be cautious with ephemeral models. They can reduce database objects, but they make debugging harder because there's no relation you can query directly.

---

6. Don't create intermediate models just for the sake of decomposition

This is an easy trap:

int_orders_step_1
int_orders_step_2
int_orders_step_3
int_orders_step_4
int_orders_step_5
fct_orders

That's not necessarily modularity; it may just be fragmentation.

Create an intermediate model when it:

• Has a meaningful conceptual purpose
• Is reused
• Makes a complicated transformation understandable
• Creates a useful boundary between teams/domains
• Prevents a mart from becoming enormous

A 50-line model doesn't need to become five 10-line models.

---

7. Make the DAG your architecture

A healthy dependency direction looks like:

Sources
   ↓
Staging
   ↓
Intermediate
   ↓
Marts
   ↓
BI / ML / applications

Try hard to prevent:

mart → staging
mart_a → mart_b → mart_c
staging_a → staging_b

especially the last one.

Cross-domain dependencies should also be intentional. If Finance depends directly on 17 Marketing models, you've created an architectural coupling problem.

Instead, create stable shared entities:

staging
   ↓
intermediate/core
   ↓
dim_customer
   ↓
finance
marketing
product

Think of high-value marts/models as internal APIs.

---

8. Use tests as architectural guardrails

At minimum, test:

• Primary keys
• Referential integrity
• Accepted values
• Important business invariants
• Freshness of critical sources
• Incremental model behavior

For example:

models:
  - name: fct_orders
    description: "One row per customer order."

    columns:
      - name: order_id
        description: "Unique identifier for the order."
        data_tests:
          - not_null
          - unique

      - name: customer_id
        data_tests:
          - not_null
          - relationships:
              to: ref('dim_customers')
              field: customer_id

Don't try to test every column equally. Put the strongest guarantees around business-critical interfaces.

---

9. Make CI selective

At hundreds or thousands of models, running the entire project for every PR becomes painful.

Use dbt's graph selection capabilities so that a change can trigger something like:

changed model
   ↓
downstream models
   ↓
relevant tests

rather than rebuilding unrelated domains.

Also consider tags/groupings for things like:

finance
marketing
product

daily
hourly
critical
expensive

The goal is to make it cheap to answer:

"What is the smallest safe subset I need to run?"

---

10. Establish ownership

At large scale, folder structure alone isn't enough.

Every important model should have an owner:

fct_revenue
    owner: finance-data
    SLA: daily
    criticality: high

You want to be able to answer:

• Who owns this model?
• Who consumes it?
• What's its grain?
• What's its SLA?
• What happens if it breaks?
• Which upstream source does it depend on?

This is where dbt metadata, groups, tags, documentation, and your broader data catalog can become valuable.

---

11. Don't split into multiple dbt projects too early

A single repo/project can comfortably contain a lot of models if the boundaries are good.

I'd consider splitting when you have real organizational or technical boundaries, such as:

• Independent teams
• Independent deployment lifecycles
• Strong domain ownership
• Different SLAs
• Security/regulatory boundaries
• A DAG that has become difficult to reason about
• CI/deployment becoming a bottleneck

Don't split simply because you've reached "500 models."

A badly structured five-project setup is harder to maintain than a well-structured 1,000-model project.

---

12. A practical rule of thumb

For a project growing from ~50 to ~1,000 models, I'd evolve it roughly like this:

The key is not adding organizational complexity merely because the project is large.

The architecture I'd aim for

                    ┌── Finance marts
                    │
Sources → Staging → Intermediate → Shared/core models
                    │
                    ├── Marketing marts
                    │
                    └── Product marts
                              ↓
                         BI / ML / Apps

And enforce five principles:

1. Staging is source-oriented.
2. Intermediate is transformation-oriented.
3. Marts are business/domain-oriented.
4. Tests + documentation live beside the models they describe.
5. Every dependency should have an understandable reason to exist.

If you get those five right, the project can grow surprisingly large without becoming a maze.

Instead, create stable shared entities:

staging
   ↓
intermediate/core
   ↓
dim_customer
   ↓
finance
marketing
product

Think of high-value marts/models as internal APIs.

---

8. Use tests as architectural guardrails

At minimum, test:

• Primary keys
• Referential integrity
• Accepted values
• Important business invariants
• Freshness of critical sources
• Incremental model behavior

For example:

models:
  - name: fct_orders
    description: "One row per customer order."

    columns:
      - name: order_id
        description: "Unique identifier for the order."
        data_tests:
          - not_null
          - unique

      - name: customer_id
        data_tests:
          - not_null
          - relationships:
              to: ref('dim_customers')
              field: customer_id

Don't try to test every column equally. Put the strongest guarantees around business-critical interfaces.

---

9. Make CI selective

At hundreds or thousands of models, running the entire project for every PR becomes painful.

Use dbt's graph selection capabilities so that a change can trigger something like:

changed model
   ↓
downstream models
   ↓
relevant tests

rather than rebuilding unrelated domains.

Also consider tags/groupings for things like:

finance
marketing
product

daily
hourly
critical
expensive

The goal is to make it cheap to answer:

"What is the smallest safe subset I need to run?"

---

10. Establish ownership

At large scale, folder structure alone isn't enough.

Every important model should have an owner:

fct_revenue
    owner: finance-data
    SLA: daily
    criticality: high

You want to be able to answer:

• Who owns this model?
• Who consumes it?
• What's its grain?
• What's its SLA?
• What happens if it breaks?
• Which upstream source does it depend on?

This is where dbt metadata, groups, tags, documentation, and your broader data catalog can become valuable.

---

11. Don't split into multiple dbt projects too early

A single repo/project can comfortably contain a lot of models if the boundaries are good.

I'd consider splitting when you have real organizational or technical boundaries, such as:

• Independent teams
• Independent deployment lifecycles
• Strong domain ownership
• Different SLAs
• Security/regulatory boundaries
• A DAG that has become difficult to reason about
• CI/deployment becoming a bottleneck

Don't split simply because you've reached "500 models."

A badly structured five-project setup is harder to maintain than a well-structured 1,000-model project.

---

12. A practical rule of thumb

For a project growing from ~50 to ~1,000 models, I'd evolve it roughly like this:

The key is not adding organizational complexity merely because the project is large.

The architecture I'd aim for

                    ┌── Finance marts
                    │
Sources → Staging → Intermediate → Shared/core models
                    │
                    ├── Marketing marts
                    │
                    └── Product marts
                              ↓
                         BI / ML / Apps

And enforce five principles:

1. Staging is source-oriented.
2. Intermediate is transformation-oriented.
3. Marts are business/domain-oriented.
4. Tests + documentation live beside the models they describe.
5. Every dependency should have an understandable reason to exist.

If you get those five right, the project can grow surprisingly large without becoming a maze.

For a large dbt project, optimize for clear boundaries, predictable naming, reusable transformations, and ownership rather than simply organizing files into lots of folders.

dbt Labs' current guidance still centers on staging → intermediate → marts, with modular models and a DAG that makes dependencies explicit.

1. Use three primary transformation layers

A good default is:

models/
├── staging/
│   ├── salesforce/
│   │   ├── _sources.yml
│   │   ├── _models.yml
│   │   ├── stg_salesforce__account.sql
│   │   └── stg_salesforce__opportunity.sql
│   ├── stripe/
│   │   ├── _sources.yml
│   │   ├── _models.yml
│   │   └── stg_stripe__payment.sql
│   └── postgres/
│
├── intermediate/
│   ├── customers/
│   │   ├── int_customers__joined.sql
│   │   └── int_customers__enriched.sql
│   └── orders/
│
└── marts/
    ├── finance/
    │   ├── fct_revenue.sql
    │   └── dim_accounts.sql
    ├── marketing/
    │   └── fct_campaign_performance.sql
    └── product/
        ├── fct_product_usage.sql
        └── dim_products.sql

Staging

One staging model per useful source table.

Keep it deliberately boring:

• rename columns
• cast types
• standardize timestamps
• basic cleaning
• establish consistent keys
• light source-specific normalization

Avoid putting business logic here. The benefit is that changes to a raw source get handled in one place rather than replicated across dozens of downstream models. dbt specifically recommends staging as a modular foundation for this reason.

Naming:

stg_<source>__<entity>

stg_salesforce__account
stg_salesforce__opportunity
stg_stripe__payment

Intermediate

This is where complexity goes.

Use it for:

• multi-table joins
• reusable transformations
• deduplication
• entity construction
• complicated business logic that feeds multiple marts
• breaking enormous SQL models into understandable pieces

For example:

int_customers__joined
int_customers__deduplicated
int_orders__with_customer
int_orders__attributed

The important distinction is that intermediate models are implementation details, whereas marts are generally the stable interfaces consumed by analysts and downstream systems.

Marts

These are your business-facing data products.

Organize them by business domain:

marts/
├── finance/
├── marketing/
├── sales/
├── product/
└── operations/

Use names that communicate the grain and purpose:

fct_orders
fct_payments
fct_subscription_events

dim_customers
dim_products
dim_accounts

dbt Labs similarly uses fct_ and dim_ conventions for mart models.

---

2. Organize by source in staging, by domain in marts

This is one of the most useful rules at scale:

Staging → source-system organization

staging/
├── salesforce/
├── stripe/
├── zendesk/
└── postgres/

Marts → business-domain organization

marts/
├── finance/
├── marketing/
├── customer/
└── product/

That reflects two different questions:

"Where did this data come from?"

versus

"What business concept does this data represent?"

The dbt Project Evaluator also recommends source-specific staging directories and keeping model tests/documentation close to their models.

---

3. Treat the DAG as an architecture, not just a build graph

A common mistake is creating a project that looks organized in the filesystem but has a horrible DAG.

Aim for something conceptually like:

Raw sources
    ↓
Staging
    ↓
Intermediate
    ↓
Business entities
    ↓
Marts
    ↓
BI / ML / downstream applications

Avoid:

stg_customer ─────┐
                  ├──> fct_revenue
stg_orders ───────┤
                  ├──> dashboard_model
stg_payments ─────┘

stg_customer ─────────> another_mart
stg_orders ───────────> another_mart
stg_payments ─────────> another_mart

where every mart independently reconstructs the same customer/order/payment logic.

Instead, extract genuinely reusable transformations into intermediate or entity models. dbt Labs explicitly recommends abstracting code used across multiple models into its own model.

---

4. Don't turn every SQL query into a model

Scalability doesn't mean maximum modularity.

This is bad:

stg_orders
    ↓
int_orders_a
    ↓
int_orders_b
    ↓
int_orders_c
    ↓
int_orders_d
    ↓
fct_orders

if each intermediate model contains only 10 lines and is used once.

Prefer a new model when the transformation is:

1. reused, or
2. conceptually meaningful, or
3. complex enough to deserve its own unit of testing/debugging.

Otherwise, excessive model decomposition makes the DAG harder to understand and debugging slower.

---

5. Establish conventions and enforce them automatically

For a 20-model project, conventions are helpful.

For a 500-model project, they're essential.

Define things such as:

The dbt Project Evaluator can automatically identify violations of naming, directory, testing, and other structural conventions.

I'd put those checks into CI so architectural standards don't depend on code reviewers remembering them.

---

6. Keep YAML/documentation local

Don't create one gigantic:

models/schema.yml

containing documentation for hundreds of models.

Instead:

staging/
└── stripe/
    ├── _sources.yml
    ├── _models.yml
    ├── stg_stripe__customer.sql
    └── stg_stripe__payment.sql

marts/
└── finance/
    ├── _models.yml
    ├── fct_revenue.sql
    └── dim_accounts.sql

This reduces merge conflicts and makes it obvious where documentation for a model lives. The Project Evaluator specifically recommends keeping tests and documentation in the corresponding model directory.

---

7. Make the mart layer an API

One particularly powerful mindset for large projects:

Your marts are an API.

For example:

fct_orders
dim_customers
dim_products

should have:

• clearly defined grain
• documented columns
• predictable semantics
• reliable tests
• identifiable owner
• reasonable backwards-compatibility expectations

Downstream dashboards shouldn't need to understand:

stripe
salesforce
postgres
zendesk

or know how customer identity is resolved.

They consume:

fct_orders
    ↓
BI

This dramatically reduces coupling between source systems and consumers.

---

8. Define ownership by domain

At scale, folder structure alone isn't enough.

I'd assign ownership roughly like:

Finance
 ├── finance marts
 ├── revenue definitions
 └── finance tests

Marketing
 ├── marketing marts
 ├── attribution logic
 └── marketing tests

Product
 ├── product marts
 ├── engagement definitions
 └── product tests

Then establish:

• code owners/reviewers
• model owners
• SLAs
• data-quality expectations
• escalation paths

This prevents a 15-person analytics team from effectively having "everyone owns everything."

---

9. Use materializations deliberately

A reasonable starting point is:

staging       → view
intermediate  → view/table/ephemeral depending on use
marts         → table/incremental

But don't blindly make every intermediate model ephemeral. Ephemeral models can reduce warehouse objects, but they also make debugging harder. Likewise, a frequently reused expensive intermediate transformation may be better as a table or view.

dbt's own guidance emphasizes staging models as views in many cases, while allowing materialization choices based on workload and cost.

For large fact tables, incremental models can become important—but introduce them because the workload warrants them, not simply because the project is large.

---

10. Separate "core entities" from reports

A useful distinction is:

Business entities
    ↓
Reporting models

For example:

marts/customer/
    dim_customers.sql
    fct_customer_orders.sql

marts/finance/
    fct_revenue.sql

marts/reporting/
    rpt_monthly_board_metrics.sql
    rpt_sales_pipeline.sql

Don't let dashboard-specific SQL become the canonical representation of a business concept.

If three dashboards need "customer lifetime value," you generally want one governed definition rather than:

dashboard_a_customer_ltv
dashboard_b_customer_ltv
dashboard_c_customer_ltv

---

11. Optimize for selective builds

A large dbt project should make it easy to run things like:

dbt build --select state:modified+

or domain-specific subsets such as:

dbt build --select marts.finance+

That becomes much more practical when:

• dependencies are clean
• domains are separated
• models have consistent tags/groups
• tests live with their models
• ownership is clear

Your CI should generally avoid rebuilding the entire warehouse for every pull request.

---

12. Don't split into multiple dbt projects too early

For a large organization, eventually you may have a case for multiple projects:

finance-dbt
product-dbt
marketing-dbt

But don't do this simply because you reached 200 models.

A well-organized monorepo can go surprisingly far.

Consider splitting when you have real organizational or dependency boundaries—for example, independently managed domains, separate deployment lifecycles, or a project becoming genuinely difficult to operate as one unit.

---

A practical architecture I'd use

For a mature ~500-model project, I'd start around this shape:

analytics/
│
├── dbt_project.yml
├── packages.yml
│
├── models/
│   │
│   ├── staging/
│   │   ├── salesforce/
│   │   ├── stripe/
│   │   ├── postgres/
│   │   └── zendesk/
│   │
│   ├── intermediate/
│   │   ├── customer/
│   │   ├── orders/
│   │   ├── subscriptions/
│   │   └── marketing/
│   │
│   └── marts/
│       ├── finance/
│       ├── sales/
│       ├── marketing/
│       ├── customer/
│       └── product/
│
├── macros/
├── tests/
├── snapshots/
├── seeds/
└── analyses/

And I'd enforce these architectural rules:

Raw → staging → intermediate → marts

Staging is source-oriented.

Marts are domain-oriented.

Business logic doesn't live in staging.

BI consumes marts, not raw/intermediate models.

Reusable logic gets its own model.

Every model has an owner, description, and appropriate tests.

Naming and directory conventions are CI-enforced.

The DAG should be understandable without opening every SQL file.

That's the combination that tends to make a dbt codebase scale—not any particular folder hierarchy by itself. dbt's recent guidance similarly emphasizes layered modeling, modularity, and keeping the DAG understandable as projects grow.

If you're starting from an existing messy 200–1000 model dbt project, the strategy is different: I would not reorganize everything at once. I'd first map the DAG, identify duplicate business logic and high-centrality models, establish naming/ownership rules, and then migrate domain by domain.

Structuring a large dbt (data build tool) project requires moving away from a single, chaotic folder of models and adopting a modular, layered architecture . This ensures maintainability, clear lineage, and scalability as your team and data warehouse grow.

Here is a proven framework for structuring a large dbt project:

• Layer 1: Staging (stg) — The foundation layer where you connect to your raw source tables. Models here should have a 1:1 relationship with raw sources. Focus on light cleaning, renaming columns to a consistent convention, casting data types, and handling simple derivations. No business logic or heavy joins belong here.
• Layer 2: Intermediate (int) — Optional but highly recommended for complex projects. This is where the heavy lifting happens—joins, un-pivoting, aggregates, and business logic that is shared across multiple data marts. Intermediate models should combine staging models, but they should not be exposed directly to end-users or BI tools.
• Layer 3: Marts (marts) — The final consumption layer organized by business domains or entities (e.g., finance, marketing, core ). Models here represent business concepts (like dim_customers or fct_orders ) and are optimized for BI tools and stakeholder queries.
• Layer 4: Semantics / Metrics — Defining metrics, dimensions, and semantic layers using dbt's native semantic layer configurations so definitions remain consistent across the entire organization.

Key Organizational Best Practices

• Domain-Driven Sub-folders: Inside your marts and intermediate folders, break your models down by business domain (e.g., marts/finance/, marts/marketing/ ) rather than technical type.
• Consistent Naming Conventions: Use prefixes to make model intent instantly recognizable:
  • stg_ for staging models.
  • int_ for intermediate models.
  • dim_ for dimension tables.
  • fct_ for fact tables.
• Granular dbt_project.yml Configurations: Apply folder-level configurations (like materialized views, tags, or meta properties) in your dbt_project.yml rather than repeating {{ config(...) }} macros inside every single model file.
• Robust Testing & Documentation: Enforce not just standard unique and not_null tests, but relationship tests and custom data tests. Maintain your schema.yml files close to the models they document.

Would you like me to dive deeper into:

• Folder directory trees and file layouts?
• How to configure materializations (tables vs. views vs. incremental) across these layers?
• Managing cross-project dependencies using dbt mesh?

Structuring a large dbt (data build tool) project requires a modular, layered architecture to keep code clean, scalable, and easy to maintain. The industry standard approach organizes models into distinct functional layers, moving progressively from raw data to business-ready reporting.

1. The Core Layered Architecture

Organize your models/ directory into clear functional subdirectories (schemas/layers):

• Staging (models/staging/ or staging/)
  • Purpose : The first layer. Connect directly to your raw tables/views (sources).
  • Rules : One model per raw source table. Perform light cleaning only (renaming columns to a standard convention, casting data types, handling basic nulls). No joins or heavy aggregations here.
  • Naming: stg_<source_name>__.sql (e.g., stg_stripe__customers.sql).
• Intermediate (models/intermediate/ or intermediate/)
  • Purpose : The optional, heavy-lifting layer where the complex logic lives.
  • Rules : Do your heavy lifting here—joins between staging models, un-nesting arrays, pivot operations, and business logic preparation. Models in this layer are usually building blocks (ephemeral or views) rather than final end-user tables.
  • Naming: int__<verb>_.sql (e.g., int_orders__joined_with_payments.sql).
• Marts (models/marts/ or marts/)
  • Purpose : The final business-facing layer, structured around business domains (e.g., finance, marketing, product).
  • Rules : Highly structured, dimensional models (facts and dimensions) ready for BI tools and stakeholders. Queries should be clean and readable because most complex joins were handled in the intermediate layer.
  • Naming: <domain>__fct_<name>.sql or <domain>__dim_<name>.sql (e.g., finance__fct_revenue.sql, core__dim_customers.sql).

2. Best Practices for Scalability

• Use Sources (sources.yml) : Always define your raw data inputs in a _sources.yml file within your staging folders. Use the {{ source('source_name', 'table_name') }} macro in your staging models instead of hardcoding raw table names.
• Leverage Ephemeral Materialization : For intermediate models that are only used once or twice downstream, use {{ config(materialized='ephemeral') }} . This compiles them as CTEs in the downstream models rather than cluttering your data warehouse with physical intermediate tables/views.
• Isolate Tests and Documentation : Keep your schema.yml (or domain-specific yaml files) right next to the models they document. Define primary key tests (unique and not_null ) for every single dimension and fact table.
• Use Tags and Groups : Apply tags (e.g., finance, daily, hourly ) to models so you can run subsets of your DAG efficiently using dbt run --select tag:finance.
• Adopt a Naming Convention : Consistency is key. Establish a strict prefix/suffix standard across your team for staging (stg_ ), intermediate (int_ ), dimensions (dim_ ), and facts (fct_).

If you'd like to dive deeper, let me know:

• What data warehouse are you using (Snowflake, BigQuery, Databricks, etc.)?
• Are you migrating an existing monolithic project or starting a brand new one?

I can share a custom directory tree or specific configuration examples for your setup.

A large dbt project stays maintainable when the structure reflects data flow, ownership, and reuse rather than just mirroring every source system or dashboard. A common scalable pattern is:

1. Staging → clean source-aligned data
2. Intermediate → reusable business logic
3. Marts → business-facing datasets

This layered approach is widely used because it limits duplication and makes lineage easier to understand.

A practical structure:

dbt_project/
├── models/
│   ├── staging/
│   │   ├── stripe/
│   │   │   ├── stg_stripe__customers.sql
│   │   │   ├── stg_stripe__payments.sql
│   │   │   └── stripe.yml
│   │   └── salesforce/
│   │       ├── stg_salesforce__accounts.sql
│   │       └── salesforce.yml
│   │
│   ├── intermediate/
│   │   ├── customers/
│   │   │   ├── int_customers__unified.sql
│   │   │   └── customers.yml
│   │   └── revenue/
│   │       └── int_revenue__monthly.sql
│   │
│   └── marts/
│       ├── finance/
│       │   ├── fct_revenue.sql
│       │   └── dim_accounts.sql
│       ├── marketing/
│       │   └── dim_campaigns.sql
│       └── product/
│           └── fct_usage.sql
│
├── macros/
├── snapshots/
├── seeds/
├── tests/
├── analyses/
└── dbt_project.yml

1. Organize staging by source

Staging should answer: "How do I make raw data consistent?"

Examples:

stg_stripe__payments
stg_salesforce__accounts
stg_app__events

Responsibilities:

• Rename columns
• Cast types
• Standardize timestamps
• Clean obvious source issues
• Add simple keys

Avoid:

• Business metrics
• Complex joins
• Customer segmentation logic

A staging model should usually represent one source table.

---

2. Organize intermediate models by business capability

Intermediate models answer: "How do we transform clean data into reusable business concepts?"

Examples:

int_customer__lifetime_value
int_orders__with_returns
int_subscription__status_history

Good uses:

• Joining multiple staging models
• Deduplicating entities
• Creating reusable calculations
• Preparing data for multiple marts

Avoid exposing intermediate models directly to analysts unless there is a strong reason.

---

3. Organize marts by business domain

Marts answer: "What datasets do consumers actually use?"

Typical domains:

marts/
├── finance/
├── marketing/
├── sales/
├── product/
└── operations/

Common naming:

dim_customer
dim_product
fct_orders
fct_revenue

Dimensions describe entities; facts describe measurable events.

---

4. Use naming conventions aggressively

A large project should make model purpose obvious from the name.

Example:

Avoid names like:

final_orders.sql
customer_model.sql
new_table.sql

---

5. Keep YAML close to models

Put documentation and tests near the models they describe:

marts/
└── finance/
    ├── fct_revenue.sql
    └── finance.yml

Example:

models:
  - name: fct_revenue
    description: Monthly revenue by customer
    columns:
      - name: revenue_id
        tests:
          - unique
          - not_null

This improves discoverability and reduces forgotten documentation.

---

6. Use dbt_project.yml for defaults

Avoid repeating configs everywhere.

Example:

models:
  analytics:
    staging:
      +materialized: view

    intermediate:
      +materialized: ephemeral

    marts:
      +materialized: table

Then override only when needed:

{{ config(
    materialized='incremental',
    unique_key='order_id'
) }}

Common pattern:

• Staging → views
• Intermediate → ephemeral/views
• Large facts → incremental
• Marts → tables

---

7. Add ownership and governance

For a large team, metadata matters.

Useful additions:

meta:
  owner: finance_team
  contains_pii: true
  criticality: high

Then use it for:

• Ownership routing
• Alerts
• Access policies
• Documentation

---

8. Avoid these scaling problems

❌ One giant models folder

Bad:

models/
├── customers.sql
├── orders.sql
├── revenue.sql
├── ...
(500 files)

Hard to navigate and own.

---

❌ Marts directly reading raw sources

Bad:

fct_sales.sql
    |
    raw_orders

Better:

fct_sales
    |
int_orders
    |
stg_orders
    |
raw_orders

---

❌ Putting all logic in marts

A 1,000-line mart becomes impossible to test. Move reusable pieces into intermediate models.

---

9. Consider splitting projects when very large

A single dbt project can work for hundreds of models, but very large organizations often move toward domain ownership:

dbt_finance/
dbt_marketing/
dbt_product/

with shared packages or cross-project dependencies.

---

A good rule of thumb

For a mature dbt project:

• Staging: mirrors sources
• Intermediate: mirrors reusable transformations
• Marts: mirrors business questions
• Folders: show ownership
• Names: explain purpose
• Tests/docs: ship with every model

The filesystem should let a new engineer answer: "What is this model, who owns it, and what breaks if I change it?" without opening the SQL.

Structuring a large dbt (data build tool) project requires moving away from a single, chaotic directory of models and adopting a modular, tiered architecture . This mirrors software engineering best practices by separating concerns, enforcing data contracts, and making lineage easy to trace.

Here is how to structure a large-scale, maintainable dbt project:

• Staging Layer (models/staging/) : The first stop for raw data. Here, you connect to your sources. Models in this layer should have a 1-to-1 relationship with source tables.
  • Rules : Rename columns to a consistent standard (e.g., snake_case), cast data types (e.g., ensuring IDs are strings and timestamps are proper datetimes), and do light cleaning. Do not do heavy aggregations or complex joins here.
  • Organization : Subdivide by source system (e.g., models/staging/stripe/, models/staging/salesforce/).
• Intermediate Layer (models/intermediate/) : Optional, but crucial for large projects. This is where you clean, prep, and join data before building final business entities.
  • Rules : Perform business logic prep, early joins (e.g., joining order items to orders), and filtering. Models here are modular building blocks and are usually not exposed directly to end-users or BI tools.
  • Organization : Subdivide by business domain or entity (e.g., models/intermediate/finance/, models/intermediate/marketing/).
• Marts Layer (models/marts/) : The final destination. This layer contains business-ready, highly polished data models (dimensional models like facts and dimensions, or wide analytical tables) consumed by BI tools, stakeholders, or reverse-ETL pipelines.
  • Rules : High-level aggregations, final business logic, and dimensional modeling. Models here should be clean, well-documented, and heavily tested.
  • Organization : Subdivide by business domain (e.g., models/marts/core/, models/marts/finance/, models/marts/product/).
• Supporting Directories:
  • *sources/ * : Keep source definitions (and freshness configs) clean, either in a dedicated _sources.yml file per staging directory or a centralized folder.
  • *seeds/ * : Static CSV files managed by dbt (e.g., country codes, mapping tables).
  • *macros/ * : Reusable SQL or Jinja snippets. Keep them organized by function (e.g., generic tests, custom SQL operations).
  • *snapshots/ * : For implementing Type 2 Slowly Changing Dimensions (SCD) on mutable source tables.
• Best Practices for Scalability:
  • Use Source Freshness : Set up freshness blocks in your source YAMLs to catch broken upstream pipelines early.
  • Enforce Tests & Documentation : Leverage not_null and unique on primary keys in your marts, and define descriptions in schema.yml files alongside your models.
  • Leverage Tags and Groups : Use tags to run subsets of your project (e.g., --select tag:hourly ) and groups/access controls if using dbt Mesh for multi-project architectures.

If you'd like to dive deeper, let me know:

• Are you using a multi-project architecture (dbt Mesh) or a single monolithic project?
• What data warehouse (Snowflake, BigQuery, Databricks, etc.) are you targeting?

I can provide a visual folder tree template or specific naming convention guidelines tailored to your setup.

For a large dbt project, optimize for three things above all: clear ownership, predictable lineage, and small blast radius when something changes.

A good default is the classic staging → intermediate → marts architecture. dbt itself recommends these layers as progressively higher-level transformations.

1. Organize by layer, then by domain

A structure like this scales well:

models/
├── staging/
│   ├── stripe/
│   │   ├── _stripe__sources.yml
│   │   ├── _stripe__models.yml
│   │   ├── stg_stripe__customers.sql
│   │   └── stg_stripe__payments.sql
│   ├── salesforce/
│   │   ├── _salesforce__sources.yml
│   │   └── ...
│   └── shopify/
│       └── ...
│
├── intermediate/
│   ├── customers/
│   │   ├── int_customers__enriched.sql
│   │   └── int_customers__deduped.sql
│   ├── orders/
│   │   └── int_orders__with_customer.sql
│   └── ...
│
└── marts/
    ├── finance/
    │   ├── dim_customers.sql
    │   ├── fct_payments.sql
    │   └── _finance__models.yml
    ├── sales/
    │   ├── dim_products.sql
    │   ├── fct_orders.sql
    │   └── _sales__models.yml
    └── marketing/
        └── ...

The important distinction is:

• Staging = source-shaped. Rename, cast, standardize, lightly clean.
• Intermediate = reusable transformation logic. Joins, deduplication, complex business transformations, grain changes.
• Marts = consumer-facing business models. Facts, dimensions, aggregates, or other models that analysts/BI tools actually consume.

This prevents business logic from leaking into source-specific models and makes the DAG tell a coherent story.

2. Make staging roughly 1:1 with source tables

For example:

raw.stripe.customers
        ↓
stg_stripe__customers
        ↓
int_customers__enriched
        ↓
dim_customers

A staging model should establish your canonical representation of the source:

select
    id                 as customer_id,
    email,
    created_at,
    cast(is_deleted as boolean) as is_deleted
from {{ source('stripe', 'customers') }}

Then downstream models don't have to know that Stripe called the field id, or that its source data type was inconvenient.

This also gives you a useful boundary: source-system changes are absorbed in staging rather than propagated throughout the project.

3. Treat marts as an API

This is one of the most useful mental models for a big project.

Your mart models are effectively the public API of your warehouse.

That means:

• stable names
• well-defined grain
• documented columns
• explicit tests
• minimal surprises
• consumers shouldn't need to understand your intermediate models

For example:

fct_orders
  grain: one row per order

dim_customers
  grain: one row per customer

fct_order_items
  grain: one row per order item

Make the grain explicit in model documentation. Grain mistakes are among the most damaging problems in analytics projects.

4. Organize ownership around business domains

Once you have hundreds of models, a purely technical organization becomes painful.

Prefer:

marts/
├── finance/
├── sales/
├── marketing/
├── product/
└── operations/

rather than:

marts/
├── tables/
├── aggregates/
├── joins/
└── reports/

The former lets someone answer "where does the revenue model live?" immediately.

Folders should describe the business problem/domain the model serves, rather than which engineer happened to create it.

5. Keep YAML close to the models

Don't create one gigantic schema.yml containing thousands of models.

Instead:

staging/stripe/
    _stripe__sources.yml
    _stripe__models.yml

marts/finance/
    _finance__models.yml

marts/sales/
    _sales__models.yml

Folder-level YAML is a useful compromise: documentation and tests remain discoverable without creating one YAML file per model.

At minimum, important production models should document:

• purpose
• grain
• owner
• important columns
• upstream sources
• freshness expectations
• known caveats

6. Establish strict dependency rules

I'd enforce rules like:

staging     → sources only
intermediate → staging + other intermediate
marts       → intermediate + staging

Avoid:

staging → marts
marts   → staging
sales   → marketing → sales

especially when these dependencies create cycles of business logic.

A useful rule is:

Business logic flows downstream; it doesn't flow sideways and backward.

There will be exceptions, but requiring an explicit reason for them keeps the DAG sane.

7. Don't create intermediate models just for the sake of layers

A common failure mode is turning this:

source → transformation → mart

into:

source → stg → int_1 → int_2 → int_3 → int_4 → mart

That's technically organized but operationally terrible.

Create an intermediate model when it provides something meaningful:

• reusable logic
• a meaningful business concept
• a useful grain
• a performance/materialization boundary
• a testable transformation

If a transformation is tiny and used once, it may belong directly in the downstream model.

8. Have a deliberate materialization strategy

Don't let every developer independently choose table, view, incremental, or ephemeral.

A reasonable starting policy is:

Then override based on actual workload.

For example:

models:
  my_project:
    staging:
      +materialized: view

    intermediate:
      +materialized: view

    marts:
      +materialized: table

Large append-heavy fact tables are good candidates for incremental models, but don't automatically make everything incremental—the complexity and correctness requirements aren't free.

9. Make testing proportional to business risk

At minimum, important entities should have tests around:

primary key → unique + not_null
foreign key → relationships
categorical fields → accepted_values
source → freshness

But don't stop at generic tests.

For example, if:

fct_orders
grain = one row per order

you should have a test that actually verifies that assumption.

Put tests and documentation near the model they describe so they're difficult to forget when the model changes. dbt's project-evaluator guidance specifically emphasizes keeping model tests/documentation organized with their corresponding directories.

10. Use contracts at important boundaries

For a large organization, I'd distinguish between:

internal implementation models

and

stable consumer-facing models.

Your marts/semantic-facing models are where contracts become especially valuable. Consumers should be able to depend on:

fct_orders.order_id
fct_orders.customer_id
fct_orders.order_total
fct_orders.order_status

without caring whether you internally rewrote five intermediate models.

That dramatically reduces the blast radius of refactoring.

11. Scale CI around the DAG

Running the entire project for every pull request doesn't scale.

A mature setup typically has:

PR
 ↓
lint / parse
 ↓
modified models + relevant parents
 ↓
tests
 ↓
merge
 ↓
production build

Use state-aware/slim CI where appropriate so a developer changing one model doesn't have to rebuild thousands of unrelated models.

12. Add automated architectural guardrails

Once you have hundreds or thousands of models, conventions shouldn't live only in a wiki.

Automate checks for things such as:

• model naming
• missing descriptions
• missing primary-key tests
• staging models referencing other staging models inappropriately
• marts depending directly on raw sources
• excessive DAG depth
• unused models
• undocumented sources
• inconsistent folder structure

dbt-project-evaluator is one option specifically designed to evaluate structural conventions in dbt projects.

---

The architecture I'd recommend

For most large teams, I'd start here:

dbt_project/
│
├── models/
│   ├── staging/
│   │   ├── stripe/
│   │   ├── salesforce/
│   │   └── shopify/
│   │
│   ├── intermediate/
│   │   ├── customers/
│   │   ├── orders/
│   │   └── products/
│   │
│   └── marts/
│       ├── finance/
│       ├── sales/
│       ├── marketing/
│       └── product/
│
├── macros/
├── tests/
├── snapshots/
├── seeds/
├── analyses/
├── dbt_project.yml
└── README.md

And enforce these principles:

Source systems → staging → reusable business logic → domain marts → consumers

The biggest scalability win isn't a clever directory tree. It's establishing clear boundaries so that developers know where logic belongs, what they can depend on, and what they are allowed to change without breaking everyone downstream.